GitHub Actions Security Review: test results

Verdicts

Date

2026-10-19

Strong · claude-sonnet-5-5 (Claude Code alias "sonnet")

Right on all 3 cases, read by hand. On an issue-triage workflow it named the issue title pasted into a shell step as one line and left out the issue body passed through an env variable and the first-party checkout pinned to a tag. On a pull_request_target workflow it gave exactly two lines, the checkout of the pull request head that is then built and tested, and the community action pinned to a tag, and left out the action pinned to a commit hash. On a clean deploy workflow it answered exactly No findings.

Weak · claude-haiku-5-5 (Claude Code alias "haiku")

Right on 2 of 3 cases, read by hand. It found the title injection and kept the safe env-passed body out, and it answered exactly No findings on the clean workflow. On the pull_request_target workflow it gave both expected lines and a third, a secret exposure for the token the checkout step leaves in the git config; the skill defines that code for printed secrets and secrets handed to code you do not control, so the check counts the extra line as a miss. Read by hand it is a defensible remark, but it is outside the definition, and it is recorded as a wrong answer.

Note

Light test: three cases (two planted, one clean), one run per model; no comparison without the skill. The three workflows were written by us; each planted one carries a safe look-alike (untrusted text passed through an env variable, first-party actions pinned to a version tag, a community action pinned to a commit hash), and neither planted case has a comment pointing at the defect. Facts come from the public security guidance of the platform, read on 19 October 2026; there are no owner-measured facts.

What was not measured

• Models other than the two named above were not run.
• Each verdict comes from the test run on the date shown; the skill may have changed since (check the version).
• Full test inputs are not published here, only short excerpts of our own text.
• Results on your own texts, languages and domains can differ.