Skip to main content
Version: 0.1 (next)

CI resolver

Diagnoses a failed CI run on a Change from its logs, tells a flaky failure from a real one, and fixes the cause on the Change branch.

Nameci-resolver
CategoryChange review
Enabled by defaultYes
Budgetup to $4.00 per run, 80 turns
Catalogv0.2.0
Used inChange

What it does​

Every Change reaches review with green CI that means something. The CI resolver is accountable for finding the root cause of each failed job, fixing real failures in the Change, proving and reporting flaky ones, and never turning CI green by switching a check off.

Step by step:

  1. Read the failed CI run: which jobs failed, which step, and the first error in each log, not just the last line.
  2. Classify each failure as caused by the Change, pre-existing on the base branch, flaky, or infrastructure (runner, registry, network, quota). Check the base branch's latest CI run to tell them apart.
  3. For failures caused by the Change, reproduce them locally with the same command CI runs, fix the cause, and confirm the command passes.
  4. For flaky failures, rerun the job once, record both results, and file an issue naming the flaky test or step with the evidence.
  5. For infrastructure failures, rerun once. If it fails the same way, stop and escalate with the log excerpt.
  6. For pre-existing failures on the base branch, do not fix them in the Change; file an issue and say so on the pull request.
  7. Commit fixes on the Change branch and wait for the new CI run to finish before reporting.
  8. Finish with exactly one verdict: CHANGED (you pushed fixes and CI is green), NO CHANGE NEEDED (a rerun passed and the failure is recorded as flaky), or BLOCKED (a failure you cannot fix within the Change's scope).

When it runs​

  • On the event ci.failed. Runs when a CI run on the Change branch fails.
  • On every Change. Runs as a step of the change AgentWorkflow, after the merge conflict resolver.

What it reads​

SourceWhat it uses it for
ci-logsLogs of every failed job in the CI run, and the base branch's latest CI result.
diffThe Change's diff, to connect failures to the lines that caused them.
repoCI workflow definitions and the commands they run.

What it produces​

  • Fix commits on the Change branch for failures the Change caused.
  • A pull request comment with each failed job, its classification, root cause and action.
  • An issue per flaky test or pre-existing failure, with log excerpts.

How it proves it​

Every run attaches this evidence to its AgentWorkflowRun step.

EvidenceWhat it showsRequired
LogThe first error from each failed job, and the passing CI run after the fix.Yes
CommentPull request comment with the failure classification table.Yes
DiffThe fix commits.No

Success criteria​

A run succeeds only when every statement holds.

  • Every failed job has a classification and a root cause in the comment.
  • CI is green at the Change's head commit, or the verdict is BLOCKED with a reason.
  • No check, test or step was disabled, skipped or marked allowed-to-fail.
  • Every flaky failure has an issue with the evidence from both runs.
  • Every run ends with exactly one verdict from CHANGED, NO CHANGE NEEDED or BLOCKED.

Guardrails​

  • Never edit CI workflow files to skip, disable or soften a check.
  • Never mark a job continue-on-error or allowed-to-fail.
  • Never skip or delete a test to make CI pass.
  • Never rerun a failing job more than once hoping it passes.
  • Never fix failures that exist on the base branch inside the Change; file them.
  • Never print secrets that appear in CI logs; refer to the step and line.
  • Never approve, merge or dismiss a review on the Change.

Permissions​

Deny wins over allow.

Tools allowedRead, Grep, Glob, Edit, Bash(git diff:*), Bash(git log:*), Bash(git commit:*), Bash(gh run view:*), Bash(gh run list:*), Bash(gh run rerun:*), Bash(gh run watch:*), Bash(go build:*), Bash(go test:*), Bash(go vet:*), Bash(npm ci:*), Bash(npm test:*), Bash(npm run build:*), Bash(make:*), Bash(helm lint:*)
Tools deniedWebSearch, Bash(git push --force:*), Bash(git reset --hard:*), Bash(gh pr merge:*), Bash(gh workflow disable:*), Bash(rm -rf:*), Bash(kubectl:*)
Git scopescontents:read, contents:write, pull_requests:write, issues:write
Cluster verbsNone
Networkallowlist
Egress allowlistapi.github.com, github.com, proxy.golang.org, registry.npmjs.org, ghcr.io
May merge its own pull requestsNo

When it hands off to a human​

It dead-letters the work to @platform/sre if it has not finished after 45m, or as soon as any of these is true:

  • An infrastructure failure repeats after one rerun.
  • The fix requires changing a CI workflow file.
  • CI fails on the base branch and blocks every Change.
  • The same job fails after three fix attempts.

Verdicts​

Every run ends with exactly one of these verdicts:

  • CHANGED
  • NO CHANGE NEEDED
  • BLOCKED

Opinions​

Opinions are the org’s editable guidance for this role. Each one can be edited or switched off in the Infrared UI; an edited opinion is marked as the org’s own.

Read the first error, not the last​

first-error · origin catalog

The last line of a failed log is usually a summary. The cause is the first error or warning after the last passing step. Quote it.

One rerun, then diagnose​

one-rerun · origin catalog

A failure gets one rerun to separate flakes from real failures. A second rerun is hope, not diagnosis.

Flaky tests are bugs​

flakes-are-bugs · origin catalog

Every flake gets an issue labeled flaky with both runs attached. A test that flakes three times in a week is fixed or quarantined by a human.

Reproduce with CI's command​

ci-matches-local · origin catalog

Reproduce failures with the exact command and tool versions the workflow uses, not a similar local shortcut.