The QA engineer

The QA engineer is the catalog AgentRole ci-resolver, shown as QA engineer. It manages CI and never runs it: CI decides whether a check passes, and the QA engineer reads the result, the logs, the diff and the issue, and decides what the failure means. It holds no cloud credential and cannot start, retry or skip a conformance run or a cluster apply.

Its one question, for every failure a Change caused: is the code wrong, or the test?

• The test is wrong, stale or missing for what the Change rightly did: the QA engineer fixes the test on the Change branch, runs the command CI runs, and ends TEST CHANGED. Infrared then waits for CI on that commit; the step succeeds only when CI passes. The quality reviewer, not the QA engineer, reviews that test change.
• The code is wrong and the test caught it: the QA engineer changes no product code. It ends CODE CHANGE NEEDED with the failing check, the cause, the file and line, and what the code should do, and the Change goes back to the builder.

It reads the issue's acceptance criteria and design notes, written by the architect when the issue was hydrated, to know what the Change is meant to do. When they don't settle it, it ends BLOCKED with the question for a person. A flake gets an issue labelled flaky with its log, and an infrastructure failure (runner, registry, network, quota) an escalation: both end BLOCKED, and a person reruns CI if they choose. A failure already on the base branch gets an issue and a note on the pull request, and stays out of the Change. The QA engineer never runs, reruns or watches CI, and never deletes, skips or loosens a test to make CI pass.

Inside a Change

In Change (quick) the QA engineer is the qa step, after the builder and before the reviewers. In Change it is the ci step.

How it reads CI

Infrared, not the agent, waits for CI. When the step's turn comes, it reads the pull request's check runs and commit statuses on its head commit:

CI on the pull request	What happens
Still running	The step waits, showing which checks run, up to the step's timeout (30 minutes by default). No agent runs and no tokens are spent.
Every check passed	The step ends NO CHANGE NEEDED with no agent at all.
A check failed	The QA engineer starts, given each failed check: its name and conclusion, its link, its annotations (file, line and message), the first errors and the last lines of its job's log, and whether the base branch fails the same check.
No check reports within two minutes	The repo has no CI for pull requests. The QA engineer starts and runs the repo's own tests itself.

Here the pull request's one check passed, so the step ended in no time, with no agent and no cost:

After TEST CHANGED (or CHANGED), Infrared waits for CI on the QA engineer's commit. When CI passes, the step succeeds with that verdict; when it still fails, the QA engineer runs again with the new failures, until the step's maxAttempts are spent and a person decides.

Back to the builder

Both QA steps send the run back to their builder on CODE CHANGE NEEDED:

1. The builder runs again on the same branch and pull request, with the QA engineer's finding as its first instruction: fix the code so the failing check passes for the right reason, and don't change the test to match.
2. Every step from the builder on runs again: Infrared waits for the new CI run and hands it to the QA engineer (or ends the step if it passes), then the reviewers.
3. This happens at most twice. A third CODE CHANGE NEEDED ends the step and hands the Change to a person, as any step's failure does. A Change a person opened has no builder step to go back to, so the finding goes to that person straight away.

When it finishes, its record shows what it found: here, from a run before the repo had CI, no change needed, with the test output it kept as evidence and its success criteria. The four failing tests it found failed on the base branch too, so it left them out of the Change:

When it sends the Change back, the Change's page shows that too. Above the steps, a note says how often the QA engineer sent the Change back. Each step's header names its round, and an open step lists how it ended in each earlier round:

What each round spent stays with its step, so the Change's cost counts every round. Through the API, status.returnRounds on the AgentWorkflowRun counts the rounds, and status.steps[].rounds holds each step's earlier rounds. A step sends the run back when it names returnTo and ends CODE CHANGE NEEDED; maxReturns (2 by default) bounds it.

- name: qa
  type: agentRole
  agentRole: ci-resolver
  returnTo: product-feature-builder
  maxReturns: 2

Failures on the default branch

A check can also fail where no Change is in flight: on a Product repo's default branch, after merges. Infrared reads the checks on each Product repo's default branch every ten minutes and raises the event ci.failed for a failed head commit. The CI triage AgentWorkflow answers it with one QA engineer run that has no issue and no pull request. It reads the failure and the Changes merged since the last green commit, decides code or test, and opens one issue a builder can act on. That issue goes through Change (quick) like any other, with the QA engineer and the builder working it out on one branch.

Nothing answers a failure twice, and no agent wakes another:

What happened	What Infrared does
A check fails on a Change's branch	Nothing new: that Change's own QA step answers it. ci.failed is never raised for a Change branch.
A check fails on the default branch, first time	One CI triage run. The Changes merged since the last green commit are named in it, and never reopened or woken.
The same checks fail again while the triage's issue is open	A comment on that issue. No new triage.
The same checks fail again within 14 days of the issue closing	The issue is reopened, labelled infrared:needs-person, and a person decides. No new triage.
The failing commit was merged by a Change the QA engineer's issue started	That issue is reopened for a person. A QA-born Change that can't get CI green never starts another triage.
The triage found a flake and opened nothing	The same failure is triaged again only after a day.

The watch reads GitHub repos of every Product but Infrared's own, through the org's GitHub App and its Checks: read permission. It keeps what it has seen in the ConfigMap infrared-ci-watch in the org namespace.

The weekly sweep

The QA weekly sweep runs on Mondays at 07:00 in the org's time zone, once per Product. The QA engineer reads the last week of CI on the default branch and of its own issues: flaky tests, steps slowing down or nearing their timeouts, checks that never fail, gaps in what the tests cover. It writes a report and opens at most one issue for the most important finding. It reruns nothing to find out.

The sweep is on by default. Unlike other scheduled runs, it runs whether or not the org's scheduled runs are on: its budget bounds what it spends. Disable the AgentWorkflow in Agents → AgentWorkflows to stop it.

Its budget

What	Cap
One CI triage run	$2 in model tokens (maxBudgetUSD on the step)
One weekly sweep run	$3
CI triage and the weekly sweep together, per org	$15 in any seven days
The QA step inside a Change	Counts against that Change, like every step; the role's own cap ($4) per attempt

The weekly cap is the AgentWorkflows' budget: every AgentWorkflow with budget.group: qa shares it. A run that would start once the group's runs spent the cap in the last seven days fails before any step runs, and says so. Raise budget.weeklyUSD on both to give QA more room.

budget:
  group: qa
  weeklyUSD: "15.00"
  scheduled: true   # the sweep only: runs with the org's scheduled runs off