The QA engineer

The QA engineer is the catalog AgentRole ci-resolver, shown as QA engineer. It reads CI; it never runs it. CI decides whether a check passes, and the QA engineer reads the result, the logs, the diff and the issue to decide what a failure means. It holds no cloud credential and can't start, retry or skip a conformance run or a cluster apply.

For every failure a Change caused, it answers one question: is the code wrong, or the test?

• The test is wrong, stale or missing for what the Change rightly did. The QA engineer fixes the test on the Change branch, runs the command CI runs, and ends TEST CHANGED. Infrared then waits for CI on that commit, and the step succeeds only when CI passes. The quality reviewer, not the QA engineer, reviews the test change.
• The code is wrong and the test caught it. The QA engineer changes no product code. It ends CODE CHANGE NEEDED with the failing check, the cause, the file and line, and what the code should do, and the Change goes back to the builder.

To know what the Change is meant to do, it reads the issue's acceptance criteria and design notes, which the architect wrote when the issue was hydrated. When they don't settle it, it ends BLOCKED with the question for a person.

Some failures aren't the Change's to fix:

• A flaky test gets an issue labelled flaky, with its log.
• An infrastructure failure (the runner, the registry, the network, a quota) is escalated to a person.
• A failure already on the base branch gets an issue and a note on the pull request, and stays out of the Change.

Flakes and infrastructure failures end BLOCKED, and a person reruns CI if they choose. The QA engineer never runs, reruns or watches CI, and never deletes, skips or loosens a test to make CI pass.

Inside a Change

In Change (quick) the QA engineer is the qa step, after the builder and before the reviewers. In Change it is the ci step.

How it reads CI

Infrared, not the agent, waits for CI. When the step's turn comes, Infrared reads the check runs and commit statuses on the pull request's head commit:

CI on the pull request	What happens
Still running	The step waits, showing which checks are running, up to the step's timeout (30 minutes by default). No agent runs and no tokens are spent.
Every check passed	The step ends NO CHANGE NEEDED with no agent at all.
A check failed	The QA engineer starts, with each failed check: its name, conclusion and link, its annotations (file, line and message), the first errors and the last lines of its job's log, and whether the base branch fails the same check.
No check reports within two minutes	The repo has no CI for pull requests. The QA engineer starts and runs the repo's own tests itself.

When CI passes, the step costs nothing:

After TEST CHANGED (or CHANGED), Infrared waits for CI on the QA engineer's commit. If CI passes, the step succeeds with that verdict. If it still fails, the QA engineer runs again with the new failures, until the step's maxAttempts are spent and a person decides.

Back to the builder

On CODE CHANGE NEEDED, both QA steps send the run back to their builder:

1. The builder runs again on the same branch and pull request, with the QA engineer's finding as its first instruction: fix the code so the failing check passes for the right reason, and don't change the test to match.
2. Every step from the builder on runs again. Infrared waits for the new CI run and hands it to the QA engineer (or ends the step if it passes), then the reviewers run.
3. This happens at most twice. A third CODE CHANGE NEEDED ends the step and hands the Change to a person, as any failed step does. A Change a person opened has no builder step to go back to, so the finding goes straight to that person.

When the QA engineer finishes, its record shows what it found, the evidence it kept and its success criteria. In this run, the repo had no CI yet, so it ran the tests itself. The four tests that failed also failed on the base branch, so it left them out of the Change:

When it sends a Change back, the Change's page says so. A note above the steps counts how often the QA engineer sent the Change back, each step's header names its round, and an open step lists how it ended in each earlier round. What each round spent stays with its step, so the Change's cost includes every round.

Through the API, status.returnRounds on the AgentWorkflowRun counts the rounds, and status.steps[].rounds holds each step's earlier rounds. A step sends the run back when it names returnTo and ends CODE CHANGE NEEDED; maxReturns (2 by default) bounds it.

- name: qa
  type: agentRole
  agentRole: ci-resolver
  returnTo: product-feature-builder
  maxReturns: 2

Failures on the default branch

A check can also fail where no Change is in flight: on a Product repo's default branch, after merges. Every ten minutes, Infrared reads the checks on each Product repo's default branch, and raises the event ci.failed when the head commit failed. The CI triage AgentWorkflow answers it with one QA engineer run that has no issue and no pull request. The QA engineer reads the failure and the Changes merged since the last green commit, decides whether the code or the test is wrong, and opens one issue a builder can act on. That issue goes through Change (quick) like any other.

You get one issue to look at, not a pile of them. Nothing answers a failure twice, and no agent wakes another:

What happened	What Infrared does
A check fails on a Change's branch	Nothing new: that Change's own QA step answers it. ci.failed is never raised for a Change branch.
A check fails on the default branch for the first time	One CI triage run. It names the Changes merged since the last green commit, and never reopens or wakes them.
The same checks fail again while the triage's issue is open	A comment on that issue, and no new triage.
The same checks fail again within 14 days of the issue closing	The issue is reopened and labelled infrared:needs-person, for a person to decide. No new triage.
The failing commit was merged by a Change that the QA engineer's own issue started	That issue is reopened for a person. A Change the QA engineer started that can't get CI green never starts another triage.
The triage found a flake and opened nothing	The same failure is triaged again only after a day.

The watch reads the GitHub repos of every Product except Infrared's own, through the org's GitHub App and its Checks: read permission. It keeps what it has seen in the ConfigMap infrared-ci-watch in the org's namespace.

The weekly sweep

The QA weekly sweep runs on Mondays at 07:00 in the org's time zone, once per Product. The QA engineer reads the last week of CI on the default branch and its own issues, looking for flaky tests, steps slowing down or nearing their timeouts, checks that never fail, and gaps in what the tests cover. It writes a report and opens at most one issue, for the most important finding. It reruns nothing to find out.

The sweep is on by default. Unlike other scheduled runs, it runs whether or not the org's scheduled runs are on, because its budget bounds what it spends. To stop it, disable the AgentWorkflow under Workforce → Agents → AgentWorkflows.

Its budget

What	Cap
One CI triage run	$2 in model tokens (maxBudgetUSD on the step)
One weekly sweep run	$3
CI triage and the weekly sweep together, per org	$15 in any seven days
The QA step inside a Change	Counts toward that Change like every step, with the AgentRole's own cap of $4 per attempt

The weekly cap is the AgentWorkflows' budget: every AgentWorkflow with budget.group: qa shares it. Once the group's runs have spent the cap in the last seven days, a new run fails before any step runs, and says why. The Usage page shows the group's budget and what is left of it. To give QA more room, raise budget.weeklyUSD on both AgentWorkflows.

budget:
  group: qa
  weeklyUSD: "15.00"
  scheduled: true   # the sweep only: runs with the org's scheduled runs off