Back up and restore
Infrared backs up your whole install to object storage outside its cluster, on a schedule, so losing the cluster doesn't mean losing your work. Each backup is one encrypted artifact holding Infrared's objects (orgs, Products, AgentRoles, AgentWorkflows, AgentWorkflowRuns and the rest), Gitea's data when the install runs Gitea, and dumps of the platform's Postgres databases. Beside it, Infrared mirrors the buckets of the cluster's object store.
A backup that a mirror run has followed is a point. A restore brings a point back whole, onto new servers.
Only platform admins see and change backups, on Settings → Backups in the platform org.
Set up backups
Before the first backup, Infrared needs three things: a destination, the destination's key, and at least one recipient to encrypt to. Set them on Settings → Backups, or in the setup wizard's optional Set up backups step, which shows the same form once the gitops repo is hydrated. Skip for now leaves it for later.
Until all three are set, the panel says No backup is made yet and names what's missing.
Destination
| Field | What to enter |
|---|---|
| Provider | Linode Object Storage, Amazon S3 or another S3-compatible store, or Google Cloud Storage |
| Region | For Linode, the region in the bucket's endpoint, such as us-lax-4. For S3, the region requests are signed for, such as us-east-1. For Google Cloud Storage, the bucket's location; empty means us-central1 |
| Endpoint | For S3 only: the store's endpoint, https:// and a host with no path, such as https://s3.us-east-1.amazonaws.com. Linode's follows from the region, and Google Cloud Storage's is always https://storage.googleapis.com |
| Bucket | The bucket's name. The install writes only under its prefix |
| Prefix | The path everything is kept under in the bucket. It defaults to the install's name |
| Access key ID and Secret access key | For Linode and S3: a key that can list, read, write and delete in the bucket |
Google Cloud Storage takes no key. Infrared reaches the bucket as its own Kubernetes ServiceAccounts through Workload Identity, as on GKE. Grant them roles/storage.objectUser on the bucket before you save. Postgres's WAL archive isn't available on Google Cloud Storage.
The key is write-only. Once it is set, the panel says when it was set, never what it is. Replace the access key lets you enter a new pair, and Keep the key set cancels that.
Before it saves a destination, Infrared checks it with a real write: it lists the prefix, then writes, reads and deletes one object. Check destination runs the same check on the destination already set. A destination that fails isn't saved, and the panel lists each step, list, write, read and delete, with what the store answered, so you can see exactly what the key is missing.
Schedule and recipients
| Field | Default | What it does |
|---|---|---|
| Backup schedule | 5 * * * * | When a backup is made: five cron fields, in UTC |
| Mirror schedule | 17 * * * * | When the object store's buckets are mirrored: five cron fields, in UTC. A backup becomes a point at the first mirror run after it |
| Retention | 7d | How long backups are kept, in whole days from 1d to 999d. The newest complete backup is never deleted |
| Recipients | none | Up to 16 age public keys (age1…). Each backup is encrypted to all of them, and any one of their private keys opens it |
| Archive Postgres's WAL | off | Also keeps Postgres's write-ahead log in the bucket, beside the dump in each backup. Not available on Google Cloud Storage |
Add recipient adds a line. Save backups saves the destination and the settings together, checking the destination first and sending only what you changed. Without a recipient, no backup is made.
Paste only the public half of a key. If you paste a private key (AGE-SECRET-KEY-…), the form empties the line and tells you it wasn't kept.
To make a key pair, run age-keygen and keep its output in your vault, or run ir backup key new <install>, which writes the private key to a file only you can read and prints its public half. Keep at least one private key somewhere that survives losing the cluster: without one, no backup can be opened.
The settings can also come from the chart's backup values at install. Infrared copies them into its own settings once, while those are empty. After that, change them on Settings → Backups.
See your backups
Once backups are set up, the panel shows What the operator last saw: whether backups are healthy (BackupsHealthy), the newest backup, the newest mirror run, the newest point and the operator's last message. Settings → Installation shows the newest point too, so anyone in the platform org can see at a glance that the install is covered.
Backups in the destination lists the newest 24 backups, read straight from the bucket:
| Column | What it says |
|---|---|
| Backup | Its stamp, such as 20261006T010500Z, and how long ago it was made |
| Size | The artifact's size |
| Parts | What it holds: Infrared's objects, Gitea's data and the Postgres databases dumped |
| Mirror run | The mirror run that followed it |
| Complete | Whether the artifact and its manifest are both there and match |
| Point | Whether a restore can bring it back, or why not yet |
Back up now makes a backup straight away, followed by a mirror run, so you have a fresh point before a risky change. The panel shows Backing up until both are done. The button is off while a scheduled backup runs.
From a terminal, ir backup status, ir backup list, ir backup get <stamp> and ir backup now do the same.
Restore an install
A restore brings an install back from one point, onto new servers, before Infrared runs there. You need:
- a cluster with nothing of Infrared on it, reached by its own kubeconfig context;
- the install's usual install file and values, with the same
backupblock, so the restore reads the same bucket; - the age private key of one of the backup's recipients;
- the bucket's key.
ir restore does the whole restore:
ir backup list # pick a point
ir restore --backup-artifact 20261006T010500Z --context <new-cluster> -f infrared.yaml
It checks the backup's manifest, that your key opens it, and that a mirror run followed it. Then it creates the Secret infrared-backup-identity from your key file, runs one helm install with restore.enabled=true and restore.point=<stamp>, and waits until the restore is complete and every Application is Synced and Healthy, printing each change as it goes. --timeout, 2 hours by default, bounds the wait.
Without ir, do the same by hand. Create the Secret infrared-backup-identity, key identity, in the infrared namespace with kubectl, then helm install with your values plus restore.enabled=true and either restore.point=<stamp> or restore.from=<RFC 3339 time>, which picks the newest complete backup at or before that time. With Gitea, add gitea.replicaCount=0: Gitea starts once its data is back. Never pass the private key as a chart value, because Helm keeps every value in its release record. The operator deletes the Secret once the restore is complete.
While a restore runs, the setup page shows it in place of the wizard: Plan the restore, Restore Gitea and Infrared's objects, Restore Postgres, then Bring back the stores. Anything that would start work, such as a run, a deployment or a setup step, waits and says A restore is in progress, with Follow the restore. If the restore can't go on, for example because no complete backup matches, it stops and says so, and you decide what happens next.
Keep a copy outside the bucket
ir backup pull <stamp> --out <dir> -f infrared.yaml downloads a backup's artifact and manifest, and with --full the mirror as well. It reads the bucket directly, so it works even after the install is gone. ir restore --backup-artifact <dir>/<stamp>.irbackup then uploads whatever the bucket lacks before it restores.
API
| Call | What it does |
|---|---|
GET /v1/backup | The destination (without the key), the settings in force, what's missing, a backup in progress, the operator's view and the newest 24 backups |
PUT /v1/backup/destination | Set where backups go, checked with a real write first |
POST /v1/backup/destination/check | Check the destination already set |
PUT /v1/backup/settings | Set the schedules, retention, recipients and WAL archive |
POST /v1/backup/copies | Back up now |
GET /v1/backup/copies/{stamp} | One backup: its manifest and the mirror run that made it a point |
Every call is for platform admins. The calls that change something answer 409 while a restore is in progress.