Deliver a single app

This guide delivers one app, one repo and one image, through two zones: a Pre-release zone (<app>-rc) and a Release zone (<app>-prod). It follows the practices Infrared is built around. Every step is a pull request to the gitops repo, so the repo is always the record of what runs where.

:::note Phase 1 In phase 1 you open the pull requests yourself. Product delivery (phase 4) opens the same pin PRs for you and adds the Releases and Zones views to the Product. :::

What you need

• Infrared installed with builds.registry set, for example 123456789012.dkr.ecr.us-east-1.amazonaws.com/<org>. The gitops template then adds the builds component: a kpack ClusterBuilder infrared-builder on Paketo buildpacks, the service account builds/builder, and jobs that keep registry and GitHub credentials fresh.
• Registry repositories for the builder and the app: <registry>/kpack-builder and <registry>/<app>. ECR doesn't create repositories on push.
• Push rights to <registry>/* for the management cluster's nodes (on AWS, through the node's IAM role).
• Infrared's GitHub App installed on the app's repo, so kpack can clone it and Argo CD can read its chart.

1. Make the app deliverable

An app that follows these rules is safe to build once and promote unchanged.

Practice	Why	How
One image, every zone	The image that passed rc is the image that runs in prod	Read configuration from environment variables; never bake a zone into the image
Structured logs	Logs you can filter, count and correlate	One JSON line per request with a request ID, route, status and duration, plus version and zone on every line
Metrics	Know the request rate, errors and latency per zone	Serve Prometheus metrics on /metrics; include a build info gauge with version and zone
Separate liveness and readiness	Restarts only when stuck; traffic only when ready	/healthz for liveness, /readyz for readiness
Graceful shutdown	No dropped requests during rollouts	On SIGTERM, fail readiness, wait a few seconds for endpoints to drain, then finish in-flight requests
Buildpacks, not a Dockerfile	Reproducible, minimal, non-root images that can be rebased	A project.toml for the Paketo buildpacks (for Go, BP_GO_TARGETS)

2. Give it a chart

Keep the chart in the app's repo, charts/<app>, so the app and its deployment are versioned together.

• Pin by digest. Render the image as repository:tag@sha256:... and refuse to render without one.
• Run with least privilege. Non-root, read-only root filesystem, all capabilities dropped, no service account token, seccompProfile: RuntimeDefault. The zone namespaces enforce Pod Security restricted.
• Roll out safely. maxUnavailable: 0, a PodDisruptionBudget when there's more than one replica, topology spread across nodes, terminationGracePeriodSeconds longer than the drain.
• Probe and limit. Startup, liveness and readiness probes; CPU and memory requests; a memory limit.
• Observe. A VMServiceScrape for /metrics, which the gitops catalog's VictoriaMetrics picks up.
• Contain. A NetworkPolicy that admits traffic to the app port only.
• Validate. A values.schema.json, so a bad value fails the sync instead of the rollout.

3. Lay it out in the gitops repo

products/<app>/
  README.md
  build/image.yaml              # the kpack Image
  zones/<app>-rc/values.yaml    # the rc zone's values, including the pin
  zones/<app>-prod/values.yaml  # the prod zone's values, including the pin
registry/clusters/<cluster>/components/
  products-project.yaml         # AppProject products
  product-<app>-build.yaml      # Application for products/<app>/build
  product-<app>-rc.yaml         # Application for the rc zone
  product-<app>-prod.yaml       # Application for the prod zone

Name your files product-<app>-*.yaml. Hydration from the gitops template never deletes files and never touches files it doesn't render, so your product files survive template upgrades.

The products AppProject limits what product Applications can do: sources only from your org's repos (https://github.com/<org>/*), destinations only builds and the product's zone namespaces.

4. Build

The kpack Image builds the app's main branch with the builder service account and pushes to <registry>/<app>:main:

apiVersion: kpack.io/v1alpha2
kind: Image
metadata:
  name: <app>
  namespace: builds
spec:
  tag: <registry>/<app>:main
  serviceAccountName: builder
  builder:
    kind: ClusterBuilder
    name: infrared-builder
  source:
    git:
      url: https://github.com/<org>/<app>
      revision: main

Merge the PR that adds it. Every push to main now builds, and kpack rebuilds when the builder's buildpacks or stack change. Watch a build:

kubectl -n builds get builds.kpack.io -l image.kpack.io/image=<app>

5. Release

A release is a semver git tag on the commit a build came from, and the same tag on that build's digest:

# the digest kpack built from the commit you're releasing
kubectl -n builds get build <build> -o jsonpath='{.spec.source.git.revision} {.status.latestImage}{"\n"}'

git tag -a v0.1.0 <commit> -m "<app> v0.1.0" && git push origin v0.1.0
crane tag <registry>/<app>@sha256:<digest> v0.1.0

Check that the build's revision is the commit you tagged before you promote it.

6. Promote to rc, verify, promote to prod

Each zone Application takes the chart from the app's repo at the release tag, and the zone's values from the gitops repo:

spec:
  project: products
  sources:
    - repoURL: https://github.com/<org>/<app>
      targetRevision: v0.1.0
      path: charts/<app>
      helm:
        releaseName: <app>
        valueFiles:
          - $values/products/<app>/zones/<app>-rc/values.yaml
    - repoURL: https://github.com/<org>/gitops
      targetRevision: main
      ref: values
  destination:
    server: https://kubernetes.default.svc
    namespace: <app>-rc
  syncPolicy:
    automated: {prune: true, selfHeal: true}
    managedNamespaceMetadata:
      labels:
        pod-security.kubernetes.io/enforce: restricted
    syncOptions: [CreateNamespace=true]

The zone's values.yaml holds the pin:

zone: <app>-rc
image:
  repository: <registry>/<app>
  tag: v0.1.0
  digest: sha256:<digest>
replicas: 1

Promotion is a PR that changes the pin for one zone. Promote to rc, then verify:

kubectl -n <app>-rc logs -l app.kubernetes.io/name=<app> --tail=20      # JSON request logs
kubectl -n <app>-rc port-forward svc/<app> 8081:80                      # then curl the app
kubectl -n <app>-rc rollout restart deploy/<app>                        # watch it drain cleanly

In VictoriaMetrics, up{namespace="<app>-rc"} should be 1 and your request metrics should appear. Then open the same pin PR for <app>-prod. Prod runs the digest rc ran, so what you verified is exactly what ships.

Roll back

Revert the pin PR for the zone. Argo CD rolls back to the previous digest, which is still in the registry: Infrared's registry lifecycle keeps every image with a release tag.