Upgrade

Upgrade the team server and the runner without downtime, including the runner StatefulSet change and the one-time reseal.

Upgrades are rolling, one replica at a time. The server migrates its store layout at the first start of the new version, under a lock, so replicas starting together migrate once. Older replicas keep working with a layout they know until they are replaced. A store written by a newer server is refused: roll forward, not back.

docker compose

  1. Back up.
  2. Change the image tags in .env (OXYSDB_SERVER_IMAGE, OXYSDB_RUNNER_IMAGE).
  3. docker compose -f FILE pull (or build), then docker compose -f FILE up -d. In the HA profiles, for no downtime, replace one replica at a time: docker compose -f FILE up -d --no-deps server-1, wait until it is healthy, then server-2, and the same for runner-1 and runner-2.
  4. MongoDB, FerretDB, PostgreSQL and Caddy follow their own upgrade procedures.

Helm

helm upgrade oxysdb deploy/helm/oxysdb -n oxysdb -f my-values.yaml
  • migrations.enabled=true runs oxysdb-server migrate-schema as a pre-upgrade hook, so a failing migration stops the upgrade before any pod is replaced.
  • Generated secrets never change: they are read back from the cluster. An upgrade that finds the store but not its secret fails: restore the Secret.
  • Changing global.database.mode keeps the old store and its secrets. Migrate the data first, then delete the old store by hand once the new one is verified.

Runner: Deployments become StatefulSets (chart 0.2.0)

In database mode, runner pods (API replicas and pull-only pools) now run as StatefulSets with a small volume per pod (runner.persistence.perReplica, 256Mi, kept on uninstall and scale-down). Each pod keeps there the newest runner state it has seen, so a pod restarted or rescheduled still refuses an older state put back in the database.

  • The upgrade deletes the runner Deployments and creates the StatefulSets: expect a short gap while the new pods start. Schedule it outside busy hours.
  • runner.persistence.perReplica.enabled=false keeps Deployments with an emptyDir (each new pod then starts without that protection).

Runner 0.1 → 0.2: reseal once

Runner 0.2 authenticates every task and the token hash with the master password and seals connections with their version. What 0.1 wrote is refused until it is signed: scheduled runs fail with send the task again. Once all runner pods run 0.2, run once:

# Kubernetes: the pod's environment holds the database URL and master password
kubectl exec statefulset/<release>-oxysdb-runner -- oxysdb-runner reseal
# docker compose
docker compose -f FILE exec runner-1 oxysdb-runner reseal

It prints the tasks it signed: check them. Sending every task again from the desktop has the same effect. The same step applies when upgrading from a 0.2 runner built before 2026-10-30.

Other moves

MoveCommand
SQLite server → MongoDB/FerretDBstop the single server, then oxysdb-server migrate-to-mongodb --database /data/oxysdb-server.sqlite with OXYSDB_SERVER_DATABASE_URL_FILE set; then switch the store
File-mode runner → database modeoxysdb-runner migrate-to-database --data-dir DIR; then switch the store and scale out. The old data directory is marked and refuses file mode afterwards
Dev StatefulSet replica countchange global.database.statefulset.replicas; scale down one member at a time (removing a majority at once is refused)

Troubleshooting

MessageFix
send the task again on scheduled runsrun oxysdb-runner reseal once, or send the tasks again
The runner database was rolled backsee Restoring a runner database
changed while reseal ransomeone wrote meanwhile; run reseal again, it resumes
server refuses a newer layoutan older image started on an upgraded store: deploy the new version
explicit terminationGracePeriodSeconds refusedremove it from your values: the chart derives it (65 s for the server)