Deploy on Kubernetes with Helm

The chart deploy/helm/oxysdb/ deploys the team server, and the remote task runner as the subchart oxysdb-runner (runner.enabled, off by default). It needs Kubernetes 1.30 or later and Helm 3. Its README.md has the full values table; this page covers the decisions you have to make.

Install

helm install oxysdb deploy/helm/oxysdb -n oxysdb --create-namespace \
  --set image.registry=registry.example.com/oxys
read -rs PW   # typed without echo, kept out of the shell history
kubectl -n oxysdb exec -i deploy/oxysdb -- oxysdb-server add-account --email [email protected] <<< "$PW"; unset PW
helm test oxysdb -n oxysdb

Pin the images with image.digest in production.

Choose the store (global.database.mode)

Only Secret names and keys appear in values, never URLs with passwords: the pods read their URL from a mounted file, so External Secrets, Sealed Secrets or a vault injector work unchanged.

FerretDB with CloudNativePG

Install the CloudNativePG operator first. The chart then creates the PostgreSQL Cluster with the DocumentDB image (global.database.ferretdb.postgresql.cnpg.*: instances 3, storageSize 10Gi by default), FerretDB (global.database.ferretdb.replicas, 2 by default), and separate DocumentDB users for the server and the runner (never PostgreSQL’s superuser). As with compose, DocumentDB users are not confined to their database: a runner sharing this FerretDB is refused unless global.database.allowSharedCredentials=true. To keep the runner away from the team server’s data, give it a store of its own with runner.database.mode: external.

Operators

• flavour: mck: the operator writes the connection strings into <fullname>-db-url and <fullname>-runner-db-url; server pods wait in ContainerCreating until it has reconciled.
• flavour: psmdb: global.database.mongodbOperator.image is required and crVersion must match your operator. Add backups through extraSpec.

The chart never installs an operator.

Key values

Runner subchart (runner.*)

Losing the master password makes the runner’s stored connections unreadable. The generated master password and token are kept on uninstall.

High availability

• At least 2 server replicas (any mode but sqlite), with the PDB and topology spread on.
• The server keeps no state in the process: any replica answers any request, no sticky sessions.
• On MongoDB replica sets the server uses multi-document transactions; on FerretDB it serializes writes with a lock document (a few milliseconds per request).
• Database-mode runners elect a leader that computes schedules; each occurrence runs once even if two replicas briefly both lead. An on-demand run is claimed by exactly one runner of its pool. A runner killed during a run leaves the run interrupted; the next occurrence runs elsewhere.
• Shutdown is graceful: the chart derives terminationGracePeriodSeconds (65 s for the server, 35 s for the runner) and refuses a smaller explicit value.

NetworkPolicies

networkPolicy.enabled is on by default and needs a CNI that enforces NetworkPolicies. It allows: API ingress from the ingress controller only, metrics from the monitoring namespace only, egress to DNS, the in-cluster store and networkPolicy.egress.allowlist ({cidr, except, ports}). The in-cluster stores get their own policies. Pull-only runner pools accept no ingress at all.

The render checks where API traffic comes from, so a default never cuts the API off silently:

• fromAnywhere: true, or a LoadBalancer/NodePort Service, opens the API port to everyone; externalSourceCidrs narrows it.
• An open API port next to server.trustedProxies is refused unless acceptForgeableForwardedFor: true: any pod in the trusted range could then forge client addresses and escape the per-client sign-in throttling.
• DNS: networkPolicy.dns.preset is kube-dns (default), openshift, nodelocal (NodeLocal DNSCache; give cidrs when the render cannot read the cluster) or custom.
• With an Ingress, the render fails if the controller’s namespace does not exist (checkControllerNamespace: false skips the lookup for accounts that cannot read namespaces).

GitOps (Argo CD, helm template)

Generated credentials are created once by helm install and read back with lookup. A render without cluster access fails instead of inventing new ones: give every credential as an existingSecret (the chart README lists the keys). global.oxysdb.previewSecrets=true renders throwaway values for CI previews; never apply that output. With fullnameOverride/nameOverride, set global.oxysdb.parentFullname.

Uninstall

helm uninstall removes pods and Services but keeps the data: operator resources, the CloudNativePG Cluster, generated credentials and the SQLite PVC (helm.sh/resource-policy: keep, Argo CD Prune=false,Delete=false). A later install with the same release name and namespace adopts them. To really delete the data, back it up, then delete those resources by hand.

Troubleshooting

deploy/test/helm-lint.sh (lint, every mode, schema validation) and deploy/test/kind-e2e.sh (a kind cluster, every mode, helm test) validate the chart.