Skip to main content
Version: v0.11.0

Sizing and scaling

Preview. For P6 on Kubernetes. No capacity measurements are published. SH_TURNS_PER_WORKER is documented as an output of an experiment (E8) whose results are not in this repository, so the defaults below are trial values. Do not read them as recommendations.

What a supervisor pod can do​

One supervisor pod runs SH_WORKERS workers, each allowing SH_TURNS_PER_WORKER concurrent turns. The shipped manifest pins 2 workers and 4 turns per worker, so a pod serves up to 8 concurrent turns; beyond that it answers 429 {"error":"overloaded"} with Retry-After: 1 before reading the request. Only POST /turn and POST /v1/turn count toward the cap, but the supervisor checks saturation once per connection, before reading any bytes, so while a pod is saturated every connection on the data port gets the 429, /v1/runs included. See Session multiplexing.

Change the worker count, the CPU limit and the memory request together: the manifest pins W with a CPU limit of 2 (about one CPU per worker) and a memory request of at least W x 256Mi (a manifest test enforces that), plus about 256Mi of headroom, which is advice in the deploy README and not tested. If SH_WORKERS were unset, W would follow the pod's CPU limit.

Resource defaults​

WorkloadRequestsLimits
Supervisor (with workers)500m CPU / 768Mi2 CPU / 1Gi
Control plane100m / 256Mi500m / 512Mi
Relay50m / 256Mimemory 512Mi, no CPU limit
Redis50m / 64Mimemory 256Mi
Each sandbox10m / 64Mimemory 512Mi
ghostunnel (OpenShift)10m / 32Mimemory 64Mi

Adding capacity​

  • More concurrent turns: add supervisor pods (replicas in an overlay). Workers share only Redis. The Service spreads connections, not load, and the default leastInFlight policy balances only within a pod, so one pod can 429 while another has room (inferred from the per-pod admission; not stated in the repository).
  • Or raise W and S in one pod, with CPU and memory raised to match (above).
  • One user cannot take every slot: each authenticated subject is capped at SH_TURNS_PER_SUBJECT in-flight turns per worker (default half of S), refused with 429 subject_quota_exceeded and Retry-After — a different code from the pool-wide overloaded 429. The cap is per worker, so a subject's host-wide share is cap/S, not the knob value itself. Async turns count against the same number (#391): an entry claimed past its owner's cap is handed back to the queue unrun, counted as async_deferred_subject, so an owner whose fan-out is running cannot also fill the worker with interactive turns. It does not pace an async-only drain: a worker runs one queued leaf at a time, so async work alone never reaches the cap, and the queue is one FIFO stream — one owner's deep backlog still delays another owner's async turn.
  • ...across workers, too, if you set it: MOCA_ASYNC_RUNNING_PER_OWNER caps how many async leaves one owner may have running in the whole deployment (#391). This is the one that bounds an async-only drain, because the count is in Redis rather than in a worker's memory. Unset by default, and off: a cap that appeared on upgrade would start pacing a single-owner deployment's own fan-out, which is the common case. Set the same value on every worker — each enforces it independently, so differing values leave the effective bound to whichever claimed first. An entry over the cap takes the same hand-back path as above (async_deferred_subject), so it waits out the 90 s reclaim window; size the cap for that latency, not just for the share. Per-owner streams would remove the hand-back altogether, and are still open.
  • More sandboxes: set SH_SANDBOX_COUNT for setup.sh (default 2, sticky). Each pod's sandbox id is its pod name. Each sandbox accepts up to KAGENTI_SANDBOX_CAP concurrent leases (the manifest sets 4; the code default is 20), so container capacity is roughly sandboxes x 4 concurrent turns. A pool with no free slot returns 503 sandbox_unavailable with Retry-After. Scaling the count down removes the workspaces of sessions pinned to the removed pods.
  • There is no autoscaling: no HPA, no KEDA.

What does not scale​

  • Control plane: keep one replica. The bundle-budget check is serialized in process.
  • Relay: exactly one replica (Recreate); a restart drops every attached sandbox.
  • Redis: one instance, no Sentinel or cluster mode.
  • Sticky routing: stickyBySession pins within one pod only.
  • L4 only: put no Ingress, edge Route, mesh sidecar or other HTTP proxy in front of the supervisor port; a proxy that reuses upstream connections defeats the per-connection hand-off and the turn cap. On OpenShift the data route is a passthrough to a TLS sidecar.

Limits to watch​

LimitDefaultNotes
Redis memory256Mi limit, 1Gi volumeSessions never expire; per-session memory is unmeasured
Config bundles16 MiB per user, 64 MiB total, 8 MiB each429 bundle_quota_exceeded when the per-user or total budget is full; a single bundle over 8 MiB gets 400 invalid_request (and mocactl promote refuses it locally); bundles expire after 30 days
Detachable turns1800 s unwatched, 86400 s log retentionSH_TURN_DETACHED_MAX_S, SH_TURN_LOG_TTL_S
Sandbox affinity86400 s memory, 60 s graceSH_SANDBOX_AFFINITY_TTL_SECONDS, SH_SANDBOX_AFFINITY_GRACE_SECONDS
Token lifetimesAPI 900 s, session 300 sRefresh 30 days idle, 90 days absolute
Harness config cache (/tmp/sh-config)Grows with each distinct bundle digestNo cleanup yet (a known ADR-0038 follow-up)

The cold-turn cost of a promoted bundle is unmeasured.