Scaling and limits
Verified against v0.10.0 · cmd/parley/main.go, internal/api/router.go, internal/api/passcode.go, internal/db/db.go, internal/db/migrate.go, internal/hub/hub.go
Parley runs on more than one replica
Section titled “Parley runs on more than one replica”It used to refuse. An advisory lock at boot made the second instance exit, because the things that had to be shared across a room were held in one process’s memory. They are not any more:
- WebSocket fanout goes through Postgres
LISTEN/NOTIFY, so a mutation on one pod reaches every connection in that room on every pod. - Presence — who is in a session — is rows in Postgres, not a per-process view.
- The room-code attempt throttle counts in Postgres, so the eight-guess budget is shared however many processes are serving.
The WebSocket hub itself is still in process memory, but it is now only the set of sockets this pod is holding. Nothing depends on one pod seeing all of them.
The advisory lock that used to enforce one instance is gone, and with it both the pooled connection it held for the life of the process and the split-brain hazard it opened around a database failover.
Scaling up is opt-in
Section titled “Scaling up is opt-in”The chart ships replicaCount: 1, so upgrading it never silently doubles a
running install’s pods — or the database connections that come with them. Ask
for more when you want rollout headroom and a survivable node failure:
helm upgrade parley oci://ghcr.io/lets-parley/charts/parley --version 0.10.0 \ --reuse-values --set replicaCount=2Multi-replica needs chart 0.4.1 or newer. Every published chart up to and
including 0.3.0 fails at helm template time with “Parley is currently
single-replica” if replicaCount is above 1, and the binaries they deploy hold
a boot-time advisory lock that makes the second pod exit. Pin --version here
for the same reason you pin it on install: without it Helm takes whatever the
registry currently calls newest.
At two or more the chart also renders a PodDisruptionBudget; below that it does
not, because a budget in front of a single pod blocks every voluntary eviction
and hangs the kubectl drain it was meant to survive. Size max_connections
for the new count first — see Sizing the database.
Migrations when several pods boot together
Section titled “Migrations when several pods boot together”Every pod runs migrations at startup. Migrate takes a blocking
pg_advisory_lock on its own dedicated connection before touching the schema,
so simultaneous boots serialize: the first pod migrates, the rest wait and then
find nothing to do. The lock is released when migration finishes, on the success
path and on every error path — the connection is closed in a defer, and ending
the session drops the lock regardless.
That id is deliberately its own constant, distinct from every other advisory lock Parley takes. A blocking lock on an id the same process already holds would deadlock every pod against itself, on every boot.
What a rolling update looks like
Section titled “What a rolling update looks like”The chart uses RollingUpdate with maxSurge: 1 and maxUnavailable: 0, so
capacity is never below the replica count and the database sees at most one
extra pod’s worth of connections.
One known and accepted behaviour: the new pod migrates the database before the old pods are gone, so for the length of the rollout the old version is running against a newer schema. Parley’s migrations are additive, so the old pods keep serving. If an old pod restarts in that window it comes back normally — the version check only refuses a database ahead of a binary by a migration that binary does not know, and it is replaced by the new version moments later anyway. Roll forward rather than back; rolling back onto an image older than the migrations that have run is still refused, by design.
Sizing the database
Section titled “Sizing the database”Each replica opens its own pool of up to 10 connections, plus one dialled
outside the pool for the fanout listener. Size Postgres max_connections for
replicas × 11, not for 10.
A booting pod peaks at 12: the migration lock dials a connection of its own,
outside the pool and separate from the listener’s, and holds it only until
migrations finish. Short-lived, but a cluster whose pods all restart at once
will briefly want replicas × 12.
On a Postgres shared with other tenants, max_connections is a budget you draw
from rather than one you set — find out what is actually spare before scaling
up. Running out does not look like a failed start: the pod comes up and serves
requests, and what breaks is the fanout listener’s reconnect, so replicas stop
hearing each other’s votes while each one still reports itself healthy.
Connection pool
Section titled “Connection pool”| Max connections | 10 | Fixed, not configurable |
|---|---|---|
| Usable for requests | 10 | Per replica. The fanout listener and the migration lock each dial their own connection outside the pool: 11 per replica steady-state, 12 briefly at boot |
| Connect timeout | 5s | |
| Startup retries | 6 | Doubling backoff, roughly a minute total |
| Statement timeout | none | pgx defaults; set one on the database role if you want one |
Sizing Postgres max_connections means counting replicas × 11: ten pooled
connections plus the fanout listener’s own — and replicas × 12 for the window
at boot in which the migration lock holds a connection too.
Server timeouts
Section titled “Server timeouts”| ReadHeaderTimeout | 10s | Time allowed for request headers |
|---|---|---|
| ReadTimeout | 30s | HTTP request read deadline |
| WriteTimeout | 30s | HTTP response write deadline |
| IdleTimeout | 120s | Keep-alive idle deadline |
| Shutdown grace | 10s | WebSockets close first; in-flight HTTP receives the grace period |
| Session revalidation | 30s | Every live WebSocket re-checks its token against the database on this interval. Not configurable: it costs one pooled query per connection per interval. A logout through Parley is fanned out to every replica immediately, so this is the ceiling only for a revocation this process never hears about — a token deleted directly in SQL, or a notification missed while a replica's listener was reconnecting |
The HTTP timeouts protect ordinary requests. After a successful WebSocket upgrade the hijacked connection uses its own ping, pong, frame-write, and token validation deadlines; your reverse proxy still controls its upstream idle timeout. Parley pings every 25 seconds, so the 75-second timeout in the reverse proxy examples is safe.
Request and payload limits
Section titled “Request and payload limits”| API request body | 64 KiB | Global cap on every /api request, including bodies an endpoint does not consume |
|---|---|---|
| Optional-body endpoints | 4 KiB | The room-code join and passcode-rotate paths |
| WebSocket frame read | 4096 bytes | Clients send almost nothing over the socket |
| Room-code attempts | 8 per 60s | Per client address, per space |
| Open identities | 10/address/hour; 500/instance/hour | Configurable backstop for open mode |
| Persistent resources | 50 spaces/identity; 500 sessions/space; 500 stories/session | Configurable hard caps |
Everything else — field lengths, token lifetimes, the facilitator grace period — is in Limits and defaults.
How far does it go?
Section titled “How far does it go?”There is no published benchmark, and inventing one here would be dishonest, nor has any of this been exercised on a real multi-node cluster. What can be said from the design: state is in Postgres, broadcasts are whole-session payloads sent to every connection in a room, and the hub holds one goroutine pair per connection.
The scaling pressure is therefore rooms × people × mutation rate, and the first thing to watch is the database, not the Go process — adding replicas adds connections and fanout traffic to the same Postgres. For a team, or several teams, on modest hardware, none of this is a concern.