Grounded assessment that the relay service is pre-production (every cross-package seam is a fail-closed stub, no server/TLS/DB/orchestration). Documents the target topology + trust boundaries, the 7 wiring gaps to close (P1 socket server, P5 injection into P1/P3, pg/redis stores, real KMS CA custody, agent/relay-web build, F2 principal + F6 replay-epoch residuals), a phased build order (dev-run → staging → prod), an accurate per-service env reference, infra checklist, and an ops runbook.
11 KiB
Deploying the rendezvous-relay service — build & ops plan
Status: PRE-PRODUCTION — not runnable as-is. The 7 relay packages (P1–P6 + contracts) are libraries wired through injection seams that currently hold fail-closed stubs. There is no server process, no TLS/subdomain ingress, no Postgres/Redis/KMS wiring, and no orchestration. "Deploying" the relay means building the integration + infra layer the packages were designed to receive. This doc is the buildable spec: architecture, the seam-by-seam gaps, a phased order, the env/config reference, and the ops runbook. Security posture of the packages themselves is covered in REVIEW_RELAY_SECURITY.md (findings F1–F6 fixed).
1. Target architecture
A rendezvous relay: the agent runs on the user's own host (no inbound ports there) and dials out to the relay; the browser connects to a per-tenant subdomain; the relay muxes the two and does an opaque byte splice — it only ever sees ciphertext (INV2). End-to-end crypto (P4) is between browser and agent; auth/tenant-isolation (P5) gates every upgrade; the control-plane (P3) is the registry + CA + revocation publisher.
Browser (P6, xterm) User's host machine
│ WSS <sub>.term.<domain> │
│ capability token (subprotocol/cookie) + DPoP │ shell / node-pty
▼ ▲ loopback (opaque, INV11)
┌──────────────────────────┐ agent mux (mTLS, ┌────┴───────────────┐
│ RELAY DATA PLANE (P1) │◀──SPIFFE cert)─────────│ AGENT (P2) │
│ term-relay/data-plane │ dial-OUT │ dist/cli.js, ws │
│ • TLS + WSS ingress │ │ E2E host endpoint│
│ • onUpgrade → P5 authz │ │ replay sealer │
│ • subdomain → hostId │ └────────────────────┘
│ • OPAQUE splice (INV2) │ ▲ enroll/pair
│ • Redis sub relay:revoc. │ │
└───────┬──────────────────┬┘ │
│ P5 authz (lib) │ Redis pub/sub │
▼ ▼ │
┌──────────────────┐ ┌─────────┐ ┌──────────────────────────┴─┐
│ relay-auth (P5) │ │ Redis │◀──│ CONTROL PLANE (P3) │
│ tokens+DPoP, │ │ relay: │ │ control-plane, Fastify │
│ step-up, mTLS/ │ │ revoc. │ │ • accounts/hosts/sessions │
│ SPIFFE, revoke │ └─────────┘ │ • pairing + /enroll (CA) │──▶ Postgres
└──────────────────┘ │ • issues agent SPIFFE certs│──▶ KMS (CA key)
│ • publishes KillSignals │
└─────────────────────────────┘
Trust boundaries: the relay is UNTRUSTED for payload confidentiality (E2E, INV2); it IS trusted for availability + authz enforcement. The control-plane holds the CA (must be KMS-custodied). Agents authenticate to the relay with SPIFFE mTLS; browsers authenticate with short-lived capability tokens (minted by P5 after human auth) bound to a DPoP key. Never expose any port without TLS.
2. Components & runtime
| Pkg | Role | Runtime today | External deps |
|---|---|---|---|
| relay-contracts | frozen schemas/codecs | library | — |
| relay-e2e (P4) | browser↔agent E2E crypto | library (isomorphic) | @noble/* |
| relay-auth (P5) | authz, tokens+DPoP, step-up, mTLS, revoke | library ✅ audited | — |
| term-relay (P1) | stateless relay data-plane + mux | library — no server | (needs ws, tls) |
| control-plane (P3) | accounts/hosts/pairing/CA/revocation | Fastify app, in-memory default | fastify, pg, ioredis, KMS |
| agent (P2) | host agent: dial, pair, mTLS, E2E, replay | CLI web-terminal-agent→dist/cli.js |
ws |
| relay-web (P6) | browser bundle (xterm, preview) | needs build + static serve |
@xterm/* |
3. The wiring gaps (what must be BUILT to deploy)
Each is an injected seam currently holding a safe stub. In rough dependency order:
- P1 socket server (biggest gap).
term-relay/data-plane/relay-node.tstakesWebSocketLikeAgentListeneras injected seams — there is no actual listener. Build arelay-run/entry that: (a) terminates browser WSS onBIND_PORTand adapts each socket toWebSocketLike; (b) accepts agent mTLS connections onAGENT_BIND_PORT(verify SPIFFE client certs againstAGENT_CA_CHAIN_PATH) and adapts them toAgentListener/mux; (c) callsauthorizeUpgrade(→ P5onUpgrade) and splices. Wire the Redisrelay:revocationssubscriber tocloseStream.
- Inject P5 into P1 + P3. P1's
Authorizermust call relay-authonUpgrade/onReattachwith a realEnforceDeps(registries backed by P3, revocation store, buckets, audit). P3'srefuseAllVerifiermust be replaced with relay-authverifyCapabilityToken+ the realCAPABILITY_SIGN_PUBKEY_B64. Close F2 here: the P1 upgrade adapter currently hardcodesprincipal: null— for any host with a step-up policy it must resolve + pass the authenticated principal, else those hosts (correctly) 403. See relay-stepup-needs-principal-wiring. - P3 durable stores. Swap
createMemoryStoresfor the Postgres adapter (accounts/hosts/sessions registries) + run migrations; point the revocation bus at real Redis (ioredis). - Real KMS for the CA key.
boot/ca-wiring.tsuses a dev in-process Ed25519 signer ("NOT a real KMS"). Provide aKmsResolverbacked by cloud KMS/HSM; the intermediate key must be non-exportable (§3.1).CA_INTERMEDIATE_KMS_KEY_REFis the ref. - Agent build + config. Build
dist/cli.js; configureRELAY_URL/ENROLL_URL/SUBDOMAIN/HOST_ID/LOCAL_TARGET_URL/STATE_DIR; enroll (redeem pairing code → SPIFFE cert +hostContentSecret). - relay-web build + serve.
npm --prefix relay-web run build; serve the bundle from each tenant subdomain (same origin as the WSS endpoint so Origin/CSWSH checks pass). - F6 replay transport (residual). The recoverable-replay ring buffer must persist each
generation's
epochand serve the epoch matching those exact frames (relay-web/.../manage-page.tsloadReplayis still a throwing stub). Until wired, the preview/replay feature is inert (fail-closed).
4. Phased build order
- Phase 0 — single-host dev run (one process, self-signed). Reuse the
e2e/harnesswiring: awsserver hosting the relay-node, control-plane with in-memory stores, P4/P5 wired for real, self-signed TLS, agent dialinglocalhost, relay-web served locally. Goal: an actually-running end-to-end relay to click through. No pg/redis/KMS. - Phase 1 — single-tenant staging. Real Postgres + Redis; real (staging) TLS cert for one subdomain; close F2 principal wiring; agent on a second machine dialing in; behind Tailscale.
- Phase 2 — multi-tenant production. Wildcard TLS
*.term.<domain>+ subdomain routing; real KMS CA custody; horizontal relay nodes (stateless, share Redis); migrations/rollback; metering; F6 replay transport; monitoring/alerting on the audit sink + revocation latency budget.
5. Environment / config reference (accurate to the code)
control-plane (P3) — control-plane/src/env.ts, all required unless defaulted:
PG_URL, REDIS_URL, CAPABILITY_SIGN_PUBKEY_B64 (P5 verify pubkey), CA_INTERMEDIATE_KMS_KEY_REF,
CA_INTERMEDIATE_CERT_PATH, NODE_MTLS_TRUST_BUNDLE_PATH, BASE_DOMAIN, HEARTBEAT_TTL_SEC=15,
PAIRING_TTL_SEC=600, PAIRING_MAX_REDEEM_ATTEMPTS=5.
Routes: POST /accounts, POST /accounts/:id/pairing-codes, POST /accounts/:id/status,
GET /accounts/:id/hosts, DELETE /hosts/:hostId, POST /enroll. (No start script / listen entry
exists yet — add one.)
term-relay data-plane (P1) — term-relay/data-plane/config.ts:
BASE_DOMAIN, BIND_HOST, BIND_PORT (browser WSS), TLS_CERT_PATH, TLS_KEY_PATH,
AGENT_BIND_PORT, AGENT_CA_CERT_PATH, AGENT_CA_CHAIN_PATH (verify agent mTLS), RELAY_NODE_ID,
HEARTBEAT_INTERVAL_MS, INITIAL_WINDOW_BYTES, MAX_FRAME_BYTES, ROUTE_TTL_MS. (Library — the
process that reads this config and listens must be built, gap #1.)
relay-auth (P5) — RELAY_AUTH_VERIFY_PUBKEY (base64url raw Ed25519) via loadVerifyKeyFromEnv;
RELAY_TRUST_DOMAIN for SPIFFE.
agent (P2) — RELAY_URL, ENROLL_URL, HOST_ID, SUBDOMAIN, LOCAL_TARGET_URL, STATE_DIR.
6. Infra checklist
- TLS: browser-facing wildcard cert
*.term.<domain>(Phase 2) or per-subdomain (Phase 1); separate CA + trust bundle for agent mTLS (AGENT_CA_*/NODE_MTLS_TRUST_BUNDLE_PATH). - DNS: wildcard
*.term.<domain>→ relay node(s);<control-plane host>. - Postgres: accounts/hosts/sessions schema + migrations (adapter unbuilt).
- Redis:
relay:revocationspub/sub (P3 publishes, P1 subscribes) — the INV12 teardown path. - KMS/HSM: non-exportable CA intermediate signing key (
CA_INTERMEDIATE_KMS_KEY_REF). - Ingress for agents: agents dial OUT (mTLS) — expose only
AGENT_BIND_PORT+ browserBIND_PORT. - Orchestration: Dockerfiles + compose/k8s; relay nodes are stateless (crash-safe, INV7).
7. Security/ops runbook (deploy invariants)
- Origin/CSWSH: the browser WSS handshake MUST validate Origin against the tenant subdomain
(relay-auth
onUpgradestep 1 enforces it — pass the correctallowedOrigins). Never disable. - Never expose without TLS; the relay hands a shell to whoever authenticates. Prefer Tailscale for the agent path.
- Key custody (INV9): P5 signing key and the CA key never in process memory / logs; KMS refs only.
- Revocation latency (INV12): monitor time from
revoke()→ stream teardown; alert on budget miss. - mTLS trust anchors: agent certs verified against a pinned CA bundle — never a peer-supplied
chain (relay-auth
verifyAgentCertgates on the registry too). - Audit sink (INV10): zero-payload; ship to append-only storage; alert on
denyspikes /cross-tenant-attempt. - Close before prod: F2 principal wiring (gap #2) and F6 replay-epoch transport (gap #7).
8. TL;DR
There is no one-command deploy today. Minimum to get a running relay = Phase 0 (gaps #1, #2, #5, #6 with in-memory stores + self-signed TLS). Production = all gaps + Postgres/Redis/KMS/wildcard-TLS
- orchestration. The security-critical logic is done and audited; what remains is integration + infra.