docs(relay): deployment build & ops plan (DEPLOY_RELAY.md)
Grounded assessment that the relay service is pre-production (every cross-package seam is a fail-closed stub, no server/TLS/DB/orchestration). Documents the target topology + trust boundaries, the 7 wiring gaps to close (P1 socket server, P5 injection into P1/P3, pg/redis stores, real KMS CA custody, agent/relay-web build, F2 principal + F6 replay-epoch residuals), a phased build order (dev-run → staging → prod), an accurate per-service env reference, infra checklist, and an ops runbook.
This commit is contained in:
Yaojia Wang
168
docs/DEPLOY_RELAY.md
Normal file
168
docs/DEPLOY_RELAY.md
Normal file
@@ -0,0 +1,168 @@
|
||||
# Deploying the rendezvous-relay service — build & ops plan
|
||||
|
||||
> **Status: PRE-PRODUCTION — not runnable as-is.** The 7 relay packages (P1–P6 + contracts) are
|
||||
> libraries wired through **injection seams that currently hold fail-closed stubs**. There is no
|
||||
> server process, no TLS/subdomain ingress, no Postgres/Redis/KMS wiring, and no orchestration.
|
||||
> "Deploying" the relay means **building the integration + infra layer** the packages were designed
|
||||
> to receive. This doc is the buildable spec: architecture, the seam-by-seam gaps, a phased order,
|
||||
> the env/config reference, and the ops runbook. Security posture of the packages themselves is
|
||||
> covered in [REVIEW_RELAY_SECURITY.md](./REVIEW_RELAY_SECURITY.md) (findings F1–F6 fixed).
|
||||
|
||||
---
|
||||
|
||||
## 1. Target architecture
|
||||
|
||||
A **rendezvous relay**: the agent runs on the user's own host (no inbound ports there) and **dials
|
||||
out** to the relay; the browser connects to a per-tenant subdomain; the relay muxes the two and does
|
||||
an **opaque byte splice** — it only ever sees ciphertext (INV2). End-to-end crypto (P4) is between
|
||||
browser and agent; auth/tenant-isolation (P5) gates every upgrade; the control-plane (P3) is the
|
||||
registry + CA + revocation publisher.
|
||||
|
||||
```
|
||||
Browser (P6, xterm) User's host machine
|
||||
│ WSS <sub>.term.<domain> │
|
||||
│ capability token (subprotocol/cookie) + DPoP │ shell / node-pty
|
||||
▼ ▲ loopback (opaque, INV11)
|
||||
┌──────────────────────────┐ agent mux (mTLS, ┌────┴───────────────┐
|
||||
│ RELAY DATA PLANE (P1) │◀──SPIFFE cert)─────────│ AGENT (P2) │
|
||||
│ term-relay/data-plane │ dial-OUT │ dist/cli.js, ws │
|
||||
│ • TLS + WSS ingress │ │ E2E host endpoint│
|
||||
│ • onUpgrade → P5 authz │ │ replay sealer │
|
||||
│ • subdomain → hostId │ └────────────────────┘
|
||||
│ • OPAQUE splice (INV2) │ ▲ enroll/pair
|
||||
│ • Redis sub relay:revoc. │ │
|
||||
└───────┬──────────────────┬┘ │
|
||||
│ P5 authz (lib) │ Redis pub/sub │
|
||||
▼ ▼ │
|
||||
┌──────────────────┐ ┌─────────┐ ┌──────────────────────────┴─┐
|
||||
│ relay-auth (P5) │ │ Redis │◀──│ CONTROL PLANE (P3) │
|
||||
│ tokens+DPoP, │ │ relay: │ │ control-plane, Fastify │
|
||||
│ step-up, mTLS/ │ │ revoc. │ │ • accounts/hosts/sessions │
|
||||
│ SPIFFE, revoke │ └─────────┘ │ • pairing + /enroll (CA) │──▶ Postgres
|
||||
└──────────────────┘ │ • issues agent SPIFFE certs│──▶ KMS (CA key)
|
||||
│ • publishes KillSignals │
|
||||
└─────────────────────────────┘
|
||||
```
|
||||
|
||||
**Trust boundaries:** the relay is UNTRUSTED for payload confidentiality (E2E, INV2); it IS trusted
|
||||
for availability + authz enforcement. The control-plane holds the CA (must be KMS-custodied). Agents
|
||||
authenticate to the relay with SPIFFE mTLS; browsers authenticate with short-lived capability tokens
|
||||
(minted by P5 after human auth) bound to a DPoP key. Never expose any port without TLS.
|
||||
|
||||
---
|
||||
|
||||
## 2. Components & runtime
|
||||
|
||||
| Pkg | Role | Runtime today | External deps |
|
||||
|---|---|---|---|
|
||||
| **relay-contracts** | frozen schemas/codecs | library | — |
|
||||
| **relay-e2e (P4)** | browser↔agent E2E crypto | library (isomorphic) | @noble/* |
|
||||
| **relay-auth (P5)** | authz, tokens+DPoP, step-up, mTLS, revoke | library ✅ audited | — |
|
||||
| **term-relay (P1)** | stateless relay data-plane + mux | **library — no server** | (needs `ws`, `tls`) |
|
||||
| **control-plane (P3)** | accounts/hosts/pairing/CA/revocation | Fastify app, **in-memory default** | fastify, **pg**, **ioredis**, KMS |
|
||||
| **agent (P2)** | host agent: dial, pair, mTLS, E2E, replay | CLI `web-terminal-agent`→`dist/cli.js` | ws |
|
||||
| **relay-web (P6)** | browser bundle (xterm, preview) | needs `build` + static serve | @xterm/* |
|
||||
|
||||
---
|
||||
|
||||
## 3. The wiring gaps (what must be BUILT to deploy)
|
||||
|
||||
Each is an injected seam currently holding a safe stub. In rough dependency order:
|
||||
|
||||
1. **P1 socket server (biggest gap).** `term-relay/data-plane/relay-node.ts` takes `WebSocketLike`
|
||||
+ `AgentListener` as **injected seams** — there is no actual listener. Build a `relay-run/` entry
|
||||
that: (a) terminates browser **WSS** on `BIND_PORT` and adapts each socket to `WebSocketLike`;
|
||||
(b) accepts agent **mTLS** connections on `AGENT_BIND_PORT` (verify SPIFFE client certs against
|
||||
`AGENT_CA_CHAIN_PATH`) and adapts them to `AgentListener`/mux; (c) calls `authorizeUpgrade`
|
||||
(→ P5 `onUpgrade`) and splices. Wire the Redis `relay:revocations` subscriber to `closeStream`.
|
||||
2. **Inject P5 into P1 + P3.** P1's `Authorizer` must call relay-auth `onUpgrade`/`onReattach` with a
|
||||
real `EnforceDeps` (registries backed by P3, revocation store, buckets, audit). P3's
|
||||
`refuseAllVerifier` must be replaced with relay-auth `verifyCapabilityToken` + the real
|
||||
`CAPABILITY_SIGN_PUBKEY_B64`. **Close F2 here:** the P1 upgrade adapter currently hardcodes
|
||||
`principal: null` — for any host with a step-up policy it must resolve + pass the authenticated
|
||||
principal, else those hosts (correctly) 403. See [[relay-stepup-needs-principal-wiring]].
|
||||
3. **P3 durable stores.** Swap `createMemoryStores` for the Postgres adapter (accounts/hosts/sessions
|
||||
registries) + run migrations; point the revocation bus at real Redis (`ioredis`).
|
||||
4. **Real KMS for the CA key.** `boot/ca-wiring.ts` uses a dev in-process Ed25519 signer ("NOT a real
|
||||
KMS"). Provide a `KmsResolver` backed by cloud KMS/HSM; the intermediate key must be
|
||||
non-exportable (§3.1). `CA_INTERMEDIATE_KMS_KEY_REF` is the ref.
|
||||
5. **Agent build + config.** Build `dist/cli.js`; configure `RELAY_URL`/`ENROLL_URL`/`SUBDOMAIN`/
|
||||
`HOST_ID`/`LOCAL_TARGET_URL`/`STATE_DIR`; enroll (redeem pairing code → SPIFFE cert +
|
||||
`hostContentSecret`).
|
||||
6. **relay-web build + serve.** `npm --prefix relay-web run build`; serve the bundle from each tenant
|
||||
subdomain (same origin as the WSS endpoint so Origin/CSWSH checks pass).
|
||||
7. **F6 replay transport (residual).** The recoverable-replay ring buffer must **persist each
|
||||
generation's `epoch` and serve the epoch matching those exact frames** (`relay-web/.../manage-page.ts`
|
||||
`loadReplay` is still a throwing stub). Until wired, the preview/replay feature is inert (fail-closed).
|
||||
|
||||
---
|
||||
|
||||
## 4. Phased build order
|
||||
|
||||
- **Phase 0 — single-host dev run (one process, self-signed).** Reuse the `e2e/harness` wiring:
|
||||
a `ws` server hosting the relay-node, control-plane with in-memory stores, P4/P5 wired for real,
|
||||
self-signed TLS, agent dialing `localhost`, relay-web served locally. Goal: an actually-running
|
||||
end-to-end relay to click through. No pg/redis/KMS.
|
||||
- **Phase 1 — single-tenant staging.** Real Postgres + Redis; real (staging) TLS cert for one
|
||||
subdomain; close F2 principal wiring; agent on a second machine dialing in; behind Tailscale.
|
||||
- **Phase 2 — multi-tenant production.** Wildcard TLS `*.term.<domain>` + subdomain routing; real
|
||||
KMS CA custody; horizontal relay nodes (stateless, share Redis); migrations/rollback; metering;
|
||||
F6 replay transport; monitoring/alerting on the audit sink + revocation latency budget.
|
||||
|
||||
---
|
||||
|
||||
## 5. Environment / config reference (accurate to the code)
|
||||
|
||||
**control-plane (P3)** — `control-plane/src/env.ts`, all required unless defaulted:
|
||||
`PG_URL`, `REDIS_URL`, `CAPABILITY_SIGN_PUBKEY_B64` (P5 verify pubkey), `CA_INTERMEDIATE_KMS_KEY_REF`,
|
||||
`CA_INTERMEDIATE_CERT_PATH`, `NODE_MTLS_TRUST_BUNDLE_PATH`, `BASE_DOMAIN`, `HEARTBEAT_TTL_SEC`=15,
|
||||
`PAIRING_TTL_SEC`=600, `PAIRING_MAX_REDEEM_ATTEMPTS`=5.
|
||||
Routes: `POST /accounts`, `POST /accounts/:id/pairing-codes`, `POST /accounts/:id/status`,
|
||||
`GET /accounts/:id/hosts`, `DELETE /hosts/:hostId`, `POST /enroll`. (No `start` script / listen entry
|
||||
exists yet — add one.)
|
||||
|
||||
**term-relay data-plane (P1)** — `term-relay/data-plane/config.ts`:
|
||||
`BASE_DOMAIN`, `BIND_HOST`, `BIND_PORT` (browser WSS), `TLS_CERT_PATH`, `TLS_KEY_PATH`,
|
||||
`AGENT_BIND_PORT`, `AGENT_CA_CERT_PATH`, `AGENT_CA_CHAIN_PATH` (verify agent mTLS), `RELAY_NODE_ID`,
|
||||
`HEARTBEAT_INTERVAL_MS`, `INITIAL_WINDOW_BYTES`, `MAX_FRAME_BYTES`, `ROUTE_TTL_MS`. (Library — the
|
||||
process that reads this config and listens must be built, gap #1.)
|
||||
|
||||
**relay-auth (P5)** — `RELAY_AUTH_VERIFY_PUBKEY` (base64url raw Ed25519) via `loadVerifyKeyFromEnv`;
|
||||
`RELAY_TRUST_DOMAIN` for SPIFFE.
|
||||
|
||||
**agent (P2)** — `RELAY_URL`, `ENROLL_URL`, `HOST_ID`, `SUBDOMAIN`, `LOCAL_TARGET_URL`, `STATE_DIR`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Infra checklist
|
||||
|
||||
- [ ] **TLS**: browser-facing wildcard cert `*.term.<domain>` (Phase 2) or per-subdomain (Phase 1);
|
||||
separate CA + trust bundle for **agent mTLS** (`AGENT_CA_*` / `NODE_MTLS_TRUST_BUNDLE_PATH`).
|
||||
- [ ] **DNS**: wildcard `*.term.<domain>` → relay node(s); `<control-plane host>`.
|
||||
- [ ] **Postgres**: accounts/hosts/sessions schema + migrations (adapter unbuilt).
|
||||
- [ ] **Redis**: `relay:revocations` pub/sub (P3 publishes, P1 subscribes) — the INV12 teardown path.
|
||||
- [ ] **KMS/HSM**: non-exportable CA intermediate signing key (`CA_INTERMEDIATE_KMS_KEY_REF`).
|
||||
- [ ] **Ingress for agents**: agents dial OUT (mTLS) — expose only `AGENT_BIND_PORT` + browser `BIND_PORT`.
|
||||
- [ ] **Orchestration**: Dockerfiles + compose/k8s; relay nodes are stateless (crash-safe, INV7).
|
||||
|
||||
## 7. Security/ops runbook (deploy invariants)
|
||||
|
||||
- **Origin/CSWSH**: the browser WSS handshake MUST validate Origin against the tenant subdomain
|
||||
(relay-auth `onUpgrade` step 1 enforces it — pass the correct `allowedOrigins`). Never disable.
|
||||
- **Never expose without TLS**; the relay hands a shell to whoever authenticates. Prefer Tailscale
|
||||
for the agent path.
|
||||
- **Key custody (INV9)**: P5 signing key and the CA key never in process memory / logs; KMS refs only.
|
||||
- **Revocation latency (INV12)**: monitor time from `revoke()` → stream teardown; alert on budget miss.
|
||||
- **mTLS trust anchors**: agent certs verified against a **pinned** CA bundle — never a peer-supplied
|
||||
chain (relay-auth `verifyAgentCert` gates on the registry too).
|
||||
- **Audit sink (INV10)**: zero-payload; ship to append-only storage; alert on `deny` spikes /
|
||||
`cross-tenant-attempt`.
|
||||
- **Close before prod**: F2 principal wiring (gap #2) and F6 replay-epoch transport (gap #7).
|
||||
|
||||
---
|
||||
|
||||
## 8. TL;DR
|
||||
|
||||
There is no one-command deploy today. Minimum to get a *running* relay = **Phase 0** (gaps #1, #2, #5,
|
||||
#6 with in-memory stores + self-signed TLS). Production = **all gaps** + Postgres/Redis/KMS/wildcard-TLS
|
||||
+ orchestration. The security-critical *logic* is done and audited; what remains is integration + infra.
|
||||
Reference in New Issue
Block a user