# RUNBOOK — deploy the rendezvous-relay to VPS `8.138.1.192` (Alibaba Cloud, mainland)

> **Staging, durable single-tenant** (PLAN_RELAY_PHASE1 §0). Real Postgres + Redis (Docker Compose on
> the VPS), real Let's Encrypt TLS on `:443` against an **already ICP-filed** domain, agent dials OUT
> from the operator's laptop. Dev in-process CA signer is accepted for staging (KMS → Phase 2).
>
> Everything below is a **staging template** — replace every `<placeholder>` and confirm each path on
> the box. Config is **env-only**; no host/port/secret is hardcoded in code or units.

Two independent trust chains — never cross-wire them:

| Chain | Issued by | Protects | Script |
|---|---|---|---|
| **Public web PKI** | Let's Encrypt | browser `:443` WSS (`TLS_CERT_PATH`) | `issue-tls-cert.sh` |
| **Private enrollment CA** | your offline root | agent mTLS + relay agent-server cert (`AGENT_CA_*`) | `gen-agent-ca.sh` |

And one shared key: the **P5 capability keypair** — control-plane **signs**, relay + admin API
**verify** (`gen-capability-key.sh`). `CAPABILITY_SIGN_PUBKEY_B64` (CP) and `RELAY_AUTH_VERIFY_PUBKEY`
(relay) are the **same public key**, two encodings.

---

## 0. Prerequisites (on `8.138.1.192`)

```bash
# Docker Engine + Compose plugin
curl -fsSL https://get.docker.com | sh
sudo systemctl enable --now docker

# Node 20 (satisfies control-plane>=18, relay-run>=20, agent>=18) via NodeSource
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo bash -
sudo apt-get install -y nodejs git openssl

# Clone (this runbook assumes /opt/web-terminal; adjust the systemd --prefix if you change it)
sudo git clone <REPO_URL> /opt/web-terminal
cd /opt/web-terminal
npm ci                          # root deps
npm --prefix control-plane ci
npm --prefix relay-run ci
npm --prefix relay-web ci

# Dedicated non-root service user + secret dir
sudo useradd --system --home /opt/web-terminal relay || true
sudo mkdir -p /etc/relay
```

**DNS (do this first — LE HTTP-01 and the browser both need it):** create an A-record
`<SUBDOMAIN>.<BASE_DOMAIN>` → `8.138.1.192`. The domain must be **ICP-filed** (mainland Aliyun blocks
`:80/:443` on unfiled domains; if unfiled, you must use DNS-01 in step 2).

---

## 1. Bring up Postgres + Redis (Docker Compose, loopback-only)

```bash
cd /opt/web-terminal
cp deploy/.env.example deploy/.env        # gitignored; fill POSTGRES_PASSWORD (strong random)
docker compose --env-file deploy/.env -f deploy/docker-compose.yml up -d
docker compose -f deploy/docker-compose.yml ps      # both healthy; bound to 127.0.0.1 only
```

Both services publish to `127.0.0.1` only — never the public interface (they are the relay's private
state).

---

## 2. Generate the CA, the capability key, and the TLS cert

```bash
# (a) PRIVATE enrollment CA (agent mTLS) — NOT Let's Encrypt.
sudo AGENT_FACING_HOST="<SUBDOMAIN>.<BASE_DOMAIN>" \
     AGENT_FACING_IP="8.138.1.192" \
     RELAY_TRUST_DOMAIN="<RELAY_TRUST_DOMAIN>" \
     bash deploy/scripts/gen-agent-ca.sh
# writes /etc/relay/ca/{root,intermediate,agent-ca.*,relay-agent-server.*}

# (b) P5 capability keypair — prints BOTH public-key encodings; private key -> 0600 file.
sudo bash deploy/scripts/gen-capability-key.sh
#   -> note CAPABILITY_SIGN_PUBKEY_B64 (base64) and RELAY_AUTH_VERIFY_PUBKEY (base64url); same key.

# (c) PUBLIC TLS cert for the browser :443 (Let's Encrypt).
#     http-01 needs inbound :80 open DURING issuance (step 4); dns-01 needs no inbound port.
sudo BASE_DOMAIN="<BASE_DOMAIN>" SUBDOMAIN="<SUBDOMAIN>" ACME_EMAIL="<you@example.com>" \
     ACME_METHOD="http-01" \
     TLS_CERT_PATH="/etc/relay/tls/fullchain.pem" TLS_KEY_PATH="/etc/relay/tls/privkey.pem" \
     bash deploy/scripts/issue-tls-cert.sh

sudo chown -R relay:relay /etc/relay
```

> **Staging CA seam (A-wave, not this task):** the control-plane's dev in-process signer must sign
> agent leaves with `/etc/relay/ca/intermediate.key.pem` so they chain to `agent-ca.bundle.pem`.
> `CA_INTERMEDIATE_KMS_KEY_REF=dev-local` selects the dev signer; point it at that key. Phase 2 = KMS.
> After first boot, move `/etc/relay/ca/root.key.pem` **off** the host (offline anchor).

---

## 3. Fill the two env files

Copy the relevant blocks of `deploy/.env.example` into two 0600 files. **The linchpin:**
`CAPABILITY_SIGN_PUBKEY_B64` (control-plane) and `RELAY_AUTH_VERIFY_PUBKEY` (relay) are the SAME key
from step 2b — paste both.

```bash
sudo install -m 600 -o relay -g relay /dev/null /etc/relay/control-plane.env
sudo install -m 600 -o relay -g relay /dev/null /etc/relay/relay.env
```

`/etc/relay/control-plane.env` (P3):

```ini
PG_URL=postgres://relay:<POSTGRES_PASSWORD>@127.0.0.1:5432/relay
REDIS_URL=redis://127.0.0.1:6379
CAPABILITY_SIGN_PUBKEY_B64=<base64 from step 2b>
CA_INTERMEDIATE_KMS_KEY_REF=dev-local
CA_INTERMEDIATE_CERT_PATH=/etc/relay/ca/intermediate.cert.pem
NODE_MTLS_TRUST_BUNDLE_PATH=/etc/relay/ca/agent-ca.bundle.pem
BASE_DOMAIN=<BASE_DOMAIN>
CP_BIND_HOST=127.0.0.1
CP_BIND_PORT=8080
# HEARTBEAT_TTL_SEC / PAIRING_TTL_SEC / PAIRING_MAX_REDEEM_ATTEMPTS use defaults if unset
```

`/etc/relay/relay.env` (P1 data-plane):

```ini
BASE_DOMAIN=<BASE_DOMAIN>
BIND_HOST=0.0.0.0
BIND_PORT=443
TLS_CERT_PATH=/etc/relay/tls/fullchain.pem
TLS_KEY_PATH=/etc/relay/tls/privkey.pem
AGENT_BIND_PORT=8444
AGENT_CA_CERT_PATH=/etc/relay/ca/agent-ca.cert.pem
AGENT_CA_CHAIN_PATH=/etc/relay/ca/agent-ca.bundle.pem
RELAY_NODE_ID=relay-1
RELAY_AUTH_VERIFY_PUBKEY=<base64url from step 2b — SAME key as CAPABILITY_SIGN_PUBKEY_B64>
RELAY_TRUST_DOMAIN=<RELAY_TRUST_DOMAIN>
PG_URL=postgres://relay:<POSTGRES_PASSWORD>@127.0.0.1:5432/relay
REDIS_URL=redis://127.0.0.1:6379
```

---

## 4. Open the Aliyun security group (inbound)

Open **only** these, in the ECS instance's security group:

| Port | Who | Note |
|---|---|---|
| `443/tcp` | browsers (WSS) | permanent |
| `8444/tcp` (`AGENT_BIND_PORT`) | agents (mTLS) | permanent |
| `80/tcp` | Let's Encrypt HTTP-01 | **temporary** — only during cert issue/renew; skip entirely if using DNS-01 |

**Keep loopback-only (never open):** Postgres `5432`, Redis `6379`, control-plane admin
`8080`. Reach the admin API from your laptop via SSH tunnel: `ssh -L 8080:127.0.0.1:8080 root@8.138.1.192`.

---

## 5. Build the browser bundle

```bash
cd /opt/web-terminal
npm --prefix relay-web run build        # emits relay-web/public/build/, served same-origin by D1
```

---

## 6. Enable + start both units

```bash
sudo cp deploy/systemd/relay-control-plane.service deploy/systemd/relay-data-plane.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now relay-control-plane.service   # migrates PG, listens on 127.0.0.1:8080
sudo systemctl enable --now relay-data-plane.service      # :443 browser WSS + :8444 agent mTLS
sudo systemctl status relay-control-plane.service relay-data-plane.service
journalctl -u relay-data-plane.service -f                 # watch for bind + verify-key load
```

---

## 7. Create an account + pairing code (control-plane admin)

The admin API is **deny-by-default**: `POST /accounts` and `POST /accounts/:id/pairing-codes` require a
**capability token** with the `manage` right whose **`aud` equals `BASE_DOMAIN`** (control-plane
`authz.ts` / `main.ts` `expectedAud: env.baseDomain`). `accountId` is taken ONLY from the verified
token (INV3) — never from the body.

Mint the token with the **capability PRIVATE key** from step 2b (the B6 token-mint owns the signing
helper; it imports `/etc/relay/capability/capability-sign.key.pem`). Token claims:
`rights=['manage']`, `aud=<BASE_DOMAIN>`. Send it as `Authorization: Bearer <token>`.

Bootstrap has a two-token nuance (grounded in `provision.ts`):

1. `POST /accounts` checks `manage` only (no account-ownership check) → mint a `manage` token, create
   the account, read back its `id`.
2. `POST /accounts/:id/pairing-codes` additionally enforces `token.sub == :id` (own-account). Mint a
   second `manage` token with `sub=<the new accountId>`, then request the pairing code.

Over the SSH tunnel from step 4:

```bash
# 1) create account
curl -sS -X POST http://127.0.0.1:8080/accounts \
  -H "Authorization: Bearer <MANAGE_TOKEN aud=BASE_DOMAIN>" \
  -H 'content-type: application/json' -d '{"plan":"personal"}'
#   -> { "id": "<ACCOUNT_ID>", ... }

# 2) pairing code (token.sub must equal <ACCOUNT_ID>)
curl -sS -X POST http://127.0.0.1:8080/accounts/<ACCOUNT_ID>/pairing-codes \
  -H "Authorization: Bearer <MANAGE_TOKEN sub=ACCOUNT_ID aud=BASE_DOMAIN>"
#   -> { "code": "<PAIRING_CODE>", ... }   (single-use, TTL 600s default)
```

---

## 8. On the LAPTOP — build the agent, pair, run

The agent dials OUT; nothing inbound is opened on the laptop.

```bash
cd <repo>/agent
npm ci && npm run build                 # emits dist/cli.js (C1)
export RELAY_URL="wss://<SUBDOMAIN>.<BASE_DOMAIN>:8444"     # AGENT_BIND_PORT
export ENROLL_URL="https://<SUBDOMAIN>.<BASE_DOMAIN>/enroll"
export HOST_ID="<pick-a-host-id>"
export SUBDOMAIN="<SUBDOMAIN>"
export LOCAL_TARGET_URL="ws://127.0.0.1:3000"              # the base web-terminal app
export STATE_DIR="$HOME/.web-terminal-agent"

node dist/cli.js pair <PAIRING_CODE>    # redeems the code -> SPIFFE mTLS cert + hostContentSecret
node dist/cli.js run                    # dials the relay, holds the mux tunnel
```

`pair` redeems the single-use code and stores the enrolled SPIFFE cert under `STATE_DIR`; `run` opens
the outbound mTLS tunnel and keeps it alive (heartbeat/backoff).

---

## 9. Browser — log in and click through to the shell

1. Open `https://<SUBDOMAIN>.<BASE_DOMAIN>` (served same-origin as the WSS endpoint).
2. Operator login (B6) mints a short-lived connect capability token.
3. Pick the host → the relay splices browser ↔ agent. The relay only sees **ciphertext** (INV2); the
   E2E is browser ↔ agent.

---

## 10. Verify restart-safety + revocation teardown

```bash
# Restart-safety (INV7): bounce the relay while a shell is open — the PTY and host registration survive.
sudo systemctl restart relay-data-plane.service
#   the browser auto-reconnects; the session is still there (state is in PG/Redis, not the process).

# Revocation teardown (INV12): revoke the host and watch the tunnel drop within budget.
curl -sS -X DELETE http://127.0.0.1:8080/hosts/<HOST_ID> \
  -H "Authorization: Bearer <MANAGE_TOKEN sub=ACCOUNT_ID aud=BASE_DOMAIN>"
#   -> control-plane publishes a KillSignal on Redis relay:revocations; the data-plane subscriber
#      closes the spliced stream (browser WS closes 4403). Measure revoke -> close latency.
```

---

## Troubleshooting

| Symptom | Meaning | Fix |
|---|---|---|
| Browser WS closes **1013** | `WS_TRY_LATER` — **no agent tunnel** for that host (agent offline / not dialed) | Start the agent (`node dist/cli.js run`); check `RELAY_URL` host+port; confirm `:8444` open; watch `journalctl -u relay-data-plane`. |
| Browser WS closes **401** | **bad Origin** (CSWSH exact-match failed) — the `Origin` header is not in the relay's `allowedOrigins` | Open via `https://<SUBDOMAIN>.<BASE_DOMAIN>` (not the raw IP, not `http:`, no stray port). `allowedOrigins` is the port-less `https://<sub>.<BASE_DOMAIN>` on `:443`; must match EXACTLY. Verify `BASE_DOMAIN` in `relay.env`. |
| Agent mTLS closes **4401** | cert chains to the CA but the **pubkey is not enrolled** in the host registry (INV14 registry-gated) | The host was never enrolled or was revoked. Re-run `pair <CODE>` with a fresh code; confirm the CP signed the leaf with `/etc/relay/ca/intermediate.key.pem` (chains to `agent-ca.bundle.pem`). |
| Browser WS closes **4403** | `WS_REVOKED` — the host/session was revoked (expected after step 10) | Re-enroll the host to reconnect. |
| CP fails to boot: "CAPABILITY_SIGN_PUBKEY_B64 must decode to 32 bytes" | wrong key encoding in `control-plane.env` | Use the **base64** value (not base64url) from step 2b for the CP; base64url is the relay's `RELAY_AUTH_VERIFY_PUBKEY`. |
| Admin API returns **401** on `POST /accounts` | missing/expired/foreign-`aud` capability token | Mint a `manage` token with `aud=<BASE_DOMAIN>`, signed by the step-2b private key; send `Authorization: Bearer`. |
| Admin API returns **403** on `/pairing-codes` | token lacks `manage` right, or `token.sub != :id` | Mint with `rights:['manage']` and `sub=<ACCOUNT_ID>` (own-account rule). |
| LE issuance fails (timeout/connection) | HTTP-01 `:80` not reachable, or domain not ICP-filed | Open `:80` in the security group for issuance, or switch `ACME_METHOD=dns-01`; confirm the A-record + ICP filing. |
| `docker compose ps` unhealthy | PG/Redis not up | `docker compose -f deploy/docker-compose.yml logs`; confirm `POSTGRES_PASSWORD` set in `deploy/.env`. |
| Mixed-content / `ws:` blocked in browser | page is HTTPS but WS scheme resolved to `ws:` | Serve over HTTPS; the client follows page scheme (`wss:` on HTTPS, M6). Confirm the LE cert is valid for the FQDN. |
