fix(tunnel): break the expired-leaf renewal deadlock
`POST /renew` is authenticated by mTLS with the very leaf it renews, so once that leaf lapsed the host could never renew it and the tunnel stayed down until an operator re-paired by hand. Production hit exactly this: the Mac slept through its 8h renewal window, the 24h leaf expired, and the agent then logged `client certificate has expired; renew before dialling` 6380 times over 8 days without recovering. Three layers independently refused an expired leaf, so all three had to move: - agent `buildTlsOptions` gains an opt-in `expiredGraceMs`. Absent/0 keeps the historical fail-closed behaviour, and the TUNNEL dial never passes it — only the renew transport does. Past the window it throws the new `CertExpiredBeyondGraceError`. - agent rotator routes an already-expired leaf to a separate recovery endpoint and treats "beyond grace" as TERMINAL: report once via `onExhausted`, stop retrying, and name the fix (re-pair) instead of spamming warnings forever. - control-plane `assertPresentedCertTrusted` grants a bounded grace on `notAfter` only. Chain validation, SPIFFE identity, `notBefore`, and the registry `active`/account checks are all unchanged, so revocation still bites. - new `deploy/nginx/recover-mtls.conf` (:8472). The strict enroll vhost cannot host this: under `ssl_verify_client optional` nginx answers a bare 400 as soon as a presented cert fails verification, so an expired leaf never reaches the location — and the directive is server-level, not per-location. Grace defaults to 30 days on both ends and is configurable (`RECOVER_URL`, `expiredRenewGraceMs`). The honest trade is recorded in the code: a stale stolen leaf stays reusable for the window, which widens an existing exposure (an unexpired stolen leaf already renews indefinitely) rather than opening a new one. Verified: agent 296/296, control-plane 286/286, tsc clean on both.
This commit is contained in:
65
deploy/nginx/recover-mtls.conf
Normal file
65
deploy/nginx/recover-mtls.conf
Normal file
@@ -0,0 +1,65 @@
|
||||
# recover-mtls.conf — nginx :8472 RECOVERY vhost: re-issue a leaf that has ALREADY EXPIRED.
|
||||
#
|
||||
# Install target: /etc/nginx/conf.d/recover-mtls.conf
|
||||
# SNI route to merge into the stream map (see stream-sni-additions.md):
|
||||
# recover.terminal.yaojia.wang 127.0.0.1:8472;
|
||||
#
|
||||
# ── WHY THIS EXISTS ───────────────────────────────────────────────────────────────────────────────
|
||||
# `POST /renew` is authenticated by mTLS with the very leaf it renews. Once that leaf lapses the host
|
||||
# can no longer renew it, so the tunnel stays down until an operator re-pairs by hand. That is not
|
||||
# hypothetical: a laptop slept through its renewal window, the leaf expired, and the agent then
|
||||
# logged `client certificate has expired; renew before dialling` 6380 times over 8 days without ever
|
||||
# recovering.
|
||||
#
|
||||
# The enroll vhost cannot host the fix. It runs `ssl_verify_client optional`, and nginx answers a
|
||||
# bare `400 Bad Request` the moment a PRESENTED client cert fails verification — an expired leaf
|
||||
# never reaches the `location`, so it can never be forwarded to the control-plane. `ssl_verify_client`
|
||||
# is a server-level directive and cannot be relaxed per-location, hence a separate server block.
|
||||
#
|
||||
# ── THE TRADE (read before changing anything here) ────────────────────────────────────────────────
|
||||
# This server runs `optional_no_ca`: nginx forwards the presented cert WITHOUT validating its chain.
|
||||
# The control-plane is therefore the SOLE and FULL verifier on this path. `api/renew.ts` does, in
|
||||
# order: real X.509 path validation to the frp-client-CA anchors (`assertPresentedCertTrusted` →
|
||||
# relay-auth `verifyChain`), SPIFFE SAN parse, `notBefore` check (NEVER graced), `notAfter` + bounded
|
||||
# grace (`DEFAULT_EXPIRED_RENEW_GRACE_MS`, 30d), then a registry lookup requiring an `active`,
|
||||
# account-consistent host — so REVOCATION IS STILL ENFORCED, via registry status rather than the CRL.
|
||||
# Do not weaken those checks on the assumption that nginx is still gating this port. It is not.
|
||||
#
|
||||
# Blast radius is deliberately tiny: ONLY `POST /renew` is reachable; every other path 404s, and the
|
||||
# strict enroll vhost (:8471) is left untouched so the normal renewal path keeps its nginx-level
|
||||
# chain + CRL enforcement.
|
||||
|
||||
# Recovery is a once-in-a-blue-moon call per host, but it is reachable pre-authentication and costs
|
||||
# the control-plane an X.509 path validation, so bound it. CP additionally rate-limits per identity.
|
||||
limit_req_zone $binary_remote_addr zone=renew_recover:1m rate=10r/m;
|
||||
|
||||
server {
|
||||
listen 127.0.0.1:8472 ssl;
|
||||
server_name recover.terminal.yaojia.wang;
|
||||
|
||||
ssl_certificate /etc/relay/frp-tls/fullchain.pem; # LE wildcard *.terminal.yaojia.wang
|
||||
ssl_certificate_key /etc/relay/frp-tls/privkey.pem;
|
||||
|
||||
# Ask for a client cert and forward whatever is presented — INCLUDING an expired one, which is
|
||||
# the entire point. `ssl_client_certificate` is kept only so the CertificateRequest carries a CA
|
||||
# hint and clients auto-select the right identity; it does NOT gate anything under optional_no_ca.
|
||||
ssl_verify_client optional_no_ca;
|
||||
ssl_client_certificate /etc/relay/frp-client-ca/frp-client-ca.cert.pem;
|
||||
|
||||
location = /renew {
|
||||
# Under optional_no_ca `$ssl_client_verify` is `NONE` even for a good cert, so presence of the
|
||||
# cert itself is the only thing nginx can meaningfully assert here.
|
||||
if ($ssl_client_escaped_cert = "") { return 403; }
|
||||
|
||||
limit_req zone=renew_recover burst=5 nodelay;
|
||||
|
||||
proxy_pass http://127.0.0.1:8080;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header x-client-cert $ssl_client_escaped_cert;
|
||||
proxy_read_timeout 60s;
|
||||
}
|
||||
|
||||
# Everything else — including /enroll, /device/*, /auth/login — stays on the strict vhost.
|
||||
location / { return 404; }
|
||||
}
|
||||
@@ -17,6 +17,18 @@ frp.terminal.yaojia.wang 127.0.0.1:7000; # frps control channel (TLS passthro
|
||||
*.terminal.yaojia.wang 127.0.0.1:8470; # nginx :8470 TLS-term + device-CA mTLS (frp-mtls.conf)
|
||||
```
|
||||
|
||||
### Later addition — expired-leaf recovery (recover-mtls.conf)
|
||||
|
||||
```nginx
|
||||
recover.terminal.yaojia.wang 127.0.0.1:8472; # expired-leaf re-issue (recover-mtls.conf)
|
||||
```
|
||||
|
||||
**Order IS load-bearing for this one**: it must sit ABOVE the `*.terminal.yaojia.wang` wildcard, the
|
||||
same way `enroll.terminal.yaojia.wang` does. `hostnames` gives exact matches priority over wildcards
|
||||
in nginx's `map`, so a correctly-placed line wins regardless — but keep the exact-match lines grouped
|
||||
above the wildcard so a future editor cannot mis-read the intent. No DNS work is needed: the zone is
|
||||
already served by the wildcard record `*.terminal.yaojia.wang → <vps>`.
|
||||
|
||||
## Coexistence warning (do NOT retype the existing lines)
|
||||
|
||||
The existing `map` body already contains — and MUST be preserved **verbatim**:
|
||||
|
||||
Reference in New Issue
Block a user