Correction to the previous commit. Its recovery vhost could not work: nginx
will not forward an expired client certificate in ANY `ssl_verify_client` mode.
`optional` answers a bare `400 The SSL certificate error` before any location
runs, and `optional_no_ca` only tolerates CHAIN failures — see nginx's
`ngx_ssl_verify_error_optional()`, which covers self-signed / unknown-issuer /
unverifiable-leaf and NOT `X509_V_ERR_CERT_HAS_EXPIRED`. Verified live against
the deployed :8472 server, which rejected the real expired leaf.
So recovery drops mTLS instead of trying to bend it:
- agent: `buildTlsOptions` and the mTLS renew transport go back to being
strictly fail-closed on expiry — the relaxation is gone from the TLS layer
entirely. The rotator now decides per attempt: valid → mTLS `/renew`,
expired-inside-grace → plain `/recover`, expired-beyond-grace → terminal
`onExhausted` with no request issued at all.
- new `recoverCert` POSTs `{cert, csr}` with no client certificate. Possession
of the private key is still proven: the CSR is self-signed by it and the host
signer already enforces CSR PoP plus `CSR key == registered key`, so a
replayed (public) cert without the key yields at most a certificate the
attacker cannot authenticate with.
- control-plane: `/renew` is strict again (grace 0, matching the terminator).
The grace lives on the new `POST /recover`, which reads the cert from the BODY
and ignores the `x-client-cert` header, then runs the identical trust
pipeline: X.509 path validation to the frp-client-CA anchors, SPIFFE parse,
`notBefore` (never graced), registry `active` + account match. Revocation
still bites.
- deploy: no new vhost, no new SNI, no DNS. One `location = /recover` merged
into the existing enroll vhost, documented in
`deploy/nginx/enroll-recover-location.md` with the nginx source citation.
The load-bearing new test is "a self-signed cert with a FORGED SPIFFE SAN is
refused → 401": nginx no longer validates the chain on this path, so that
assertion is what keeps `/recover` from being a cert vending machine.
Verified: agent 289/289, control-plane 290/290, tsc clean on both.
`POST /renew` is authenticated by mTLS with the very leaf it renews, so once
that leaf lapsed the host could never renew it and the tunnel stayed down until
an operator re-paired by hand. Production hit exactly this: the Mac slept
through its 8h renewal window, the 24h leaf expired, and the agent then logged
`client certificate has expired; renew before dialling` 6380 times over 8 days
without recovering.
Three layers independently refused an expired leaf, so all three had to move:
- agent `buildTlsOptions` gains an opt-in `expiredGraceMs`. Absent/0 keeps the
historical fail-closed behaviour, and the TUNNEL dial never passes it — only
the renew transport does. Past the window it throws the new
`CertExpiredBeyondGraceError`.
- agent rotator routes an already-expired leaf to a separate recovery endpoint
and treats "beyond grace" as TERMINAL: report once via `onExhausted`, stop
retrying, and name the fix (re-pair) instead of spamming warnings forever.
- control-plane `assertPresentedCertTrusted` grants a bounded grace on
`notAfter` only. Chain validation, SPIFFE identity, `notBefore`, and the
registry `active`/account checks are all unchanged, so revocation still bites.
- new `deploy/nginx/recover-mtls.conf` (:8472). The strict enroll vhost cannot
host this: under `ssl_verify_client optional` nginx answers a bare 400 as soon
as a presented cert fails verification, so an expired leaf never reaches the
location — and the directive is server-level, not per-location.
Grace defaults to 30 days on both ends and is configurable (`RECOVER_URL`,
`expiredRenewGraceMs`). The honest trade is recorded in the code: a stale stolen
leaf stays reusable for the window, which widens an existing exposure (an
unexpired stolen leaf already renews indefinitely) rather than opening a new one.
Verified: agent 296/296, control-plane 286/286, tsc clean on both.
