web-terminal/deploy/nginx/enroll-recover-location.md

Enroll vhost addition — /recover (expired-leaf recovery)

One location block to merge into the existing enroll vhost on the VPS (/etc/nginx/conf.d/enroll.conf, the 127.0.0.1:8471 server). No new server, no new SNI route, no DNS work.

This is a MERGE, not a file to ship. The enroll vhost also carries /enroll, /device/enroll, /auth/login, /crl/, /renew and /device/:id/renew — do not replace it.

Why the route exists

POST /renew is mTLS-authenticated by the very leaf it renews, so once that leaf lapses the host can never renew it and the tunnel stays down until an operator re-pairs. That is not hypothetical: a laptop slept through its 8h renewal window, its 24h leaf expired, and the agent then logged client certificate has expired; renew before dialling 6380 times over 8 days without recovering.

Why it cannot be fixed on /renew itself

nginx will not forward an expired client certificate, under any ssl_verify_client mode.

• optional → nginx answers a bare 400 The SSL certificate error as soon as verification fails. The request never reaches the location, so no if ($ssl_client_verify …) can rescue it.
• optional_no_ca → does not help either. It only tolerates chain failures; see nginx's ngx_ssl_verify_error_optional(), which covers DEPTH_ZERO_SELF_SIGNED_CERT, SELF_SIGNED_CERT_IN_CHAIN, UNABLE_TO_GET_ISSUER_CERT_LOCALLY and UNABLE_TO_VERIFY_LEAF_SIGNATURE — and not X509_V_ERR_CERT_HAS_EXPIRED.
• ssl_verify_client is a server-level directive, so it cannot be relaxed per-location anyway.

So recovery drops mTLS: /recover takes no client certificate, and the lapsed cert travels in the request body instead.

Why that is still authenticated

A certificate is public, so the body alone proves nothing — possession of the private key does, and it is still proven end to end:

• the accompanying CSR is self-signed by that key, and the host signer's delegated gate enforces CSR proof-of-possession plus CSR key == registered key (control-plane/src/ca/csr.ts verifyCsrPoP);
• control-plane/src/api/renew.ts runs the same trust pipeline as /renew — real X.509 path validation to the frp-client-CA anchors, SPIFFE SAN parse, notBefore, and a registry lookup requiring an active, account-consistent host — differing only in a bounded overrun allowance on notAfter (DEFAULT_EXPIRED_RENEW_GRACE_MS, 30 days).

Worst case for a replayed cert without the key: the attacker receives a certificate they cannot authenticate with. Revocation still bites, via registry status.

The block to add

    # Expired-leaf recovery: NO client cert (see enroll-recover-location.md). The control-plane is
    # the sole verifier; strip any client-supplied cert header so only the body can speak.
    location = /recover {
        limit_req zone=renew_recover burst=5 nodelay;
        proxy_pass http://127.0.0.1:8080;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header x-client-cert "";
        proxy_read_timeout 60s;
    }

And once, in the http context (top of enroll.conf is fine) — the route is reachable pre-authentication and costs the control-plane an X.509 path validation, so bound it. The control-plane additionally rate-limits per identity (createRenewRateLimiter, 30/hour):

limit_req_zone $binary_remote_addr zone=renew_recover:1m rate=10r/m;

Deploy gate

cp /etc/nginx/conf.d/enroll.conf{,.bak.$(date +%s)}   # snapshot first
# ...merge the block above...
nginx -t && systemctl restart nginx                   # NEVER reload on a failed -t

restart, not reload: a graceful reload has been observed keeping old workers alive for seconds, which makes post-deploy verification race the change.

Verify

# no cert needed — an in-grace expired leaf is re-issued
curl -sS --resolve enroll.terminal.yaojia.wang:443:<vps> \
  -X POST -H 'content-type: application/json' \
  -d "{\"cert\":\"$(base64 -w0 expired.cert.pem)\",\"csr\":\"$(base64 -w0 new.csr.der)\"}" \
  https://enroll.terminal.yaojia.wang/recover        # → 201 {cert,caChain,notAfter}

# a forged self-signed cert with a correct-looking SPIFFE SAN must still be refused
# → 401   (this is the load-bearing check; nginx is no longer validating the chain here)