Adversarial review of the previous commit found ten defects. The critical one
would have destroyed exactly the work this feature exists to protect.
CRITICAL — `#{session_activity}` is a CLIENT clock, not an output clock.
It advances on attach and on keypresses, and never moves for a detached session
however much its shell is printing. Measured on tmux 3.6a: a detached session
running `while true; do echo tick; sleep 1; done` still reported
session_activity == session_created after 6 s, while `#{window_activity}` tracked
the output exactly. So "idle for 7 days" was really "nobody has typed for 7 days",
and `Clean up idle 7d+` would have killed long-running unattended sessions that
were busy working — the walk-away case the whole app is built around.
Measured against this host's 69 sessions: the old clock condemned 36, the new one
condemns 33, so three sessions doing real work were one click from deletion.
Now reads window_activity, with a test that fails if the format string regresses.
HIGH — no timeout on any tmux call. maxBuffer bounds a flood but nothing bounded
the wait, and a synchronous exec cannot be interrupted by the event loop, so a
tmux server stopped mid-syscall hung the whole server permanently. All five calls
(including the three that predate this feature) now carry one.
HIGH — listSessions and capturePane were synchronous on a POLLED path. Measured
73 ms and 58 ms per call on this host; every home screen on every device refreshes
on a 5 s timer, and each of those was 73 ms with the event loop stopped, i.e.
73 ms in which no PTY byte reached any terminal anywhere. Both are async now, so
the manager's listOrphans/captureOrphan/killOrphansIdleSince are too. The server
is a byte-shuttle first: nothing polled may block it.
HIGH — GET /orphan-sessions had no rate limiter although, unlike every other read
route, it spawns a subprocess. It shares the preview bucket now.
HIGH — the cleanup confirmation counted CARDS, and the grid is capped at 24 while
the cleanup matches across every session the server enumerates. On this host it
would have said "24" and ended 33 shells. New GET /orphan-sessions/count-idle
answers the real question, and the dialog states that number, or refuses to
proceed if it cannot be determined.
MEDIUM — fetchOrphanSessions collapsed every failure into [], so one 429 or 500
read as "all recovered sessions are gone", tearing down every mounted xterm and
rebuilding it on the next success. It returns null for failure now, and the
refresh leaves the section untouched.
MEDIUM — the preview latch was "card was created", not "preview succeeded", so a
card that lost its single request stayed blank forever. Latches on success.
MEDIUM — overlapping refreshes. refresh() is entered from a 5 s timer, from
killOrphan and from cleanup, with awaits inside and no guard, so a slow poll could
resurrect cards for sessions already killed or re-mount them into a root that
setVisible(false) had just cleared. A generation counter, bumped on teardown and
checked after every await, invalidates stale runs.
MEDIUM — renderPreview could write to a disposed xterm (it throws "Object has
been disposed"). PreviewCard carries an explicit `disposed` flag set by a new
disposeCard() helper, so no caller can dispose a terminal and forget the flag.
Deliberately NOT el.isConnected: a card not yet appended is alive, and two
existing preview-grid tests correctly caught that conflation.
LOW — kill and cleanup discarded their results, so a refusal (403 behind a proxy
whose Origin differs, 404 on a lost race) looked like a dead button. Both report.
Tests: 2209 unit, 27 e2e, 8 orphan integration. Verified live against all 69 real
sessions with none harmed; the window_activity fix confirmed on an isolated tmux
socket (-L) so the developer's sessions were never involved.
A `web_*` tmux session that outlives the process which created it was unreachable
from the UI, permanently. Nothing enumerated tmux — src/session/tmux.ts had
hasSession but no list — so recovery only worked if a client still had the id in
localStorage. list() walks the in-memory table, and so does reapIdle, so such a
session could not be listed, previewed, joined or killed, and never aged out.
On this host that was 69 tmux sessions with 5 clients: 64 unreachable, the oldest
from 26 Jun, one of them still running a Claude Code session with 7 agents.
This is the "catalogue" approach: make them visible and actionable WITHOUT
adopting them. Nothing is attached, because attaching makes tmux resize the window
to the new client's dimensions and SIGWINCH whatever is running inside — doing
that to dozens of live TUIs at startup is exactly the outcome to avoid. Adoption
stays an explicit user action: opening an orphan re-attaches by id through the
existing re-attach path, and it becomes an ordinary live session.
src/session/tmux.ts listSessions() + parseSessionList() (pure, so the filter
rules are unit-testable) and capturePane(), which reads a
session's current screen via `capture-pane -p -e` — no
client, no resize, colour preserved.
src/session/manager.ts listOrphans/captureOrphan/killOrphan/killOrphansIdleSince
src/server.ts GET /orphan-sessions, GET /orphan-sessions/:id/preview,
DELETE /orphan-sessions/:id, DELETE /orphan-sessions
?idleDays=N
public/ a "Recoverable" section under the home grid, reusing the
existing card/thumbnail machinery via orphanAsLive()
Guards, all enforced in the manager so no route can forget one:
- the tmux name must be `web_` + a UUID v4 (SESSION_ID_RE, the same gate the
attach protocol applies). This is what stops a hostile or malformed name from
reaching `tmux -t` — a literal `-t`, or path traversal — and it means a tmux
session the user created for their own work is never enumerated, never
previewed and never offered for deletion.
- an id the table already owns is refused everywhere: killing it via tmux would
end the shell behind a live PTY's back and leave a zombie table entry. Those go
through killById.
- every function is inert when cfg.useTmux is off, so the non-tmux deployment is
byte-for-byte unchanged.
- bulk cleanup skips ATTACHED sessions and requires idleDays >= 1: there is no
"delete everything" form of the route. "This server does not track it" is not
evidence that it is abandoned — a plain `tmux attach` and a second server
process both report as attached.
- reads carry no Origin guard (same threat model as /live-sessions); both DELETEs
do. Preview gets its own rate-limit bucket since each call spawns a process.
Two things the live run exposed and this fixes: previews were loading
sequentially (69 tmux spawns per 5 s refresh) — now once per card, fire-and-forget
— and the grid is capped at 24 cards because each thumbnail is an xterm instance.
The remainder is stated in the UI with the tmux command to reach it, never
silently truncated.
Separate wire type from LiveSessionInfo on purpose: an orphan supports none of the
live-session operations, and the Android/iOS clients decode /live-sessions, so a
variant shape there would break them.
Tests: 2203 unit (+42: parse/filter rules, all four manager guards, tmux-off
inertness) and 8 integration against real tmux — create a throwaway session,
list it, capture it while asserting it stays unattached, reject non-UUID ids,
reject a foreign Origin, kill it, 404 the second time. The integration file never
issues the bulk DELETE: it runs against the host's real tmux server and would end
the developer's own sessions. Verified live against all 69 with none harmed.
`npm run dist:mac` failed at the signing step:
codesign --timestamp .../node_modules/node-gyp/gyp/pylib/gyp/generator/\
__pycache__/make.cpython-313.pyc
→ "A timestamp was expected but was not found."
node-gyp only COMPILES native modules. Nothing loads it at runtime — node-pty
resolves its prebuilt .node directly, and a grep over the packaged tree finds no
require of it outside the package itself. But it vendors a Python tree, and
`npmRebuild: true` runs it and leaves __pycache__/*.pyc behind. Those got copied
into Contents/Resources and codesign then tried to sign Python bytecode, which
takes the entire build down.
Excluding it removes both the dead weight (5 MB) and the failure surface. Also
excludes **/__pycache__ outright, so no Python bytecode can reach the bundle by
another route.
Verified on a real build: signature verifies --strict, node-gyp absent, zero .pyc
in the bundle, public/style.css and build/main.js byte-identical to the repo,
express/ws/web-push/qrcode/google-auth-library all still resolve from the
packaged tree, and node-pty's Electron-ABI pty.node is in place.
Note: the immediate failure was transient — a plain retry of the same config
succeeded (Apple's timestamp server). So this is the durable fix for a landmine
that would otherwise re-arm on every npmRebuild, not the cause of that one run.
Deviation from the one-worktree-per-session rule: the desktop build packages from
desktop/node_modules and its script does `cd .. && npm run build`, so it can only
be exercised in the main checkout. Committed on develop directly.
The low-severity leftovers from the panel audit, plus the tooling trap that made
reviewing this work harder than it needed to be.
.gitattributes (new) — `public/projects.ts diff`
That file embeds literal NUL bytes on purpose: the namespace sentinels
'\0active' / '\0other' are prefixed so they cannot collide with a real
`First.Second` key. git therefore classified it as binary and printed
"Binary files differ" for every diff, and `git diff --stat` reported
"Bin 66815 -> 68701 bytes" instead of line counts — the file was effectively
unreviewable. The `diff` attribute makes git diff it as text without setting
`text`, so no line endings are normalised and the bytes on disk are untouched.
(Same trap in the shell: grep prints only "Binary file ... matches", which
reads as a no-match. The comment in the file says to use grep -a.)
Detached HEAD now shows its short sha
The Head cell held the "detached HEAD" chip and nothing else: it said what was
wrong and nothing about where you were. SyncState gains an optional `head`,
populated from the .git/HEAD text readSyncState ALREADY reads for the detached
test — so this costs no extra I/O and no extra git call. Guarded by a hex regex
and rendered as textContent.
One geometry for the ghost button role
The design defines two button roles. The outline one had drifted to three
different sizes (.proj-wt-open at 11.5px/4px 12px/6px, .proj-diff-toggle at
12.5px/7px 15px/7px) and both used full-strength --text instead of the muted
label. All four outline buttons are now declared once, together, at the
design's geometry; the duplicate .proj-diff-toggle block further down the file
is gone, since a second block at equal specificity is exactly how the role
drifted in the first place.
Header branch chip joins the chip language
.proj-branch was the only chip with no border and no mono face, and 1px tighter
all round, so the detail header's branch did not match the branch chips in the
worktree list right below it.
"Changes" heading over the View Diff toggle
Every other block on the page is introduced by a section heading. Without one
the toggle hung 4px off the bottom of the sync band and read as part of it.
Tests: 2167 unit (4 new — detached sha present/absent on both the server and the
panel) and 27 integration, twice.
An audit of the project-detail panel against docs/mockups/project-detail-git.html
turned up 59 differences. The ones that changed what you see:
Controls that had class names but no rules, so they rendered as raw OS widgets:
- .proj-wt-form had no layout at all — the branch input and Create Worktree
button sat flush together, and the button jumped to its own line the moment a
validation error appeared between them.
- .proj-wt-error / .proj-wt-actions-error had no rule, so a validation failure
rendered as 16px near-white body text, indistinguishable from content.
- .proj-wt-remove (the worktree ✕) and .proj-wt-prune had no rule either.
- .proj-fanout-mode never reset `appearance`, so WebKit kept its own light
dropdown — the one control in the panel that stayed a native grey widget.
- --danger was never defined, so every validation message fell back to an
off-palette red.
Cascade bugs — a later rule at equal specificity was silently winning:
- .proj-wt-row-current's amber wash and border were completely dead. "You are
here" rendered identically to every other row.
- .proj-wt-branch inherited the 16px user-agent default in the UI sans, making a
branch name the largest text on the page. Now 13.5px mono.
Layout:
- .proj-syncband dropped flex-wrap. Wrapping produced ragged half-empty rows
between 721px and 1000px — at 900px the Fetch button dropped alone onto a
second row. One cell now absorbs the shrink so the band stays a single row
down to the 720px column break.
- Commit rows are a 4-column grid, not flex, and the ↑ mark cell is rendered on
every row instead of only unpushed ones. Previously no two lines agreed on
where the sha started, so the unpushed group did not read as a group.
- Pushed commit subjects step down to --text-dim, which is what makes the
upstream boundary mean anything.
Honesty of the sync band — the design's two hard rules:
- Green "✓ in sync" was reachable when ahead/behind were UNKNOWN, because
`?? 0` folded undefined into zero. Unknown counts now render "↑ —" / "↓ —"
and are marked unverified. Green requires real zeros and a fresh fetch.
- A stale ↓ painted the digit warning-red. Since a missing FETCH_HEAD is the
default state, the largest glyph in the band was almost always red and the
panel read as "this repo is broken". The doubt now sits on the `unverified`
chip and the footnote, never on the number.
Also: semantic tokens (--sunk/--warn/--ok/--dirty/--mono-font) replacing hex
literals, one chip language across the panel, section headings as tracked caps
with the count as a separate lighter element, and a ≤720px rule that drops the
age column — placed after the base rule, since a media query adds no specificity.
Verified in a real browser at 1280/1000/900/780/721/600/420px: the sync band
holds one row down to 721px and stacks below it, commit columns align on every
row, and nothing overflows or scrolls the page horizontally.
margin-left:auto was correct for the old single-line row, where it pushed the
path to the far right of a flex row. The two-line card stacks branch over path
in a column, so the same declaration shoved the path to the opposite edge from
the branch it labels. Removed, with a note saying why it must not come back.
Same shape as the .proj-wt-row duplicate earlier in this series: an element
reused in a new layout kept a positioning declaration that only made sense in
the old one. Caught by screenshot; the DOM assertions could not see it.
Two things, both invisible in code review and obvious on screen.
1. Eight controls in the project detail had class names but no CSS rule at
all, so they rendered as raw user-agent widgets — white input boxes and grey
buttons sitting inside a dark panel: View Diff, the new-branch input, Create
Worktree, and the five fan-out fields. Gave them the skin the panel already
established, with two button roles: filled accent for the control that
commits a form (matching Fetch), outline for one that only reveals or
toggles (matching Open). Layout stays in each element's existing rule; this
block sets only the shared skin, so no property is declared twice.
2. The w6 CSS referenced --fg, --fg-dim, --fg-faint, --bg-sunk and --mono,
none of which exist in :root. Every one of those 19 references was silently
falling through to its hardcoded fallback, which happened to match the
default amber theme — so it looked correct while being wired to nothing.
Under any other theme the sync band would have kept the default colours
while the rest of the app changed. Rewired to the real tokens: --text,
--text-dim, --text-faint, --surface-1, --on-accent, and the mono stack the
file already uses.
The fallback values are what hid this: a var() fallback turns a typo into a
silent success. Worth remembering when reaching for one.
Verified: tsc clean, npm test green (78 files / 2161, e2e 27), bundle rebuilt.
Two layout defects the screenshots caught that the DOM tests could not.
1. `.proj-wt-row` was declared twice — my two-line-card rule earlier in the
file, the original flex rule later. Same specificity, so the later copy won
and the Open button sat against the text instead of at the card's right
edge. Folded the grid into the ORIGINAL rule and left a note where the
duplicate was. One rule per selector; a second declaration of a layout
property is a silent override waiting to happen.
2. The unpushed count rendered as its own line under "Recent commits". The
design has it ON the heading, because it qualifies that heading rather than
making a separate statement. renderGitLog now takes the heading element and
writes the badge there, clearing any previous badge first so a re-render
cannot stack duplicates.
Both were invisible to the existing assertions: they checked that the elements
exist and carry the right text, which was true in each case. Added a test for
the no-duplicate-badge path; the right-edge alignment is a pure CSS outcome and
is verified by screenshot, not by the suite.
Closes the three deviations between the shipped git panel and
docs/mockups/project-detail-git.html: the four-cell sync band with its
explanatory footnotes, the dirty count on the title line, and worktree rows as
two-line cards with a working Open button. Adds structural assertions so
layout drift is caught by the suite, not by eye.
The shipped panel was functionally right — every number matched git — but it
was not the approved design. I implemented from the plan's semantics (which
state is green, when to flag stale) and let the mock's layout drift, and
nothing caught it: the tests asserted behaviour, and no assertion covered
structure. Three deviations, now closed against docs/mockups/project-detail-git.html.
1. The sync band was compressed into one row of chips. Restored to the four
labelled cells: Upstream / To push / To pull / Fetch, with display-size
numerals and a footnote under each count. The footnotes are load-bearing,
not decoration — "Last fetched 19d ago — this number cannot be trusted" is
what tells a reader WHY the zero is suspect. Compressed to
"fetched 19d ago", the causal link was left for the reader to infer, and
not having to infer it is the entire point of the panel. The unverified
state is a chip on the count again, so the doubt attaches to the digit.
2. The working-tree count sat inside the band. Moved to the title line beside
the branch, where the design puts it: the band is about the remote, and the
dirty count is not. Reads "● 3 uncommitted" instead of a bare dot; the dot
remains as the fallback when a server sends no count.
3. Worktree rows were the old single line with the path right-aligned against
the chips. Rebuilt as the two-line card the design calls for — identity and
state on top, path underneath — plus the Open button, which was specified
and simply missing, so a worktree row could be read but not entered. Open
reuses the detail view pointed at the worktree path: a linked worktree is a
project directory, so this needs no new concept and no new route.
Also added the unpushed count beside the commit-list heading, so the number is
readable before the eye walks the rows hunting for rails.
Tests now assert structure, not just behaviour: cell captions exist, the stale
footnote says why, counts render at display size, the path is outside the chip
row, and Open fires with its worktree. That is the gap that let the drift
through, so it is the gap that is now covered.
Verified: tsc and build clean; npm test green (unit 78 files / 2159, e2e 27).
Adds the sync band (upstream, ↑ to push, ↓ to pull, staleness), the unpushed
boundary in the commit list, a reworked worktree panel with per-row state and
session counts, and a read-only POST /projects/git/fetch so the numbers can be
refreshed without a pull.
The governing rule throughout: only one state may look reassuring — even with
upstream and a fetch inside the hour. behind is derived from a locally cached
ref, so a stale zero is a guess and is marked as one; no upstream renders as
'no upstream', never as synced.
Also fixes the test suite's long-standing flakiness (5-11 shifting failures per
run) so the feature could actually be gated on it.
`npm test` was failing 5-11 tests per run with the set shifting between runs,
which made it useless as a gate. Two independent causes, neither of them a
product defect.
1. Fixtures had outgrown vitest's 5 s default. The git ones spawn 6-12
sequential `git` processes (init/config/commit/clone/push); the real-server
ones boot a server and a shell. Alone they finish easily; under a full
parallel run they do not, and the failure reads `Test timed out in 5000ms`.
Gave those describes an explicit 30 s ceiling, and the real-PTY waits a
named PTY_WAIT_MS. The deliberate short bounds in the "must be rejected"
handshake tests are left alone — there a timeout IS the assertion.
(The same rebudgeting also touched the test files in the preceding commit.)
2. test/integration/server.test.ts cannot share the machine with the rest of
the suite. It asserts on real prompt output within seconds, which is not
achievable while ~8 workers saturate the box: it passed alone (27/27,
repeatedly) and flaked in the full run. Raising the numbers further only
moved the flake around, so this is fixed as a scheduling problem — `npm
test` now runs two passes, `test:unit` (everything else, parallel) then
`test:e2e` (that file on its own). `vitest run` still runs everything at
once for anyone who wants that.
Also bounded the srv.close() in H1's finally. Unbounded, it swallowed whatever
really went wrong in the body — the inner waits threw, control jumped to the
finally, close() blocked, and the case reported a bare timeout instead of its
own diagnosis. The root-causing above only became possible after making that
failure legible.
Note on the `[needs real PTY (sandbox-off)]` labels: those cases were NOT
skipping here. PTY_AVAILABLE is true on this machine, so they were running and
failing on timing, not being gated out by a sandbox.
Verified: npm test green end to end across repeated runs (unit 78 files /
2147 tests, e2e 27).
The page carried one bare `●` for "dirty" and nothing else, so "do I have
commits I haven't pushed" and "which worktree am I in" still meant dropping
into a terminal. Design mock: docs/mockups/project-detail-git.html; plan and
task breakdown (G1-G7): docs/plans/w6-project-git-panel.md.
One rule drives the whole feature: ahead/behind compare against `@{u}`, a
LOCALLY CACHED remote ref that only a fetch moves. This repo was the live
example while building — `↑9` true, `↓0` false, because FETCH_HEAD had not
moved in 19 days. So:
- `ahead` needs only local refs and is never flagged.
- `behind` is flagged `stale` once FETCH_HEAD is older than an hour.
- Exactly ONE state may render green: ↑0 ↓0 AND a fresh fetch. Green means
"I checked, ignore this"; getting it wrong is lying to the user.
- No upstream (the normal state of a fresh worktree branch) leaves ahead and
behind undefined — it renders an explicit `no upstream`, never the green
path. That fall-through is the easiest bug to ship here.
What landed:
G1 SyncState (upstream/ahead/behind/lastFetchMs/detached) + ProjectDetail.sync
and .dirtyCount. All additive and optional — the Android and iOS clients
decode these shapes. The ahead/behind helper already existed for the list
view; buildProjectDetail had simply never called it.
Fixes a pre-existing bug on the way: readBranch read <repo>/.git/HEAD
directly, so it returned nothing inside a LINKED worktree, where .git is a
file. resolveGitDirs now resolves both the per-worktree gitdir (HEAD) and
the shared common dir (FETCH_HEAD).
G2 POST /projects/git/fetch. Same discipline as push: the remote is derived
server-side and no remote or refspec is ever read from the body, so a
client cannot aim it at an arbitrary URL. Touches refs/remotes only — no
working tree, no index, no merge; it is not a pull. Own rate-limit bucket
so refreshes cannot eat the budget a real push needs. On failure
lastFetchMs is left alone, so the UI keeps saying "stale" instead of
pretending it refreshed.
G3 makeSyncBand replaces the bare dot: upstream name, ↑n, ↓n, stale flag,
dirty count, Fetch button (disabled on a detached HEAD).
G4 The commit list marks unpushed commits and draws the upstream boundary
once, after the last of them. Marking is server-side from `rev-list`,
deliberately NOT "the first N rows": `git log` is date-ordered, so merging
an older branch interleaves unpushed commits BELOW pushed ones, and that
shortcut fails in the dangerous direction — calling an unpushed commit
pushed. A regression test builds exactly that backdated-merge shape.
G5 The worktree section is always "Worktrees (n)" (it used to rename itself
to "Branch" at n=1) and the current row carries its own state chips.
G6 Cost control. The plan called for a .git-mtime cache; that was dropped
during implementation because a fingerprint over HEAD/index/reflog does
NOT move when push updates a remote-tracking ref — the cached `ahead`
would still claim "9 to push" right after a successful push, which is the
exact lie the feature exists to prevent. Replaced with three measures that
cannot go stale: in-flight coalescing (N devices watching one repo cost
one probe, entry dropped as it settles, nothing cached across time),
skipping the re-render when the payload is byte-identical (this also stops
the 5 s re-mount of the commit log, two more git spawns per tick), and
pausing the timer while the document is hidden.
G7 Per-worktree state via GET /projects/worktree/state, kept narrower than
/projects/detail so N rows do not pay for worktree listing and CLAUDE.md
reads nothing renders. Needed an unplanned prerequisite: ProjectSessionRef
carried no cwd, so sessions could not be attributed to a worktree. Added
it, plus countSessionsByWorktree, which matches DEEPEST-first because
.claude/worktrees/<name> lives INSIDE the main checkout and prefix
matching would count every worktree session against the parent repo too.
Out of scope, unchanged: no reset, no checkout, no clean, no rebase, no
force-push. stage/commit/push stay exactly as they were.
Verified: tsc and build clean; 46 new tests.
`worktree.baseRef: head` is the setting CLAUDE.md's session workflow depends on:
without it EnterWorktree defaults to `fresh` and branches from `origin/main`,
which trails `develop` by ~75 commits. Shared config, so it belongs in the repo —
personal overrides still go in the ignored `.claude/settings.local.json`.
Sessions that change files now work in their own git worktree and merge back to
`develop`, instead of developing directly in the main checkout. Documents the
tool's actual behaviour rather than the assumed one: `EnterWorktree` prefixes the
branch as `worktree-<name>`, branches from the last commit (so uncommitted edits
do not carry over), and `ExitWorktree({action: "remove"})` deletes the branch
with the directory — hence the merge must happen before the cleanup.
Base ref is pinned to `head` in `.claude/settings.json` because `develop` runs
well ahead of `origin/main`, and the default `fresh` would branch from the stale
release trunk.
Also ignores `.claude/worktrees/`, which lives on disk per session and is never
committed.
Closes the three leftovers called out after the renewal-deadlock fix: a source
file the blanket `dist/` ignore rule kept out of every checkout, tunnel logs
that named no host (including all 6380 warnings during the outage), and the
missing phone-track counterpart to /recover.
1. `.gitignore` swallowed a source file. `agent/src/dist/buildBinary.ts` is the
packaging config, not build output, but the blanket `dist/` rule meant it was
never committed — a fresh clone could neither typecheck `agent/src/index.ts`
nor import it from the committed `agent/test/buildBinary.test.ts`. Re-included
the DIRECTORY first (git does not descend into an excluded one, so un-ignoring
just the file would not have worked) and committed the file. Verified build
output under `agent/dist/`, `dist/`, `public/build/` is still ignored.
2. Tunnel logs named no host. `pair` learned hostId/subdomain from the enroll
response and threw them away, so the long-running `run` process logged
`{"subdomain":null,"hostId":null}` — including all 6380 warnings during the
8-day outage, at exactly the moment you want to know which host. New
`config/hostRecord.ts` persists them at enrol; `resolveHostIdentity` resolves
config > record > the subdomain embedded in the leaf's SPIFFE SAN, so hosts
enrolled before the record existed get an identifier back without re-pairing.
3. The phone track had no recovery path. Added `POST /device/:id/recover`,
mirroring the host route: cert in the body, device-CA path validation, SPIFFE
parse, `notBefore` never graced, and the full registry check (active + same
account + `:id` matches the cert + same key) — only `notAfter` is relaxed.
Verified: agent 300/300, control-plane 296/296, tsc clean on both.
NOTE: the iOS/Android clients are not wired to call /device/:id/recover yet —
the server capability exists, the client-side trigger does not.
The tunnel had been down 8 days because /renew is mTLS-authenticated by the very
leaf it renews, so a lapsed leaf could never renew itself. Recovery now runs over
plain HTTPS with the expired cert in the body (CSR self-signature still proves key
possession), bounded by a 30-day grace; past that the agent stops retrying and
names the re-pair. Verified live end-to-end: an 8-day-expired leaf self-healed and
the tunnel returned HTTP 200 with no human action.
They were local worktree conveniences (symlinks into the main checkout so vitest
could resolve the file: workspace deps), swept in by a broad `git add agent`.
Correction to the previous commit. Its recovery vhost could not work: nginx
will not forward an expired client certificate in ANY `ssl_verify_client` mode.
`optional` answers a bare `400 The SSL certificate error` before any location
runs, and `optional_no_ca` only tolerates CHAIN failures — see nginx's
`ngx_ssl_verify_error_optional()`, which covers self-signed / unknown-issuer /
unverifiable-leaf and NOT `X509_V_ERR_CERT_HAS_EXPIRED`. Verified live against
the deployed :8472 server, which rejected the real expired leaf.
So recovery drops mTLS instead of trying to bend it:
- agent: `buildTlsOptions` and the mTLS renew transport go back to being
strictly fail-closed on expiry — the relaxation is gone from the TLS layer
entirely. The rotator now decides per attempt: valid → mTLS `/renew`,
expired-inside-grace → plain `/recover`, expired-beyond-grace → terminal
`onExhausted` with no request issued at all.
- new `recoverCert` POSTs `{cert, csr}` with no client certificate. Possession
of the private key is still proven: the CSR is self-signed by it and the host
signer already enforces CSR PoP plus `CSR key == registered key`, so a
replayed (public) cert without the key yields at most a certificate the
attacker cannot authenticate with.
- control-plane: `/renew` is strict again (grace 0, matching the terminator).
The grace lives on the new `POST /recover`, which reads the cert from the BODY
and ignores the `x-client-cert` header, then runs the identical trust
pipeline: X.509 path validation to the frp-client-CA anchors, SPIFFE parse,
`notBefore` (never graced), registry `active` + account match. Revocation
still bites.
- deploy: no new vhost, no new SNI, no DNS. One `location = /recover` merged
into the existing enroll vhost, documented in
`deploy/nginx/enroll-recover-location.md` with the nginx source citation.
The load-bearing new test is "a self-signed cert with a FORGED SPIFFE SAN is
refused → 401": nginx no longer validates the chain on this path, so that
assertion is what keeps `/recover` from being a cert vending machine.
Verified: agent 289/289, control-plane 290/290, tsc clean on both.
`POST /renew` is authenticated by mTLS with the very leaf it renews, so once
that leaf lapsed the host could never renew it and the tunnel stayed down until
an operator re-paired by hand. Production hit exactly this: the Mac slept
through its 8h renewal window, the 24h leaf expired, and the agent then logged
`client certificate has expired; renew before dialling` 6380 times over 8 days
without recovering.
Three layers independently refused an expired leaf, so all three had to move:
- agent `buildTlsOptions` gains an opt-in `expiredGraceMs`. Absent/0 keeps the
historical fail-closed behaviour, and the TUNNEL dial never passes it — only
the renew transport does. Past the window it throws the new
`CertExpiredBeyondGraceError`.
- agent rotator routes an already-expired leaf to a separate recovery endpoint
and treats "beyond grace" as TERMINAL: report once via `onExhausted`, stop
retrying, and name the fix (re-pair) instead of spamming warnings forever.
- control-plane `assertPresentedCertTrusted` grants a bounded grace on
`notAfter` only. Chain validation, SPIFFE identity, `notBefore`, and the
registry `active`/account checks are all unchanged, so revocation still bites.
- new `deploy/nginx/recover-mtls.conf` (:8472). The strict enroll vhost cannot
host this: under `ssl_verify_client optional` nginx answers a bare 400 as soon
as a presented cert fails verification, so an expired leaf never reaches the
location — and the directive is server-level, not per-location.
Grace defaults to 30 days on both ends and is configurable (`RECOVER_URL`,
`expiredRenewGraceMs`). The honest trade is recorded in the code: a stale stolen
leaf stays reusable for the window, which widens an existing exposure (an
unexpired stolen leaf already renews indefinitely) rather than opening a new one.
Verified: agent 296/296, control-plane 286/286, tsc clean on both.
A launchd/Finder-launched server has no LANG/LC_* at all, so tmux — which
decides per client whether the terminal is UTF-8 capable purely from those
variables — fell back to its non-UTF-8 mode and rewrote the byte stream
server-side, before it ever reached xterm:
- every wide character became `_` (中文测试 → ________)
- `✓` / `⏺` / emoji became `_`
- `═║╔╗` was downgraded to DEC Special Graphics (ESC ( 0), so rounded
corners vanished and boxes rendered as loose horizontal lines
The `-l` login shell cannot fix this: tmux is the PTY's root process and has
already made its decision by the time the shell sources ~/.zprofile.
withUtf8Locale() fills a locale in on the spawn env, and only when needed —
an existing UTF-8 LC_ALL / LC_CTYPE / LANG (zh_CN.UTF-8, ja_JP.UTF-8, …) is
left untouched; only a missing or non-UTF-8 charset is replaced.
Existing tmux sessions do not need recreating: re-attaching with a UTF-8
client restores correct output (already-scrolled `_` stays mangled).
- wire the enroll flow into the app UI (EnrollmentScreen + ViewModel + Hilt DI +
host-menu "自动获取证书"), mirroring iOS — Android previously only had manual
.p12 import; the enroll library was built but unreachable.
- renew is now mTLS-only ({csr}-only body, no Authorization header) matching the
/device/:id/renew contract (the enroll bearer is minutes-lived → silent
rotation would have thrown weeks later).
- enroll refreshes the identity-repository cache so a mid-session-enrolled cert
is presented on the next mTLS handshake without a process restart.
gradle :app:assembleDebug + api-client/client-tls-android unit tests + koverVerify
green. On-device QA (keygen/enroll/present) is the operator's step.
launchd has no default log destination (unlike systemd's journald), so a unit
that fails to start (bare node, missing env) was silent — which hid the EX_CONFIG
+ loadConfig failures during this deploy. Route both units' stdout+stderr to
<stateDir>/{base-app,agent}.log. 281 tests pass.
Two real-deploy renew bugs (found running the live /renew):
- the /renew mTLS request pinned the enroll caChain as the server CA → TLS
'unable to get local issuer certificate' (the LE-fronted CP is publicly
trusted). Verify the server against SYSTEM roots (drop ca), keep the client
cert + rejectUnauthorized:true. (TlsClientOptions.ca now optional.)
- renewCert parsed {cert, caChain:string}, but the CP returns cert=base64(DER)
+ caChain=base64(DER)[]; normalize to PEM (shared certs/pem.ts, reused by
native enroll). Verified live: cert rotated 13:41→next-day, frpc restarted,
tunnel stayed up. 281 tests pass.
nginx has no njs deployed, so the mTLS terminator forwards the verified client
cert via $ssl_client_escaped_cert (URL-encoded PEM). headerPresentedCert now
normalizes base64-DER, PEM, and escaped-PEM alike to raw DER. 281 tests pass.
Fixes found deploying `pair --install` against the live control-plane:
- native enroll tolerates hostContentSecret:null and converts the base64-DER
cert + caChain[] the CP returns into PEM for frpc/keystore
- launchd/systemd units use an absolute node (process.execPath) + a PATH with
/usr/local/bin (bare `node` died with EX_CONFIG 78)
- the agent unit now carries ENROLL_URL/RELAY_URL/STATE_DIR/LOCAL_TARGET_URL
(`run` loadConfig()-validates them; missing → supervisor exited 1)
281 tests pass. Follow-up: add unit tests for the native-enroll tolerance +
agent-unit env (only install node-path is covered so far).
Wire the prod boot to issue frp-client + device leaves from the existing on-disk
P-256 CAs: NATIVE_* env vars, a file-backed KmsResolver (loads the PEM key,
validates prime256v1, never logs key material), buildFileBackedNativeCas threaded
into buildControlPlane via a nativeCas override. Fail-closed on partial NATIVE_*.
281 tests pass.
When the always-on tunnel daemon already serves :3000, the app reuses it
(close() is a no-op) rather than spawning a second server on another port, so
the GUI and the tunnel share one base app.
Wire the SecureEnclave enroll library into a real flow (login->bearer->CSR->
/device/enroll->keychain identity), presented on the existing mTLS path; add a
rotation scheduler. Atomic keychain replace (add-before-delete); renew body is
{csr}-only; renewal-failing surfaced in the UI. ClientTLS 48 tests pass.
Wire createCertRotator/renewCert into superviseNative so the frp-client cert
renews silently at ~2/3 TTL over mTLS /renew and frpc restarts onto the fresh
leaf; failures retry with backoff, never crash the supervisor. mTLS transport
has a 15s timeout + 64KB response cap. 281 tests pass.
Unblocks the phone-enrollment track: an operator-password login mints a
short-lived device:enroll capability token that POST /device/enroll requires.
Constant-time (SHA-256 fixed-length) compare, per-client rate-limit, fail-closed
when unset. 260 tests pass.
Execution plan grounded in current VPS reality: pre-flight checks, Track A
(host auto-enroll via the agent) and Track B (phone auto-enroll), risks, and
the honest one-bootstrap-tap constraint. R1 (CA material) verified present on VPS.
An OPTIONAL shared token so the app can be used off-LAN (via the relay/tunnel) more
safely than "anyone who reaches the port gets a shell". Sits IN FRONT OF the existing
Origin/CSRF model — never replacing it — and is fully inert when unset.
- src/http/auth.ts (new, pure): constantTimeEqual hashes both inputs to sha256 (32
bytes) then crypto.timingSafeEqual — no `===`, no length side-channel, never throws.
parseCookieHeader / cookieIsAuthed / buildSetCookie / isAuthEnabled.
- WEBTERM_TOKEN in config: 16–512 URL/cookie-safe chars or the server refuses to start.
- GET /?token=<t> or POST /auth (rate-limited 10/min) validates → sets
HttpOnly; SameSite=Strict; Secure-when-https cookie; public/login.html (no <script>).
- When enabled: the WS handshake (AFTER the Origin check, before handleUpgrade) + a
central authGate over all remote HTTP require the cookie. Open: /login, /auth.
Loopback bypass scoped to /hook* ONLY (tighter than the plan — gates the local
browser too).
- Unset ⇒ isAuthEnabled false ⇒ pure passthrough (LAN zero-config unchanged).
Honest tradeoff (in code + CLAUDE.md + login page): a bar-raiser, NOT a TLS/Tailscale
substitute — on bare ws:// the token is cleartext + replayable; only hardens the
TLS-terminated relay path. Verified: typecheck + build:web clean, 2118 pass at
--test-timeout=30000 (disabled-mode regression proven; only the known tmux flake red);
auth.ts 100% line coverage.
Fan ONE prompt across N branch/agent lanes: N worktrees, N Claude sessions on the
same prompt, watched side-by-side in the split-grid board, approve/kill per lane,
🏆 keep the winner (losers' worktrees removed). ~90% composition of shipped parts.
- src/http/session-groups.ts (new, pure, no git exec): deriveRepoRoot /
groupSessionsByRepo (cluster sessions by their <repo>-worktrees parent).
- GET /live-sessions/grouped (read-only; registered BEFORE /live-sessions/:id so
"grouped" isn't captured as an id). MAX_FANOUT_LANES env (default 6, = grid-6 cap).
- public/fanout.ts (new, pure): buildFanoutCmd shell-quotes the prompt (single-quote
wrap with '\''-escaping so $(...)/backticks/;/&& can't execute) + collapses newlines
+ caps 4000 chars; laneBranch/slugify. Effective N = min(lanes, maxFanoutLanes, 6),
≥2; the maxSessions cap is enforced server-side ("Started K of N" banner).
- public/tabs.ts launchFanout (N× createWorktree → openProject w/ prompt pre-injected
via the existing initialInput) + keepFanoutWinner; public/projects.ts renderFanoutForm
+ extracted shared createWorktreeReq (DRY). Byte-shuttle preserved (lane = own PTY).
One-click merge deferred (winner session stays open for a manual merge).
Reused unchanged: createWorktree/removeWorktree, addEntry/initialInput, split-grid +
per-quadrant approve/maximize/monitor + gauges. Verified: typecheck + build:web clean,
2063 pass at --test-timeout=30000 (only the known tmux/PTY flake red). Fixed 3
/config/ui exact-shape tests to include the new maxFanoutLanes field.
