fix(session): give the PTY a UTF-8 locale so tmux stops mangling CJK

A launchd/Finder-launched server has no LANG/LC_* at all, so tmux — which
decides per client whether the terminal is UTF-8 capable purely from those
variables — fell back to its non-UTF-8 mode and rewrote the byte stream
server-side, before it ever reached xterm:

  - every wide character became `_`  (中文测试 → ________)
  - `✓` / `⏺` / emoji became `_`
  - `═║╔╗` was downgraded to DEC Special Graphics (ESC ( 0), so rounded
    corners vanished and boxes rendered as loose horizontal lines

The `-l` login shell cannot fix this: tmux is the PTY's root process and has
already made its decision by the time the shell sources ~/.zprofile.

withUtf8Locale() fills a locale in on the spawn env, and only when needed —
an existing UTF-8 LC_ALL / LC_CTYPE / LANG (zh_CN.UTF-8, ja_JP.UTF-8, …) is
left untouched; only a missing or non-UTF-8 charset is replaced.

Existing tmux sessions do not need recreating: re-attaching with a UTF-8
client restores correct output (already-scrolled `_` stays mangled).
This commit is contained in:
Yaojia Wang
2026-07-29 08:09:48 +02:00
parent 1dbed54581
commit b1bc50ccd1
4 changed files with 155 additions and 1 deletions

View File

@@ -24,6 +24,22 @@
> 新会话读到的第一块。保持准确,只描述"此刻"。 > 新会话读到的第一块。保持准确,只描述"此刻"。
### 🐛 [x] Mac 版中文全变 `_` + 框线渲染错乱 —— PTY 缺 UTF-8 locale(2026-07-28)
- **现象**(用户截图,Mac 版):Claude Code TUI 里所有中文变成 `_`;banner/输入框只剩零散横线,圆角边框消失。
- **根因**(实测确认,不是字体也不是前端):`src/session/session.ts` spawn PTY 时用 `env: {...process.env}`,而**服务端由 launchd/Finder 启动,环境里根本没有 `LANG`/`LC_*`**(`~/Library/LaunchAgents/com.web-terminal.base-app.plist``EnvironmentVariables` 只有 PATH/PORT/BIND_HOST)。tmux 逐 client 从 `LC_ALL`/`LC_CTYPE`/`LANG` 判断 UTF-8 能力,全空 → 进入**非 UTF-8 模式**,在**服务端就把字节改写掉**:宽字符→`_``✓ ⏺` 等→`_``═║╔╗` 降级成 DEC Special Graphics(`ESC ( 0` + `qxlk`)。所以前端 xterm 收到的已经是残骸,改字体/renderer 救不回来。
- 用户 `~/.zshrc` 里其实**有** `LANG=sv_SE.UTF-8`,但没用:tmux 是 PTY 的根进程,它做完 UTF-8 判定之后 shell 才 source profile。登录 shell(`-l`)救不了 tmux。
- node-pty 探针实证:无 LANG → `MARK ________ ==BOX== \e(0qxlk`;有 LANG → `MARK 中文测试 ==BOX== ═║╔╗ ✓`
- **修复**: 新增 `src/session/locale.ts``withUtf8Locale(env)`,在 `createSession()` 的 spawn env 上先过一道。POSIX 优先级 `LC_ALL > LC_CTYPE > LANG`;**已是 UTF-8 就原样不动**(用户的 `zh_CN.UTF-8`/`ja_JP.UTF-8` 保留),只在缺失或非 UTF-8(`C`/`POSIX`/`ISO8859-1`)时补 `en_US.UTF-8`(并清掉会盖过它的非 UTF-8 `LC_ALL`/`LC_CTYPE`)。
- **验证**:
- `test/locale.test.ts` 9 例(TDD:先 RED 再 GREEN);`tsc --noEmit` 干净。
- 端到端:用 `env -u LANG -u LC_ALL -u LC_CTYPE` 复现 launchd 环境跑 server → 新建 session → `中文测试:你好,世界!`/`🎉 ✅`/`╭─╮ ═║╔╗ █▓▒░ ⏺ ⎿` **DOM 里逐字符核对全部正确**(修复前同一条命令得到 `________`)。
- 之后 `npm run build` + `launchctl kickstart` 让真正的 launchd 单元(仍无 LANG)接管,再复验一次通过。
- 回归:`test/integration/server.test.ts`(含 ⑥ CJK ring-buffer replay、H1 tmux 重启存活)+ git-ops + worktree + projects 共 **107 例全绿**。(全量 79 文件并跑时有 4 例超时 flake,单跑全过,与本改动无关。)
- 存量 tmux session **无需重建**:探针证明用带 LANG 的新 client 重新 attach 一个"非 UTF-8 模式下创建"的旧 session,新输出即恢复正常(只有已滚进 scrollback 的旧 `_` 救不回)。
- **遗留**: `/Applications/Web Terminal.app/Contents/Resources/dist/` 里打包的是 7/18 的旧代码,**仍未修**——桌面 Electron 版要 `cd desktop && npm run dist:mac` 重打包(或就地补 `dist/session/*.js`,注意 adhoc 签名会失效)。仓库内 `dist/` 与 launchd base-app 已修好。
- **文件**: `src/session/locale.ts`(新)· `src/session/session.ts`(spawn env)· `test/locale.test.ts`(新)。
### 🔐 零接触隧道 Enrollment 产品化 + Control Panel(2026-07-18~23,当前活跃) ### 🔐 零接触隧道 Enrollment 产品化 + Control Panel(2026-07-18~23,当前活跃)
- **需求**: 用户要"本机 app 一跑就自动连服务器、自动管证书;手机连接时自动搞定证书"——即**产品级零接触**(引导时一次动作,之后自动签发+续期,永不再管)。诚实边界:第一次签发无法零人类动作(否则谁碰端点谁拿证书),所以"引导一次,之后免管"。 - **需求**: 用户要"本机 app 一跑就自动连服务器、自动管证书;手机连接时自动搞定证书"——即**产品级零接触**(引导时一次动作,之后自动签发+续期,永不再管)。诚实边界:第一次签发无法零人类动作(否则谁碰端点谁拿证书),所以"引导一次,之后免管"。

64
src/session/locale.ts Normal file
View File

@@ -0,0 +1,64 @@
/**
* src/session/locale.ts — guarantee the spawned PTY has a UTF-8 locale.
*
* The bug this fixes: when the server is launched from Finder / launchd (the Mac
* desktop app, or the launchd-managed tunnel agent) the process environment has
* NO `LANG` / `LC_*` at all — unlike a shell started from Terminal.app. tmux
* decides per client whether the terminal is UTF-8 capable purely from those
* variables, so with none set it falls back to its non-UTF-8 mode and:
*
* - rewrites every wide character to `_` → `中文测试` renders as `________`
* - rewrites other non-Latin-1 glyphs to `_` (`✓`, `⏺`, emoji …)
* - downgrades double-line box drawing (`═║╔╗`) to DEC Special Graphics
* (`ESC ( 0` + `qxlk`), i.e. single lines
*
* That mangling happens on the server side, *before* the bytes reach xterm.js,
* so no amount of frontend font work can recover it. The `-l` login shell can't
* fix it either: tmux is the PTY's root process and has already made its
* decision by the time the shell sources `~/.zprofile`.
*
* Policy: leave the environment alone whenever the effective charset is already
* UTF-8 — a user who picked `zh_CN.UTF-8` or `ja_JP.UTF-8` keeps it. Only a
* missing or non-UTF-8 charset is replaced. Precedence follows POSIX:
* `LC_ALL` > `LC_CTYPE` > `LANG`.
*/
/** Fallback locale when the environment specifies no UTF-8 charset. */
export const DEFAULT_UTF8_LOCALE = 'en_US.UTF-8';
/** Matches the charset suffix of a locale: `.UTF-8`, `.utf8`, or a bare `UTF-8`. */
const UTF8_PATTERN = /utf-?8$/i;
/** Environment mapping as node-pty / process.env exposes it. */
type Env = Readonly<Record<string, string | undefined>>;
function isUtf8(value: string | undefined): boolean {
return value !== undefined && value !== '' && UTF8_PATTERN.test(value);
}
/**
* Return a copy of `env` that is guaranteed to select a UTF-8 charset.
*
* Existing values are never rewritten — if the effective locale is already
* UTF-8 the env comes back unchanged (aside from being a fresh object).
*/
export function withUtf8Locale(env: Env): Record<string, string | undefined> {
// LC_ALL outranks everything; LC_CTYPE outranks LANG for character handling.
// If whichever one is in effect is already UTF-8, there is nothing to do.
const effective =
env.LC_ALL !== undefined && env.LC_ALL !== ''
? env.LC_ALL
: env.LC_CTYPE !== undefined && env.LC_CTYPE !== ''
? env.LC_CTYPE
: env.LANG;
if (isUtf8(effective)) return { ...env };
// Setting LANG (the lowest-precedence variable) is the least invasive fix, but
// it only takes effect if nothing above it is overriding — so clear a
// non-UTF-8 LC_ALL / LC_CTYPE that would otherwise win.
const next: Record<string, string | undefined> = { ...env, LANG: DEFAULT_UTF8_LOCALE };
if (env.LC_ALL !== undefined && env.LC_ALL !== '') delete next.LC_ALL;
if (env.LC_CTYPE !== undefined && env.LC_CTYPE !== '') delete next.LC_CTYPE;
return next;
}

View File

@@ -39,6 +39,7 @@ import { WS_OPEN } from '../types.js';
import { serialize } from '../protocol.js'; import { serialize } from '../protocol.js';
import { createRingBuffer } from './ring-buffer.js'; import { createRingBuffer } from './ring-buffer.js';
import { tmuxName, killSession } from './tmux.js'; import { tmuxName, killSession } from './tmux.js';
import { withUtf8Locale } from './locale.js';
/** Send a server message to `ws` only if it is OPEN (M5). Never throws on a /** Send a server message to `ws` only if it is OPEN (M5). Never throws on a
* closed socket; forwarding to a dead ws is simply a no-op. */ * closed socket; forwarding to a dead ws is simply a no-op. */
@@ -105,8 +106,11 @@ export function createSession(
cols: dims.cols, cols: dims.cols,
rows: dims.rows, rows: dims.rows,
cwd: cwd ?? cfg.homeDir, cwd: cwd ?? cfg.homeDir,
// A Finder/launchd-launched server has NO LANG/LC_* — which puts tmux in its
// non-UTF-8 mode and turns every CJK/wide character into `_` before the bytes
// ever reach xterm. withUtf8Locale() fills one in (and only then).
env: { env: {
...process.env, ...withUtf8Locale(process.env),
WEBTERM_SESSION: id, WEBTERM_SESSION: id,
WEBTERM_HOOK_URL: `http://127.0.0.1:${cfg.port}/hook`, WEBTERM_HOOK_URL: `http://127.0.0.1:${cfg.port}/hook`,
// B2: statusLine scripts POST telemetry here; WEBTERM_NTFY_* vars forwarded // B2: statusLine scripts POST telemetry here; WEBTERM_NTFY_* vars forwarded

70
test/locale.test.ts Normal file
View File

@@ -0,0 +1,70 @@
import { describe, it, expect } from 'vitest';
import { withUtf8Locale, DEFAULT_UTF8_LOCALE } from '../src/session/locale.js';
/**
* locale.ts — UTF-8 locale guarantee for the spawned PTY.
*
* Why this exists: when the server is launched from Finder / launchd (the Mac
* desktop app) the process env has NO `LANG`/`LC_*`. tmux then runs in its
* non-UTF-8 mode and rewrites every wide character to `_` and downgrades
* double-line box drawing (`═║╔╗`) to DEC Special Graphics single lines — so all
* CJK output in the Claude Code TUI is destroyed before it ever reaches xterm.
*/
describe('withUtf8Locale', () => {
it('adds LANG when no locale variable is set at all', () => {
const out = withUtf8Locale({ PATH: '/usr/bin' });
expect(out.LANG).toBe(DEFAULT_UTF8_LOCALE);
expect(out.PATH).toBe('/usr/bin');
});
it('keeps a UTF-8 LANG the user already has', () => {
const out = withUtf8Locale({ LANG: 'zh_CN.UTF-8' });
expect(out.LANG).toBe('zh_CN.UTF-8');
});
it('accepts lowercase / hyphen-less spellings of utf8', () => {
expect(withUtf8Locale({ LANG: 'en_US.utf8' }).LANG).toBe('en_US.utf8');
expect(withUtf8Locale({ LANG: 'C.UTF8' }).LANG).toBe('C.UTF8');
});
it('replaces a non-UTF-8 LANG (C / POSIX / latin1) with the default', () => {
expect(withUtf8Locale({ LANG: 'C' }).LANG).toBe(DEFAULT_UTF8_LOCALE);
expect(withUtf8Locale({ LANG: 'POSIX' }).LANG).toBe(DEFAULT_UTF8_LOCALE);
expect(withUtf8Locale({ LANG: 'en_US.ISO8859-1' }).LANG).toBe(DEFAULT_UTF8_LOCALE);
});
it('respects LC_ALL — it outranks LANG, so a UTF-8 LC_ALL is enough', () => {
const out = withUtf8Locale({ LC_ALL: 'ja_JP.UTF-8', LANG: 'C' });
expect(out.LC_ALL).toBe('ja_JP.UTF-8');
expect(out.LANG).toBe('C'); // untouched — LC_ALL already decides the charset
});
it('respects a UTF-8 LC_CTYPE (macOS Terminal.app sets only this)', () => {
const out = withUtf8Locale({ LC_CTYPE: 'UTF-8' });
expect(out.LC_CTYPE).toBe('UTF-8');
expect(out.LANG).toBeUndefined();
});
it('overrides LANG when LC_CTYPE is present but not UTF-8', () => {
const out = withUtf8Locale({ LC_CTYPE: 'C' });
expect(out.LANG).toBe(DEFAULT_UTF8_LOCALE);
});
it('does not mutate the input env (immutability)', () => {
const input = { PATH: '/usr/bin' };
const out = withUtf8Locale(input);
expect(input).toEqual({ PATH: '/usr/bin' });
expect(out).not.toBe(input);
});
it('ignores an empty-string locale (unset in practice)', () => {
expect(withUtf8Locale({ LANG: '' }).LANG).toBe(DEFAULT_UTF8_LOCALE);
expect(withUtf8Locale({ LC_ALL: '' }).LANG).toBe(DEFAULT_UTF8_LOCALE);
});
});