feat: voice command mapping — context-gated approve/reject over voice
While a tool-permission gate is held on the active tab, spoken confirm phrases resolve it via the existing approve/reject WS channel — no server change (byte-shuttle intact). Everything else stays ordinary dictation. Safety: whole-utterance-exact matching + leading-negation guard + reject precedence + confidence gate (approve only) + a cancellable 1.5s confirm window whose commit re-validates gate identity/epoch/connectivity (TOCTOU guard) + stale-gate epoch bound at PTT-start. New pure modules public/voice-commands.ts and public/voice-confirm.ts at 100% coverage; wiring in voice/terminal-session/tabs/main. 1307 tests pass, typecheck clean, build:web OK. Independent code + security reviews flagged a confirm-window TOCTOU and an unwired cancel path — both fixed and covered. Plan: docs/PLAN_VOICE_COMMANDS.md.
This commit is contained in:
Yaojia Wang
222
docs/PLAN_VOICE_COMMANDS.md
Normal file
222
docs/PLAN_VOICE_COMMANDS.md
Normal file
@@ -0,0 +1,222 @@
|
||||
# Voice Command Mapping — Implementation Plan (v1, LOCKED)
|
||||
|
||||
Context-gated **confirm-class** voice commands: while a permission is HELD on the
|
||||
active tab, spoken confirm/deny phrases resolve the gate via the existing
|
||||
`approve`/`reject` WS channel. Produced by a multi-agent plan+review pass
|
||||
(understand → FE/BE plan → 4 adversarial review lenses → synthesis) and hardened
|
||||
against 2 CRITICAL + 1 HIGH findings. This doc is the source of intent for build.
|
||||
|
||||
## 0. Locked decisions
|
||||
|
||||
| # | Decision | Choice |
|
||||
|---|----------|--------|
|
||||
| 1 | Disambiguation | **Context-aware** — command only when active tab `pendingApproval===true`; else ordinary dictation (byte-for-byte unchanged) |
|
||||
| 2 | Scope v1 | **Confirm-class only** — affirmative→`{type:'approve'}`, negative→`{type:'reject'}`, reusing the existing H3/A1 channel (never raw bytes) |
|
||||
| 3 | Danger keys | **Not wired** — Ctrl+C/D/kill out of scope; two-step affordance built but only used for the approve confirm window (B) |
|
||||
| A | Plan-gate | **tool-gate only** — `gate!=='tool'` falls through to dictation |
|
||||
| B | Approve confirm window | **Adopted (B2-lite)** — approve arms a ~1.5s cancellable window, then commits; reject fires immediately (fail-safe) |
|
||||
| C | WS-not-open feedback | Flash "未连接 / not connected" (the confirm overlay already exists) |
|
||||
| D | Confidence gate | **Adopted** — `voice.ts` forwards ASR confidence; approve requires `≥ MIN_APPROVE_CONFIDENCE` (0.6); reject unaffected |
|
||||
| E | Recognizer language | `navigator.language` (unchanged); document that zh/en sets are each only reachable under the matching engine language |
|
||||
| F | tabs.ts overage | tabs.ts is already 869 lines (>800 max) — keep the delta minimal, **log a follow-up extraction ticket**, do not silently normalize |
|
||||
|
||||
## 1. Backend verdict — NONE
|
||||
|
||||
Zero changes to `src/server.ts`, `src/protocol.ts`, `src/types.ts`. The byte-shuttle stays intact.
|
||||
|
||||
- **Context signal already delivered:** a held `POST /hook/permission` broadcasts `{type:'status',status:'waiting',pending:true,gate:'tool'|'plan'}` (server.ts:446) and replays it on attach (server.ts:739-744); the client tracks it as `TerminalSession.pendingApproval` / `pendingGate` (terminal-session.ts:297-299).
|
||||
- **Resolve channel already exists:** `{type:'approve'}` / `{type:'reject'}` are first-class `ClientMessage`s (types.ts:91-92), whitelisted (protocol.ts:27), parsed non-throwing (protocol.ts:77-82), resolved server-side keyed by `boundSessionId` (server.ts:761-771). No per-request token on the WS path.
|
||||
- **Trust identical to the on-screen button:** a voice approve is one user action on the already-Origin-validated, attach-bound socket, same authority as the existing Approve button. No new WS rate-limit/capability.
|
||||
- **Regression obligation (no new BE code):** `test/protocol.test.ts:111-117` and `test/integration/server.test.ts:608-646` must stay green.
|
||||
|
||||
## 2. Files & ownership
|
||||
|
||||
| File | Action | Lane |
|
||||
|------|--------|------|
|
||||
| `public/voice-commands.ts` | **create** — pure matcher (types, phrase sets, `normalizeTranscript`, `matchCommand`) | V1 |
|
||||
| `test/voice-commands.test.ts` | **create** — pure unit tests (node env) | V1 |
|
||||
| `public/voice-confirm.ts` | **create** — small cancellable-window controller for the approve confirm (B); timer injectable for tests | V1 |
|
||||
| `test/voice-confirm.test.ts` | **create** — arm/commit/cancel/re-arm with fake timers | V1 |
|
||||
| `public/voice.ts` | **modify** — forward `confidence` as 2nd `onTranscript` arg (still transcription-only) | V2 |
|
||||
| `public/terminal-session.ts` | **modify** — add `pendingEpoch` nonce (stale-gate guard) | V2 |
|
||||
| `public/tabs.ts` | **modify** — wire matcher into `onTranscript`; PTT-start epoch/session capture; dispatch; arm confirm window; WS-not-open flash | V2 |
|
||||
| `public/main.ts` | **modify** — pass `onVoiceTrigger` to un-dormant the 🎤 | V2 |
|
||||
| `test/tabs.test.ts` | **modify** — jsdom integration via `vi.mock('../public/voice.js')` | V2 |
|
||||
| `vitest.config.ts` | **modify (orchestrator only)** — add `public/voice-commands.ts` + `public/voice-confirm.ts` to `coverage.include` | orch |
|
||||
| `public/keybar.ts`, `src/**` | **untouched** | — |
|
||||
|
||||
V1 exports must be frozen before V2 consumes them. `PROGRESS_LOG.md` is orchestrator-only (subagents return a ready-to-paste entry, G1).
|
||||
|
||||
## 3. `public/voice-commands.ts` — pure matcher (DOM-free, WS-free)
|
||||
|
||||
```ts
|
||||
export type VoiceAction = 'approve' | 'reject' | 'text'
|
||||
|
||||
export interface VoiceMatchContext {
|
||||
readonly pendingApproval: boolean // active session held-permission flag
|
||||
readonly gate: 'tool' | 'plan' | null // v1 acts ONLY on 'tool'
|
||||
readonly confidence?: number // optional ASR confidence (D)
|
||||
}
|
||||
|
||||
export const MIN_APPROVE_CONFIDENCE = 0.6
|
||||
|
||||
export const APPROVE_PHRASES: readonly string[] // normalized whole utterances
|
||||
export const REJECT_PHRASES: readonly string[]
|
||||
export function normalizeTranscript(raw: string): string
|
||||
export function matchCommand(transcript: string, ctx: VoiceMatchContext): VoiceAction
|
||||
```
|
||||
|
||||
**`normalizeTranscript`** (pure): `trim → toLowerCase → strip PUNCTUATION_RE (ASCII + CJK punctuation + apostrophes so don't→dont) → collapse ASCII \s+ → trim`. Keeps CJK + ASCII letters/digits.
|
||||
|
||||
**`matchCommand`** — whole-utterance-exact, reject-first, negation-guarded:
|
||||
|
||||
```
|
||||
norm = normalizeTranscript(transcript)
|
||||
if (!ctx.pendingApproval || norm === '') return 'text' // decision 1 + empty guard
|
||||
if (ctx.gate !== 'tool') return 'text' // decision A: plan → dictation
|
||||
if (startsWithNegation(norm)) // leading 不/别/没/未/勿, no/not/dont/cannot/wont/never
|
||||
return REJECT_PHRASES.includes(norm) ? 'reject' : 'text' // negation can NEVER approve
|
||||
if (REJECT_PHRASES.includes(norm)) return 'reject' // reject precedence (fail-safe)
|
||||
if (APPROVE_PHRASES.includes(norm)) {
|
||||
if (ctx.confidence !== undefined && ctx.confidence < MIN_APPROVE_CONFIDENCE) return 'text' // D
|
||||
return 'approve'
|
||||
}
|
||||
return 'text' // dictation fallthrough
|
||||
```
|
||||
|
||||
Whole-utterance-exact is the load-bearing fix: it kills both CRITICALs (`不可以`≠`可以`; `ok let's not do that`≠`ok`) at the root while keeping the module trivially testable. Cost: `yes please`→`text` (must say the command word alone or an enumerated multi-word phrase). Accepted precision-first trade-off for a shell-granting gate.
|
||||
|
||||
- `APPROVE_PHRASES`: `确认 批准 同意 通过 允许 可以 好 好的 好吧 好啊 是 是的 对 行 继续 回车 yes yeah yep ok okay confirm approve accept proceed "go ahead"`
|
||||
- `REJECT_PHRASES`: `拒绝 取消 不行 不要 不用 不同意 不批准 不可以 不允许 不通过 中断 停止 算了 别 不 no nope reject deny cancel abort decline stop`
|
||||
|
||||
Invariants (tested): every phrase satisfies `normalizeTranscript(p)===p`; the two sets are disjoint; no `APPROVE_PHRASES` entry starts with a negation prefix.
|
||||
|
||||
## 4. `public/voice-confirm.ts` — approve confirm window (B)
|
||||
|
||||
Small controller with an injectable timer (so tests use fake timers, no DOM coupling in the logic):
|
||||
|
||||
```ts
|
||||
export interface ApproveConfirm {
|
||||
arm(onHeard: () => void, onCommit: () => void): void // show "听到:批准", start window
|
||||
cancel(): void // user aborted before commit
|
||||
isArmed(): boolean
|
||||
}
|
||||
export function createApproveConfirm(windowMs?: number): ApproveConfirm // default 1500
|
||||
```
|
||||
|
||||
`arm` starts a `windowMs` timer that calls `onCommit` unless `cancel()` runs first; re-arming clears any prior timer. tabs.ts owns the tiny DOM overlay (a `#voice-confirm` sibling that `hideInterim` does not clobber) and calls `session.approve()` in `onCommit`.
|
||||
|
||||
**Cancel triggers (corrected during impl):** (a) **tapping the overlay**, (b) **a new utterance superseding** the armed window (the top of `handleVoiceTranscript` cancels first), and (c) **a reject utterance** (cancels the armed approve, then rejects). Note: "releasing the mic" is NOT a cancel trigger — under push-to-talk the mic release is exactly what delivers the final transcript that *arms* the window, so cancelling on release would make approve impossible.
|
||||
|
||||
**Commit-time re-validation (TOCTOU fix):** `onCommit` re-checks `armedSession.pendingApproval && pendingGate==='tool' && pendingEpoch===armedEpoch && connected` immediately before `approve()`; if the held gate resolved and a new one was raised (epoch bump), or the WS dropped, during the ~1.5s window, the commit is dropped rather than approving a different/unseen gate.
|
||||
|
||||
## 5. `public/terminal-session.ts` — stale-gate epoch
|
||||
|
||||
```ts
|
||||
private pendingEpochValue = 0
|
||||
get pendingEpoch(): number { return this.pendingEpochValue }
|
||||
// inside status handling: count false→true flips
|
||||
if (nextPending && !this.pendingApprovalValue) this.pendingEpochValue++
|
||||
```
|
||||
|
||||
## 6. `public/tabs.ts` / `public/voice.ts` / `public/main.ts` — wiring (minimal delta)
|
||||
|
||||
- `voice.ts`: `onresult` forwards `result[0][0].confidence` as a 2nd arg to `onTranscript`. Still transcription-only.
|
||||
- `tabs.ts`:
|
||||
- On PTT start, capture `voicePttSession = active.session` and `voicePttEpoch = session.pendingEpoch`.
|
||||
- `createVoiceInput((text, confidence) => this.handleVoiceTranscript(text, confidence), { onInterim })`.
|
||||
- `handleVoiceTranscript`:
|
||||
```
|
||||
session = active.session; if (!session) return sendToActive(text)
|
||||
action = matchCommand(text, { pendingApproval: session.pendingApproval===true, gate: session.pendingGate, confidence })
|
||||
if (action!=='text' && (session!==voicePttSession || session.pendingEpoch!==voicePttEpoch)) return sendToActive(text) // stale → dictation
|
||||
if (action==='reject') { session.reject(); updateApprovalBar(); return }
|
||||
if (action==='approve') { approveConfirm.arm(showHeard, () => { session.approve(); updateApprovalBar() }); return } // B
|
||||
sendToActive(text) // UNCHANGED dictation path
|
||||
```
|
||||
- WS-not-open (`session.approve/reject` dropped by `sendMsg`): flash "未连接" (C).
|
||||
- Drop the earlier `flashVoiceCommand`/`VOICE_HINT_MS` idea — it duplicated the approval-bar vanish and raced with `hideInterim`.
|
||||
- `main.ts:46`: `mountKeybar((d)=>app.sendToActive(d), { onVoiceTrigger:(a)=>app.handleVoiceTrigger(a) })` — activates the whole voice path (dictation + commands); note in log.
|
||||
|
||||
## 7. Security safeguards adopted
|
||||
|
||||
1. Whole-utterance-exact matching (no substring/token) — no auto-approve from incidental dictation.
|
||||
2. Leading-negation override — negated phrases can never approve.
|
||||
3. Reject precedence — fail-safe direction wins ties.
|
||||
4. tool-gate-only (A) — avoids plan-gate semantic mismatch.
|
||||
5. Stale-gate epoch (session+epoch bound at PTT-start) — a slow transcript can't approve a newer gate / another tab.
|
||||
6. Approve confirm window (B) — cancellable ~1.5s before the irreversible approve commits.
|
||||
7. Confidence gate on approve only (D) — asymmetric-risk aware.
|
||||
8. Strict `pendingApproval===true` gate — never triggers the server's spurious `status:'working'` broadcast.
|
||||
9. XSS-safe UI — `textContent` only.
|
||||
10. SEC-L2 disclosure extended: voice *control* of permission gates (not just dictation) streams audio to the browser's cloud speech provider.
|
||||
|
||||
## 8. TDD steps (RED-first)
|
||||
|
||||
1. **RED** `test/voice-commands.test.ts` — normalize + matchCommand incl. the negation/filler regression set → fails (module absent).
|
||||
2. **GREEN** implement `public/voice-commands.ts` → `npx vitest run voice-commands` green; `npm run typecheck` clean.
|
||||
3. **RED** `test/voice-confirm.test.ts` (fake timers: arm→commit after window, cancel before window, re-arm clears prior) → fails.
|
||||
4. **GREEN** implement `public/voice-confirm.ts` → green.
|
||||
5. **RED** `test/tabs.test.ts` — `vi.mock('../public/voice.js')` capturing `onTranscript`; integration cases → fails.
|
||||
6. **GREEN** edit `voice.ts`, `terminal-session.ts`, `tabs.ts`, `main.ts` → `npx vitest run tabs` green.
|
||||
7. **REFACTOR** — `matchCommand`<50 lines, no console.log, no mutation, `readonly` types; log the pre-existing tabs.ts overage + extraction follow-up.
|
||||
8. **COVERAGE** (orchestrator) — add modules to `vitest.config.ts` `coverage.include`; `npx vitest run --coverage` meets 80%×4.
|
||||
9. **SHIP-CHECK** — `npm run build:web` bundles the new modules via the tabs.ts import.
|
||||
|
||||
### Key test cases
|
||||
|
||||
- **Context gating:** `matchCommand('yes',{pendingApproval:false,...})`==='text'; same for '确认'.
|
||||
- **Held tool approve:** 确认/批准/同意/好/好的/好吧/行/可以/yes/ok/confirm/approve/"go ahead" → 'approve'.
|
||||
- **Held tool reject:** 拒绝/取消/不行/no/nope/cancel/deny/abort/stop → 'reject'.
|
||||
- **Negation/filler regression (MUST NOT approve):** 不可以→'reject'; 不允许/不通过/不同意/不批准→'reject'; 不继续→'text'; "don't approve"/"do not confirm"/"cannot proceed"/"won't continue"→'text'; "ok let's not do that"→'text'; "ok can you also check the logs"→'text'; 我觉得可以先看看→'text'; 请继续→'text' (≠继续); now→'text' (≠no); 我不知道→'text'.
|
||||
- **Plan gate (A):** `{pendingApproval:true,gate:'plan'}` → 'text' for both 'confirm' and 'stop'.
|
||||
- **Confidence (D):** confirm@0.4→'text'; confirm@0.9→'approve'; no@0.2→'reject' (reject unaffected).
|
||||
- **Normalize:** 'YES'→'yes'; 'confirm.'→'confirm'; '「确认」'→'确认'; "don't"→'dont'; ' '→''.
|
||||
- **tabs integration:** pending+tool+'confirm'→arms confirm window, commits `session.approve()` once after window, `send` NOT called; +'拒绝'→`session.reject()` immediately; +'deploy to prod'→`send('deploy to prod')`; pending=false+'yes'→`send('yes')`; plan gate+'confirm'→`send('confirm')`; stale-gate (epoch flip mid-utterance)→dictation; tab-switch mid-utterance→dictation; WS-not-open→no throw + "未连接" flash.
|
||||
|
||||
## 9. Verification
|
||||
|
||||
```bash
|
||||
npx vitest run voice-commands # pure matcher
|
||||
npx vitest run voice-confirm # confirm-window controller
|
||||
npx vitest run tabs # wiring/integration (jsdom + voice.js mock)
|
||||
npx vitest run protocol # BE regression: approve/reject parse (stay green)
|
||||
npx vitest run integration/server # BE regression: held-permission → pending → approve resolves
|
||||
npm test # full suite
|
||||
npm run typecheck # frozen src/types.ts unchanged
|
||||
npx vitest run --coverage # 80%×4 after coverage.include update
|
||||
npm run build:web # esbuild bundles new modules via tabs.ts import
|
||||
```
|
||||
|
||||
## 10. Manual verification (real device)
|
||||
|
||||
The command-dispatch logic (matchCommand → approve/reject/dictation, confirm window,
|
||||
stale-gate, WS-drop) is fully covered by jsdom integration tests via the `voice.js`
|
||||
mock. What CANNOT be automated is the browser's Web Speech engine itself (it needs
|
||||
real microphone audio; there is no reliable headless E2E for it) — so the end-to-end
|
||||
path is verified by hand on a real device.
|
||||
|
||||
**Setup:** Chrome on a phone (Web Speech / `webkitSpeechRecognition` supported), on the
|
||||
same LAN/Tailscale as the host. Open a session and give Claude Code a task that raises a
|
||||
**tool** permission gate (e.g. a shell command it must ask to run).
|
||||
|
||||
**Checklist:**
|
||||
1. **Approve (happy path):** when the approve/reject bar appears, hold 🎤, say "确认"
|
||||
(or "confirm"), release → overlay shows **"听到：批准（点此取消）"** → after ~1.5s the
|
||||
gate is approved and the bar disappears.
|
||||
2. **Tap-to-cancel:** say "确认", release, then **tap the overlay** within 1.5s → overlay
|
||||
clears, gate is **not** approved.
|
||||
3. **Reject:** say "拒绝" (or "no") → rejected immediately (no window).
|
||||
4. **Reject cancels a pending approve:** say "确认", then within 1.5s say "拒绝" → rejected,
|
||||
and the pending approve does **not** fire afterwards.
|
||||
5. **Context gate (dictation preserved):** with **no** gate pending, say "确认" → the literal
|
||||
text "确认" is dictated into the terminal, not treated as a command.
|
||||
6. **Plan gate falls through:** on a *plan* gate, say "确认" → dictated as text, not
|
||||
auto-approved (use the on-screen buttons for plan gates).
|
||||
7. **Negation safety:** with a tool gate pending, say "不可以" / "不用" → rejected or
|
||||
dictated, **never** approved.
|
||||
8. **Disconnect feedback:** kill Wi-Fi briefly (WS reconnecting), say "确认" → overlay
|
||||
flashes **"未连接"**, nothing is approved.
|
||||
|
||||
Report the device + Chrome version with results, since real recognition of the zh vs en
|
||||
phrase sets depends on the engine's active language.
|
||||
@@ -24,6 +24,9 @@
|
||||
|
||||
> 新会话读到的第一块。保持准确,只描述"此刻"。
|
||||
|
||||
### ✅ VC 语音命令映射 — 语音直接 approve/reject 权限门(完成 — 2026-07-01,分支 `v0.6-projects`,未提交)
|
||||
上下文门控:仅当活动 tab 有 held 权限门(`pendingApproval===true` 且 `gate==='tool'`)时,语音「确认/批准/yes…」→ 复用现有 `{type:'approve'}` WS 通道、「拒绝/取消/no…」→ `{type:'reject'}`;其余一律口述不变。**后端零改动**。安全设计:整句精确匹配(否定/口水词攻不破)+ 前导否定守卫 + reject 优先 + 置信度门(仅 approve)+ **可撤销确认窗口(1.5s,提交时复检门身份/连接,防 TOCTOU)** + stale-gate epoch。新增 `public/voice-commands.ts`(纯匹配器 100% 覆盖)+ `public/voice-confirm.ts`(确认窗口 100%),接线 `tabs/voice/terminal-session/main.ts`。**1307 tests 绿**,双 review(code+security)判 CRITICAL 已全修复并补测。规划 `docs/PLAN_VOICE_COMMANDS.md`,详见 Detailed Log 首条。**待办**:confidence fail-open 取舍留给用户;tabs.ts 968 行待拆分。
|
||||
|
||||
### ✅ v0.6 Projects 命名空间自动分组 + 跨设备 prefs(完成 — 2026-07-01,分支 `v0.6-projects`,未提交)
|
||||
Projects 页去杂乱:按点分隔命名空间自动折叠成 section(`Billo.Platform ·8` / `Billo.Infrastructure ·2` / `Other`)+ 置顶 "Active now" + 无前缀时退化为扁平网格。收藏/折叠态改**服务端持久化**(`GET/PUT /prefs` → `~/.web-terminal-prefs.json`,离线回退 localStorage 镜像 + 迁移旧 `proj-favs`)跨设备同步。1227 tests 绿,真浏览器验证折叠跨 reload 从服务端载回。详见 Detailed Log 首条。
|
||||
|
||||
@@ -149,7 +152,25 @@ Projects 页去杂乱:按点分隔命名空间自动折叠成 section(`Billo.Pla
|
||||
- **commit**: <hash 或 N/A>
|
||||
======================================================= -->
|
||||
|
||||
### 2026-07-01 · v0.6 Projects 页去杂乱 — 命名空间自动分组 + "Active now" + 跨设备 prefs
|
||||
### 2026-07-01 · VC 语音命令映射 — 上下文门控 confirm/reject(多 agent 编排 + 双 review)
|
||||
|
||||
- **状态**: `[x]` DONE(未提交,分支 `v0.6-projects`)
|
||||
- **背景**: 已有 v0.7 A2 push-to-talk 只做「语音→文字」口述;本次加**语音命令层**,让语音直接 approve/reject 权限门。规划见 `docs/PLAN_VOICE_COMMANDS.md`(多 agent:理解→前后端计划→4 路对抗 review→综合)。锁定决策:①上下文感知(仅当活动 tab `pendingApproval===true` 才当命令,否则口述不变)②只做 confirm 类,复用现有 `approve`/`reject` WS 通道(**后端零改动**)③危险键不接线,two-step 组件仅用于 approve 确认窗口。
|
||||
- **改动**:
|
||||
- 新增 `public/voice-commands.ts`(纯匹配器,**100% 覆盖**):`matchCommand` = 整句精确 + 前导否定守卫 + reject 优先 + 置信度门(仅 approve);中英确认/否定词表。
|
||||
- 新增 `public/voice-confirm.ts`(确认窗口控制器,**100% 覆盖**):`createApproveConfirm(1500)` arm/cancel/isArmed,纯计时(注入 setTimeout,无 DOM)。
|
||||
- `public/voice.ts`:`onTranscript` 多传 `confidence`(仍只转写)。`public/terminal-session.ts`:加 `pendingEpoch`(false→true 翻转自增,stale-gate nonce)。`public/tabs.ts`:`handleVoiceTranscript` 分派 + 确认窗口 arm + WS 断开闪「未连接」+ 单独 `#voice-confirm` overlay(避 hideInterim 清屏)。`public/main.ts`:接 `onVoiceTrigger` 点亮 🎤。`vitest.config.ts`:两新模块入 coverage.include(orchestrator 协调编辑)。
|
||||
- 测试:`test/voice-commands.test.ts`/`test/voice-confirm.test.ts` 新建;`test/tabs.test.ts` 加 VC 集成(含 6 条 review-fix 用例);`test/voice.test.ts` 4 处旧断言随 `(text,confidence)` 新签名更新 + 补 confidence 转发断言。
|
||||
- **验证**: `npm run typecheck` 干净;`npm test` → **1307 passed / 43 files**;`build:web` OK;覆盖率 voice-commands/voice-confirm **100×4**、tabs.ts 94/84/89/97、TOTAL 91.3/84.6/91.7/93.2(80×4 通过)。`git diff src/` 空(后端 NONE 成立)。
|
||||
- **决策 / 偏离 PLAN**:
|
||||
- **实现 workflow 首跑失败**(review 阶段某 agent 6 次 stall)——但 Build V1/V2 已落文件;改为 orchestrator 亲自跑验证 + 用普通 agent 并行做 code/security review(避开 workflow 卡死)。
|
||||
- **两份 review 独立收敛,判 CRITICAL/BLOCK**,均已修复并补测:①**确认窗口 TOCTOU**:`onCommit` 现复检 `pendingApproval && gate==='tool' && pendingEpoch===armedEpoch && connected`,否则丢弃;②**`cancel()` 从未接线**:接入新话语顶替(handler 顶部)、reject 分支、`#voice-confirm` 点击;③**WS 断开延迟提交错位**:提交前复检连接。
|
||||
- **偏离 PLAN §4**:「松开麦克风取消」在 PTT 时序下不成立(松麦克风正是触发最终转录/arm 的动作),改为 **tap + 新话语顶替 + reject 取消** 三条真实取消路径。
|
||||
- **`startsWithNegation` 重写**(review MEDIUM):原 `prefix+rest[0]` 实际只测 prefix[0](巧对但脆弱),改为显式测 `rest`。
|
||||
- **遗留 / 待办**:
|
||||
- **confidence fail-open**(review MEDIUM,**保留**):`confidence===undefined` 时置信度门短路(符合 §D「有置信度时才门控」),靠确认窗口兜底误听;若要 fail-safe(undefined→不批准)是一行改动,但会让不报告置信度的引擎(部分非 Chromium)语音批准失效——留给用户定。
|
||||
- **`public/tabs.ts` 已 968 行 > 800 上限**(既有债 + 本次 ~99 行):本次仅加最小 diff,**待后续抽 `voice-controller` 拆分**(decision F)。
|
||||
- **commit**: N/A(未提交)
|
||||
|
||||
- **状态**: `[x]` DONE。双 typecheck 干净 · `build:web` OK · 全量 `vitest run` **1227 passed (41 files)** · 真浏览器端到端验证通过。
|
||||
- **需求**(用户): Projects 页扁平网格太乱(50–100 repo)。多 agent 探索(4 路:自动分组/手动文件夹/密度分段/竞品调研)结论一致 → **主方向=按点分隔命名空间自动分组**(repo 名如 `Billo.Platform.Payment` 天然带层级),配 **Active-now 置顶**;手动文件夹只是加分项非主线。用户选:**范围=自动分组+Active band**,**持久化=服务端**(收藏/折叠态跨设备同步,修掉 localStorage 每设备各一份的缺陷)。
|
||||
|
||||
@@ -42,8 +42,13 @@ if (joinId) {
|
||||
let settings: Settings = loadSettings()
|
||||
app.applySettings(settings)
|
||||
|
||||
// Key bar (mobile + desktop) sends to whichever tab is active.
|
||||
mountKeybar((data) => app.sendToActive(data))
|
||||
// Key bar (mobile + desktop) sends to whichever tab is active. onVoiceTrigger
|
||||
// activates the full voice path (dictation + context-gated confirm/reject, VC)
|
||||
// and un-dormants the 🎤 button (createVoiceInput / matchCommand are otherwise
|
||||
// unreachable dead code without this wiring).
|
||||
mountKeybar((data) => app.sendToActive(data), {
|
||||
onVoiceTrigger: (action) => app.handleVoiceTrigger(action),
|
||||
})
|
||||
|
||||
// Multi-device: when this device regains focus, re-assert the active tab's size
|
||||
// so a shared session snaps to THIS screen (latest-writer-wins) — full-screen on
|
||||
|
||||
130
public/tabs.ts
130
public/tabs.ts
@@ -32,6 +32,8 @@ import { mountPushToggle } from './push.js'
|
||||
import { mountQuickReply } from './quick-reply.js'
|
||||
import { mountTimeline, type TimelineHandle } from './timeline.js'
|
||||
import { createVoiceInput, type VoiceInput } from './voice.js'
|
||||
import { matchCommand, type VoiceMatchContext } from './voice-commands.js'
|
||||
import { createApproveConfirm, type ApproveConfirm } from './voice-confirm.js'
|
||||
|
||||
const TABS_KEY = 'web-terminal:tabs'
|
||||
const ACTIVE_KEY = 'web-terminal:active'
|
||||
@@ -42,6 +44,9 @@ const PERMISSION_MODE_KEY = 'web-terminal:permission-mode'
|
||||
* (class `tg-stale`) once telemetry is older than this (B2). */
|
||||
const STATUSLINE_TTL_MS = 30_000
|
||||
|
||||
/** VC decision C: how long the "未连接" not-connected flash stays visible. */
|
||||
const VOICE_NOT_CONNECTED_FLASH_MS = 1500
|
||||
|
||||
/** All `--permission-mode` values (B4). 'auto' is high-risk and only offered
|
||||
* when the server's ALLOW_AUTO_MODE allows it (SEC-M5, gated via /config/ui). */
|
||||
const ALL_PERMISSION_MODES: readonly PermissionMode[] = ['default', 'acceptEdits', 'plan', 'auto']
|
||||
@@ -102,6 +107,11 @@ export class TabApp {
|
||||
private timelineBtn: HTMLButtonElement | null = null
|
||||
private voice: VoiceInput | null = null // A2: push-to-talk recognizer
|
||||
private voiceOverlay: HTMLElement | null = null // A2: interim-transcript overlay
|
||||
// VC: context-gated confirm/reject command mapping (PLAN_VOICE_COMMANDS.md).
|
||||
private readonly approveConfirm: ApproveConfirm = createApproveConfirm() // decision B
|
||||
private voiceConfirmOverlay: HTMLElement | null = null // "听到：批准" / "未连接" (B, C)
|
||||
private voicePttSession: TerminalSession | null = null // session captured at PTT-start (safeguard 5)
|
||||
private voicePttEpoch = 0 // session.pendingEpoch captured at PTT-start (safeguard 5)
|
||||
|
||||
constructor(paneHost: HTMLElement, tabBar: HTMLElement) {
|
||||
this.paneHost = paneHost
|
||||
@@ -173,13 +183,23 @@ export class TabApp {
|
||||
document.body.appendChild(this.timelinePanel)
|
||||
}
|
||||
|
||||
/** A2: overlay that shows the live interim transcript while dictating. */
|
||||
/** A2: overlay that shows the live interim transcript while dictating. VC: a
|
||||
* SEPARATE overlay carries "听到：批准" / "未连接" so hideInterim() (which
|
||||
* fires on PTT release) never clobbers it mid-confirm-window (§6). */
|
||||
private setupVoiceOverlay(): void {
|
||||
const overlay = document.createElement('div')
|
||||
overlay.id = 'voice-interim'
|
||||
overlay.style.display = 'none'
|
||||
document.body.appendChild(overlay)
|
||||
this.voiceOverlay = overlay
|
||||
|
||||
const confirmOverlay = document.createElement('div')
|
||||
confirmOverlay.id = 'voice-confirm'
|
||||
confirmOverlay.style.display = 'none'
|
||||
// Tap-to-cancel an armed approve confirm window (§4 cancel trigger).
|
||||
confirmOverlay.addEventListener('click', () => this.cancelArmedApprove())
|
||||
document.body.appendChild(confirmOverlay)
|
||||
this.voiceConfirmOverlay = confirmOverlay
|
||||
}
|
||||
|
||||
/** B4: fetch the server UI config (allowAutoMode). Best-effort, never throws. */
|
||||
@@ -351,11 +371,17 @@ export class TabApp {
|
||||
|
||||
private startVoice(): void {
|
||||
if (!this.voice) {
|
||||
this.voice = createVoiceInput((text) => this.sendToActive(text), {
|
||||
onInterim: (t) => this.showInterim(t),
|
||||
})
|
||||
this.voice = createVoiceInput(
|
||||
(text, confidence) => this.handleVoiceTranscript(text, confidence),
|
||||
{ onInterim: (t) => this.showInterim(t) },
|
||||
)
|
||||
}
|
||||
if (!this.voice) return // SpeechRecognition unsupported — no-op (AC-A2.3)
|
||||
// VC safeguard 5: bind this utterance to the CURRENTLY active session/epoch
|
||||
// so a slow transcript can't approve a newer gate or a different tab.
|
||||
const active = this.tabs[this.activeIndex]?.session ?? null
|
||||
this.voicePttSession = active
|
||||
this.voicePttEpoch = active?.pendingEpoch ?? 0
|
||||
this.showInterim('')
|
||||
this.voice.start()
|
||||
}
|
||||
@@ -375,6 +401,102 @@ export class TabApp {
|
||||
if (this.voiceOverlay) this.voiceOverlay.style.display = 'none'
|
||||
}
|
||||
|
||||
/**
|
||||
* VC — dispatch a final voice transcript (PLAN_VOICE_COMMANDS.md §6):
|
||||
* context-gated confirm/reject resolve the held permission via the existing
|
||||
* approve()/reject() WS channel; everything else (including a stale gate or
|
||||
* a tab switch mid-utterance) falls through to ordinary dictation, byte-for-
|
||||
* byte unchanged. Raw bytes are NEVER synthesized for approve/reject.
|
||||
*/
|
||||
private handleVoiceTranscript(text: string, confidence?: number): void {
|
||||
// A NEW utterance supersedes any pending approve confirm window: the user
|
||||
// has spoken again, so never let a stale timer commit behind a fresh
|
||||
// command (this is the "say another word" cancel trigger, §4). Combined
|
||||
// with tap-to-cancel and the commit-time re-check below, it closes the
|
||||
// TOCTOU race where a delayed approve could resolve a different gate.
|
||||
this.cancelArmedApprove()
|
||||
|
||||
const session = this.tabs[this.activeIndex]?.session
|
||||
if (!session) {
|
||||
this.sendToActive(text)
|
||||
return
|
||||
}
|
||||
const ctx: VoiceMatchContext = {
|
||||
pendingApproval: session.pendingApproval,
|
||||
gate: session.pendingGate,
|
||||
confidence,
|
||||
}
|
||||
const action = matchCommand(text, ctx)
|
||||
// Stale-gate guard: a command can only resolve the SAME session+gate it
|
||||
// was spoken against; anything else (tab switch, a new pending flip) is
|
||||
// downgraded to plain dictation rather than silently dropped.
|
||||
const stale = session !== this.voicePttSession || session.pendingEpoch !== this.voicePttEpoch
|
||||
if (action !== 'text' && stale) {
|
||||
this.sendToActive(text)
|
||||
return
|
||||
}
|
||||
if (action === 'reject') {
|
||||
if (!this.isVoiceConnected(session)) return this.flashVoiceStatus('未连接', VOICE_NOT_CONNECTED_FLASH_MS)
|
||||
session.reject()
|
||||
this.updateApprovalBar()
|
||||
return
|
||||
}
|
||||
if (action === 'approve') {
|
||||
if (!this.isVoiceConnected(session)) return this.flashVoiceStatus('未连接', VOICE_NOT_CONNECTED_FLASH_MS)
|
||||
// Decision B: arm a cancellable window instead of committing immediately.
|
||||
// The commit RE-VALIDATES the gate identity (TOCTOU fix): if the held
|
||||
// permission resolved and a NEW gate was raised during the ~1.5s window
|
||||
// (pendingEpoch bumps), or the tab/WS changed, drop the approve — never
|
||||
// silently approve a different, unseen/unheard gate.
|
||||
const armedSession = session
|
||||
const armedEpoch = session.pendingEpoch
|
||||
this.approveConfirm.arm(
|
||||
() => this.flashVoiceStatus('听到：批准（点此取消）'),
|
||||
() => {
|
||||
this.hideVoiceStatus()
|
||||
const fresh =
|
||||
armedSession.pendingApproval &&
|
||||
armedSession.pendingGate === 'tool' &&
|
||||
armedSession.pendingEpoch === armedEpoch &&
|
||||
this.isVoiceConnected(armedSession)
|
||||
if (!fresh) return // gate changed / WS dropped during the window → drop
|
||||
armedSession.approve()
|
||||
this.updateApprovalBar()
|
||||
},
|
||||
)
|
||||
return
|
||||
}
|
||||
this.sendToActive(text) // UNCHANGED dictation path
|
||||
}
|
||||
|
||||
/** Cancel a still-armed approve confirm window and clear its overlay. Invoked
|
||||
* on a new utterance, a reject, and a tap on the overlay (§4 cancel triggers). */
|
||||
private cancelArmedApprove(): void {
|
||||
if (!this.approveConfirm.isArmed()) return
|
||||
this.approveConfirm.cancel()
|
||||
this.hideVoiceStatus()
|
||||
}
|
||||
|
||||
/** Decision C: approve()/reject() are silently dropped by sendMsg when the WS
|
||||
* isn't open — flash feedback instead of a silent no-op. */
|
||||
private isVoiceConnected(session: TerminalSession): boolean {
|
||||
return session.status === 'connected'
|
||||
}
|
||||
|
||||
/** Transient voice-status text ("听到：批准" / "未连接") on a DIFFERENT overlay
|
||||
* than the interim transcript, so hideInterim() (PTT release) never clears
|
||||
* it mid-confirm-window. XSS-safe: textContent only. */
|
||||
private flashVoiceStatus(text: string, autoHideMs?: number): void {
|
||||
if (!this.voiceConfirmOverlay) return
|
||||
this.voiceConfirmOverlay.textContent = text
|
||||
this.voiceConfirmOverlay.style.display = 'block'
|
||||
if (autoHideMs !== undefined) setTimeout(() => this.hideVoiceStatus(), autoHideMs)
|
||||
}
|
||||
|
||||
private hideVoiceStatus(): void {
|
||||
if (this.voiceConfirmOverlay) this.voiceConfirmOverlay.style.display = 'none'
|
||||
}
|
||||
|
||||
/** Toggle the activity-timeline panel for the active session (A4). */
|
||||
private toggleTimeline(): void {
|
||||
this.timelineOpen = !this.timelineOpen
|
||||
|
||||
@@ -95,6 +95,11 @@ export class TerminalSession {
|
||||
private pendingToolValue: string | undefined = undefined
|
||||
private telemetryValue: StatusTelemetry | null = null
|
||||
private pendingGateValue: PermissionGate | null = null
|
||||
// VC: nonce that increments on every false→true pendingApproval flip — lets a
|
||||
// caller (e.g. a voice command captured at PTT-start) detect a stale gate: a
|
||||
// slow transcript must not resolve a NEWER held permission than the one it
|
||||
// was spoken against (PLAN_VOICE_COMMANDS.md §5, safeguard 5).
|
||||
private pendingEpochValue = 0
|
||||
|
||||
private ws: WebSocket | null = null
|
||||
private sessionId: string | null
|
||||
@@ -189,6 +194,11 @@ export class TerminalSession {
|
||||
return this.pendingGateValue
|
||||
}
|
||||
|
||||
/** Nonce counting false→true pendingApproval flips (VC stale-gate guard, §5). */
|
||||
get pendingEpoch(): number {
|
||||
return this.pendingEpochValue
|
||||
}
|
||||
|
||||
private setStatus(s: SessionStatus): void {
|
||||
this.statusValue = s
|
||||
this.onStatus?.(s)
|
||||
@@ -293,9 +303,11 @@ export class TerminalSession {
|
||||
break
|
||||
}
|
||||
case 'status': {
|
||||
const nextPending = msg.pending === true
|
||||
if (nextPending && !this.pendingApprovalValue) this.pendingEpochValue++
|
||||
this.claudeStatusValue = msg.status
|
||||
this.pendingApprovalValue = msg.pending === true
|
||||
this.pendingToolValue = msg.pending === true ? msg.detail : undefined
|
||||
this.pendingApprovalValue = nextPending
|
||||
this.pendingToolValue = nextPending ? msg.detail : undefined
|
||||
this.pendingGateValue = msg.gate ?? null
|
||||
this.onClaudeStatus?.(msg.status, msg.detail)
|
||||
break
|
||||
|
||||
91
public/voice-commands.ts
Normal file
91
public/voice-commands.ts
Normal file
@@ -0,0 +1,91 @@
|
||||
/**
|
||||
* public/voice-commands.ts — pure, DOM-free/WS-free voice command matcher.
|
||||
*
|
||||
* Context-gated confirm-class command mapping: while the active tab has a
|
||||
* HELD tool-gate permission (`pendingApproval===true`, `gate==='tool'`),
|
||||
* spoken confirm/deny phrases resolve to `approve`/`reject`; every other
|
||||
* transcript (or context) falls through to ordinary dictation (`text`),
|
||||
* byte-for-byte unchanged. See docs/PLAN_VOICE_COMMANDS.md §3/§7 for the
|
||||
* locked decisions and security rationale — this module is intentionally
|
||||
* whole-utterance-exact (no substring/token matching) so incidental
|
||||
* dictation can never auto-approve a shell-granting gate.
|
||||
*/
|
||||
|
||||
export type VoiceAction = 'approve' | 'reject' | 'text'
|
||||
|
||||
export interface VoiceMatchContext {
|
||||
readonly pendingApproval: boolean // active session held-permission flag
|
||||
readonly gate: 'tool' | 'plan' | null // v1 acts ONLY on 'tool'
|
||||
readonly confidence?: number // optional ASR confidence (decision D)
|
||||
}
|
||||
|
||||
/** Approve requires ASR confidence at or above this threshold; reject is unaffected. */
|
||||
export const MIN_APPROVE_CONFIDENCE = 0.6
|
||||
|
||||
/** Normalized whole-utterance affirmative phrases (see normalizeTranscript). */
|
||||
export const APPROVE_PHRASES: readonly string[] = [
|
||||
'确认', '批准', '同意', '通过', '允许', '可以', '好', '好的', '好吧', '好啊',
|
||||
'是', '是的', '对', '行', '继续', '回车',
|
||||
'yes', 'yeah', 'yep', 'ok', 'okay', 'confirm', 'approve', 'accept', 'proceed', 'go ahead',
|
||||
]
|
||||
|
||||
/** Normalized whole-utterance negative phrases (see normalizeTranscript). */
|
||||
export const REJECT_PHRASES: readonly string[] = [
|
||||
'拒绝', '取消', '不行', '不要', '不用', '不同意', '不批准', '不可以', '不允许', '不通过',
|
||||
'中断', '停止', '算了', '别', '不',
|
||||
'no', 'nope', 'reject', 'deny', 'cancel', 'abort', 'decline', 'stop',
|
||||
]
|
||||
|
||||
/** Leading tokens that negate an utterance; a match can never approve. */
|
||||
export const NEGATION_PREFIXES: readonly string[] = [
|
||||
'不', '别', '没', '未', '勿',
|
||||
'no', 'not', 'dont', 'cannot', 'wont', 'never',
|
||||
]
|
||||
|
||||
/** ASCII + CJK punctuation and quote marks, plus straight/curly apostrophes. */
|
||||
const PUNCTUATION_RE =
|
||||
/['’.,!?;:，。！？；：、「」『』“”‘’（）()[\]{}【】〈〉《》…—\-_/\\|"`~@#$%^&*+=<>]/g
|
||||
|
||||
/**
|
||||
* `trim → toLowerCase → strip punctuation (so don't→dont) → collapse ASCII
|
||||
* whitespace → trim`. Keeps CJK + ASCII letters/digits.
|
||||
*/
|
||||
export function normalizeTranscript(raw: string): string {
|
||||
return raw
|
||||
.trim()
|
||||
.toLowerCase()
|
||||
.replace(PUNCTUATION_RE, '')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim()
|
||||
}
|
||||
|
||||
/** True if a normalized transcript begins with a negation token (word-boundary for ASCII). */
|
||||
export function startsWithNegation(norm: string): boolean {
|
||||
return NEGATION_PREFIXES.some((prefix) => {
|
||||
if (!norm.startsWith(prefix)) return false
|
||||
// CJK negation particles (不/别/没/未/勿) have no word boundary, so any
|
||||
// following character still counts as negation. ASCII negators require a
|
||||
// word boundary after them, so "now" is NOT negated by the "no" prefix.
|
||||
if (!/^[a-z0-9]/.test(prefix)) return true
|
||||
const rest = norm.slice(prefix.length)
|
||||
return rest === '' || !/^[a-z0-9]/.test(rest)
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Whole-utterance-exact, reject-first, negation-guarded command match.
|
||||
* Falls through to `'text'` (ordinary dictation) for anything not an exact,
|
||||
* gate-eligible, confidence-sufficient command phrase.
|
||||
*/
|
||||
export function matchCommand(transcript: string, ctx: VoiceMatchContext): VoiceAction {
|
||||
const norm = normalizeTranscript(transcript)
|
||||
if (!ctx.pendingApproval || norm === '') return 'text'
|
||||
if (ctx.gate !== 'tool') return 'text'
|
||||
if (startsWithNegation(norm)) return REJECT_PHRASES.includes(norm) ? 'reject' : 'text'
|
||||
if (REJECT_PHRASES.includes(norm)) return 'reject'
|
||||
if (APPROVE_PHRASES.includes(norm)) {
|
||||
if (ctx.confidence !== undefined && ctx.confidence < MIN_APPROVE_CONFIDENCE) return 'text'
|
||||
return 'approve'
|
||||
}
|
||||
return 'text'
|
||||
}
|
||||
65
public/voice-confirm.ts
Normal file
65
public/voice-confirm.ts
Normal file
@@ -0,0 +1,65 @@
|
||||
/**
|
||||
* public/voice-confirm.ts — Approve confirm window controller (VC / V1b, decision B).
|
||||
*
|
||||
* PLAN_VOICE_COMMANDS.md §4: a spoken "approve" is irreversible (grants shell
|
||||
* command execution via the held permission gate), so instead of committing
|
||||
* immediately it arms a short cancellable window. `arm()` fires `onHeard`
|
||||
* synchronously (so the caller can show a "听到:批准" style affordance) and
|
||||
* starts a `windowMs` timer; if `cancel()` is not called first, `onCommit`
|
||||
* fires when the timer elapses.
|
||||
*
|
||||
* Pure timing logic — NO DOM here. The caller (tabs.ts, lane V2) owns the
|
||||
* overlay UI and calls `session.approve()` inside `onCommit`. Kept dependency-
|
||||
* free (setTimeout/clearTimeout only) so tests can use vi.useFakeTimers
|
||||
* without any DOM/WS coupling.
|
||||
*/
|
||||
|
||||
/** Default confirm-window duration in milliseconds (PLAN_VOICE_COMMANDS §4). */
|
||||
const DEFAULT_WINDOW_MS = 1500
|
||||
|
||||
/** A cancellable approve confirm window. Obtain via createApproveConfirm(). */
|
||||
export interface ApproveConfirm {
|
||||
/**
|
||||
* Arm the confirm window: calls `onHeard` immediately, then starts a
|
||||
* `windowMs` timer that calls `onCommit` unless `cancel()` runs first.
|
||||
* Re-arming (calling `arm` again while already armed) clears the prior
|
||||
* timer, so only the latest arm's `onCommit` can ever fire.
|
||||
*/
|
||||
arm(onHeard: () => void, onCommit: () => void): void
|
||||
/** Cancel the pending window, if any. No-op (does not throw) if not armed. */
|
||||
cancel(): void
|
||||
/** True while a confirm window is pending (armed and not yet committed/cancelled). */
|
||||
isArmed(): boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Create an approve confirm window controller.
|
||||
*
|
||||
* @param windowMs Duration of the cancellable window in ms. Default: 1500 (B).
|
||||
*/
|
||||
export function createApproveConfirm(windowMs: number = DEFAULT_WINDOW_MS): ApproveConfirm {
|
||||
let timer: ReturnType<typeof setTimeout> | undefined
|
||||
|
||||
return {
|
||||
arm(onHeard: () => void, onCommit: () => void): void {
|
||||
if (timer !== undefined) {
|
||||
clearTimeout(timer)
|
||||
}
|
||||
onHeard()
|
||||
timer = setTimeout(() => {
|
||||
timer = undefined
|
||||
onCommit()
|
||||
}, windowMs)
|
||||
},
|
||||
|
||||
cancel(): void {
|
||||
if (timer === undefined) return
|
||||
clearTimeout(timer)
|
||||
timer = undefined
|
||||
},
|
||||
|
||||
isArmed(): boolean {
|
||||
return timer !== undefined
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -108,11 +108,13 @@ function getSRConstructor(): SpeechRecognitionCtor | undefined {
|
||||
* Returns null when the browser does not support SpeechRecognition — the
|
||||
* caller should hide the mic UI in that case (AC-A2.3).
|
||||
*
|
||||
* @param onTranscript Receives the final recognised text (+ '\r' if autoSend).
|
||||
* @param onTranscript Receives the final recognised text (+ '\r' if autoSend),
|
||||
* plus the top alternative's ASR confidence (0-1) as an
|
||||
* optional 2nd arg when the engine reports one (VC decision D).
|
||||
* @param opts See VoiceInputOptions.
|
||||
*/
|
||||
export function createVoiceInput(
|
||||
onTranscript: (text: string) => void,
|
||||
onTranscript: (text: string, confidence?: number) => void,
|
||||
opts?: VoiceInputOptions,
|
||||
): VoiceInput | null {
|
||||
const SRClass = getSRConstructor()
|
||||
@@ -130,11 +132,17 @@ export function createVoiceInput(
|
||||
if (disposed) return
|
||||
let interimText = ''
|
||||
let finalText = ''
|
||||
let finalConfidence: number | undefined
|
||||
for (let i = event.resultIndex; i < event.results.length; i++) {
|
||||
const result = event.results[i]
|
||||
const text = result?.[0]?.transcript ?? ''
|
||||
const alt = result?.[0]
|
||||
const text = alt?.transcript ?? ''
|
||||
if (result?.isFinal) {
|
||||
finalText += text
|
||||
// VC (decision D): forward the top alternative's confidence so the
|
||||
// caller can gate risk-asymmetric decisions (e.g. approve). Still
|
||||
// transcription-only — no decision-making happens in this module.
|
||||
finalConfidence = alt?.confidence ?? finalConfidence
|
||||
} else {
|
||||
interimText += text
|
||||
}
|
||||
@@ -143,7 +151,7 @@ export function createVoiceInput(
|
||||
opts.onInterim(interimText)
|
||||
}
|
||||
if (finalText) {
|
||||
onTranscript(opts?.autoSend ? finalText + '\r' : finalText)
|
||||
onTranscript(opts?.autoSend ? finalText + '\r' : finalText, finalConfidence)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -22,6 +22,8 @@ class FakeTerminalSession {
|
||||
pendingTool: string | undefined = undefined
|
||||
/** B4: which gate the held approval is — 'tool' (two buttons) or 'plan' (three). */
|
||||
pendingGate: 'tool' | 'plan' | null = null
|
||||
/** VC: nonce that counts false→true pendingApproval flips (stale-gate guard). */
|
||||
pendingEpoch = 0
|
||||
/** B2: latest statusLine telemetry (single source of truth). */
|
||||
telemetry: unknown = null
|
||||
/** Captured so tests can assert initialInput ends with \r (Finding 1). */
|
||||
@@ -86,11 +88,17 @@ const timelineDispose = vi.fn()
|
||||
const mountTimeline = vi.fn(() => ({ dispose: timelineDispose }))
|
||||
vi.mock('../public/timeline.js', () => ({ mountTimeline }))
|
||||
|
||||
const voice: { onTranscript?: (t: string) => void; onInterim?: (t: string) => void } = {}
|
||||
const voice: {
|
||||
onTranscript?: (t: string, confidence?: number) => void
|
||||
onInterim?: (t: string) => void
|
||||
} = {}
|
||||
const voiceStart = vi.fn()
|
||||
const voiceStop = vi.fn()
|
||||
const createVoiceInput = vi.fn(
|
||||
(onTranscript: (t: string) => void, opts?: { onInterim?: (t: string) => void }) => {
|
||||
(
|
||||
onTranscript: (t: string, confidence?: number) => void,
|
||||
opts?: { onInterim?: (t: string) => void },
|
||||
) => {
|
||||
voice.onTranscript = onTranscript
|
||||
voice.onInterim = opts?.onInterim
|
||||
return { start: voiceStart, stop: voiceStop, dispose: vi.fn(), isActive: () => false }
|
||||
@@ -790,3 +798,237 @@ describe('TabApp — A2 push-to-talk voice', () => {
|
||||
expect(voiceStart).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
describe('TabApp — VC voice command mapping (context-gated confirm/reject)', () => {
|
||||
/** Open one tab, hold a tool-gate approval, start PTT, and return its session. */
|
||||
function pendingToolSession(app: InstanceType<typeof TabApp>): FakeTerminalSession {
|
||||
app.newTab()
|
||||
const session = FakeTerminalSession.instances[0]!
|
||||
session.status = 'connected'
|
||||
session.pendingApproval = true
|
||||
session.pendingGate = 'tool'
|
||||
app.handleVoiceTrigger('start')
|
||||
return session
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers()
|
||||
})
|
||||
|
||||
it('a held tool-gate "confirm" arms the approve window, then commits once after it elapses (send NOT called)', () => {
|
||||
vi.useFakeTimers()
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app)
|
||||
|
||||
voice.onTranscript?.('confirm', 0.9)
|
||||
expect(session.approve).not.toHaveBeenCalled() // armed, not yet committed
|
||||
|
||||
vi.advanceTimersByTime(1500)
|
||||
expect(session.approve).toHaveBeenCalledTimes(1)
|
||||
expect(session.send).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('a held tool-gate "拒绝" rejects immediately (fail-safe, no confirm window)', () => {
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app)
|
||||
|
||||
voice.onTranscript?.('拒绝')
|
||||
expect(session.reject).toHaveBeenCalledTimes(1)
|
||||
expect(session.send).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('an unmatched phrase ("deploy to prod") falls through to dictation even while pending', () => {
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app)
|
||||
|
||||
voice.onTranscript?.('deploy to prod')
|
||||
expect(session.send).toHaveBeenCalledWith('deploy to prod')
|
||||
expect(session.approve).not.toHaveBeenCalled()
|
||||
expect(session.reject).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('pending=false: "yes" is ordinary dictation (decision 1 — context gate)', () => {
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
app.newTab()
|
||||
const session = FakeTerminalSession.instances[0]!
|
||||
session.status = 'connected'
|
||||
session.pendingApproval = false
|
||||
app.handleVoiceTrigger('start')
|
||||
|
||||
voice.onTranscript?.('yes')
|
||||
expect(session.send).toHaveBeenCalledWith('yes')
|
||||
expect(session.approve).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('plan gate: "confirm" falls through to dictation (decision A — tool-gate only)', () => {
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
app.newTab()
|
||||
const session = FakeTerminalSession.instances[0]!
|
||||
session.status = 'connected'
|
||||
session.pendingApproval = true
|
||||
session.pendingGate = 'plan'
|
||||
app.handleVoiceTrigger('start')
|
||||
|
||||
voice.onTranscript?.('confirm')
|
||||
expect(session.send).toHaveBeenCalledWith('confirm')
|
||||
expect(session.approve).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('stale gate: an epoch flip after PTT-start downgrades an approve to dictation (safeguard 5)', () => {
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app) // captures pendingEpoch=0 at PTT-start
|
||||
session.pendingEpoch = 1 // a new pending flip landed mid-utterance
|
||||
|
||||
voice.onTranscript?.('confirm', 0.9)
|
||||
expect(session.send).toHaveBeenCalledWith('confirm')
|
||||
expect(session.approve).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('tab switch: activating a different tab after PTT-start downgrades an approve to dictation (safeguard 5)', () => {
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session0 = pendingToolSession(app) // PTT started while tab 0 is active
|
||||
|
||||
app.newTab() // tab 1 becomes active mid-utterance
|
||||
const session1 = FakeTerminalSession.instances[1]!
|
||||
session1.status = 'connected'
|
||||
session1.pendingApproval = true
|
||||
session1.pendingGate = 'tool'
|
||||
|
||||
voice.onTranscript?.('confirm', 0.9)
|
||||
expect(session1.send).toHaveBeenCalledWith('confirm')
|
||||
expect(session1.approve).not.toHaveBeenCalled()
|
||||
expect(session0.approve).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('WS not connected: approve is never sent and never throws; flashes "未连接" (decision C)', () => {
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
app.newTab()
|
||||
const session = FakeTerminalSession.instances[0]!
|
||||
session.status = 'connecting' // not connected
|
||||
session.pendingApproval = true
|
||||
session.pendingGate = 'tool'
|
||||
app.handleVoiceTrigger('start')
|
||||
|
||||
expect(() => voice.onTranscript?.('确认', 0.9)).not.toThrow()
|
||||
expect(session.approve).not.toHaveBeenCalled()
|
||||
const overlay = document.getElementById('voice-confirm')
|
||||
expect(overlay?.textContent).toBe('未连接')
|
||||
expect(overlay?.style.display).toBe('block')
|
||||
})
|
||||
|
||||
it('WS not connected: reject is never sent and never throws (decision C)', () => {
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
app.newTab()
|
||||
const session = FakeTerminalSession.instances[0]!
|
||||
session.status = 'reconnecting' // not connected
|
||||
session.pendingApproval = true
|
||||
session.pendingGate = 'tool'
|
||||
app.handleVoiceTrigger('start')
|
||||
|
||||
expect(() => voice.onTranscript?.('拒绝')).not.toThrow()
|
||||
expect(session.reject).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('a confidence below the threshold downgrades an approve phrase to dictation (decision D)', () => {
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app)
|
||||
|
||||
voice.onTranscript?.('confirm', 0.4)
|
||||
expect(session.send).toHaveBeenCalledWith('confirm')
|
||||
expect(session.approve).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
// ── confirm-window cancellation + commit-time re-validation (review fixes) ──
|
||||
|
||||
it('a "拒绝" spoken during the confirm window cancels the pending approve (reject wins)', () => {
|
||||
vi.useFakeTimers()
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app)
|
||||
|
||||
voice.onTranscript?.('confirm', 0.9) // arm the window
|
||||
voice.onTranscript?.('拒绝') // change of mind mid-window
|
||||
expect(session.reject).toHaveBeenCalledTimes(1)
|
||||
|
||||
vi.advanceTimersByTime(1500)
|
||||
expect(session.approve).not.toHaveBeenCalled() // stale timer must NOT commit
|
||||
})
|
||||
|
||||
it('tapping the confirm overlay cancels the pending approve before it commits', () => {
|
||||
vi.useFakeTimers()
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app)
|
||||
|
||||
voice.onTranscript?.('confirm', 0.9)
|
||||
document.getElementById('voice-confirm')!.click() // tap-to-cancel (§4)
|
||||
|
||||
vi.advanceTimersByTime(1500)
|
||||
expect(session.approve).not.toHaveBeenCalled()
|
||||
expect(document.getElementById('voice-confirm')!.style.display).toBe('none')
|
||||
})
|
||||
|
||||
it('a new dictation utterance during the window supersedes (cancels) the pending approve', () => {
|
||||
vi.useFakeTimers()
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app)
|
||||
|
||||
voice.onTranscript?.('confirm', 0.9)
|
||||
voice.onTranscript?.('deploy to prod') // user moved on
|
||||
|
||||
vi.advanceTimersByTime(1500)
|
||||
expect(session.approve).not.toHaveBeenCalled()
|
||||
expect(session.send).toHaveBeenCalledWith('deploy to prod')
|
||||
})
|
||||
|
||||
it('an epoch flip DURING the confirm window drops the commit (TOCTOU re-validation)', () => {
|
||||
vi.useFakeTimers()
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app) // epoch 0 at PTT-start AND at dispatch
|
||||
|
||||
voice.onTranscript?.('confirm', 0.9) // arms against epoch 0
|
||||
session.pendingEpoch = 1 // gate A resolved + gate B raised during the window
|
||||
|
||||
vi.advanceTimersByTime(1500)
|
||||
expect(session.approve).not.toHaveBeenCalled() // must not approve the new gate
|
||||
})
|
||||
|
||||
it('a WS drop DURING the confirm window drops the commit (no local/server desync)', () => {
|
||||
vi.useFakeTimers()
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app)
|
||||
|
||||
voice.onTranscript?.('confirm', 0.9)
|
||||
session.status = 'reconnecting' // socket dropped mid-window
|
||||
|
||||
vi.advanceTimersByTime(1500)
|
||||
expect(session.approve).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('re-confirming within the window re-arms and commits exactly once', () => {
|
||||
vi.useFakeTimers()
|
||||
const { paneHost, tabBar } = makeHosts()
|
||||
const app = new TabApp(paneHost, tabBar)
|
||||
const session = pendingToolSession(app)
|
||||
|
||||
voice.onTranscript?.('confirm', 0.9)
|
||||
vi.advanceTimersByTime(500)
|
||||
voice.onTranscript?.('confirm', 0.9) // supersede first, re-arm
|
||||
|
||||
vi.advanceTimersByTime(1500)
|
||||
expect(session.approve).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
})
|
||||
|
||||
207
test/voice-commands.test.ts
Normal file
207
test/voice-commands.test.ts
Normal file
@@ -0,0 +1,207 @@
|
||||
import { describe, it, expect } from 'vitest'
|
||||
import {
|
||||
APPROVE_PHRASES,
|
||||
REJECT_PHRASES,
|
||||
MIN_APPROVE_CONFIDENCE,
|
||||
NEGATION_PREFIXES,
|
||||
normalizeTranscript,
|
||||
startsWithNegation,
|
||||
matchCommand,
|
||||
type VoiceMatchContext,
|
||||
} from '../public/voice-commands.js'
|
||||
|
||||
/** Default context: a held tool-gate permission (the common case under test). */
|
||||
function pendingToolCtx(overrides: Partial<VoiceMatchContext> = {}): VoiceMatchContext {
|
||||
return { pendingApproval: true, gate: 'tool', ...overrides }
|
||||
}
|
||||
|
||||
describe('normalizeTranscript', () => {
|
||||
it('lowercases', () => {
|
||||
expect(normalizeTranscript('YES')).toBe('yes')
|
||||
})
|
||||
|
||||
it('strips trailing ASCII punctuation', () => {
|
||||
expect(normalizeTranscript('confirm.')).toBe('confirm')
|
||||
})
|
||||
|
||||
it('strips CJK punctuation/quotes', () => {
|
||||
expect(normalizeTranscript('「确认」')).toBe('确认')
|
||||
})
|
||||
|
||||
it('strips apostrophes so contractions collapse (dont)', () => {
|
||||
expect(normalizeTranscript("don't")).toBe('dont')
|
||||
})
|
||||
|
||||
it('collapses whitespace and trims', () => {
|
||||
expect(normalizeTranscript(' ')).toBe('')
|
||||
expect(normalizeTranscript(' go ahead ')).toBe('go ahead')
|
||||
})
|
||||
|
||||
it('is idempotent (already-normalized input passes through)', () => {
|
||||
expect(normalizeTranscript('go ahead')).toBe('go ahead')
|
||||
expect(normalizeTranscript('确认')).toBe('确认')
|
||||
})
|
||||
})
|
||||
|
||||
describe('startsWithNegation', () => {
|
||||
it('detects CJK negation prefixes', () => {
|
||||
expect(startsWithNegation('不可以')).toBe(true)
|
||||
expect(startsWithNegation('别继续')).toBe(true)
|
||||
expect(startsWithNegation('没关系')).toBe(true)
|
||||
expect(startsWithNegation('未确认')).toBe(true)
|
||||
expect(startsWithNegation('勿动')).toBe(true)
|
||||
})
|
||||
|
||||
it('detects English negation prefixes', () => {
|
||||
expect(startsWithNegation('no way')).toBe(true)
|
||||
expect(startsWithNegation('not now')).toBe(true)
|
||||
expect(startsWithNegation('dont approve')).toBe(true)
|
||||
expect(startsWithNegation('cannot proceed')).toBe(true)
|
||||
expect(startsWithNegation('wont continue')).toBe(true)
|
||||
expect(startsWithNegation('never mind')).toBe(true)
|
||||
})
|
||||
|
||||
it('does not flag ordinary phrases', () => {
|
||||
expect(startsWithNegation('确认')).toBe(false)
|
||||
expect(startsWithNegation('yes')).toBe(false)
|
||||
expect(startsWithNegation('now')).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('matchCommand — context gating (decision 1)', () => {
|
||||
it('returns text when pendingApproval is false, even for a command word', () => {
|
||||
expect(matchCommand('yes', { pendingApproval: false, gate: null })).toBe('text')
|
||||
expect(matchCommand('确认', { pendingApproval: false, gate: null })).toBe('text')
|
||||
})
|
||||
|
||||
it('returns text for an empty/whitespace transcript', () => {
|
||||
expect(matchCommand(' ', pendingToolCtx())).toBe('text')
|
||||
expect(matchCommand('', pendingToolCtx())).toBe('text')
|
||||
})
|
||||
})
|
||||
|
||||
describe('matchCommand — plan gate (decision A)', () => {
|
||||
it('falls through to dictation when gate is plan, not tool', () => {
|
||||
expect(matchCommand('confirm', { pendingApproval: true, gate: 'plan' })).toBe('text')
|
||||
expect(matchCommand('stop', { pendingApproval: true, gate: 'plan' })).toBe('text')
|
||||
})
|
||||
})
|
||||
|
||||
describe('matchCommand — held tool approve', () => {
|
||||
const approvePhrases = [
|
||||
'确认', '批准', '同意', '好', '好的', '好吧', '行', '可以',
|
||||
'yes', 'ok', 'confirm', 'approve', 'go ahead',
|
||||
]
|
||||
for (const phrase of approvePhrases) {
|
||||
it(`approves "${phrase}"`, () => {
|
||||
expect(matchCommand(phrase, pendingToolCtx())).toBe('approve')
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
describe('matchCommand — held tool reject', () => {
|
||||
const rejectPhrases = ['拒绝', '取消', '不行', 'no', 'nope', 'cancel', 'deny', 'abort', 'stop']
|
||||
for (const phrase of rejectPhrases) {
|
||||
it(`rejects "${phrase}"`, () => {
|
||||
expect(matchCommand(phrase, pendingToolCtx())).toBe('reject')
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
describe('matchCommand — negation/filler regression (MUST NOT approve)', () => {
|
||||
it('不可以 rejects (negated approve phrase is a listed reject phrase)', () => {
|
||||
expect(matchCommand('不可以', pendingToolCtx())).toBe('reject')
|
||||
})
|
||||
|
||||
it('不允许/不通过/不同意/不批准 reject', () => {
|
||||
expect(matchCommand('不允许', pendingToolCtx())).toBe('reject')
|
||||
expect(matchCommand('不通过', pendingToolCtx())).toBe('reject')
|
||||
expect(matchCommand('不同意', pendingToolCtx())).toBe('reject')
|
||||
expect(matchCommand('不批准', pendingToolCtx())).toBe('reject')
|
||||
})
|
||||
|
||||
it('不继续 is negated but not a listed reject phrase -> text', () => {
|
||||
expect(matchCommand('不继续', pendingToolCtx())).toBe('text')
|
||||
})
|
||||
|
||||
it('English negated phrases fall through to text', () => {
|
||||
expect(matchCommand("don't approve", pendingToolCtx())).toBe('text')
|
||||
expect(matchCommand('do not confirm', pendingToolCtx())).toBe('text')
|
||||
expect(matchCommand('cannot proceed', pendingToolCtx())).toBe('text')
|
||||
expect(matchCommand("won't continue", pendingToolCtx())).toBe('text')
|
||||
})
|
||||
|
||||
it('"ok let\'s not do that" is not a whole-utterance match -> text', () => {
|
||||
expect(matchCommand("ok let's not do that", pendingToolCtx())).toBe('text')
|
||||
})
|
||||
|
||||
it('"ok can you also check the logs" is not a whole-utterance match -> text', () => {
|
||||
expect(matchCommand('ok can you also check the logs', pendingToolCtx())).toBe('text')
|
||||
})
|
||||
|
||||
it('我觉得可以先看看 is not a whole-utterance match -> text', () => {
|
||||
expect(matchCommand('我觉得可以先看看', pendingToolCtx())).toBe('text')
|
||||
})
|
||||
|
||||
it('请继续 is not a whole-utterance match (unequal to 继续) -> text', () => {
|
||||
expect(matchCommand('请继续', pendingToolCtx())).toBe('text')
|
||||
})
|
||||
|
||||
it('now is not a listed phrase (unequal to no) -> text', () => {
|
||||
expect(matchCommand('now', pendingToolCtx())).toBe('text')
|
||||
})
|
||||
|
||||
it('我不知道 is not a whole-utterance match -> text', () => {
|
||||
expect(matchCommand('我不知道', pendingToolCtx())).toBe('text')
|
||||
})
|
||||
})
|
||||
|
||||
describe('matchCommand — confidence gate (decision D)', () => {
|
||||
it('low-confidence approve falls through to text', () => {
|
||||
expect(matchCommand('confirm', pendingToolCtx({ confidence: 0.4 }))).toBe('text')
|
||||
})
|
||||
|
||||
it('high-confidence approve succeeds', () => {
|
||||
expect(matchCommand('confirm', pendingToolCtx({ confidence: 0.9 }))).toBe('approve')
|
||||
})
|
||||
|
||||
it('confidence at exactly MIN_APPROVE_CONFIDENCE is sufficient', () => {
|
||||
expect(matchCommand('confirm', pendingToolCtx({ confidence: MIN_APPROVE_CONFIDENCE }))).toBe(
|
||||
'approve',
|
||||
)
|
||||
})
|
||||
|
||||
it('reject is unaffected by low confidence', () => {
|
||||
expect(matchCommand('no', pendingToolCtx({ confidence: 0.2 }))).toBe('reject')
|
||||
})
|
||||
|
||||
it('approve with no confidence supplied succeeds (confidence optional)', () => {
|
||||
expect(matchCommand('confirm', pendingToolCtx())).toBe('approve')
|
||||
})
|
||||
})
|
||||
|
||||
describe('phrase-set integrity invariants', () => {
|
||||
it('every phrase is already normalized (normalize(p) === p)', () => {
|
||||
for (const p of [...APPROVE_PHRASES, ...REJECT_PHRASES]) {
|
||||
expect(normalizeTranscript(p)).toBe(p)
|
||||
}
|
||||
})
|
||||
|
||||
it('APPROVE_PHRASES and REJECT_PHRASES are disjoint', () => {
|
||||
const rejectSet = new Set(REJECT_PHRASES)
|
||||
const overlap = APPROVE_PHRASES.filter((p) => rejectSet.has(p))
|
||||
expect(overlap).toEqual([])
|
||||
})
|
||||
|
||||
it('no APPROVE_PHRASES entry starts with a negation prefix', () => {
|
||||
for (const p of APPROVE_PHRASES) {
|
||||
expect(startsWithNegation(p)).toBe(false)
|
||||
}
|
||||
})
|
||||
|
||||
it('NEGATION_PREFIXES is non-empty and covers both CJK and English', () => {
|
||||
expect(NEGATION_PREFIXES.length).toBeGreaterThan(0)
|
||||
expect(NEGATION_PREFIXES).toContain('不')
|
||||
expect(NEGATION_PREFIXES).toContain('no')
|
||||
})
|
||||
})
|
||||
168
test/voice-confirm.test.ts
Normal file
168
test/voice-confirm.test.ts
Normal file
@@ -0,0 +1,168 @@
|
||||
/**
|
||||
* test/voice-confirm.test.ts — Unit tests for voice-confirm.ts (VC / V1b).
|
||||
*
|
||||
* Pure timing-logic controller for the approve confirm window (PLAN_VOICE_COMMANDS §4,
|
||||
* decision B). Uses vi.useFakeTimers — no DOM, no WS; setTimeout/clearTimeout only.
|
||||
*
|
||||
* Covered behaviours:
|
||||
* - arm(onHeard, onCommit) fires onHeard synchronously/immediately.
|
||||
* - onCommit fires once the windowMs timer elapses.
|
||||
* - cancel() before the window elapses → onCommit is NEVER called.
|
||||
* - re-arming clears the prior timer — only the latest arm's onCommit fires.
|
||||
* - isArmed() reflects true while pending, false after commit/cancel/before-arm.
|
||||
* - default windowMs is 1500 when not provided.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
|
||||
import { createApproveConfirm } from '../public/voice-confirm.js'
|
||||
|
||||
describe('voice-confirm', () => {
|
||||
beforeEach(() => {
|
||||
vi.useFakeTimers()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers()
|
||||
})
|
||||
|
||||
it('fires onHeard immediately when armed', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm(1500)
|
||||
const onHeard = vi.fn()
|
||||
const onCommit = vi.fn()
|
||||
|
||||
// Act
|
||||
confirm.arm(onHeard, onCommit)
|
||||
|
||||
// Assert
|
||||
expect(onHeard).toHaveBeenCalledTimes(1)
|
||||
expect(onCommit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('fires onCommit after windowMs elapses', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm(1500)
|
||||
const onHeard = vi.fn()
|
||||
const onCommit = vi.fn()
|
||||
|
||||
// Act
|
||||
confirm.arm(onHeard, onCommit)
|
||||
vi.advanceTimersByTime(1499)
|
||||
expect(onCommit).not.toHaveBeenCalled()
|
||||
vi.advanceTimersByTime(1)
|
||||
|
||||
// Assert
|
||||
expect(onCommit).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('never calls onCommit when cancel() runs before the window elapses', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm(1500)
|
||||
const onHeard = vi.fn()
|
||||
const onCommit = vi.fn()
|
||||
confirm.arm(onHeard, onCommit)
|
||||
|
||||
// Act
|
||||
vi.advanceTimersByTime(500)
|
||||
confirm.cancel()
|
||||
vi.advanceTimersByTime(10_000)
|
||||
|
||||
// Assert
|
||||
expect(onCommit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('cancel() before any arm() is a no-op that does not throw', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm(1500)
|
||||
|
||||
// Act / Assert
|
||||
expect(() => confirm.cancel()).not.toThrow()
|
||||
})
|
||||
|
||||
it('re-arming clears the prior timer so only the latest arm commits', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm(1500)
|
||||
const firstHeard = vi.fn()
|
||||
const firstCommit = vi.fn()
|
||||
const secondHeard = vi.fn()
|
||||
const secondCommit = vi.fn()
|
||||
|
||||
// Act
|
||||
confirm.arm(firstHeard, firstCommit)
|
||||
vi.advanceTimersByTime(1000)
|
||||
confirm.arm(secondHeard, secondCommit)
|
||||
// Enough time for the FIRST window to have elapsed (1000 + 1000 = 2000 > 1500)
|
||||
// but not enough for the second window (armed at t=1000, needs t=2500).
|
||||
vi.advanceTimersByTime(1000)
|
||||
|
||||
// Assert: first window's commit never fires, second hasn't fired yet either.
|
||||
expect(firstCommit).not.toHaveBeenCalled()
|
||||
expect(secondCommit).not.toHaveBeenCalled()
|
||||
expect(secondHeard).toHaveBeenCalledTimes(1)
|
||||
|
||||
// Advance past the second window's own deadline.
|
||||
vi.advanceTimersByTime(500)
|
||||
expect(secondCommit).toHaveBeenCalledTimes(1)
|
||||
expect(firstCommit).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('isArmed() is false before any arm()', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm(1500)
|
||||
|
||||
// Assert
|
||||
expect(confirm.isArmed()).toBe(false)
|
||||
})
|
||||
|
||||
it('isArmed() is true immediately after arm() and while the window is pending', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm(1500)
|
||||
|
||||
// Act
|
||||
confirm.arm(vi.fn(), vi.fn())
|
||||
|
||||
// Assert
|
||||
expect(confirm.isArmed()).toBe(true)
|
||||
vi.advanceTimersByTime(500)
|
||||
expect(confirm.isArmed()).toBe(true)
|
||||
})
|
||||
|
||||
it('isArmed() is false after onCommit fires', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm(1500)
|
||||
confirm.arm(vi.fn(), vi.fn())
|
||||
|
||||
// Act
|
||||
vi.advanceTimersByTime(1500)
|
||||
|
||||
// Assert
|
||||
expect(confirm.isArmed()).toBe(false)
|
||||
})
|
||||
|
||||
it('isArmed() is false after cancel()', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm(1500)
|
||||
confirm.arm(vi.fn(), vi.fn())
|
||||
|
||||
// Act
|
||||
confirm.cancel()
|
||||
|
||||
// Assert
|
||||
expect(confirm.isArmed()).toBe(false)
|
||||
})
|
||||
|
||||
it('uses a default windowMs of 1500 when not provided', () => {
|
||||
// Arrange
|
||||
const confirm = createApproveConfirm()
|
||||
const onCommit = vi.fn()
|
||||
|
||||
// Act
|
||||
confirm.arm(vi.fn(), onCommit)
|
||||
vi.advanceTimersByTime(1499)
|
||||
expect(onCommit).not.toHaveBeenCalled()
|
||||
vi.advanceTimersByTime(1)
|
||||
|
||||
// Assert
|
||||
expect(onCommit).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
})
|
||||
@@ -200,7 +200,7 @@ describe('createVoiceInput', () => {
|
||||
const mock = MockRecognition.last!
|
||||
voice?.start()
|
||||
mock.onresult?.(makeResultEvent('hello world', true))
|
||||
expect(onTranscript).toHaveBeenCalledWith('hello world')
|
||||
expect(onTranscript).toHaveBeenCalledWith('hello world', expect.any(Number))
|
||||
})
|
||||
|
||||
it('autoSend=true appends \\r to the transcript (AC-A2.4)', () => {
|
||||
@@ -209,7 +209,7 @@ describe('createVoiceInput', () => {
|
||||
const mock = MockRecognition.last!
|
||||
voice?.start()
|
||||
mock.onresult?.(makeResultEvent('hello', true))
|
||||
expect(onTranscript).toHaveBeenCalledWith('hello\r')
|
||||
expect(onTranscript).toHaveBeenCalledWith('hello\r', expect.any(Number))
|
||||
})
|
||||
|
||||
it('autoSend=false does not append \\r (AC-A2.4)', () => {
|
||||
@@ -218,7 +218,7 @@ describe('createVoiceInput', () => {
|
||||
const mock = MockRecognition.last!
|
||||
voice?.start()
|
||||
mock.onresult?.(makeResultEvent('hello', true))
|
||||
expect(onTranscript).toHaveBeenCalledWith('hello')
|
||||
expect(onTranscript).toHaveBeenCalledWith('hello', expect.any(Number))
|
||||
})
|
||||
|
||||
it('omitting autoSend does not append \\r', () => {
|
||||
@@ -227,7 +227,16 @@ describe('createVoiceInput', () => {
|
||||
const mock = MockRecognition.last!
|
||||
voice?.start()
|
||||
mock.onresult?.(makeResultEvent('hello', true))
|
||||
expect(onTranscript).toHaveBeenCalledWith('hello')
|
||||
expect(onTranscript).toHaveBeenCalledWith('hello', expect.any(Number))
|
||||
})
|
||||
|
||||
it('forwards the top alternative ASR confidence as a 2nd arg (VC decision D)', () => {
|
||||
const onTranscript = vi.fn()
|
||||
const voice = createVoiceInput(onTranscript)
|
||||
const mock = MockRecognition.last!
|
||||
voice?.start()
|
||||
mock.onresult?.(makeResultEvent('hello', true))
|
||||
expect(onTranscript).toHaveBeenCalledWith('hello', 1.0)
|
||||
})
|
||||
|
||||
it('calls onInterim with interim transcript text', () => {
|
||||
|
||||
@@ -22,6 +22,8 @@ export default defineConfig({
|
||||
'public/tabs.ts',
|
||||
'public/preview-grid.ts',
|
||||
'public/title-util.ts',
|
||||
'public/voice-commands.ts',
|
||||
'public/voice-confirm.ts',
|
||||
],
|
||||
thresholds: {
|
||||
lines: 80,
|
||||
|
||||
Reference in New Issue
Block a user