feat: voice command mapping — context-gated approve/reject over voice

While a tool-permission gate is held on the active tab, spoken confirm
phrases resolve it via the existing approve/reject WS channel — no server
change (byte-shuttle intact). Everything else stays ordinary dictation.

Safety: whole-utterance-exact matching + leading-negation guard + reject
precedence + confidence gate (approve only) + a cancellable 1.5s confirm
window whose commit re-validates gate identity/epoch/connectivity (TOCTOU
guard) + stale-gate epoch bound at PTT-start.

New pure modules public/voice-commands.ts and public/voice-confirm.ts at
100% coverage; wiring in voice/terminal-session/tabs/main. 1307 tests pass,
typecheck clean, build:web OK. Independent code + security reviews flagged a
confirm-window TOCTOU and an unwired cancel path — both fixed and covered.

Plan: docs/PLAN_VOICE_COMMANDS.md.
This commit is contained in:
Yaojia Wang
2026-07-01 10:34:51 +02:00
parent 03612323c0
commit e4c327e25e
13 changed files with 1193 additions and 19 deletions

View File

@@ -108,11 +108,13 @@ function getSRConstructor(): SpeechRecognitionCtor | undefined {
* Returns null when the browser does not support SpeechRecognition — the
* caller should hide the mic UI in that case (AC-A2.3).
*
* @param onTranscript Receives the final recognised text (+ '\r' if autoSend).
* @param onTranscript Receives the final recognised text (+ '\r' if autoSend),
* plus the top alternative's ASR confidence (0-1) as an
* optional 2nd arg when the engine reports one (VC decision D).
* @param opts See VoiceInputOptions.
*/
export function createVoiceInput(
onTranscript: (text: string) => void,
onTranscript: (text: string, confidence?: number) => void,
opts?: VoiceInputOptions,
): VoiceInput | null {
const SRClass = getSRConstructor()
@@ -130,11 +132,17 @@ export function createVoiceInput(
if (disposed) return
let interimText = ''
let finalText = ''
let finalConfidence: number | undefined
for (let i = event.resultIndex; i < event.results.length; i++) {
const result = event.results[i]
const text = result?.[0]?.transcript ?? ''
const alt = result?.[0]
const text = alt?.transcript ?? ''
if (result?.isFinal) {
finalText += text
// VC (decision D): forward the top alternative's confidence so the
// caller can gate risk-asymmetric decisions (e.g. approve). Still
// transcription-only — no decision-making happens in this module.
finalConfidence = alt?.confidence ?? finalConfidence
} else {
interimText += text
}
@@ -143,7 +151,7 @@ export function createVoiceInput(
opts.onInterim(interimText)
}
if (finalText) {
onTranscript(opts?.autoSend ? finalText + '\r' : finalText)
onTranscript(opts?.autoSend ? finalText + '\r' : finalText, finalConfidence)
}
}