feat: initial project setup with planning docs

Smart Support - AI customer service action layer framework.
Includes design doc, CEO plan, eng review, test plan, and README.
This commit is contained in:
Yaojia Wang
2026-03-29 21:11:36 +02:00
commit f93e8baef1
8 changed files with 762 additions and 0 deletions

47
eng-review-test-plan.md Normal file
View File

@@ -0,0 +1,47 @@
# Test Plan
Generated by /plan-eng-review on 2026-03-29
Branch: unknown
Repo: smart-support
## Affected Pages/Routes
- WebSocket `/ws` — Main chat endpoint. Test connection, message flow, streaming, interrupt responses, reconnection
- GET `/` — Chat UI serving. Test static file serving
- GET `/api/replay/{thread_id}` — Conversation replay. Test pagination, 404, structured timeline JSON
- GET `/api/analytics` — Analytics dashboard. Test resolution rate, agent usage, escalation %, zero state
- POST (webhook URL) — Escalation webhook. Test payload shape, retry on failure
## Key Interactions to Verify
- Send message via WebSocket → receive streaming tokens back
- Send message triggering write action → receive interrupt prompt → send approval → receive confirmation
- Send message triggering write action → receive interrupt prompt → send rejection → receive cancellation
- Multi-turn: send order lookup, then "cancel that one" → agent resolves reference from context
- Paste OpenAPI spec URL → tools generated → agents registered → chat works with new tools
- View conversation replay → see step-by-step agent decisions and tool calls
- View analytics dashboard → see resolution rate, agent usage metrics
## Edge Cases
- Empty message → graceful error response
- Very long message (10K+ chars) → handled without crash
- Rapid-fire messages → no race condition in graph execution
- WebSocket disconnect mid-stream → server cleans up gracefully, client reconnects + resumes interrupt
- LLM API timeout → error message returned to client
- Invalid YAML agent config → clear startup error with file/line reference
- MCP tool timeout → timeout error returned to user with agent name
- Cancel already-cancelled order → appropriate error message
- Ambiguous intent with no context → clarifying question asked
- OpenAPI spec with 100+ endpoints → generation completes without timeout
- Invalid/malformed OpenAPI spec → clear error with what's wrong
- SSRF attempt (private IP, localhost, 169.254.x) → blocked with clear error
- DNS rebinding attack → blocked
- Replay of thread with 200+ turns → pagination works, no slow query
- Analytics with no conversations → zero state displayed correctly
- Webhook URL unreachable → retry with backoff, log failure
- Multi-intent request ("cancel my order and give me a discount") → sequenced correctly
## Critical Paths (E2E)
- Happy path: "What's the status of order 1042?" → lookup → answer
- Cancel with approval: "Cancel order 1042" → interrupt → approve → cancel confirmed
- Cancel with rejection: "Cancel order 1042" → interrupt → reject → no action taken
- Multi-turn context: "Check order 1042" then "cancel that one" → correct entity resolution
- OpenAPI import: paste spec URL → tools generated → use new tool in chat
- Conversation replay: select completed conversation → step-by-step replay renders correctly