feat: initial project setup with planning docs
Smart Support - AI customer service action layer framework. Includes design doc, CEO plan, eng review, test plan, and README.
This commit is contained in:
47
eng-review-test-plan.md
Normal file
47
eng-review-test-plan.md
Normal file
@@ -0,0 +1,47 @@
|
||||
# Test Plan
|
||||
Generated by /plan-eng-review on 2026-03-29
|
||||
Branch: unknown
|
||||
Repo: smart-support
|
||||
|
||||
## Affected Pages/Routes
|
||||
- WebSocket `/ws` — Main chat endpoint. Test connection, message flow, streaming, interrupt responses, reconnection
|
||||
- GET `/` — Chat UI serving. Test static file serving
|
||||
- GET `/api/replay/{thread_id}` — Conversation replay. Test pagination, 404, structured timeline JSON
|
||||
- GET `/api/analytics` — Analytics dashboard. Test resolution rate, agent usage, escalation %, zero state
|
||||
- POST (webhook URL) — Escalation webhook. Test payload shape, retry on failure
|
||||
|
||||
## Key Interactions to Verify
|
||||
- Send message via WebSocket → receive streaming tokens back
|
||||
- Send message triggering write action → receive interrupt prompt → send approval → receive confirmation
|
||||
- Send message triggering write action → receive interrupt prompt → send rejection → receive cancellation
|
||||
- Multi-turn: send order lookup, then "cancel that one" → agent resolves reference from context
|
||||
- Paste OpenAPI spec URL → tools generated → agents registered → chat works with new tools
|
||||
- View conversation replay → see step-by-step agent decisions and tool calls
|
||||
- View analytics dashboard → see resolution rate, agent usage metrics
|
||||
|
||||
## Edge Cases
|
||||
- Empty message → graceful error response
|
||||
- Very long message (10K+ chars) → handled without crash
|
||||
- Rapid-fire messages → no race condition in graph execution
|
||||
- WebSocket disconnect mid-stream → server cleans up gracefully, client reconnects + resumes interrupt
|
||||
- LLM API timeout → error message returned to client
|
||||
- Invalid YAML agent config → clear startup error with file/line reference
|
||||
- MCP tool timeout → timeout error returned to user with agent name
|
||||
- Cancel already-cancelled order → appropriate error message
|
||||
- Ambiguous intent with no context → clarifying question asked
|
||||
- OpenAPI spec with 100+ endpoints → generation completes without timeout
|
||||
- Invalid/malformed OpenAPI spec → clear error with what's wrong
|
||||
- SSRF attempt (private IP, localhost, 169.254.x) → blocked with clear error
|
||||
- DNS rebinding attack → blocked
|
||||
- Replay of thread with 200+ turns → pagination works, no slow query
|
||||
- Analytics with no conversations → zero state displayed correctly
|
||||
- Webhook URL unreachable → retry with backoff, log failure
|
||||
- Multi-intent request ("cancel my order and give me a discount") → sequenced correctly
|
||||
|
||||
## Critical Paths (E2E)
|
||||
- Happy path: "What's the status of order 1042?" → lookup → answer
|
||||
- Cancel with approval: "Cancel order 1042" → interrupt → approve → cancel confirmed
|
||||
- Cancel with rejection: "Cancel order 1042" → interrupt → reject → no action taken
|
||||
- Multi-turn context: "Check order 1042" then "cancel that one" → correct entity resolution
|
||||
- OpenAPI import: paste spec URL → tools generated → use new tool in chat
|
||||
- Conversation replay: select completed conversation → step-by-step replay renders correctly
|
||||
Reference in New Issue
Block a user