feat: complete phase 2 -- multi-agent routing, interrupt TTL, escalation, templates
- Intent classification with LLM structured output (single/multi/ambiguous) - Discount agent with apply_discount and generate_coupon tools - Interrupt manager with 30-min TTL auto-expiration and retry prompts - Webhook escalation module with exponential backoff retry (max 3) - Three vertical industry templates (e-commerce, SaaS, fintech) - Template loading in AgentRegistry - Enhanced supervisor prompt with dynamic agent descriptions - 153 tests passing, 90.18% coverage
This commit is contained in:
Yaojia Wang
@@ -276,6 +276,9 @@ Smart Support 是一个 AI 客服行动层框架。核心价值主张: "粘贴
|
||||
|
||||
## Phase 2: 多 Agent 路由 + 安全层 (第 3-4 周)
|
||||
|
||||
> Status: COMPLETED (2026-03-30)
|
||||
> Dev log: [Phase 2 Dev Log](phases/phase-2-dev-log.md)
|
||||
|
||||
### 目标
|
||||
|
||||
完善 Supervisor 的意图分类和多 Agent 路由能力, 实现 Webhook 升级、垂直行业模板、中断超时处理。
|
||||
@@ -289,22 +292,22 @@ Smart Support 是一个 AI 客服行动层框架。核心价值主张: "粘贴
|
||||
|
||||
#### 2.1 Supervisor 路由增强 (预计 2 天)
|
||||
|
||||
- [ ] **2.1.1** 实现 LLM 结构化输出的意图分类 (基于 Agent 描述选择)
|
||||
- [x] **2.1.1** 实现 LLM 结构化输出的意图分类 (基于 Agent 描述选择)
|
||||
- 文件: `backend/app/graph.py` (增强)
|
||||
- 工作量: M (4 小时)
|
||||
- 依赖: Phase 1 完成
|
||||
- 风险: 中 -- 路由准确率需要评估
|
||||
- [ ] **2.1.2** 实现多意图请求处理 ("取消订单并给我一个折扣" -> 顺序执行)
|
||||
- [x] **2.1.2** 实现多意图请求处理 ("取消订单并给我一个折扣" -> 顺序执行)
|
||||
- 文件: `backend/app/graph.py` (增强)
|
||||
- 工作量: M (6 小时)
|
||||
- 依赖: 2.1.1
|
||||
- 风险: 高 -- 多意图原子性问题 (全部成功 vs. 部分失败升级)
|
||||
- [ ] **2.1.3** 实现歧义意图处理 (无法分类时询问澄清问题)
|
||||
- [x] **2.1.3** 实现歧义意图处理 (无法分类时询问澄清问题)
|
||||
- 文件: `backend/app/agents/fallback.py` (增强)
|
||||
- 工作量: S (2 小时)
|
||||
- 依赖: 2.1.1
|
||||
- 风险: 低
|
||||
- [ ] **2.1.4** 编写路由测试 (正确路由、多意图、歧义、回退)
|
||||
- [x] **2.1.4** 编写路由测试 (正确路由、多意图、歧义、回退)
|
||||
- 文件: `backend/tests/test_routing.py`
|
||||
- 工作量: M (4 小时)
|
||||
- 依赖: 2.1.1, 2.1.2, 2.1.3
|
||||
@@ -312,12 +315,12 @@ Smart Support 是一个 AI 客服行动层框架。核心价值主张: "粘贴
|
||||
|
||||
#### 2.2 Mock 折扣 Agent (预计 0.5 天)
|
||||
|
||||
- [ ] **2.2.1** 创建 Mock 折扣 Agent + 工具 (apply_discount, generate_coupon)
|
||||
- [x] **2.2.1** 创建 Mock 折扣 Agent + 工具 (apply_discount, generate_coupon)
|
||||
- 文件: `backend/app/agents/discount.py`
|
||||
- 工作量: S (2 小时)
|
||||
- 依赖: Phase 1
|
||||
- 风险: 低
|
||||
- [ ] **2.2.2** 更新 agents.yaml 添加折扣 Agent 配置
|
||||
- [x] **2.2.2** 更新 agents.yaml 添加折扣 Agent 配置
|
||||
- 文件: `backend/agents.yaml`
|
||||
- 工作量: S (30 分钟)
|
||||
- 依赖: 2.2.1
|
||||
@@ -325,17 +328,17 @@ Smart Support 是一个 AI 客服行动层框架。核心价值主张: "粘贴
|
||||
|
||||
#### 2.3 中断超时处理 (预计 1 天)
|
||||
|
||||
- [ ] **2.3.1** 实现 30 分钟 TTL 自动取消机制
|
||||
- [x] **2.3.1** 实现 30 分钟 TTL 自动取消机制
|
||||
- 文件: `backend/app/interrupt_manager.py`
|
||||
- 工作量: M (4 小时)
|
||||
- 依赖: Phase 1 (interrupt 基础)
|
||||
- 风险: 中 -- 定时器精度和状态一致性
|
||||
- [ ] **2.3.2** 实现过期后重试提示 (重新评估当前状态后重新发起)
|
||||
- [x] **2.3.2** 实现过期后重试提示 (重新评估当前状态后重新发起)
|
||||
- 文件: `backend/app/interrupt_manager.py` (扩展)
|
||||
- 工作量: M (3 小时)
|
||||
- 依赖: 2.3.1
|
||||
- 风险: 中
|
||||
- [ ] **2.3.3** 编写中断超时测试
|
||||
- [x] **2.3.3** 编写中断超时测试
|
||||
- 文件: `backend/tests/test_interrupt.py`
|
||||
- 工作量: S (2 小时)
|
||||
- 依赖: 2.3.1, 2.3.2
|
||||
@@ -343,17 +346,17 @@ Smart Support 是一个 AI 客服行动层框架。核心价值主张: "粘贴
|
||||
|
||||
#### 2.4 Webhook 升级 (预计 1 天)
|
||||
|
||||
- [ ] **2.4.1** 实现 Webhook 升级模块 (HTTP POST 到配置的 URL, 包含完整对话上下文)
|
||||
- [x] **2.4.1** 实现 Webhook 升级模块 (HTTP POST 到配置的 URL, 包含完整对话上下文)
|
||||
- 文件: `backend/app/escalation.py`
|
||||
- 工作量: M (3 小时)
|
||||
- 依赖: Phase 1
|
||||
- 风险: 低
|
||||
- [ ] **2.4.2** 实现 Webhook 重试机制 (指数退避, 最多 3 次)
|
||||
- [x] **2.4.2** 实现 Webhook 重试机制 (指数退避, 最多 3 次)
|
||||
- 文件: `backend/app/escalation.py` (扩展)
|
||||
- 工作量: S (2 小时)
|
||||
- 依赖: 2.4.1
|
||||
- 风险: 低
|
||||
- [ ] **2.4.3** 编写 Webhook 测试 (成功发送、目标不可达、重试)
|
||||
- [x] **2.4.3** 编写 Webhook 测试 (成功发送、目标不可达、重试)
|
||||
- 文件: `backend/tests/test_escalation.py`
|
||||
- 工作量: S (2 小时)
|
||||
- 依赖: 2.4.1, 2.4.2
|
||||
@@ -361,22 +364,22 @@ Smart Support 是一个 AI 客服行动层框架。核心价值主张: "粘贴
|
||||
|
||||
#### 2.5 垂直行业模板 (预计 0.5 天)
|
||||
|
||||
- [ ] **2.5.1** 创建电商模板 YAML (订单查询、订单操作、折扣)
|
||||
- [x] **2.5.1** 创建电商模板 YAML (订单查询、订单操作、折扣)
|
||||
- 文件: `backend/templates/e-commerce.yaml`
|
||||
- 工作量: S (1 小时)
|
||||
- 依赖: 1.2.2
|
||||
- 风险: 低
|
||||
- [ ] **2.5.2** 创建 SaaS 模板 YAML (账户查询、订阅管理、计费)
|
||||
- [x] **2.5.2** 创建 SaaS 模板 YAML (账户查询、订阅管理、计费)
|
||||
- 文件: `backend/templates/saas.yaml`
|
||||
- 工作量: S (1 小时)
|
||||
- 依赖: 1.2.2
|
||||
- 风险: 低
|
||||
- [ ] **2.5.3** 创建 Fintech 模板 YAML (交易查询、争议处理)
|
||||
- [x] **2.5.3** 创建 Fintech 模板 YAML (交易查询、争议处理)
|
||||
- 文件: `backend/templates/fintech.yaml`
|
||||
- 工作量: S (1 小时)
|
||||
- 依赖: 1.2.2
|
||||
- 风险: 低
|
||||
- [ ] **2.5.4** 实现模板加载逻辑 (选择模板 -> 覆盖 agents.yaml)
|
||||
- [x] **2.5.4** 实现模板加载逻辑 (选择模板 -> 覆盖 agents.yaml)
|
||||
- 文件: `backend/app/registry.py` (扩展)
|
||||
- 工作量: S (2 小时)
|
||||
- 依赖: 2.5.1, 2.5.2, 2.5.3
|
||||
|
||||
76
docs/phases/phase-2-dev-log.md
Normal file
76
docs/phases/phase-2-dev-log.md
Normal file
@@ -0,0 +1,76 @@
|
||||
# Phase 2: Multi-Agent Routing + Safety Layer -- Development Log
|
||||
|
||||
> Status: COMPLETED
|
||||
> Phase branch: `phase-2/multi-agent-safety`
|
||||
> Date started: 2026-03-30
|
||||
> Date completed: 2026-03-30
|
||||
> Related plan section: [Phase 2 in DEVELOPMENT-PLAN](../DEVELOPMENT-PLAN.md#phase-2-多-agent-路由--安全层-第-3-4-周)
|
||||
|
||||
## What Was Built
|
||||
|
||||
- **Intent Classification** (`app/intent.py`): LLM structured output-based intent classifier with Pydantic models (`IntentTarget`, `ClassificationResult`). Supports single-intent, multi-intent, and ambiguity detection with configurable confidence threshold.
|
||||
- **Discount Agent** (`app/agents/discount.py`): Mock agent with `apply_discount` (write + interrupt) and `generate_coupon` (read) tools. Validates discount range (1-100%).
|
||||
- **Interrupt Manager** (`app/interrupt_manager.py`): TTL-based interrupt tracking with 30-minute auto-expiration. Provides `register`, `check_status`, `resolve`, `cleanup_expired`, and `generate_retry_prompt` methods. Complements SessionManager.
|
||||
- **Webhook Escalation** (`app/escalation.py`): HTTP POST escalation with exponential backoff retry (max 3 attempts). Includes `WebhookEscalator` and `NoOpEscalator` implementations behind `EscalationService` protocol.
|
||||
- **Enhanced Supervisor Routing** (`app/graph.py`): Supervisor prompt now includes dynamic agent descriptions. Intent classifier attached to graph for use by ws_handler routing layer. Multi-intent hint injection for sequential execution.
|
||||
- **Vertical Templates**: Three industry YAML templates (e-commerce, SaaS, fintech) in `backend/templates/`.
|
||||
- **Template Loading** (`app/registry.py`): `load_template()` and `list_templates()` class methods for template-based agent configuration.
|
||||
- **WebSocket Integration** (`app/ws_handler.py`): Ambiguous intent sends clarification message. Interrupt TTL checked before resume -- expired interrupts return retry prompt. Interrupt manager registration on interrupt detection.
|
||||
|
||||
## Code Structure
|
||||
|
||||
New files:
|
||||
- `backend/app/intent.py` -- Intent classification models and LLM classifier
|
||||
- `backend/app/agents/discount.py` -- Discount agent tools
|
||||
- `backend/app/interrupt_manager.py` -- Interrupt TTL management
|
||||
- `backend/app/escalation.py` -- Webhook escalation with retry
|
||||
- `backend/templates/e-commerce.yaml` -- E-commerce agent template
|
||||
- `backend/templates/saas.yaml` -- SaaS agent template
|
||||
- `backend/templates/fintech.yaml` -- Fintech agent template
|
||||
|
||||
Modified files:
|
||||
- `backend/app/graph.py` -- Intent classifier integration, dynamic supervisor prompt
|
||||
- `backend/app/agents/__init__.py` -- Registered discount tools
|
||||
- `backend/app/agents/fallback.py` -- Updated capability list
|
||||
- `backend/app/registry.py` -- Template loading methods
|
||||
- `backend/app/config.py` -- Webhook, template settings
|
||||
- `backend/app/ws_handler.py` -- Interrupt manager + intent classification integration
|
||||
- `backend/app/main.py` -- Wiring new modules, template loading, version bump to 0.2.0
|
||||
- `backend/agents.yaml` -- Added discount agent
|
||||
- `backend/pyproject.toml` -- Added httpx to main dependencies
|
||||
|
||||
Test files added:
|
||||
- `tests/unit/test_intent.py` (11 tests)
|
||||
- `tests/unit/test_discount.py` (13 tests)
|
||||
- `tests/unit/test_interrupt_manager.py` (14 tests)
|
||||
- `tests/unit/test_escalation.py` (11 tests)
|
||||
- `tests/unit/test_templates.py` (9 tests)
|
||||
|
||||
Test files updated:
|
||||
- `tests/unit/test_graph.py` -- Tests for classifier attachment and classify_intent
|
||||
- `tests/unit/test_ws_handler.py` -- Tests for interrupt manager and clarification flow
|
||||
- `tests/unit/test_main.py` -- Updated version check
|
||||
|
||||
## Test Coverage
|
||||
|
||||
- Total tests: 153 (87 Phase 1 + 66 Phase 2)
|
||||
- Overall coverage: 90.18%
|
||||
- New module coverage:
|
||||
- intent.py: 100%
|
||||
- discount.py: 96%
|
||||
- interrupt_manager.py: 100%
|
||||
- escalation.py: 100%
|
||||
- graph.py: 100%
|
||||
- registry.py: 97%
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
- Multi-intent handling uses supervisor prompt hint injection rather than a fully custom pre-routing graph node. This is simpler and leverages the existing `langgraph-supervisor` routing rather than fighting it.
|
||||
- Webhook escalation is wired to main.py app.state but not yet connected to a specific agent tool (escalation trigger). The module is ready for use -- integration with fallback agent's escalation path is straightforward but deferred to avoid scope creep.
|
||||
- The `escalate_to_human` tool mentioned in the plan was not created. The escalation module works standalone and can be triggered from ws_handler or agent tools in Phase 5.
|
||||
|
||||
## Known Issues / Tech Debt
|
||||
|
||||
- SaaS and fintech templates reference tool names (`get_account_status`, `change_plan`, etc.) that don't have implementations. These are configuration blueprints for future use.
|
||||
- Interrupt manager cleanup is not called on a schedule -- `cleanup_expired()` exists but no periodic task invokes it. Consider adding a background task in Phase 5.
|
||||
- `main.py` coverage is 44% due to lifespan requiring real DB connection. Integration tests would cover this.
|
||||
Reference in New Issue
Block a user