Clean up

2025-11-15 08:58:48 +01:00
parent 4479c9ef91
commit 34a379750f
32 changed files with 7537 additions and 24 deletions
--- a/docs/plans/sprint_5_story_17.md
+++ b/docs/plans/sprint_5_story_17.md
@@ -0,0 +1,438 @@
+---
+story_id: story_17
+sprint_id: sprint_5
+parent_story_id: story_0
+status: not_started
+priority: P0
+assignee: backend
+created_date: 2025-11-09
+estimated_days: 10
+phase: 5
+---
+
+# Story 17: Testing, Optimization & Documentation
+
+**Parent Epic**: [Story 0](sprint_5_story_0.md) - Integrate Microsoft .NET MCP SDK
+**Phase**: 5 - Testing & Optimization (Week 7-8)
+**Priority**: P0 - Critical
+**Estimated Effort**: 10 days (2 weeks)
+**Dependencies**: Story 16 (Transport migration complete)
+
+## User Story
+
+**As** a backend team lead,
+**I want** comprehensive testing, performance optimization, and documentation for MCP SDK integration,
+**So that** we have production-ready code with zero regressions and complete knowledge transfer.
+
+## Business Value
+
+- **Quality Assurance**: Zero CRITICAL bugs in production
+- **Performance Validation**: Confirm ≥20% improvement target met
+- **Security Confidence**: Pass security audit with zero HIGH vulnerabilities
+- **Team Knowledge**: Complete documentation for future maintenance
+- **Production Ready**: Confident deployment with rollback plan
+
+## Week 7: Integration Testing & Bug Fixes
+
+### 7.1 End-to-End Integration Testing (2 days)
+
+**Test Scenarios**:
+
+1. **Claude Desktop Integration** (stdio)
+   - Initialize handshake
+   - List Tools and Resources
+   - Call each Tool (10 total)
+   - Query each Resource (11 total)
+   - Verify Diff Preview workflow
+   - Test approval/rejection flow
+
+2. **Web Client Integration** (HTTP/SSE)
+   - HTTP POST requests to MCP endpoints
+   - SSE connection for notifications
+   - API Key authentication
+   - CORS preflight handling
+
+3. **Multi-Tenant Isolation**
+   - Create 2 test tenants
+   - Verify cross-tenant data access blocked
+   - Test API Keys scoped to tenants
+   - Validate Global Query Filters
+
+4. **Error Handling**
+   - Invalid API Key
+   - Missing required parameters
+   - Malformed JSON-RPC requests
+   - Network failures (timeout, disconnect)
+
+**Deliverables**:
+- Integration test suite (50+ test cases)
+- Test execution report
+- Bug tracking spreadsheet
+
+---
+
+### 7.2 Performance Testing & Optimization (2 days)
+
+**Baseline vs. Target Performance**:
+
+| Metric | Baseline (Custom) | Target (SDK) | Actual (Measured) |
+|--------|-------------------|--------------|-------------------|
+| Tool Execution (P95) | 150ms | 120ms (20% faster) | TBD |
+| Resource Query (P95) | 120ms | 85ms (30% faster) | TBD |
+| Throughput | 70 req/s | 100 req/s | TBD |
+| Memory Usage | 180 MB | 150 MB (15% less) | TBD |
+
+**Performance Tests**:
+1. **Load Testing** (Apache Bench or k6)
+   - 100 concurrent users
+   - 1000 requests total
+   - Measure P50, P95, P99 latency
+
+2. **Memory Profiling** (dotMemory)
+   - Baseline memory usage
+   - Peak memory under load
+   - Memory leak detection
+
+3. **Database Query Performance**
+   - EF Core query analysis
+   - Index usage verification
+   - N+1 query detection
+
+**Optimization Tasks** (if needed):
+- Add missing database indexes
+- Optimize Redis cache keys
+- Tune connection pool sizes
+- Enable HTTP response compression
+
+**Deliverables**:
+- Performance benchmark report
+- Optimization recommendations
+- Before/after comparison charts
+
+---
+
+### 7.3 Security Audit (1 day)
+
+**Security Tests**:
+
+1. **Authentication & Authorization**
+   - API Key brute force attempt (rate limiting)
+   - Expired API Key handling
+   - Invalid API Key handling
+   - Field-level permission bypass attempts
+
+2. **Multi-Tenant Security**
+   - Cross-tenant data access attempts
+   - SQL injection attempts (parameterized queries)
+   - JSONB injection attempts
+   - Path traversal attempts (Resource URIs)
+
+3. **Input Validation**
+   - Tool parameter validation
+   - Resource URI validation
+   - JSON-RPC message validation
+   - CSRF protection (HTTP transport)
+
+4. **Sensitive Data Exposure**
+   - Log scrubbing (no API Keys, passwords logged)
+   - Error messages (no stack traces to clients)
+   - HTTPS enforcement (production)
+
+**Security Checklist**:
+- [ ] OWASP Top 10 compliance
+- [ ] API Key BCrypt hashed (never plaintext)
+- [ ] HTTPS only (production)
+- [ ] Rate limiting enabled
+- [ ] CORS properly configured
+- [ ] SQL injection protected (EF Core parameterized)
+- [ ] XSS protection (JSON escaping)
+- [ ] Sensitive data not logged
+
+**Deliverables**:
+- Security audit report (PASS/FAIL per category)
+- Vulnerability remediation plan (if issues found)
+
+---
+
+### 7.4 Bug Fixes & Refinement (1 day)
+
+**Bug Triage Process**:
+1. Categorize bugs (CRITICAL, HIGH, MEDIUM, LOW)
+2. Fix CRITICAL and HIGH bugs immediately
+3. Defer MEDIUM/LOW bugs to backlog (if not blocking)
+
+**Expected Bug Types**:
+- Integration test failures
+- Performance bottlenecks
+- Edge case handling (null values, empty lists)
+- Error message clarity
+
+**Deliverables**:
+- Bug fix commits
+- Updated test suite (all passing)
+
+---
+
+## Week 8: Documentation & Production Readiness
+
+### 8.1 Architecture Documentation Update (2 days)
+
+**Documents to Update**:
+
+1. **MCP Server Architecture** (`docs/architecture/mcp-server-architecture.md`)
+   - Add SDK integration section (500+ lines)
+   - Update component diagrams
+   - Document hybrid architecture decisions
+   - Add SDK vs. Custom comparison table
+
+2. **SDK Migration Guide** (`docs/guides/mcp-sdk-migration.md`) - NEW
+   - Step-by-step migration instructions
+   - Code examples (before/after)
+   - Common pitfalls and solutions
+   - Troubleshooting guide
+
+3. **ADR: MCP SDK Adoption** (`docs/architecture/adr/005-mcp-sdk-adoption.md`) - NEW
+   - Decision context
+   - Options considered
+   - Decision rationale
+   - Consequences and trade-offs
+
+**Deliverables**:
+- 3 updated/new documents (1500+ lines total)
+- Architecture diagrams (Mermaid or Draw.io)
+- Code examples repository
+
+---
+
+### 8.2 API Documentation (1 day)
+
+**OpenAPI/Swagger Updates**:
+
+1. **Tool Endpoints**
+   - Document all 10 Tools
+   - Parameter schemas
+   - Response examples
+   - Error codes
+
+2. **Resource Endpoints**
+   - Document all 11 Resources
+   - URI patterns
+   - Query parameters
+   - Response formats
+
+3. **Authentication**
+   - API Key header format
+   - Error responses (401, 403)
+   - Rate limiting headers
+
+**Example Swagger Annotation**:
+```csharp
+[McpTool(Name = "create_issue")]
+[SwaggerOperation(
+    Summary = "Create a new issue",
+    Description = "Creates Epic/Story/Task with Diff Preview approval workflow"
+)]
+[SwaggerResponse(200, "Pending change created", typeof(PendingChangeDto))]
+[SwaggerResponse(400, "Invalid parameters")]
+[SwaggerResponse(401, "Unauthorized")]
+public class CreateIssueTool { ... }
+```
+
+**Deliverables**:
+- Complete Swagger/OpenAPI spec
+- Interactive API documentation (Swagger UI)
+
+---
+
+### 8.3 Code Cleanup (1 day)
+
+**Cleanup Tasks**:
+
+1. **Remove Old Code**
+   - Delete `McpProtocolMiddleware.cs`
+   - Delete `ApiKeyAuthMiddleware.cs`
+   - Delete `IMcpTool.cs` interface
+   - Delete `IMcpResource.cs` interface
+   - Remove obsolete NuGet packages
+
+2. **Update Code Comments**
+   - Add XML documentation to public APIs
+   - Update inline comments
+   - Remove TODO/FIXME comments
+
+3. **Code Style Enforcement**
+   - Run `dotnet format`
+   - Fix StyleCop warnings
+   - Consistent naming conventions
+
+4. **Dependency Audit**
+   - Remove unused NuGet packages
+   - Update vulnerable packages
+   - Lock SDK version (avoid auto-updates)
+
+**Deliverables**:
+- Clean codebase (zero warnings)
+- Updated `.csproj` files
+- Dependency lock file
+
+---
+
+### 8.4 Production Readiness & Deployment (2 days)
+
+**Production Readiness Checklist**:
+
+- [ ] **Testing**
+  - [ ] All integration tests pass (>80% coverage)
+  - [ ] Performance benchmarks meet targets
+  - [ ] Security audit passed (0 CRITICAL, 0 HIGH)
+
+- [ ] **Documentation**
+  - [ ] Architecture docs updated
+  - [ ] API docs complete
+  - [ ] Migration guide published
+
+- [ ] **Code Quality**
+  - [ ] Code reviewed and approved
+  - [ ] Zero StyleCop warnings
+  - [ ] No TODO/FIXME comments
+
+- [ ] **Configuration**
+  - [ ] Environment variables documented
+  - [ ] Secrets managed (Azure Key Vault / AWS Secrets Manager)
+  - [ ] Feature flags configured
+
+- [ ] **Deployment**
+  - [ ] Staging deployment successful
+  - [ ] Smoke tests passed
+  - [ ] Rollback plan documented
+
+**Staging Deployment**:
+1. Deploy to staging environment
+2. Run smoke tests (5 critical scenarios)
+3. Monitor logs for errors
+4. Validate performance metrics
+
+**Smoke Test Scenarios**:
+- Claude Desktop can connect (stdio)
+- Web client can connect (HTTP/SSE)
+- Create issue Tool works
+- List projects Resource works
+- Diff Preview approval flow works
+
+**Rollback Plan**:
+- Database: No schema changes (safe to rollback)
+- Code: Git tag before deployment, revert if needed
+- Configuration: Feature flag to disable SDK (fallback to custom)
+
+**Deliverables**:
+- Production readiness report (PASS/FAIL)
+- Staging deployment success confirmation
+- Rollback plan document
+
+---
+
+### 8.5 Final Code Review & Sign-Off (1 day)
+
+**Review Checklist**:
+- [ ] Architect review (architecture compliance)
+- [ ] Tech lead review (code quality)
+- [ ] Security review (security best practices)
+- [ ] QA review (test coverage)
+- [ ] Product manager sign-off (business value delivered)
+
+**Sign-Off Criteria**:
+- All acceptance criteria met (Epic level)
+- Performance targets achieved (≥20% improvement)
+- Security audit passed (0 CRITICAL, 0 HIGH)
+- Documentation complete
+- Production deployment plan approved
+
+**Deliverables**:
+- Code review approval
+- Sign-off document (all stakeholders)
+
+---
+
+## Acceptance Criteria
+
+### Testing
+- [ ] Integration test suite complete (50+ tests)
+- [ ] All integration tests pass (100%)
+- [ ] Performance benchmarks meet targets (≥20% improvement)
+- [ ] Security audit passed (0 CRITICAL, 0 HIGH)
+
+### Documentation
+- [ ] Architecture docs updated (1500+ lines)
+- [ ] API docs complete (Swagger)
+- [ ] Migration guide published
+- [ ] ADR documented
+
+### Code Quality
+- [ ] Old code removed (330+ lines)
+- [ ] Zero StyleCop warnings
+- [ ] XML documentation complete
+- [ ] Code reviewed and approved
+
+### Production Readiness
+- [ ] Staging deployment successful
+- [ ] Smoke tests passed
+- [ ] Rollback plan documented
+- [ ] All stakeholders signed off
+
+## Tasks Breakdown
+
+### Week 7: Testing & Optimization
+- [ ] [Task 1](sprint_5_story_17_task_1.md) - End-to-end integration testing - 2 days
+- [ ] [Task 2](sprint_5_story_17_task_2.md) - Performance testing and optimization - 2 days
+- [ ] [Task 3](sprint_5_story_17_task_3.md) - Security audit - 1 day
+- [ ] [Task 4](sprint_5_story_17_task_4.md) - Bug fixes and refinement - 1 day
+
+### Week 8: Documentation & Deployment
+- [ ] [Task 5](sprint_5_story_17_task_5.md) - Architecture documentation update - 2 days
+- [ ] [Task 6](sprint_5_story_17_task_6.md) - API documentation (Swagger) - 1 day
+- [ ] [Task 7](sprint_5_story_17_task_7.md) - Code cleanup - 1 day
+- [ ] [Task 8](sprint_5_story_17_task_8.md) - Production readiness and staging deployment - 2 days
+- [ ] [Task 9](sprint_5_story_17_task_9.md) - Final code review and sign-off - 1 day
+
+**Progress**: 0/9 tasks completed (0%)
+
+## Success Metrics
+
+### Quality Metrics
+- **Test Coverage**: ≥80%
+- **Integration Tests**: 50+ tests, 100% passing
+- **Security**: 0 CRITICAL, 0 HIGH vulnerabilities
+- **Code Quality**: 0 StyleCop warnings
+
+### Performance Metrics
+- **Tool Execution**: ≥20% faster (target: 120ms P95)
+- **Resource Query**: ≥30% faster (target: 85ms P95)
+- **Throughput**: ≥100 req/s (vs. 70 baseline)
+- **Memory**: ≤150 MB (vs. 180 baseline)
+
+### Documentation Metrics
+- **Architecture Docs**: 1500+ lines updated
+- **API Docs**: 100% Swagger coverage
+- **Migration Guide**: Complete step-by-step
+- **Code Examples**: 10+ before/after samples
+
+## Definition of Done
+
+- [ ] All Week 7 tasks complete (testing, optimization, security, bug fixes)
+- [ ] All Week 8 tasks complete (docs, cleanup, deployment, sign-off)
+- [ ] Integration tests pass (100%)
+- [ ] Performance targets met (≥20% improvement)
+- [ ] Security audit passed (0 CRITICAL, 0 HIGH)
+- [ ] Documentation complete (architecture, API, migration guide)
+- [ ] Code cleanup complete (old code removed, warnings fixed)
+- [ ] Staging deployment successful
+- [ ] Smoke tests passed
+- [ ] All stakeholders signed off
+
+---
+
+**Created**: 2025-11-09 by Product Manager Agent
+**Owner**: Backend Team Lead
+**Start Date**: 2026-01-06 (Week 7)
+**Target Date**: 2026-01-17 (End of Week 8)
+**Status**: Not Started