smart-support/docs/phases/phase-3-dev-log.md

Phase 3: OpenAPI Auto-Discovery -- Development Log

Status: COMPLETED Phase branch: phase-3/openapi-discovery Date started: 2026-03-30 Date completed: 2026-03-30 Related plan section: Phase 3 in DEVELOPMENT-PLAN

What Was Built

• SSRF protection module with private IP blocking, DNS rebinding defense, redirect chain validation
• OpenAPI spec fetcher with SSRF protection, JSON/YAML auto-detection, 10MB size limit
• Structural OpenAPI spec validator (3.0.x and 3.1.x)
• Endpoint parser with $ref resolution, parameter extraction, auto-generated operation IDs
• Heuristic + LLM endpoint classifier with fallback (GET=read, POST/PUT/PATCH/DELETE=write)
• Review API (FastAPI router at /api/openapi) for import jobs, classification review, approval
• Tool code generator producing @tool-decorated async functions with httpx
• Agent YAML generator grouping endpoints by classification
• Import orchestrator coordinating the full pipeline (fetch -> validate -> parse -> classify)
• In-memory job store for import state tracking

Code Structure

New files created:

File	Purpose	Lines
app/openapi/__init__.py	Module entry point	2
app/openapi/models.py	Frozen dataclasses: EndpointInfo, ClassificationResult, ImportJob, etc.	68
app/openapi/ssrf.py	SSRF protection (validate_url, safe_fetch, DNS resolution)	162
app/openapi/fetcher.py	SSRF-safe spec fetching with format auto-detection	94
app/openapi/validator.py	Structural OpenAPI spec validation	52
app/openapi/parser.py	Endpoint extraction with $ref resolution	153
app/openapi/classifier.py	HeuristicClassifier + LLMClassifier with Protocol	164
app/openapi/review_api.py	FastAPI router for import/review workflow	180
app/openapi/generator.py	@tool code generation + YAML generation	157
app/openapi/importer.py	Async import pipeline orchestrator	117

Modified files:

• app/main.py -- Wired openapi_router
• pyproject.toml -- Added openapi-spec-validator, pytest-httpx dependencies

Test files:

• tests/unit/test_ssrf.py (42 tests)
• tests/unit/openapi/test_fetcher.py (7 tests)
• tests/unit/openapi/test_validator.py (8 tests)
• tests/unit/openapi/test_parser.py (10 tests)
• tests/unit/openapi/test_classifier.py (18 tests)
• tests/unit/openapi/test_review_api.py (17 tests)
• tests/unit/openapi/test_generator.py (16 tests)
• tests/integration/test_import_pipeline.py (7 tests)

Test Coverage

• Unit tests: 118 new tests across 8 test files
• Integration tests: 7 new tests for full import pipeline
• Total: 322 tests passing (125 new + 197 existing)
• Overall coverage: 93.23% (requirement: 80%)

Per-module coverage:

• classifier.py: 98%
• fetcher.py: 84%
• generator.py: 96%
• importer.py: 100%
• models.py: 100%
• parser.py: 89%
• review_api.py: 100%
• ssrf.py: 90%
• validator.py: 88%

Deviations from Plan

1. No custom tool base class (3.0.2 skipped): Architecture doc explicitly says "do not build custom tool base class." Generated tools use @tool decorator directly.
2. Structural validator instead of openapi-spec-validator: Implemented a lightweight structural validator instead of wrapping the external library. The library is still in dependencies for potential future use.
3. In-memory job store: Used dict-based in-memory store instead of database. Can migrate to PostgreSQL in Phase 5 if needed.
4. Frontend Review UI deferred: ReviewPage.tsx not implemented in this phase; backend API is complete and testable via HTTP.

Known Issues / Tech Debt

• Frontend Review UI (3.4.2) deferred -- API is ready, UI needs Phase 5
• Generated tool code uses string templates -- works for simple REST but may need AST-based generation for complex scenarios
• LLMClassifier prompt could be tuned with real-world examples
• No rate limiting on review API endpoints yet
• openapi-spec-validator library added but not actively used (structural validator is simpler)