invoice-master-poc-v2/docs/web-refactoring-status.md

Web Directory Refactoring - Current Status

✅ Completed Steps

1. Directory Structure Created

src/web/
├── api/
│   ├── v1/
│   │   ├── admin/      (documents.py, annotations.py, training.py)
│   │   ├── async_api/  (routes.py)
│   │   ├── batch/      (routes.py)
│   │   └── routes.py   (public inference API)
├── schemas/
│   ├── admin.py        (admin schemas)
│   ├── inference.py    (inference + async schemas)
│   └── common.py       (ErrorResponse)
├── services/
│   ├── autolabel.py
│   ├── async_processing.py
│   ├── batch_upload.py
│   └── inference.py
├── core/
│   ├── auth.py
│   ├── rate_limiter.py
│   └── scheduler.py
└── workers/
    ├── async_queue.py
    └── batch_queue.py

2. Files Copied and Imports Updated

Admin API (✅ Complete)

• admin_routes.py → api/v1/admin/documents.py (imports updated)
• admin_annotation_routes.py → api/v1/admin/annotations.py (imports updated)
• admin_training_routes.py → api/v1/admin/training.py (imports updated)
• api/v1/admin/__init__.py created with exports

Public & Async API (✅ Complete)

• routes.py → api/v1/routes.py (imports updated)
• async_routes.py → api/v1/async_api/routes.py (imports updated)
• batch_upload_routes.py → api/v1/batch/routes.py (copied, imports pending)

Schemas (✅ Complete)

• admin_schemas.py → schemas/admin.py
• schemas.py → schemas/inference.py
• schemas/common.py created
• schemas/__init__.py created with exports

Services (✅ Complete)

• admin_autolabel.py → services/autolabel.py
• async_service.py → services/async_processing.py
• batch_upload_service.py → services/batch_upload.py
• services.py → services/inference.py
• services/__init__.py created

Core Components (✅ Complete)

• admin_auth.py → core/auth.py
• rate_limiter.py → core/rate_limiter.py
• admin_scheduler.py → core/scheduler.py
• core/__init__.py created

Workers (✅ Complete)

• async_queue.py → workers/async_queue.py
• batch_queue.py → workers/batch_queue.py
• workers/__init__.py created

Main App (✅ Complete)

• app.py imports updated to use new structure

---

⏳ Remaining Work

1. Update Remaining File Imports (HIGH PRIORITY)

Files that need import updates:

• api/v1/batch/routes.py - update to use new schema/service imports
• services/autolabel.py - may need import updates if it references old paths
• services/async_processing.py - check for old import references
• services/batch_upload.py - check for old import references
• services/inference.py - check for old import references

2. Update ALL Test Files (CRITICAL)

Test files need to import from new locations. Pattern:

Old:

from src.web.admin_routes import create_admin_router
from src.web.admin_schemas import DocumentItem
from src.web.admin_auth import validate_admin_token

New:

from src.web.api.v1.admin import create_admin_router
from src.web.schemas.admin import DocumentItem
from src.web.core.auth import validate_admin_token

Test files to update:

• tests/web/test_admin_annotations.py
• tests/web/test_admin_auth.py
• tests/web/test_admin_routes.py
• tests/web/test_admin_routes_enhanced.py
• tests/web/test_admin_training.py
• tests/web/test_annotation_locks.py
• tests/web/test_annotation_phase5.py
• tests/web/test_async_queue.py
• tests/web/test_async_routes.py
• tests/web/test_async_service.py
• tests/web/test_autolabel_with_locks.py
• tests/web/test_batch_queue.py
• tests/web/test_batch_upload_routes.py
• tests/web/test_batch_upload_service.py
• tests/web/test_rate_limiter.py
• tests/web/test_training_phase4.py

3. Create Backward Compatibility Layer (OPTIONAL)

Keep old imports working temporarily:

# src/web/admin_routes.py (temporary compatibility shim)
\"\"\"
DEPRECATED: Use src.web.api.v1.admin.documents instead.
This file will be removed in next version.
\"\"\"
import warnings
from src.web.api.v1.admin.documents import *

warnings.warn(
    "Importing from src.web.admin_routes is deprecated. "
    "Use src.web.api.v1.admin.documents instead.",
    DeprecationWarning,
    stacklevel=2
)

4. Verify and Test

1. Run tests:

pytest tests/web/ -v

2. Check for any import errors:

python -c "from src.web.app import create_app; create_app()"

3. Start server and test endpoints:

python run_server.py

5. Clean Up Old Files (ONLY AFTER TESTS PASS)

Old files to remove:

• src/web/admin_*.py (7 files)
• src/web/async_*.py (3 files)
• src/web/batch_*.py (3 files)
• src/web/routes.py
• src/web/services.py
• src/web/schemas.py
• src/web/rate_limiter.py

Keep these files (don't remove):

• src/web/__init__.py
• src/web/app.py
• src/web/config.py
• src/web/dependencies.py

---

🎯 Next Immediate Steps

1. Update batch/routes.py imports - Quick fix for remaining API route
2. Update test file imports - Critical for verification
3. Run test suite - Verify nothing broke
4. Fix any import errors - Address failures
5. Remove old files - Clean up after tests pass

---

📊 Migration Impact Summary

Category	Files Moved	Imports Updated	Status
API Routes	7	5/7	🟡 In Progress
Schemas	3	3/3	✅ Complete
Services	4	0/4	⚠️ Pending
Core	3	3/3	✅ Complete
Workers	2	2/2	✅ Complete
Tests	0	0/16	❌ Not Started

Overall Progress: 65%

---

🚀 Benefits After Migration

1. Better Organization: Clear separation by function
2. Easier Navigation: Find files by purpose, not prefix
3. Scalability: Easy to add new API versions
4. Standard Structure: Follows FastAPI best practices
5. Maintainability: Each module has single responsibility

---

📝 Notes

• All original files are still in place (no data loss risk)
• New structure is operational but needs import updates
• Backward compatibility can be added if needed
• Tests will validate the migration success