243 lines
5.6 KiB
Markdown
243 lines
5.6 KiB
Markdown
# Verification Loop Skill
|
|
|
|
Comprehensive verification system for Python/FastAPI development.
|
|
|
|
## When to Use
|
|
|
|
Invoke this skill:
|
|
- After completing a feature or significant code change
|
|
- Before creating a PR
|
|
- When you want to ensure quality gates pass
|
|
- After refactoring
|
|
- Before deployment
|
|
|
|
## Verification Phases
|
|
|
|
### Phase 1: Type Check
|
|
```bash
|
|
# Run mypy type checker
|
|
mypy src/ --ignore-missing-imports 2>&1 | head -30
|
|
```
|
|
|
|
Report all type errors. Fix critical ones before continuing.
|
|
|
|
### Phase 2: Lint Check
|
|
```bash
|
|
# Run ruff linter
|
|
ruff check src/ 2>&1 | head -30
|
|
|
|
# Auto-fix if desired
|
|
ruff check src/ --fix
|
|
```
|
|
|
|
Check for:
|
|
- Unused imports
|
|
- Code style violations
|
|
- Common Python anti-patterns
|
|
|
|
### Phase 3: Test Suite
|
|
```bash
|
|
# Run tests with coverage
|
|
pytest --cov=src --cov-report=term-missing -q 2>&1 | tail -50
|
|
|
|
# Run specific test file
|
|
pytest tests/test_ocr/test_machine_code_parser.py -v
|
|
|
|
# Run with short traceback
|
|
pytest -x --tb=short
|
|
```
|
|
|
|
Report:
|
|
- Total tests: X
|
|
- Passed: X
|
|
- Failed: X
|
|
- Coverage: X%
|
|
- Target: 80% minimum
|
|
|
|
### Phase 4: Security Scan
|
|
```bash
|
|
# Check for hardcoded secrets
|
|
grep -rn "password\s*=" --include="*.py" src/ 2>/dev/null | grep -v "db_password:" | head -10
|
|
grep -rn "api_key\s*=" --include="*.py" src/ 2>/dev/null | head -10
|
|
grep -rn "sk-" --include="*.py" src/ 2>/dev/null | head -10
|
|
|
|
# Check for print statements (should use logging)
|
|
grep -rn "print(" --include="*.py" src/ 2>/dev/null | head -10
|
|
|
|
# Check for bare except
|
|
grep -rn "except:" --include="*.py" src/ 2>/dev/null | head -10
|
|
|
|
# Check for SQL injection risks (f-strings in execute)
|
|
grep -rn 'execute(f"' --include="*.py" src/ 2>/dev/null | head -10
|
|
grep -rn "execute(f'" --include="*.py" src/ 2>/dev/null | head -10
|
|
```
|
|
|
|
### Phase 5: Import Check
|
|
```bash
|
|
# Verify all imports work
|
|
python -c "from src.web.app import app; print('Web app OK')"
|
|
python -c "from src.inference.pipeline import InferencePipeline; print('Pipeline OK')"
|
|
python -c "from src.ocr.machine_code_parser import parse_payment_line; print('Parser OK')"
|
|
```
|
|
|
|
### Phase 6: Diff Review
|
|
```bash
|
|
# Show what changed
|
|
git diff --stat
|
|
git diff HEAD --name-only
|
|
|
|
# Show staged changes
|
|
git diff --staged --stat
|
|
```
|
|
|
|
Review each changed file for:
|
|
- Unintended changes
|
|
- Missing error handling
|
|
- Potential edge cases
|
|
- Missing type hints
|
|
- Mutable default arguments
|
|
|
|
### Phase 7: API Smoke Test (if server running)
|
|
```bash
|
|
# Health check
|
|
curl -s http://localhost:8000/api/v1/health | python -m json.tool
|
|
|
|
# Verify response format
|
|
curl -s http://localhost:8000/api/v1/health | grep -q "healthy" && echo "Health: OK" || echo "Health: FAIL"
|
|
```
|
|
|
|
## Output Format
|
|
|
|
After running all phases, produce a verification report:
|
|
|
|
```
|
|
VERIFICATION REPORT
|
|
==================
|
|
|
|
Types: [PASS/FAIL] (X errors)
|
|
Lint: [PASS/FAIL] (X warnings)
|
|
Tests: [PASS/FAIL] (X/Y passed, Z% coverage)
|
|
Security: [PASS/FAIL] (X issues)
|
|
Imports: [PASS/FAIL]
|
|
Diff: [X files changed]
|
|
|
|
Overall: [READY/NOT READY] for PR
|
|
|
|
Issues to Fix:
|
|
1. ...
|
|
2. ...
|
|
```
|
|
|
|
## Quick Commands
|
|
|
|
```bash
|
|
# Full verification (WSL)
|
|
wsl bash -c "source ~/miniconda3/etc/profile.d/conda.sh && conda activate invoice-py311 && cd /mnt/c/Users/yaoji/git/ColaCoder/invoice-master-poc-v2 && mypy src/ --ignore-missing-imports && ruff check src/ && pytest -x --tb=short"
|
|
|
|
# Type check only
|
|
wsl bash -c "source ~/miniconda3/etc/profile.d/conda.sh && conda activate invoice-py311 && cd /mnt/c/Users/yaoji/git/ColaCoder/invoice-master-poc-v2 && mypy src/ --ignore-missing-imports"
|
|
|
|
# Tests only
|
|
wsl bash -c "source ~/miniconda3/etc/profile.d/conda.sh && conda activate invoice-py311 && cd /mnt/c/Users/yaoji/git/ColaCoder/invoice-master-poc-v2 && pytest --cov=src -q"
|
|
```
|
|
|
|
## Verification Checklist
|
|
|
|
### Before Commit
|
|
- [ ] mypy passes (no type errors)
|
|
- [ ] ruff check passes (no lint errors)
|
|
- [ ] All tests pass
|
|
- [ ] No print() statements in production code
|
|
- [ ] No hardcoded secrets
|
|
- [ ] No bare `except:` clauses
|
|
- [ ] No SQL injection risks (f-strings in queries)
|
|
- [ ] Coverage >= 80% for changed code
|
|
|
|
### Before PR
|
|
- [ ] All above checks pass
|
|
- [ ] git diff reviewed for unintended changes
|
|
- [ ] New code has tests
|
|
- [ ] Type hints on all public functions
|
|
- [ ] Docstrings on public APIs
|
|
- [ ] No TODO/FIXME for critical items
|
|
|
|
### Before Deployment
|
|
- [ ] All above checks pass
|
|
- [ ] E2E tests pass
|
|
- [ ] Health check returns healthy
|
|
- [ ] Model loaded successfully
|
|
- [ ] No server errors in logs
|
|
|
|
## Common Issues and Fixes
|
|
|
|
### Type Error: Missing return type
|
|
```python
|
|
# Before
|
|
def process(data):
|
|
return result
|
|
|
|
# After
|
|
def process(data: dict) -> InferenceResult:
|
|
return result
|
|
```
|
|
|
|
### Lint Error: Unused import
|
|
```python
|
|
# Remove unused imports or add to __all__
|
|
```
|
|
|
|
### Security: print() in production
|
|
```python
|
|
# Before
|
|
print(f"Processing {doc_id}")
|
|
|
|
# After
|
|
logger.info(f"Processing {doc_id}")
|
|
```
|
|
|
|
### Security: Bare except
|
|
```python
|
|
# Before
|
|
except:
|
|
pass
|
|
|
|
# After
|
|
except Exception as e:
|
|
logger.error(f"Error: {e}")
|
|
raise
|
|
```
|
|
|
|
### Security: SQL injection
|
|
```python
|
|
# Before (DANGEROUS)
|
|
cur.execute(f"SELECT * FROM docs WHERE id = '{user_input}'")
|
|
|
|
# After (SAFE)
|
|
cur.execute("SELECT * FROM docs WHERE id = %s", (user_input,))
|
|
```
|
|
|
|
## Continuous Mode
|
|
|
|
For long sessions, run verification after major changes:
|
|
|
|
```markdown
|
|
Checkpoints:
|
|
- After completing each function
|
|
- After finishing a module
|
|
- Before moving to next task
|
|
- Every 15-20 minutes of coding
|
|
|
|
Run: /verify
|
|
```
|
|
|
|
## Integration with Other Skills
|
|
|
|
| Skill | Purpose |
|
|
|-------|---------|
|
|
| code-review | Detailed code analysis |
|
|
| security-review | Deep security audit |
|
|
| tdd-workflow | Test coverage |
|
|
| build-fix | Fix errors incrementally |
|
|
|
|
This skill provides quick, comprehensive verification. Use specialized skills for deeper analysis.
|