feat: add 67 new endpoints across 10 feature groups
Prerequisite refactor: - Consolidate duplicate _to_dicts into shared obb_utils.to_list - Add fetch_historical and first_or_empty helpers to obb_utils Phase 1 - Local computation (no provider risk): - Group I: 12 technical indicators (ATR, ADX, Stoch, OBV, Ichimoku, Donchian, Aroon, CCI, Keltner, Fibonacci, A/D, Volatility Cones) - Group J: Sortino, Omega ratios + rolling stats (variance, stdev, mean, skew, kurtosis, quantile via generic endpoint) - Group H: ECB currency reference rates Phase 2 - FRED/Federal Reserve providers: - Group C: 10 fixed income endpoints (treasury rates, yield curve, auctions, TIPS, EFFR, SOFR, HQM, commercial paper, spot rates, spreads) - Group D: 11 economy endpoints (CPI, GDP, unemployment, PCE, money measures, CLI, HPI, FRED search, balance of payments, Fed holdings, FOMC documents) - Group E: 5 survey endpoints (Michigan, SLOOS, NFP, Empire State, BLS search) Phase 3 - SEC/stockgrid/FINRA providers: - Group B: 4 equity fundamental endpoints (management, dividends, SEC filings, company search) - Group A: 4 shorts/dark pool endpoints (short volume, FTD, short interest, OTC dark pool) - Group F: 3 index/ETF enhanced (S&P 500 multiples, index constituents, ETF N-PORT) Phase 4 - Regulators: - Group G: 5 regulatory endpoints (COT report, COT search, SEC litigation, institution search, CIK mapping) Security hardening: - Service-layer allowlists for all getattr dynamic dispatch - Regex validation on date, country, security_type, form_type params - Exception handling in fetch_historical - Callable guard on rolling stat dispatch Total: 32 existing + 67 new = 99 endpoints, all free providers.
This commit is contained in:
Yaojia Wang
216
task_plan.md
Normal file
216
task_plan.md
Normal file
@@ -0,0 +1,216 @@
|
||||
# OpenBB Feature Expansion Plan
|
||||
|
||||
> 67 new endpoints across 10 feature groups. All use free providers.
|
||||
|
||||
## Prerequisites (Do First)
|
||||
|
||||
### P0: Consolidate Shared Utilities
|
||||
- [ ] Replace duplicate `_to_dicts` in `openbb_service.py` and `macro_service.py` with `obb_utils.to_list`
|
||||
- [ ] Add `fetch_historical(symbol, days, provider)` helper to `obb_utils.py`
|
||||
- [ ] Add `serialize_dates(items)` helper to `obb_utils.py`
|
||||
- **Files:** `obb_utils.py`, `openbb_service.py`, `macro_service.py`, `technical_service.py`, `quantitative_service.py`
|
||||
- **Complexity:** S
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Local Computation (No Provider Risk)
|
||||
|
||||
### Group I: Technical Analysis Extended (14 endpoints)
|
||||
- [ ] Add generic indicator dispatcher to `technical_service.py`
|
||||
- [ ] Implement indicators: ATR, ADX, Stochastic, OBV, VWAP, Ichimoku, Donchian, Aroon, CCI, Keltner Channels, Fibonacci, A/D Line, Volatility Cones, Relative Rotation
|
||||
- [ ] Add individual endpoints to `routes_technical.py`
|
||||
- [ ] Add generic endpoint: `GET /api/v1/stock/{symbol}/technical/{indicator}`
|
||||
- **New endpoints:**
|
||||
- `GET /api/v1/stock/{symbol}/technical/atr` -- Average True Range (volatility, position sizing)
|
||||
- `GET /api/v1/stock/{symbol}/technical/adx` -- Average Directional Index (trend strength)
|
||||
- `GET /api/v1/stock/{symbol}/technical/stoch` -- Stochastic Oscillator (overbought/oversold)
|
||||
- `GET /api/v1/stock/{symbol}/technical/obv` -- On-Balance Volume (volume-price divergence)
|
||||
- `GET /api/v1/stock/{symbol}/technical/vwap` -- Volume Weighted Average Price
|
||||
- `GET /api/v1/stock/{symbol}/technical/ichimoku` -- Ichimoku Cloud (comprehensive trend)
|
||||
- `GET /api/v1/stock/{symbol}/technical/donchian` -- Donchian Channels (breakout detection)
|
||||
- `GET /api/v1/stock/{symbol}/technical/aroon` -- Aroon Indicator (trend changes)
|
||||
- `GET /api/v1/stock/{symbol}/technical/cci` -- Commodity Channel Index (cyclical trends)
|
||||
- `GET /api/v1/stock/{symbol}/technical/kc` -- Keltner Channels (volatility bands)
|
||||
- `GET /api/v1/stock/{symbol}/technical/fib` -- Fibonacci Retracement (support/resistance)
|
||||
- `GET /api/v1/stock/{symbol}/technical/ad` -- Accumulation/Distribution Line
|
||||
- `GET /api/v1/stock/{symbol}/technical/cones` -- Volatility Cones (implied vs realized vol)
|
||||
- `GET /api/v1/stock/{symbol}/technical/relative_rotation` -- RRG (sector rotation)
|
||||
- **Extend:** `technical_service.py` (+200 lines), `routes_technical.py` (+80 lines)
|
||||
- **Complexity:** L (high volume, low individual complexity)
|
||||
|
||||
### Group J: Quantitative Extended (8 endpoints)
|
||||
- [ ] Add Sortino ratio, Omega ratio
|
||||
- [ ] Add rolling statistics: variance, stdev, mean, skew, kurtosis, quantile
|
||||
- **New endpoints:**
|
||||
- `GET /api/v1/stock/{symbol}/sortino?days=365` -- Sortino ratio (downside risk only)
|
||||
- `GET /api/v1/stock/{symbol}/omega?days=365` -- Omega ratio (full distribution)
|
||||
- `GET /api/v1/stock/{symbol}/rolling/variance?days=365&window=30` -- Rolling variance
|
||||
- `GET /api/v1/stock/{symbol}/rolling/stdev?days=365&window=30` -- Rolling std deviation
|
||||
- `GET /api/v1/stock/{symbol}/rolling/mean?days=365&window=30` -- Rolling mean
|
||||
- `GET /api/v1/stock/{symbol}/rolling/skew?days=365&window=30` -- Rolling skewness
|
||||
- `GET /api/v1/stock/{symbol}/rolling/kurtosis?days=365&window=30` -- Rolling kurtosis
|
||||
- `GET /api/v1/stock/{symbol}/rolling/quantile?days=365&window=30&quantile=0.5` -- Rolling quantile
|
||||
- **Extend:** `quantitative_service.py` (+120 lines), `routes_quantitative.py` (+60 lines)
|
||||
- **Complexity:** M
|
||||
|
||||
### Group H: Currency Reference Rates (1 endpoint)
|
||||
- [ ] Add ECB reference rates to `market_service.py`
|
||||
- **New endpoint:**
|
||||
- `GET /api/v1/currency/reference-rates` -- ECB reference rates for 28 currencies
|
||||
- **Extend:** `market_service.py` (+15 lines), `routes_market.py` (+10 lines)
|
||||
- **Complexity:** S
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: FRED/Federal Reserve Providers
|
||||
|
||||
### Group C: Fixed Income (10 endpoints)
|
||||
- [ ] Create `fixed_income_service.py` -- treasury rates, yield curve, auctions, TIPS, EFFR, SOFR, HQM, commercial paper, spot rates, spreads
|
||||
- [ ] Create `routes_fixed_income.py`
|
||||
- [ ] Register router in `main.py`
|
||||
- **New endpoints:**
|
||||
- `GET /api/v1/fixed-income/treasury-rates` -- Full yield curve rates (4W-30Y)
|
||||
- `GET /api/v1/fixed-income/yield-curve?date=` -- Yield curve with maturity/rate pairs
|
||||
- `GET /api/v1/fixed-income/treasury-auctions?security_type=` -- Auction bid-to-cover, yields
|
||||
- `GET /api/v1/fixed-income/tips-yields` -- TIPS real yields by maturity
|
||||
- `GET /api/v1/fixed-income/effr` -- Effective Fed Funds Rate with percentiles
|
||||
- `GET /api/v1/fixed-income/sofr` -- SOFR rate with moving averages
|
||||
- `GET /api/v1/fixed-income/hqm` -- High Quality Market corporate bond yields
|
||||
- `GET /api/v1/fixed-income/commercial-paper` -- CP rates by maturity/type
|
||||
- `GET /api/v1/fixed-income/spot-rates` -- Corporate bond spot rates
|
||||
- `GET /api/v1/fixed-income/spreads?series=tcm` -- Treasury/corporate spreads
|
||||
- **New files:** `fixed_income_service.py` (~250 lines), `routes_fixed_income.py` (~180 lines)
|
||||
- **Complexity:** L
|
||||
|
||||
### Group D: Economy Expanded (13 endpoints)
|
||||
- [ ] Extend `macro_service.py` with structured FRED indicators (CPI, GDP, unemployment, PCE, money measures)
|
||||
- [ ] Create `economy_service.py` for non-series endpoints (fred_search, fred_regional, balance_of_payments, central_bank_holdings, primary_dealer_positioning, fomc_documents)
|
||||
- [ ] Extend `routes_macro.py` for FRED-based indicators
|
||||
- [ ] Create `routes_economy.py` for search/institutional data
|
||||
- [ ] Register new router in `main.py`
|
||||
- **New endpoints (extend routes_macro.py):**
|
||||
- `GET /api/v1/macro/cpi?country=united_states` -- Consumer Price Index (multi-country)
|
||||
- `GET /api/v1/macro/gdp?type=real` -- GDP nominal/real/forecast
|
||||
- `GET /api/v1/macro/unemployment?country=united_states` -- Unemployment rate (multi-country)
|
||||
- `GET /api/v1/macro/pce` -- Personal Consumption Expenditures (Fed preferred inflation)
|
||||
- `GET /api/v1/macro/money-measures` -- M1/M2 money supply
|
||||
- `GET /api/v1/macro/cli?country=united_states` -- Composite Leading Indicator
|
||||
- `GET /api/v1/macro/house-price-index?country=united_states` -- Housing price index
|
||||
- **New endpoints (new routes_economy.py):**
|
||||
- `GET /api/v1/economy/fred-search?query=` -- Search FRED series by keyword
|
||||
- `GET /api/v1/economy/fred-regional?series_id=®ion=` -- Regional economic data
|
||||
- `GET /api/v1/economy/balance-of-payments` -- Current/capital/financial account
|
||||
- `GET /api/v1/economy/central-bank-holdings` -- Fed SOMA portfolio
|
||||
- `GET /api/v1/economy/primary-dealer-positioning` -- Wall Street firm positions
|
||||
- `GET /api/v1/economy/fomc-documents?year=` -- FOMC meeting documents
|
||||
- **New files:** `economy_service.py` (~200 lines), `routes_economy.py` (~150 lines)
|
||||
- **Extend:** `macro_service.py` (+80 lines), `routes_macro.py` (+50 lines)
|
||||
- **Complexity:** L
|
||||
|
||||
### Group E: Economy Surveys (5 endpoints)
|
||||
- [ ] Create `surveys_service.py` -- Michigan, SLOOS, NFP, Empire State, BLS
|
||||
- [ ] Create `routes_surveys.py`
|
||||
- [ ] Register router in `main.py`
|
||||
- **New endpoints:**
|
||||
- `GET /api/v1/economy/surveys/michigan` -- Consumer Sentiment + inflation expectations
|
||||
- `GET /api/v1/economy/surveys/sloos` -- Senior Loan Officer survey (recession predictor)
|
||||
- `GET /api/v1/economy/surveys/nonfarm-payrolls` -- Detailed employment data
|
||||
- `GET /api/v1/economy/surveys/empire-state` -- NY manufacturing outlook
|
||||
- `GET /api/v1/economy/surveys/bls-search?query=` -- BLS data series search
|
||||
- **New files:** `surveys_service.py` (~130 lines), `routes_surveys.py` (~100 lines)
|
||||
- **Complexity:** M
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: SEC/Stockgrid/CFTC Providers
|
||||
|
||||
### Group B: Equity Fundamentals (4 endpoints)
|
||||
- [ ] Add management, dividends, filings, search to `openbb_service.py`
|
||||
- [ ] Add endpoints to `routes.py`
|
||||
- **New endpoints:**
|
||||
- `GET /api/v1/stock/{symbol}/management` -- Executive team, titles, compensation
|
||||
- `GET /api/v1/stock/{symbol}/dividends` -- Historical dividend records
|
||||
- `GET /api/v1/stock/{symbol}/filings?form_type=10-K` -- SEC filings (10-K, 10-Q, 8-K)
|
||||
- `GET /api/v1/search?query=` -- Company search by name (SEC/NASDAQ)
|
||||
- **Extend:** `openbb_service.py` (+60 lines), `routes.py` (+40 lines)
|
||||
- **Complexity:** S
|
||||
|
||||
### Group A: Equity Shorts & Dark Pool (4 endpoints)
|
||||
- [ ] Create `shorts_service.py` -- short volume, FTD, short interest, OTC dark pool
|
||||
- [ ] Create `routes_shorts.py`
|
||||
- [ ] Register router in `main.py`
|
||||
- **New endpoints:**
|
||||
- `GET /api/v1/stock/{symbol}/shorts/volume` -- Daily short volume & percent (stockgrid)
|
||||
- `GET /api/v1/stock/{symbol}/shorts/ftd` -- Fails-to-deliver records (SEC)
|
||||
- `GET /api/v1/stock/{symbol}/shorts/interest` -- Short interest, days to cover (FINRA)
|
||||
- `GET /api/v1/darkpool/{symbol}/otc` -- OTC/dark pool trade volume (FINRA)
|
||||
- **New files:** `shorts_service.py` (~120 lines), `routes_shorts.py` (~80 lines)
|
||||
- **Complexity:** M
|
||||
|
||||
### Group F: Index & ETF Enhanced (3 endpoints)
|
||||
- [ ] Add sp500_multiples, index_constituents, etf nport_disclosure to `market_service.py`
|
||||
- [ ] Add endpoints to `routes_market.py`
|
||||
- **New endpoints:**
|
||||
- `GET /api/v1/index/sp500-multiples?series=pe_ratio` -- Historical S&P 500 valuation (Shiller PE, P/B, P/S, dividend yield)
|
||||
- `GET /api/v1/index/{symbol}/constituents` -- Index member stocks with sector/price data
|
||||
- `GET /api/v1/etf/{symbol}/nport` -- Detailed ETF holdings from SEC N-PORT filings
|
||||
- **Extend:** `market_service.py` (+60 lines), `routes_market.py` (+50 lines)
|
||||
- **Complexity:** S
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Regulators
|
||||
|
||||
### Group G: Regulators (5 endpoints)
|
||||
- [ ] Create `regulators_service.py` -- COT, COT search, SEC litigation, institution search, CIK mapping
|
||||
- [ ] Create `routes_regulators.py`
|
||||
- [ ] Register router in `main.py`
|
||||
- **New endpoints:**
|
||||
- `GET /api/v1/regulators/cot?symbol=` -- Commitment of Traders report (commercial/speculator positions)
|
||||
- `GET /api/v1/regulators/cot/search?query=` -- Search COT report symbols
|
||||
- `GET /api/v1/regulators/sec/litigation` -- SEC litigation releases RSS feed
|
||||
- `GET /api/v1/regulators/sec/institutions?query=` -- Search institutional investors
|
||||
- `GET /api/v1/regulators/sec/cik-map?symbol=` -- Ticker to CIK mapping
|
||||
- **New files:** `regulators_service.py` (~150 lines), `routes_regulators.py` (~100 lines)
|
||||
- **Complexity:** M
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Phase | Groups | Endpoints | New Files | Complexity |
|
||||
|-------|--------|-----------|-----------|------------|
|
||||
| P0 Prereq | - | 0 | 0 | S |
|
||||
| Phase 1 | I, J, H | 23 | 0 | L+M+S |
|
||||
| Phase 2 | C, D, E | 28 | 6 | L+L+M |
|
||||
| Phase 3 | B, A, F | 11 | 2 | S+M+S |
|
||||
| Phase 4 | G | 5 | 2 | M |
|
||||
| **Total** | **10** | **67** | **10** | |
|
||||
|
||||
### File Impact
|
||||
|
||||
**New files (10):**
|
||||
- `shorts_service.py`, `routes_shorts.py`
|
||||
- `fixed_income_service.py`, `routes_fixed_income.py`
|
||||
- `economy_service.py`, `routes_economy.py`
|
||||
- `surveys_service.py`, `routes_surveys.py`
|
||||
- `regulators_service.py`, `routes_regulators.py`
|
||||
|
||||
**Extended files (12):**
|
||||
- `obb_utils.py` (shared helpers)
|
||||
- `openbb_service.py` (Group B fundamentals)
|
||||
- `routes.py` (Group B endpoints)
|
||||
- `macro_service.py` (Group D indicators)
|
||||
- `routes_macro.py` (Group D endpoints)
|
||||
- `market_service.py` (Groups F, H)
|
||||
- `routes_market.py` (Groups F, H)
|
||||
- `technical_service.py` (Group I indicators)
|
||||
- `routes_technical.py` (Group I endpoints)
|
||||
- `quantitative_service.py` (Group J metrics)
|
||||
- `routes_quantitative.py` (Group J endpoints)
|
||||
- `main.py` (register 5 new routers)
|
||||
|
||||
### Endpoint Count After Completion
|
||||
- Current: 32 endpoints
|
||||
- New: 67 endpoints
|
||||
- **Total: 99 endpoints**
|
||||
Reference in New Issue
Block a user