openbb-invest-api/task_plan.md

# OpenBB Feature Expansion Plan

> 67 new endpoints across 10 feature groups. All use free providers.

## Prerequisites (Do First)

### P0: Consolidate Shared Utilities
- [ ] Replace duplicate `_to_dicts` in `openbb_service.py` and `macro_service.py` with `obb_utils.to_list`
- [ ] Add `fetch_historical(symbol, days, provider)` helper to `obb_utils.py`
- [ ] Add `serialize_dates(items)` helper to `obb_utils.py`
- **Files:** `obb_utils.py`, `openbb_service.py`, `macro_service.py`, `technical_service.py`, `quantitative_service.py`
- **Complexity:** S

---

## Phase 1: Local Computation (No Provider Risk)

### Group I: Technical Analysis Extended (14 endpoints)
- [ ] Add generic indicator dispatcher to `technical_service.py`
- [ ] Implement indicators: ATR, ADX, Stochastic, OBV, VWAP, Ichimoku, Donchian, Aroon, CCI, Keltner Channels, Fibonacci, A/D Line, Volatility Cones, Relative Rotation
- [ ] Add individual endpoints to `routes_technical.py`
- [ ] Add generic endpoint: `GET /api/v1/stock/{symbol}/technical/{indicator}`
- **New endpoints:**
  - `GET /api/v1/stock/{symbol}/technical/atr` -- Average True Range (volatility, position sizing)
  - `GET /api/v1/stock/{symbol}/technical/adx` -- Average Directional Index (trend strength)
  - `GET /api/v1/stock/{symbol}/technical/stoch` -- Stochastic Oscillator (overbought/oversold)
  - `GET /api/v1/stock/{symbol}/technical/obv` -- On-Balance Volume (volume-price divergence)
  - `GET /api/v1/stock/{symbol}/technical/vwap` -- Volume Weighted Average Price
  - `GET /api/v1/stock/{symbol}/technical/ichimoku` -- Ichimoku Cloud (comprehensive trend)
  - `GET /api/v1/stock/{symbol}/technical/donchian` -- Donchian Channels (breakout detection)
  - `GET /api/v1/stock/{symbol}/technical/aroon` -- Aroon Indicator (trend changes)
  - `GET /api/v1/stock/{symbol}/technical/cci` -- Commodity Channel Index (cyclical trends)
  - `GET /api/v1/stock/{symbol}/technical/kc` -- Keltner Channels (volatility bands)
  - `GET /api/v1/stock/{symbol}/technical/fib` -- Fibonacci Retracement (support/resistance)
  - `GET /api/v1/stock/{symbol}/technical/ad` -- Accumulation/Distribution Line
  - `GET /api/v1/stock/{symbol}/technical/cones` -- Volatility Cones (implied vs realized vol)
  - `GET /api/v1/stock/{symbol}/technical/relative_rotation` -- RRG (sector rotation)
- **Extend:** `technical_service.py` (+200 lines), `routes_technical.py` (+80 lines)
- **Complexity:** L (high volume, low individual complexity)

### Group J: Quantitative Extended (8 endpoints)
- [ ] Add Sortino ratio, Omega ratio
- [ ] Add rolling statistics: variance, stdev, mean, skew, kurtosis, quantile
- **New endpoints:**
  - `GET /api/v1/stock/{symbol}/sortino?days=365` -- Sortino ratio (downside risk only)
  - `GET /api/v1/stock/{symbol}/omega?days=365` -- Omega ratio (full distribution)
  - `GET /api/v1/stock/{symbol}/rolling/variance?days=365&window=30` -- Rolling variance
  - `GET /api/v1/stock/{symbol}/rolling/stdev?days=365&window=30` -- Rolling std deviation
  - `GET /api/v1/stock/{symbol}/rolling/mean?days=365&window=30` -- Rolling mean
  - `GET /api/v1/stock/{symbol}/rolling/skew?days=365&window=30` -- Rolling skewness
  - `GET /api/v1/stock/{symbol}/rolling/kurtosis?days=365&window=30` -- Rolling kurtosis
  - `GET /api/v1/stock/{symbol}/rolling/quantile?days=365&window=30&quantile=0.5` -- Rolling quantile
- **Extend:** `quantitative_service.py` (+120 lines), `routes_quantitative.py` (+60 lines)
- **Complexity:** M

### Group H: Currency Reference Rates (1 endpoint)
- [ ] Add ECB reference rates to `market_service.py`
- **New endpoint:**
  - `GET /api/v1/currency/reference-rates` -- ECB reference rates for 28 currencies
- **Extend:** `market_service.py` (+15 lines), `routes_market.py` (+10 lines)
- **Complexity:** S

---

## Phase 2: FRED/Federal Reserve Providers

### Group C: Fixed Income (10 endpoints)
- [ ] Create `fixed_income_service.py` -- treasury rates, yield curve, auctions, TIPS, EFFR, SOFR, HQM, commercial paper, spot rates, spreads
- [ ] Create `routes_fixed_income.py`
- [ ] Register router in `main.py`
- **New endpoints:**
  - `GET /api/v1/fixed-income/treasury-rates` -- Full yield curve rates (4W-30Y)
  - `GET /api/v1/fixed-income/yield-curve?date=` -- Yield curve with maturity/rate pairs
  - `GET /api/v1/fixed-income/treasury-auctions?security_type=` -- Auction bid-to-cover, yields
  - `GET /api/v1/fixed-income/tips-yields` -- TIPS real yields by maturity
  - `GET /api/v1/fixed-income/effr` -- Effective Fed Funds Rate with percentiles
  - `GET /api/v1/fixed-income/sofr` -- SOFR rate with moving averages
  - `GET /api/v1/fixed-income/hqm` -- High Quality Market corporate bond yields
  - `GET /api/v1/fixed-income/commercial-paper` -- CP rates by maturity/type
  - `GET /api/v1/fixed-income/spot-rates` -- Corporate bond spot rates
  - `GET /api/v1/fixed-income/spreads?series=tcm` -- Treasury/corporate spreads
- **New files:** `fixed_income_service.py` (~250 lines), `routes_fixed_income.py` (~180 lines)
- **Complexity:** L

### Group D: Economy Expanded (13 endpoints)
- [ ] Extend `macro_service.py` with structured FRED indicators (CPI, GDP, unemployment, PCE, money measures)
- [ ] Create `economy_service.py` for non-series endpoints (fred_search, fred_regional, balance_of_payments, central_bank_holdings, primary_dealer_positioning, fomc_documents)
- [ ] Extend `routes_macro.py` for FRED-based indicators
- [ ] Create `routes_economy.py` for search/institutional data
- [ ] Register new router in `main.py`
- **New endpoints (extend routes_macro.py):**
  - `GET /api/v1/macro/cpi?country=united_states` -- Consumer Price Index (multi-country)
  - `GET /api/v1/macro/gdp?type=real` -- GDP nominal/real/forecast
  - `GET /api/v1/macro/unemployment?country=united_states` -- Unemployment rate (multi-country)
  - `GET /api/v1/macro/pce` -- Personal Consumption Expenditures (Fed preferred inflation)
  - `GET /api/v1/macro/money-measures` -- M1/M2 money supply
  - `GET /api/v1/macro/cli?country=united_states` -- Composite Leading Indicator
  - `GET /api/v1/macro/house-price-index?country=united_states` -- Housing price index
- **New endpoints (new routes_economy.py):**
  - `GET /api/v1/economy/fred-search?query=` -- Search FRED series by keyword
  - `GET /api/v1/economy/fred-regional?series_id=&region=` -- Regional economic data
  - `GET /api/v1/economy/balance-of-payments` -- Current/capital/financial account
  - `GET /api/v1/economy/central-bank-holdings` -- Fed SOMA portfolio
  - `GET /api/v1/economy/primary-dealer-positioning` -- Wall Street firm positions
  - `GET /api/v1/economy/fomc-documents?year=` -- FOMC meeting documents
- **New files:** `economy_service.py` (~200 lines), `routes_economy.py` (~150 lines)
- **Extend:** `macro_service.py` (+80 lines), `routes_macro.py` (+50 lines)
- **Complexity:** L

### Group E: Economy Surveys (5 endpoints)
- [ ] Create `surveys_service.py` -- Michigan, SLOOS, NFP, Empire State, BLS
- [ ] Create `routes_surveys.py`
- [ ] Register router in `main.py`
- **New endpoints:**
  - `GET /api/v1/economy/surveys/michigan` -- Consumer Sentiment + inflation expectations
  - `GET /api/v1/economy/surveys/sloos` -- Senior Loan Officer survey (recession predictor)
  - `GET /api/v1/economy/surveys/nonfarm-payrolls` -- Detailed employment data
  - `GET /api/v1/economy/surveys/empire-state` -- NY manufacturing outlook
  - `GET /api/v1/economy/surveys/bls-search?query=` -- BLS data series search
- **New files:** `surveys_service.py` (~130 lines), `routes_surveys.py` (~100 lines)
- **Complexity:** M

---

## Phase 3: SEC/Stockgrid/CFTC Providers

### Group B: Equity Fundamentals (4 endpoints)
- [ ] Add management, dividends, filings, search to `openbb_service.py`
- [ ] Add endpoints to `routes.py`
- **New endpoints:**
  - `GET /api/v1/stock/{symbol}/management` -- Executive team, titles, compensation
  - `GET /api/v1/stock/{symbol}/dividends` -- Historical dividend records
  - `GET /api/v1/stock/{symbol}/filings?form_type=10-K` -- SEC filings (10-K, 10-Q, 8-K)
  - `GET /api/v1/search?query=` -- Company search by name (SEC/NASDAQ)
- **Extend:** `openbb_service.py` (+60 lines), `routes.py` (+40 lines)
- **Complexity:** S

### Group A: Equity Shorts & Dark Pool (4 endpoints)
- [ ] Create `shorts_service.py` -- short volume, FTD, short interest, OTC dark pool
- [ ] Create `routes_shorts.py`
- [ ] Register router in `main.py`
- **New endpoints:**
  - `GET /api/v1/stock/{symbol}/shorts/volume` -- Daily short volume & percent (stockgrid)
  - `GET /api/v1/stock/{symbol}/shorts/ftd` -- Fails-to-deliver records (SEC)
  - `GET /api/v1/stock/{symbol}/shorts/interest` -- Short interest, days to cover (FINRA)
  - `GET /api/v1/darkpool/{symbol}/otc` -- OTC/dark pool trade volume (FINRA)
- **New files:** `shorts_service.py` (~120 lines), `routes_shorts.py` (~80 lines)
- **Complexity:** M

### Group F: Index & ETF Enhanced (3 endpoints)
- [ ] Add sp500_multiples, index_constituents, etf nport_disclosure to `market_service.py`
- [ ] Add endpoints to `routes_market.py`
- **New endpoints:**
  - `GET /api/v1/index/sp500-multiples?series=pe_ratio` -- Historical S&P 500 valuation (Shiller PE, P/B, P/S, dividend yield)
  - `GET /api/v1/index/{symbol}/constituents` -- Index member stocks with sector/price data
  - `GET /api/v1/etf/{symbol}/nport` -- Detailed ETF holdings from SEC N-PORT filings
- **Extend:** `market_service.py` (+60 lines), `routes_market.py` (+50 lines)
- **Complexity:** S

---

## Phase 4: Regulators

### Group G: Regulators (5 endpoints)
- [ ] Create `regulators_service.py` -- COT, COT search, SEC litigation, institution search, CIK mapping
- [ ] Create `routes_regulators.py`
- [ ] Register router in `main.py`
- **New endpoints:**
  - `GET /api/v1/regulators/cot?symbol=` -- Commitment of Traders report (commercial/speculator positions)
  - `GET /api/v1/regulators/cot/search?query=` -- Search COT report symbols
  - `GET /api/v1/regulators/sec/litigation` -- SEC litigation releases RSS feed
  - `GET /api/v1/regulators/sec/institutions?query=` -- Search institutional investors
  - `GET /api/v1/regulators/sec/cik-map?symbol=` -- Ticker to CIK mapping
- **New files:** `regulators_service.py` (~150 lines), `routes_regulators.py` (~100 lines)
- **Complexity:** M

---

## Summary

| Phase | Groups | Endpoints | New Files | Complexity |
|-------|--------|-----------|-----------|------------|
| P0 Prereq | - | 0 | 0 | S |
| Phase 1 | I, J, H | 23 | 0 | L+M+S |
| Phase 2 | C, D, E | 28 | 6 | L+L+M |
| Phase 3 | B, A, F | 11 | 2 | S+M+S |
| Phase 4 | G | 5 | 2 | M |
| **Total** | **10** | **67** | **10** | |

### File Impact

**New files (10):**
- `shorts_service.py`, `routes_shorts.py`
- `fixed_income_service.py`, `routes_fixed_income.py`
- `economy_service.py`, `routes_economy.py`
- `surveys_service.py`, `routes_surveys.py`
- `regulators_service.py`, `routes_regulators.py`

**Extended files (12):**
- `obb_utils.py` (shared helpers)
- `openbb_service.py` (Group B fundamentals)
- `routes.py` (Group B endpoints)
- `macro_service.py` (Group D indicators)
- `routes_macro.py` (Group D endpoints)
- `market_service.py` (Groups F, H)
- `routes_market.py` (Groups F, H)
- `technical_service.py` (Group I indicators)
- `routes_technical.py` (Group I endpoints)
- `quantitative_service.py` (Group J metrics)
- `routes_quantitative.py` (Group J endpoints)
- `main.py` (register 5 new routers)

### Endpoint Count After Completion
- Current: 32 endpoints
- New: 67 endpoints
- **Total: 99 endpoints**