Commit Graph

51 Commits

Author SHA1 Message Date
Yaojia Wang
37c46e76ae feat: add DeFi data via DefiLlama API (TDD)
7 new endpoints under /api/v1/defi/ (all free, no API key):
- GET /defi/tvl/protocols - top DeFi protocols by TVL
- GET /defi/tvl/chains - chain TVL rankings
- GET /defi/tvl/{protocol} - single protocol TVL
- GET /defi/yields - top yield pools (filter by chain/project)
- GET /defi/stablecoins - stablecoin market data
- GET /defi/volumes/dexs - DEX volume overview
- GET /defi/fees - protocol fees/revenue overview

Data source: DefiLlama API (free, no key needed)
58 new tests (33 service + 25 route). All 561 tests passing.
2026-03-19 23:03:01 +01:00
Yaojia Wang
4915f1bae4 feat: add t-SNE stock clustering and similarity search (TDD)
2 new endpoints:
- POST /portfolio/cluster - t-SNE + KMeans clustering by return
  similarity. Maps stocks to 2D coordinates with cluster labels.
- POST /portfolio/similar - find most/least similar stocks by
  return correlation against a target symbol.

Implementation:
- sklearn TSNE (method=exact) + KMeans with auto n_clusters
- Jitter handling for identical returns edge case
- 33 new tests (17 service unit + 16 route integration)
- All 503 tests passing
2026-03-19 22:53:27 +01:00
Yaojia Wang
9ee3ec9b4e feat: add A-share and HK stock data via AKShare (TDD)
5 new endpoints under /api/v1/cn/:
- GET /cn/a-share/{symbol}/quote - A-share real-time quote
- GET /cn/a-share/{symbol}/historical - A-share historical OHLCV
- GET /cn/a-share/search?query= - search A-shares by name
- GET /cn/hk/{symbol}/quote - HK stock real-time quote
- GET /cn/hk/{symbol}/historical - HK stock historical OHLCV

Features:
- Chinese column names auto-mapped to English
- Symbol validation: A-share ^[036]\d{5}$, HK ^\d{5}$
- qfq (forward-adjusted) prices by default
- 79 new tests (51 service unit + 28 route integration)
- All 470 tests passing
2026-03-19 22:44:30 +01:00
Yaojia Wang
5c7a0ee4c0 feat: add backtesting engine with 4 strategies (TDD)
Strategies:
- POST /backtest/sma-crossover - SMA crossover (short/long window)
- POST /backtest/rsi - RSI oversold/overbought signals
- POST /backtest/buy-and-hold - passive benchmark
- POST /backtest/momentum - multi-symbol momentum rotation

Returns: total_return, annualized_return, sharpe_ratio, max_drawdown,
win_rate, total_trades, equity_curve (last 20 points)

Implementation: pure pandas/numpy, no external backtesting libs.
Shared _compute_metrics helper across all strategies.
79 new tests (46 service unit + 33 route integration).
All 391 tests passing.
2026-03-19 22:35:00 +01:00
Yaojia Wang
42ba359c48 feat: add portfolio optimization and congress tracking (TDD)
Portfolio optimization (3 endpoints):
- POST /portfolio/optimize - HRP optimal weights via scipy clustering
- POST /portfolio/correlation - pairwise correlation matrix
- POST /portfolio/risk-parity - inverse-volatility risk parity weights

Congress tracking (2 endpoints):
- GET /regulators/congress/trades - congress member stock trades
- GET /regulators/congress/bills?query= - search congress bills

Implementation:
- portfolio_service.py: HRP with scipy fallback to inverse-vol
- congress_service.py: multi-provider fallback pattern
- 51 new tests (14 portfolio unit, 20 portfolio route, 12 congress
  unit, 7 congress route)
- All 312 tests passing
2026-03-19 22:27:03 +01:00
Yaojia Wang
27b131492f test: add 159 tests for all new modules
New test files (171 tests):
- test_routes_shorts.py (16) - short volume, FTD, interest, darkpool
- test_routes_fixed_income.py (34) - treasury, yield curve, SOFR, etc.
- test_routes_economy.py (44) - CPI, GDP, FRED search, Fed holdings
- test_routes_surveys.py (17) - Michigan, SLOOS, NFP, Empire State
- test_routes_regulators.py (20) - COT, SEC litigation, institutions
- test_finnhub_service_social.py (20) - social/reddit sentiment unit tests
- test_routes_sentiment_social.py (20) - social endpoints + composite

Updated:
- test_routes_sentiment.py - match new composite sentiment response shape

Total: 261 tests passing (was 102)
2026-03-19 22:12:27 +01:00
Yaojia Wang
ea72497587 docs: update README for sentiment aggregation and social endpoints
- Update endpoint count to 102
- Add social-sentiment, reddit-sentiment, reddit-trending endpoints
- Document composite sentiment scoring (4 sources, weighted)
- Add ApeWisdom to data sources table
2026-03-19 20:59:26 +01:00
Yaojia Wang
3c725c45fa feat: aggregate all sentiment sources into composite score
Redesign /stock/{symbol}/sentiment to combine 4 data sources with
weighted scoring:
- News sentiment (Alpha Vantage, 25%) - article-level bullish/bearish
- Analyst recommendations (Finnhub, 30%) - buy/sell ratio
- Upgrade/downgrade activity (yfinance, 20%) - recent actions
- Reddit buzz (ApeWisdom, 25%) - mention change trend

Returns composite_score (-1 to +1), composite_label, per-source
scores, and full detail data from each source.
2026-03-19 20:55:52 +01:00
Yaojia Wang
4eb06dd8e5 feat: add social media sentiment endpoints
- /stock/{symbol}/social-sentiment -- Finnhub Reddit+Twitter sentiment
  (requires premium, gracefully degrades)
- /stock/{symbol}/reddit-sentiment -- Reddit WSB/stocks/investing
  mentions, upvotes, rank via ApeWisdom (free, no key)
- /discover/reddit-trending -- Top 25 trending stocks on Reddit
  (free, no key)

ApeWisdom provides real-time Reddit data without API key.
Finnhub social-sentiment requires premium plan but endpoint
responds gracefully with premium_required flag.
2026-03-19 20:50:28 +01:00
Yaojia Wang
ca8d7099b3 docs: update SKILL.md for 99 endpoints
All checks were successful
continuous-integration/drone/push Build is passing
- Add all new sections: technical (14 indicators), quantitative
  extended (Sortino/Omega/rolling), shorts/dark pool, fixed income,
  macro expanded, economy/surveys, regulators, equity fundamentals
- Add recommended workflows: comprehensive analysis, macro/fixed
  income, sector rotation (RRG), short squeeze screening
- Update description to reflect 99 endpoints and 14 providers
- Add multi-country support note
2026-03-19 20:03:41 +01:00
Yaojia Wang
c5c9c7db83 docs: comprehensive README update for 99 endpoints
- Update endpoint count from 32 to 99
- Add all new endpoint sections: technical (14), quantitative extended,
  shorts/dark pool, fixed income, macro expanded, economy, surveys,
  regulators, equity fundamentals
- Add free providers table (stockgrid, FINRA, CFTC, multpl, ECB, OECD)
- Update data sources table with all 14 providers
- Update project structure with all new service/route files
- Update OpenClaw integration workflow with new endpoints
- Update quick start examples with new features
2026-03-19 19:59:44 +01:00
Yaojia Wang
a57a6835c5 test: fix broken tests after refactor
- Update test_openbb_service to import from obb_utils (to_list,
  first_or_empty) instead of removed _to_dicts/_first_or_empty
- Update test_stock_upgrades to mock openbb_service instead of
  finnhub_service (upgrades moved to yfinance)
2026-03-19 17:47:36 +01:00
Yaojia Wang
89bdc6c552 refactor: address python review findings
- Move FRED credential registration to FastAPI lifespan (was fragile
  import-order-dependent side-effect)
- Add noqa E402 annotations for imports after curl_cffi patch
- Fix all return type hints: bare dict -> dict[str, Any]
- Move yfinance import to module level (was inline in functions)
- Fix datetime.now() -> datetime.now(tz=timezone.utc) in openbb_service
- Add try/except error handling to Group B service functions
- Fix dict mutation in relative_rotation (immutable pattern)
- Extract _classify_rrg_quadrant helper function
- Fix type builtin shadow in routes_economy (type -> gdp_type)
- Fix falsy int guard (if year: -> if year is not None:)
- Remove user input echo from error messages
2026-03-19 17:40:47 +01:00
Yaojia Wang
e2cf6e2488 fix: redesign relative rotation for multi-symbol comparison
- Accept comma-separated symbols query param instead of single path param
- Move endpoint from /stock/{symbol}/technical/relative-rotation to
  /technical/relative-rotation?symbols=AAPL,MSFT,GOOGL&benchmark=SPY
- Fetch all symbols + benchmark in single obb.equity.price.historical call
- Add RRG quadrant classification (Leading/Weakening/Lagging/Improving)
- Support study parameter (price/volume/volatility)
2026-03-19 17:34:18 +01:00
Yaojia Wang
615f17a3bb feat: add remaining 5 endpoints (VWAP, relative rotation, fred-regional, primary dealer)
Complete all 67 planned endpoints:
- VWAP and Relative Rotation technical indicators
- FRED regional data (by state/county/MSA)
- Primary dealer positioning (Fed data)
2026-03-19 17:31:08 +01:00
Yaojia Wang
87260f4b10 feat: add 67 new endpoints across 10 feature groups
Prerequisite refactor:
- Consolidate duplicate _to_dicts into shared obb_utils.to_list
- Add fetch_historical and first_or_empty helpers to obb_utils

Phase 1 - Local computation (no provider risk):
- Group I: 12 technical indicators (ATR, ADX, Stoch, OBV, Ichimoku,
  Donchian, Aroon, CCI, Keltner, Fibonacci, A/D, Volatility Cones)
- Group J: Sortino, Omega ratios + rolling stats (variance, stdev,
  mean, skew, kurtosis, quantile via generic endpoint)
- Group H: ECB currency reference rates

Phase 2 - FRED/Federal Reserve providers:
- Group C: 10 fixed income endpoints (treasury rates, yield curve,
  auctions, TIPS, EFFR, SOFR, HQM, commercial paper, spot rates,
  spreads)
- Group D: 11 economy endpoints (CPI, GDP, unemployment, PCE, money
  measures, CLI, HPI, FRED search, balance of payments, Fed holdings,
  FOMC documents)
- Group E: 5 survey endpoints (Michigan, SLOOS, NFP, Empire State,
  BLS search)

Phase 3 - SEC/stockgrid/FINRA providers:
- Group B: 4 equity fundamental endpoints (management, dividends,
  SEC filings, company search)
- Group A: 4 shorts/dark pool endpoints (short volume, FTD, short
  interest, OTC dark pool)
- Group F: 3 index/ETF enhanced (S&P 500 multiples, index
  constituents, ETF N-PORT)

Phase 4 - Regulators:
- Group G: 5 regulatory endpoints (COT report, COT search, SEC
  litigation, institution search, CIK mapping)

Security hardening:
- Service-layer allowlists for all getattr dynamic dispatch
- Regex validation on date, country, security_type, form_type params
- Exception handling in fetch_historical
- Callable guard on rolling stat dispatch

Total: 32 existing + 67 new = 99 endpoints, all free providers.
2026-03-19 17:28:31 +01:00
Yaojia Wang
b6f49055ad docs: update README for yfinance upgrades/price target, curl_cffi pin
- Remove FMP from optional keys (no longer used)
- Update Finnhub description (upgrades moved to yfinance)
- Update upgrades endpoint description to note yfinance + price targets
- Update data sources table
- Fix conda setup instructions to use pip install -e .
- Add Known Issues section for curl_cffi TLS workaround
2026-03-19 16:36:03 +01:00
Yaojia Wang
ac101c663a fix: switch price target from FMP to yfinance
All checks were successful
continuous-integration/drone/push Build is passing
FMP provider requires a paid API key. yfinance provides
targetMeanPrice in ticker.info for free.
2026-03-19 15:58:17 +01:00
Yaojia Wang
f5b22deec3 fix: resolve curl_cffi TLS errors and fix FRED/upgrades endpoints
All checks were successful
continuous-integration/drone/push Build is passing
- Pin curl_cffi==0.7.4 to avoid BoringSSL bug in 0.12-0.14
- Patch curl_cffi Session to use safari TLS fingerprint instead of
  chrome, which triggers SSL_ERROR_SYSCALL on some networks
- Register FRED API key with OpenBB credentials at startup
- Fix macro overview to return latest data instead of oldest, and
  extract values by FRED series ID key
- Replace Finnhub upgrades endpoint (premium-only) with yfinance
  upgrades_downgrades which includes price target changes
- Remove redundant curl_cffi upgrade from Dockerfile
2026-03-19 15:40:41 +01:00
Yaojia Wang
b631c888a5 fix: keep libssl3 runtime dependency to prevent TLS errors in container
All checks were successful
continuous-integration/drone/push Build is passing
autoremove was removing SSL runtime libraries after purging gcc/g++,
causing curl_cffi TLS handshake failures when connecting to Yahoo Finance.
Explicitly install libssl3 as runtime dep and only purge libssl-dev.
2026-03-19 13:25:58 +01:00
Yaojia Wang
760b0a09ea Trigger build
All checks were successful
continuous-integration/drone/push Build is passing
2026-03-19 11:29:20 +01:00
Yaojia Wang
16ad276146 fix: add list and update verbs to drone RBAC for rollout restart
Some checks failed
continuous-integration/drone/push Build is failing
2026-03-19 11:10:11 +01:00
Yaojia Wang
e5820ebe4a fix: add list and update verbs to drone RBAC for rollout restart
Some checks failed
continuous-integration/drone/push Build is failing
2026-03-19 11:00:32 +01:00
Yaojia Wang
cd6158b05c fix: add list and update verbs to drone RBAC for rollout restart
Some checks failed
continuous-integration/drone/push Build is failing
2026-03-19 10:56:52 +01:00
Yaojia Wang
2446a2fde8 fix: add list and update verbs to drone RBAC for rollout restart
Some checks failed
continuous-integration/drone/push Build is failing
2026-03-19 10:44:35 +01:00
Yaojia Wang
d3c919385f fix: add libnss3 and upgrade curl_cffi to fix TLS/SSL errors
Some checks failed
continuous-integration/drone/push Build is failing
curl_cffi uses BoringSSL internally and needs libnss3 on Debian.
Also upgrade curl_cffi to latest version for compatibility.
2026-03-19 00:38:45 +01:00
Yaojia Wang
e797f8929d fix: use fmp provider for price_target endpoint
Some checks reported errors
continuous-integration/drone/push Build was killed
yfinance is not supported by obb.equity.estimates.price_target,
which only accepts benzinga or fmp.
2026-03-19 00:13:07 +01:00
Yaojia Wang
d46e8685d7 ci: add RBAC for drone to restart invest-api deployment 2026-03-18 23:50:11 +01:00
Yaojia Wang
26cd716590 ci: simplify deploy by using kubectl rollout restart after build
All checks were successful
continuous-integration/drone/push Build is passing
2026-03-18 23:34:54 +01:00
Yaojia Wang
42f25426ac fix: correct drone yaml environment syntax 2026-03-18 23:33:17 +01:00
Yaojia Wang
d4e06c71b7 ci: auto-update image tag in kustomization for ArgoCD sync
Add Drone CI step to update kustomization.yaml newTag with commit SHA
after image build, enabling ArgoCD to detect manifest changes and
auto-deploy new images.
2026-03-18 23:32:17 +01:00
Yaojia Wang
eac9fe963e Trigger build
All checks were successful
continuous-integration/drone/push Build is passing
2026-03-18 21:22:59 +01:00
Yaojia Wang
f4e98653d4 fix: add ca-certificates and libssl-dev to fix OpenSSL TLS errors
yfinance curl calls failed with OPENSSL_internal:invalid library in
python:3.12-slim. Adding ca-certificates, libssl-dev and curl ensures
the OpenSSL libraries are available at runtime.
2026-03-18 15:28:11 +01:00
Yaojia Wang
dd17c4f2ae chore: add API keys to k8s secret
All checks were successful
continuous-integration/drone/push Build is passing
Add Finnhub, FRED, and Alpha Vantage API keys to the
Kubernetes secret manifest for deployment.
2026-03-10 17:23:04 +01:00
Yaojia Wang
82ac3deee4 docs: add Kubernetes deployment section to README
Some checks failed
continuous-integration/drone/push Build is failing
2026-03-10 01:30:43 +01:00
Yaojia Wang
f9f5d37b44 feat: add ingress for invest-api.k8s.home
Some checks failed
continuous-integration/drone/push Build is failing
2026-03-10 01:28:24 +01:00
Yaojia Wang
1a0c8ea675 fix: grant write permission to openbb package dir for build lock
All checks were successful
continuous-integration/drone/push Build is passing
2026-03-10 00:16:12 +01:00
Yaojia Wang
8c682dc5dd fix: create home directory for OpenBB in Docker container
All checks were successful
continuous-integration/drone/push Build is passing
2026-03-10 00:10:54 +01:00
Yaojia Wang
e807af6395 chore: trigger drone build
All checks were successful
continuous-integration/drone/push Build is passing
2026-03-10 00:06:20 +01:00
Yaojia Wang
4565edf432 chore: trigger drone build
Some checks reported errors
continuous-integration/drone/push Build encountered an error
2026-03-10 00:05:14 +01:00
Yaojia Wang
79d9d5a012 chore: trigger drone build 2026-03-09 23:56:47 +01:00
Yaojia Wang
64fb5fc43e chore: test drone webhook 2026-03-09 23:55:45 +01:00
Yaojia Wang
faec132bbe chore: trigger drone build 2026-03-09 23:48:14 +01:00
Yaojia Wang
e7c747d297 fix: add custom event trigger for Drone manual builds 2026-03-09 23:37:16 +01:00
Yaojia Wang
ec76e92f1e fix: switch Drone pipeline from docker to kaniko for k8s runner 2026-03-09 23:30:58 +01:00
Yaojia Wang
d05cb55cb0 feat: add Docker, Drone CI, and k8s deployment manifests
- Dockerfile for Python 3.12 FastAPI app
- Drone CI pipeline to build and push to internal registry
- Kubernetes manifests (Deployment, Service, Secret, Namespace)
- ArgoCD Application for GitOps deployment
- Kustomize base configuration
2026-03-09 23:28:31 +01:00
Yaojia Wang
e3e9c1986c docs: add route_utils and obb_utils to project structure in README 2026-03-09 14:50:54 +01:00
Yaojia Wang
003c1d6ffc refactor: fix code review issues across routes and services
- Extract shared route_utils.py (validate_symbol, safe decorator)
  removing duplication from 6 route files
- Extract shared obb_utils.py (to_list, extract_single, safe_last)
  removing duplication from calendar_service and market_service
- Fix _to_list dict mutation during iteration (use comprehension)
- Fix double vars() call and live __dict__ mutation risk
- Fix route ordering: /etf/search and /crypto/search now registered
  before /{symbol} path params to prevent shadowing
- Add date format validation (YYYY-MM-DD pattern) on calendar routes
- Use timezone-aware datetime.now(tz=timezone.utc) in all services
- Add explicit type annotation for asyncio.gather results
2026-03-09 10:56:21 +01:00
Yaojia Wang
507194397e feat: integrate quantitative, calendar, market data endpoints
Add 3 new service layers and route modules:
- quantitative_service: Sharpe ratio, CAPM, normality tests, unit root tests
- calendar_service: earnings/dividends/IPO/splits calendars, estimates, SEC ownership
- market_service: ETF, index, crypto, forex, options, futures data

Total endpoints: 50. All use free providers (yfinance, SEC).
Update README with comprehensive endpoint documentation.
2026-03-09 10:28:33 +01:00
Yaojia Wang
00f2cb5e74 feat: integrate Alpha Vantage news sentiment + fix technical indicators
- Add alphavantage_service.py for per-article sentiment scores
- Add /stock/{symbol}/news-sentiment endpoint (Alpha Vantage)
- Merge Alpha Vantage data into /stock/{symbol}/sentiment
- Fix technical indicators: use close_ prefixed keys from OpenBB
- Increase historical data to 400 days for SMA_200 calculation
- Add .gitignore, handle Finnhub 403 on premium endpoints
- Add INVEST_API_ALPHAVANTAGE_API_KEY config
2026-03-09 00:37:32 +01:00