You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

23 KiB

Stock Oracle API Documentation 🔮

Comprehensive Investment Data Analysis API

Stock Oracle provides comprehensive financial, price, news, and social media data analysis through a unified REST API. Built for investors, analysts, and developers who need reliable access to SEC filings, market data, and sentiment analysis.


🚀 Quick Start

Running the API

# Start all services (first time)
docker-compose up -d

# Restart API after code changes
just run dev

Base URL

http://localhost:18001/api/v1

Authentication

Currently no authentication required. API key support coming soon.

Rate Limits

  • 100 requests per minute per IP
  • 10,000 requests per day per IP

📊 Core Data Sources

  • SEC EDGAR: Official company filings (10-K, 10-Q, N-PORT)
  • Yahoo Finance: Real-time price data and news
  • Alpaca: Market price data (OHLCV + VWAP) via official REST API (optional API key)
  • FINRA: RegSHO short sale volume data (public CDN, no API key required)
  • NewsAPI: Professional news aggregation
  • Reddit API: Social sentiment analysis

🎯 API Endpoints

Health & System

GET /health

Basic health check with system status.

Response:

{
  "status": "healthy",
  "version": "1.0.0",
  "database": "healthy",
  "cache": "healthy",
  "sec_data_available": true,
  "timestamp": "2025-08-10T17:59:27.790041"
}

GET /health/detailed

Detailed system health with component status.


Financial Data

POST /financial/data

Get comprehensive financial data for a ticker.

Request Body:

{
  "ticker": "AAPL",
  "period": "1y",
  "period_type": "quarterly",
  "include_metrics": true,
  "force_refresh": false
}

Alternative Time Specifications:

// Date Range
{
  "ticker": "AAPL",
  "start_date": "2023-01-01",
  "end_date": "2023-12-31"
}

// Specific Quarters
{
  "ticker": "AAPL", 
  "quarters": ["2024Q1", "2024Q2", "2024Q3"]
}

Response:

{
  "company": {
    "ticker": "AAPL",
    "name": "Apple Inc.",
    "cik": "320193",
    "sector": "Technology",
    "industry": "Consumer Electronics"
  },
  "financial_data": [
    {
      "period_date": "2024-06-30",
      "period_type": "quarterly",
      "revenue": 85777000000,
      "gross_profit": 35398000000,
      "operating_income": 24261000000,
      "net_income": 21448000000,
      "eps": 1.40,
      "pe_ratio": 28.5,
      "roe": 0.63,
      "debt_to_equity": 1.97,
      "market_cap": 3200000000000
    }
  ],
  "metadata": {
    "data_points": 8,
    "period_type": "quarterly",
    "last_updated": "2025-08-10T12:00:00"
  }
}

GET /financial/data/{ticker}

Simplified financial data endpoint with query parameters.

Parameters:

  • period: Time period (1d, 7d, 30d, 1m, 3m, 6m, 1y, 2y, 5y, 10y)
  • period_type: quarterly, annual, all
  • include_metrics: true/false
  • force_refresh: true/false

POST /financial/data/bulk

Get financial data for multiple tickers in a single request.

Request:

{
  "tickers": ["AAPL", "MSFT", "GOOGL"],
  "period": "1y",
  "include_metrics": true
}

Price Data

POST /price/data

Get historical price data (OHLCV) for a ticker.

Request:

{
  "ticker": "AAPL",
  "period": "30d",
  "interval": "1d",
  "force_refresh": false
}

Supported Intervals:

  • 1m, 2m, 5m, 15m, 30m, 60m, 90m (minutes)
  • 1h (hour)
  • 1d, 5d (days)
  • 1wk (week)
  • 1mo, 3mo (months)

Response:

{
  "ticker": "AAPL",
  "price_data": [
    {
      "date": "2024-08-10",
      "open": 220.05,
      "high": 225.30,
      "low": 218.75,
      "close": 224.72,
      "volume": 45234567,
      "adj_close": 224.72
    }
  ],
  "interval": "1d"
}

POST /price/data/bulk

Bulk price data for multiple tickers.


Alpaca Market Data 🆕

GET /alpaca/status

Check Alpaca API key validity and connection health.

Response:

{
  "configured": true,
  "connected": true,
  "bars_returned": 1,
  "base_url": "https://data.alpaca.markets"
}

GET /alpaca/bars/{ticker}

Fetch raw bars directly from Alpaca without DB storage.

Parameters:

  • interval: Bar interval — 1m, 5m, 15m, 1h, 1d, 1w, 1mo (default: 1d)
  • start_date: Start date (YYYY-MM-DD)
  • end_date: End date (YYYY-MM-DD)
  • limit: Max bars to return (1-10000, default: 1000)

Response:

{
  "ticker": "AAPL",
  "interval": "1d",
  "count": 5,
  "bars": [
    {
      "timestamp": "2025-01-02T05:00:00Z",
      "open": 248.93,
      "high": 249.1,
      "low": 241.82,
      "close": 243.85,
      "volume": 55740731,
      "vwap": 244.339399,
      "trade_count": 685448
    }
  ]
}

GET /alpaca/data/{ticker}

Fetch OHLCV bars from Alpaca, store in DB, and return in PriceDataResponse format.

Parameters:

  • interval: Bar interval (default: 1d)
  • start_date: Start date (required, YYYY-MM-DD)
  • end_date: End date (required, YYYY-MM-DD)
  • force_refresh: Re-fetch even if data exists (default: false)

Response: Same PriceDataResponse format as /price/data/{ticker} with data_source: "ALPACA".

GET /alpaca/intraday/{ticker}

Fetch intraday candles from Alpaca (not stored in DB).

Parameters:

  • interval: 1m, 5m, 15m, 1h (default: 1m)
  • start_date, end_date: Date range
  • limit: Max candles (default: 1000)

FINRA Short Volume Data 🆕

GET /finra/short-volume/{symbol}

Query FINRA RegSHO short sale volume for a symbol. Auto-ingests missing data.

Parameters:

  • days: Number of days to look back (1-365, default: 30)
  • limit: Max entries to return (1-1000, default: 100)

Response:

{
  "symbol": "AAPL",
  "entries": [
    {
      "date": "2025-03-11",
      "symbol": "AAPL",
      "short_volume": 3214614.5,
      "short_exempt_volume": 29855.0,
      "total_volume": 7471743.3,
      "market": "B,Q,N",
      "short_ratio": 0.430236
    }
  ],
  "total_count": 21,
  "metadata": {
    "days_requested": 30,
    "start_date": "2025-02-10",
    "end_date": "2025-03-12"
  }
}

GET /finra/short-ratio/{symbol}

Return daily short_ratio (aggregated across markets) for the last N days.

Parameters:

  • days: Number of days (1-365, default: 60)

Response:

{
  "symbol": "AAPL",
  "history": [
    {
      "date": "2025-03-09",
      "short_volume": 4040406.5,
      "short_exempt_volume": 52868.5,
      "total_volume": 11925537.0,
      "short_ratio": 0.338803
    }
  ],
  "avg_short_ratio": 0.400623,
  "metadata": { "days_requested": 5, "data_points": 3 }
}

POST /finra/admin/ingest

Download and ingest FINRA short volume file(s) for a specific date or date range.

Parameters:

  • date: Single date (YYYY-MM-DD)
  • start_date + end_date: Date range
  • force: Re-ingest even if data exists (default: false)

Response:

{
  "date": "2025-03-10",
  "records_ingested": 10433,
  "status": "completed"
}

Stock Market Data 🆕

GET /stocks/index/{index_name}

S&P 500 또는 Nasdaq 100 구성 종목을 Wikipedia에서 조회합니다.

Path Parameters:

  • index_name: sp500 또는 nasdaq100

Query Parameters:

  • force_refresh: true/false (default: false) — 캐시 무시하고 재조회

Caching:

  • TTL: 24시간 (Cache-Control: public, max-age=86400)
  • X-Cache: HIT / MISS, ETag 헤더 포함

Response:

{
  "success": true,
  "index": "sp500",
  "count": 503,
  "constituents": [
    {
      "symbol": "AAPL",
      "name": "Apple Inc.",
      "sector": "Information Technology",
      "industry": "Technology Hardware, Storage & Peripherals"
    }
  ]
}

Examples:

GET /stocks/index/sp500        → S&P 500 구성 종목 (~503개)
GET /stocks/index/nasdaq100    → Nasdaq 100 구성 종목 (~101개)
GET /stocks/index/sp500?force_refresh=true  → 캐시 무시하고 재조회
GET /stocks/index/foo          → 400 Bad Request

Error Codes:

  • 400: 지원하지 않는 index_name
  • 504: Wikipedia 응답 30초 초과

GET /stocks/gainers

오늘의 상승 상위 종목 (Yahoo Finance day_gainers preset).

조건: 등락률 >3%, 시총 ≥$2B, 주가 ≥$5, 거래량 >15,000. 등락률 내림차순 정렬.

캐시 없음 — 실시간 데이터.

Query Parameters:

  • page: 페이지 번호 (1-based, default: 1)
  • page_size: 페이지당 결과 수 (1250, default: 25)

Response:

{
  "stocks": [
    {
      "symbol": "HUT",
      "name": "Hut 8 Corp.",
      "exchange": "NASDAQ",
      "market_cap": 3200000000,
      "price": 106.46,
      "change_percent": 32.26,
      "volume": 18500000,
      "avg_volume_3m": 4200000,
      "pe_ratio": null,
      "fifty_two_week_high": 138.0,
      "fifty_two_week_low": 10.5
    }
  ],
  "total_available": 25,
  "returned_count": 25,
  "page": 1,
  "page_size": 25,
  "total_pages": 1,
  "query_time_seconds": 1.82,
  "metadata": {
    "preset": "day_gainers",
    "source": "yfinance_screen_preset"
  }
}

Examples:

GET /stocks/gainers              → 상위 25개
GET /stocks/gainers?page_size=10 → 상위 10개
GET /stocks/gainers?page=2       → 2페이지

Error Codes:

  • 503: Yahoo Finance rate limit — 3060초 후 재시도

GET /stocks/most-active

Most actively traded stocks from Yahoo Finance.

Parameters:

  • limit: Number of stocks to return (1-500). If omitted, returns all available (~170)
  • force_refresh: true/false (default: false). When true, bypasses cache and fetches fresh data

Caching:

  • Server-side cache TTL: 1 hour
  • Cache key: stocks:most-active:limit=<N|all>
  • Response headers:
    • X-Cache: HIT | MISS | BYPASS
    • Cache-Control: public, max-age=3600
    • ETag: Strong hash of the response
    • X-Data-Source: redis-cache | scraper

Examples:

# Default (cached up to 1h)
GET /stocks/most-active

# Limit results (cached per limit)
GET /stocks/most-active?limit=100

# Force fresh fetch (bypass cache)
GET /stocks/most-active?force_refresh=true

# Limit + fresh
GET /stocks/most-active?limit=50&force_refresh=true

Stock Screener

GET /screener/stocks

조건 기반 종목 필터링 (EquityQuery).

Query Parameters:

  • market_cap_min / market_cap_max: 시총 범위 (USD)
  • exchange: NYSE, NASDAQ, AMEX, NYSE_ARCA (콤마 구분)
  • min_avg_volume: 3개월 평균 거래량 최솟값
  • exclude_types: 제외할 종목 유형 (예: ETF,FUND)
  • sector: 섹터 (예: Technology, Healthcare)
  • pe_min / pe_max: Trailing P/E 범위
  • price_min / price_max: 주가 범위
  • page / page_size: 페이지네이션 (max 250)
  • sort_by: market_cap, volume, price, change_percent
  • sort_ascending: true/false (default: false)
  • force_refresh: 캐시 무시

Caching: Redis 5분

Examples:

GET /screener/stocks?market_cap_min=500000000&exchange=NYSE,NASDAQ&exclude_types=ETF,FUND
GET /screener/stocks?sector=Technology&pe_max=30&sort_by=change_percent

News & Social Media 🆕

GET /news/{ticker}

Complete news and social media data for a ticker.

Parameters:

  • days_back: Days to look back (1-30, default: 7)
  • max_articles: Max news articles (5-100, default: 20)
  • max_social_posts: Max social posts (0-100, default: 15)
  • include_social: Include social media (true/false, default: true)

Example:

GET /news/AAPL?days_back=7&max_articles=20&include_social=true

Response:

{
  "ticker": "AAPL",
  "retrieved_at": "2025-08-10T17:40:04.781906",
  "news": {
    "total_articles": 12,
    "sources": {
      "yahoo_finance": 6,
      "newsapi": 6
    },
    "articles": [
      {
        "title": "Apple Reports Strong Q3 Results",
        "summary": "Apple exceeded expectations with record iPhone sales...",
        "url": "https://finance.yahoo.com/news/apple-q3-2024",
        "source": "Yahoo Finance", 
        "published_at": "2025-08-10T14:30:00",
        "author": "John Smith",
        "tags": ["earnings", "iphone", "revenue"]
      }
    ]
  },
  "social_media": {
    "total_posts": 8,
    "platforms": {"reddit": 8},
    "posts": [
      {
        "title": "$AAPL breakout incoming? Technical analysis",
        "content": "Looking at the charts, AAPL seems to be forming...",
        "url": "https://reddit.com/r/stocks/comments/xyz",
        "platform": "Reddit",
        "author": "trader123",
        "published_at": "2025-08-10T16:20:00",
        "score": 245,
        "comments_count": 67,
        "subreddit": "stocks"
      }
    ]
  },
  "summary": {
    "total_items": 20,
    "time_range_days": 7,
    "newest_item": "2025-08-10T16:20:00",
    "oldest_item": "2025-08-03T09:15:00"
  }
}

GET /news/{ticker}/news-only

News articles only (faster response, no social media).

GET /news/{ticker}/social-only

Social media posts only.


News v2 — Multi-source Headlines & Session Aggregates 🆕

Structured, persisted, multi-source news/social ingest designed for backtest and forward-test consumers (e.g. fithia2 V49 ORB). Distinct from the legacy /news/{ticker} aggregator above, which is on-demand and not persisted.

Sources — all enabled via NEWS_INGEST_ENABLED=true in .env:

Source Status History Rate limit Sentiment
alpaca_benzinga P0 ~30 days vendor cap (cumulative archive built daily) 200 req/min None on free tier
stocktwits P1 rolling 200 req/hr/IP Bullish/Bearish tag → ±1
finnhub P2 ~12 months vendor cap 60 calls/min free None
gdelt (separate) 2017+ varies None

Unified taxonomy — vendor categories are normalized to a 22-term enum: analyst_rating_upgrade, analyst_rating_downgrade, analyst_rating_initiate, earnings_release, earnings_preannouncement, guidance_update, m_and_a, partnership, contract_award, fda_approval, fda_rejection, clinical_trial, litigation, regulatory_action, sec_filing, insider_trading, secondary_offering, buyback, management_change, restructuring, general.

Original vendor labels are preserved on vendor_categories.

Session windows (NYSE / pandas_market_calendars XNYS, holidays + early closes honored):

  • premarket = previous session's close → today 09:30 ET
  • intraday = 09:30 → 16:00 ET
  • post = 16:00 ET → next trading day 04:00 ET (disjoint from next premarket)
  • full_session = previous close → next trading day 04:00 ET

PIT safety: aggregates filter ingested_at <= window_end_utc, so backtests never see headlines that arrived after the window closed in real time.

GET /news/v2/headlines

Raw multi-source rows.

Query parameters:

  • symbols — CSV ticker list (max 50)
  • start, end — UTC ISO datetime
  • sources — CSV filter (subset of source names)
  • limit — 1500 (default 100)
  • cursorpublished_at_lt ISO datetime for pagination

Response:

{
  "items": [
    {
      "source": "alpaca_benzinga",
      "source_id": "12345",
      "ticker": "AAPL",
      "tickers_all": ["AAPL", "MSFT"],
      "published_at": "2026-04-25T13:30:00+00:00",
      "headline": "Apple announces $90B buyback",
      "summary": "...",
      "url": "https://...",
      "language": "en",
      "vendor_categories": ["Buybacks"],
      "categories": ["buyback"],
      "raw_sentiment": null,
      "is_primary": true,
      "ingested_at": "2026-04-25T13:32:11+00:00"
    }
  ],
  "next_cursor": "2026-04-25T13:30:00+00:00"
}

GET /news/v2/session_aggregate

One-ticker, one-window aggregate. Redis-cached (10 min current session, 1 day past).

Query parameters:

  • symbol (required)
  • session_date (required) — ET date YYYY-MM-DD
  • window — premarket | intraday | post | full_session (default premarket)
  • sources — CSV filter (optional)
  • force_refresh — bypass cache

Response: see batch response below (single object, not wrapped in items).

POST /news/v2/session_aggregate/batch

Many tickers in one call (V49's hot path — 20 ticker batch per session). No server-side cache — fithia2 maintains a client-side disk cache as the primary defense; Oracle absorbs only burst load.

Body:

{
  "session_date": "2026-04-25",
  "window": "premarket",
  "symbols": ["AAPL", "MSFT", "NVDA"],
  "sources": ["alpaca_benzinga", "stocktwits"]
}

Response:

{
  "items": {
    "AAPL": {
      "ticker": "AAPL",
      "session_date": "2026-04-25",
      "window": "premarket",
      "headline_count": 7,
      "primary_count": 4,
      "first_headline_at": "2026-04-24T20:15:00+00:00",
      "last_headline_at": "2026-04-25T13:01:55+00:00",
      "category_counts": {
        "analyst_rating_upgrade": 2,
        "earnings_release": 1,
        "guidance_update": 1,
        "general": 3
      },
      "sentiment_mean": 0.31,
      "sentiment_recency_weighted": 0.45,
      "social": {
        "message_count": 142,
        "bull_count": 98,
        "bear_count": 31,
        "bull_bear_ratio": 0.7597
      },
      "sources_present": ["alpaca_benzinga", "stocktwits"]
    },
    "MSFT": { "...": "..." }
  }
}

Tickers with no matching headlines are returned with zero-counts (not omitted).

GET /news/v2/coverage

Per-source ingest depth probe. Use before backtest window selection to confirm the historical archive is deep enough.

Query parameters:

  • source (required) — one of the source names
  • symbol — optional ticker filter

Response:

{
  "source": "alpaca_benzinga",
  "symbol": "AAPL",
  "earliest": "2026-03-26T00:00:00+00:00",
  "latest": "2026-04-25T13:42:00+00:00",
  "ingested_count": 31204
}

Operational notes

  • Ingest is opt-in. NEWS_INGEST_ENABLED=false (default) leaves the scheduler off; endpoints still work and return empty results until data flows in.
  • Fail-fast. When NEWS_INGEST_ENABLED=true but neither ALPACA_API_KEY/SECRET nor FINNHUB_API_KEY is set, the scheduler refuses to start (the StockTwits-only configuration is too low-signal to run silently).
  • Historical backfill. Alpaca News only exposes the last ~30 days, so cumulative depth is built by the daily backfill job from the moment ingest is enabled. Finnhub's 12-month archive is loaded once via:
    docker exec stock_oracle_api python scripts/news_backfill.py \
      --source finnhub \
      --tickers AAPL,MSFT,NVDA \
      --start 2025-04-26 --end 2026-04-26 \
      --chunk monthly
    
  • StockTwits universe. Computed daily at 09:00 ET as (last 14 days of UniverseSnapshot active tickers) (today's premarket gap movers > STOCKTWITS_PREMARKET_GAP_THRESHOLD), capped at STOCKTWITS_UNIVERSE_MAX_SIZE (default 300). Pollers read from Redis key news_v2:stocktwits:universe.

ETF Holdings

Temporarily unavailable. The ETF API is being redesigned. Previous endpoints under /etf/* have been removed and will return 404. See docs/ETF_API.md for historical reference only.


Database & Metadata

GET /database/stats

Database statistics and data coverage information.

GET /metadata/catalog

Complete data field catalog with descriptions and types.


Admin & Monitoring

GET /admin/errors/logs

Error log retrieval (admin access).

Parameters:

  • limit: Number of logs (default: 100)
  • offset: Pagination offset (default: 0)
  • min_level: Minimum log level (ERROR, WARNING, INFO)

GET /admin/errors/stats

Error statistics and trends.

POST /admin/migrate

Database migration from another Stock Oracle instance.


🔧 Python Client Usage

Installation

# Download the client from the repository
wget https://raw.githubusercontent.com/your-repo/stock-oracle/main/stock_oracle_client.py

Basic Usage

from stock_oracle_client import StockOracleClient

# Initialize client
client = StockOracleClient("http://localhost:18001")

# Check API health
health = client.get_health()
print("API Status:", health["status"])

# Get financial data
data = client.get_financial_data("AAPL", period="1y")
print(f"Found {len(data['financial_data'])} quarters of data")

# Get news and social media data
news = client.get_news_social_data("AAPL", days_back=7, max_articles=20)
print(f"Found {news['summary']['total_items']} news/social items")

# Get ETF holdings
etf = client.get_etf_holdings("QQQ", include_holdings=False)
print(f"QQQ has {etf['data']['holdings_count']} holdings")

# Bulk operations
bulk_data = client.get_bulk_financial_data(
    tickers=["AAPL", "MSFT", "GOOGL"],
    period="2y"
)

Error Handling

from stock_oracle_client import StockOracleAPIError, ETFDataNotAvailableError

try:
    data = client.get_financial_data("INVALID_TICKER")
except StockOracleAPIError as e:
    print(f"API Error: {e}")
    print(f"Status Code: {e.status_code}")

try:
    etf = client.get_etf_holdings("QQQ", as_of_date="1990-01-01")
except ETFDataNotAvailableError as e:
    print(f"ETF data not available: {e}")
    if e.availability_info:
        print(f"Available from: {e.availability_info['available_date_range']['start']}")

📈 Investment Analysis

Stock Oracle Analyzer

The included analyzer provides comprehensive investment analysis combining all data sources:

from stock_oracle_analyzer import StockOracleAnalyzer

# Initialize analyzer
analyzer = StockOracleAnalyzer("http://localhost:18001")

# Analyze single company
data = analyzer.get_company_data("AAPL", period="2y")
analysis = data['analysis_summary']

print(f"Investment Score: {analysis['investment_score']}/100")
print(f"Financial Grade: {analysis['financial_health']['grade']}")
print(f"Price Trend: {analysis['price_trends']['trend']}")
print(f"Sentiment: {analysis['sentiment_analysis']['sentiment_label']}")

# Compare multiple companies
tickers = ["AAPL", "MSFT", "GOOGL", "TSLA", "NVDA"]
results = analyzer.analyze_multiple_companies(tickers, period="1y")

# Generate comparison report
summary_df = analyzer.create_summary_report(results)
print(summary_df[['Ticker', 'Investment Score', 'Financial Grade', 'Sentiment']])

⚠️ Important Notes

Data Availability

  • Period parameters automatically use yesterday as end date to ensure data availability
  • SEC filings may have delays - latest data is typically 1-3 months behind
  • ETF holdings are updated quarterly via N-PORT filings
  • News data is real-time but may have API rate limits

Performance Tips

  • Use bulk endpoints for multiple tickers to reduce latency
  • Enable caching by avoiding force_refresh=true unless necessary
  • Use news-only endpoints for faster sentiment analysis
  • Implement client-side caching for frequently accessed data

Error Codes

  • 400: Bad Request (invalid parameters)
  • 404: Data not found (ticker not found, no filings available)
  • 429: Rate limit exceeded
  • 500: Internal server error
  • 503: Service temporarily unavailable


📞 Support

  • Issues: Report bugs and feature requests on GitHub
  • Documentation: This page is auto-generated from the latest API specification
  • Updates: Check the changelog for latest features and improvements

Stock Oracle - Empowering investment decisions with comprehensive data analysis 🔮📈