# Stock Oracle API Documentation 🔮

**Comprehensive Investment Data Analysis API**

Stock Oracle provides comprehensive financial, price, news, and social media data analysis through a unified REST API. Built for investors, analysts, and developers who need reliable access to SEC filings, market data, and sentiment analysis.

---

## 🚀 Quick Start

### Running the API

```bash
# Start all services (first time)
docker-compose up -d

# Restart API after code changes
just run dev
```

### Base URL
```
http://localhost:18001/api/v1
```

### Authentication
Currently no authentication required. API key support coming soon.

### Rate Limits
- 100 requests per minute per IP
- 10,000 requests per day per IP

---

## 📊 Core Data Sources

- **SEC EDGAR**: Official company filings (10-K, 10-Q, N-PORT)
- **Yahoo Finance**: Real-time price data and news
- **Alpaca**: Market price data (OHLCV + VWAP) via official REST API (optional API key)
- **FINRA**: RegSHO short sale volume data (public CDN, no API key required)
- **NewsAPI**: Professional news aggregation
- **Reddit API**: Social sentiment analysis

---

## 🎯 API Endpoints

### Health & System

#### `GET /health`
Basic health check with system status.

**Response:**
```json
{
  "status": "healthy",
  "version": "1.0.0",
  "database": "healthy",
  "cache": "healthy",
  "sec_data_available": true,
  "timestamp": "2025-08-10T17:59:27.790041"
}
```

#### `GET /health/detailed`
Detailed system health with component status.

---

### Financial Data

#### `POST /financial/data`
Get comprehensive financial data for a ticker.

**Request Body:**
```json
{
  "ticker": "AAPL",
  "period": "1y",
  "period_type": "quarterly",
  "include_metrics": true,
  "force_refresh": false
}
```

**Alternative Time Specifications:**
```json
// Date Range
{
  "ticker": "AAPL",
  "start_date": "2023-01-01",
  "end_date": "2023-12-31"
}

// Specific Quarters
{
  "ticker": "AAPL", 
  "quarters": ["2024Q1", "2024Q2", "2024Q3"]
}
```

**Response:**
```json
{
  "company": {
    "ticker": "AAPL",
    "name": "Apple Inc.",
    "cik": "320193",
    "sector": "Technology",
    "industry": "Consumer Electronics"
  },
  "financial_data": [
    {
      "period_date": "2024-06-30",
      "period_type": "quarterly",
      "revenue": 85777000000,
      "gross_profit": 35398000000,
      "operating_income": 24261000000,
      "net_income": 21448000000,
      "eps": 1.40,
      "pe_ratio": 28.5,
      "roe": 0.63,
      "debt_to_equity": 1.97,
      "market_cap": 3200000000000
    }
  ],
  "metadata": {
    "data_points": 8,
    "period_type": "quarterly",
    "last_updated": "2025-08-10T12:00:00"
  }
}
```

#### `GET /financial/data/{ticker}`
Simplified financial data endpoint with query parameters.

**Parameters:**
- `period`: Time period (1d, 7d, 30d, 1m, 3m, 6m, 1y, 2y, 5y, 10y)
- `period_type`: quarterly, annual, all
- `include_metrics`: true/false
- `force_refresh`: true/false

#### `POST /financial/data/bulk`
Get financial data for multiple tickers in a single request.

**Request:**
```json
{
  "tickers": ["AAPL", "MSFT", "GOOGL"],
  "period": "1y",
  "include_metrics": true
}
```

---

### Price Data

#### `POST /price/data`
Get historical price data (OHLCV) for a ticker.

**Request:**
```json
{
  "ticker": "AAPL",
  "period": "30d",
  "interval": "1d",
  "force_refresh": false
}
```

**Supported Intervals:**
- `1m`, `2m`, `5m`, `15m`, `30m`, `60m`, `90m` (minutes)
- `1h` (hour)
- `1d`, `5d` (days)
- `1wk` (week)
- `1mo`, `3mo` (months)

**Response:**
```json
{
  "ticker": "AAPL",
  "price_data": [
    {
      "date": "2024-08-10",
      "open": 220.05,
      "high": 225.30,
      "low": 218.75,
      "close": 224.72,
      "volume": 45234567,
      "adj_close": 224.72
    }
  ],
  "interval": "1d"
}
```

#### `POST /price/data/bulk`
Bulk price data for multiple tickers.

---

### Alpaca Market Data 🆕

#### `GET /alpaca/status`
Check Alpaca API key validity and connection health.

**Response:**
```json
{
  "configured": true,
  "connected": true,
  "bars_returned": 1,
  "base_url": "https://data.alpaca.markets"
}
```

#### `GET /alpaca/bars/{ticker}`
Fetch raw bars directly from Alpaca without DB storage.

**Parameters:**
- `interval`: Bar interval — `1m`, `5m`, `15m`, `1h`, `1d`, `1w`, `1mo` (default: `1d`)
- `start_date`: Start date (YYYY-MM-DD)
- `end_date`: End date (YYYY-MM-DD)
- `limit`: Max bars to return (1-10000, default: 1000)

**Response:**
```json
{
  "ticker": "AAPL",
  "interval": "1d",
  "count": 5,
  "bars": [
    {
      "timestamp": "2025-01-02T05:00:00Z",
      "open": 248.93,
      "high": 249.1,
      "low": 241.82,
      "close": 243.85,
      "volume": 55740731,
      "vwap": 244.339399,
      "trade_count": 685448
    }
  ]
}
```

#### `GET /alpaca/data/{ticker}`
Fetch OHLCV bars from Alpaca, store in DB, and return in PriceDataResponse format.

**Parameters:**
- `interval`: Bar interval (default: `1d`)
- `start_date`: Start date (required, YYYY-MM-DD)
- `end_date`: End date (required, YYYY-MM-DD)
- `force_refresh`: Re-fetch even if data exists (default: false)

**Response:** Same `PriceDataResponse` format as `/price/data/{ticker}` with `data_source: "ALPACA"`.

#### `GET /alpaca/intraday/{ticker}`
Fetch intraday candles from Alpaca (not stored in DB).

**Parameters:**
- `interval`: `1m`, `5m`, `15m`, `1h` (default: `1m`)
- `start_date`, `end_date`: Date range
- `limit`: Max candles (default: 1000)

---

### FINRA Short Volume Data 🆕

#### `GET /finra/short-volume/{symbol}`
Query FINRA RegSHO short sale volume for a symbol. Auto-ingests missing data.

**Parameters:**
- `days`: Number of days to look back (1-365, default: 30)
- `limit`: Max entries to return (1-1000, default: 100)

**Response:**
```json
{
  "symbol": "AAPL",
  "entries": [
    {
      "date": "2025-03-11",
      "symbol": "AAPL",
      "short_volume": 3214614.5,
      "short_exempt_volume": 29855.0,
      "total_volume": 7471743.3,
      "market": "B,Q,N",
      "short_ratio": 0.430236
    }
  ],
  "total_count": 21,
  "metadata": {
    "days_requested": 30,
    "start_date": "2025-02-10",
    "end_date": "2025-03-12"
  }
}
```

#### `GET /finra/short-ratio/{symbol}`
Return daily short_ratio (aggregated across markets) for the last N days.

**Parameters:**
- `days`: Number of days (1-365, default: 60)

**Response:**
```json
{
  "symbol": "AAPL",
  "history": [
    {
      "date": "2025-03-09",
      "short_volume": 4040406.5,
      "short_exempt_volume": 52868.5,
      "total_volume": 11925537.0,
      "short_ratio": 0.338803
    }
  ],
  "avg_short_ratio": 0.400623,
  "metadata": { "days_requested": 5, "data_points": 3 }
}
```

#### `POST /finra/admin/ingest`
Download and ingest FINRA short volume file(s) for a specific date or date range.

**Parameters:**
- `date`: Single date (YYYY-MM-DD)
- `start_date` + `end_date`: Date range
- `force`: Re-ingest even if data exists (default: false)

**Response:**
```json
{
  "date": "2025-03-10",
  "records_ingested": 10433,
  "status": "completed"
}
```

---

### Stock Market Data 🆕

#### `GET /stocks/index/{index_name}`
S&P 500 또는 Nasdaq 100 구성 종목을 Wikipedia에서 조회합니다.

**Path Parameters:**
- `index_name`: `sp500` 또는 `nasdaq100`

**Query Parameters:**
- `force_refresh`: `true`/`false` (default: `false`) — 캐시 무시하고 재조회

**Caching:**
- TTL: 24시간 (`Cache-Control: public, max-age=86400`)
- `X-Cache: HIT` / `MISS`, `ETag` 헤더 포함

**Response:**
```json
{
  "success": true,
  "index": "sp500",
  "count": 503,
  "constituents": [
    {
      "symbol": "AAPL",
      "name": "Apple Inc.",
      "sector": "Information Technology",
      "industry": "Technology Hardware, Storage & Peripherals"
    }
  ]
}
```

**Examples:**
```
GET /stocks/index/sp500        → S&P 500 구성 종목 (~503개)
GET /stocks/index/nasdaq100    → Nasdaq 100 구성 종목 (~101개)
GET /stocks/index/sp500?force_refresh=true  → 캐시 무시하고 재조회
GET /stocks/index/foo          → 400 Bad Request
```

**Error Codes:**
- `400`: 지원하지 않는 `index_name`
- `504`: Wikipedia 응답 30초 초과

---

#### `GET /stocks/most-active`
Most actively traded stocks from Yahoo Finance.

**Parameters:**
- `limit`: Number of stocks to return (1-500). If omitted, returns all available (~170)
- `force_refresh`: true/false (default: false). When true, bypasses cache and fetches fresh data

**Caching:**
- Server-side cache TTL: 1 hour
- Cache key: `stocks:most-active:limit=<N|all>`
- Response headers:
  - `X-Cache`: `HIT` | `MISS` | `BYPASS`
  - `Cache-Control`: `public, max-age=3600`
  - `ETag`: Strong hash of the response
  - `X-Data-Source`: `redis-cache` | `scraper`

**Examples:**
```
# Default (cached up to 1h)
GET /stocks/most-active

# Limit results (cached per limit)
GET /stocks/most-active?limit=100

# Force fresh fetch (bypass cache)
GET /stocks/most-active?force_refresh=true

# Limit + fresh
GET /stocks/most-active?limit=50&force_refresh=true
```

---

### News & Social Media 🆕

#### `GET /news/{ticker}`
Complete news and social media data for a ticker.

**Parameters:**
- `days_back`: Days to look back (1-30, default: 7)
- `max_articles`: Max news articles (5-100, default: 20)
- `max_social_posts`: Max social posts (0-100, default: 15)
- `include_social`: Include social media (true/false, default: true)

**Example:**
```
GET /news/AAPL?days_back=7&max_articles=20&include_social=true
```

**Response:**
```json
{
  "ticker": "AAPL",
  "retrieved_at": "2025-08-10T17:40:04.781906",
  "news": {
    "total_articles": 12,
    "sources": {
      "yahoo_finance": 6,
      "newsapi": 6
    },
    "articles": [
      {
        "title": "Apple Reports Strong Q3 Results",
        "summary": "Apple exceeded expectations with record iPhone sales...",
        "url": "https://finance.yahoo.com/news/apple-q3-2024",
        "source": "Yahoo Finance", 
        "published_at": "2025-08-10T14:30:00",
        "author": "John Smith",
        "tags": ["earnings", "iphone", "revenue"]
      }
    ]
  },
  "social_media": {
    "total_posts": 8,
    "platforms": {"reddit": 8},
    "posts": [
      {
        "title": "$AAPL breakout incoming? Technical analysis",
        "content": "Looking at the charts, AAPL seems to be forming...",
        "url": "https://reddit.com/r/stocks/comments/xyz",
        "platform": "Reddit",
        "author": "trader123",
        "published_at": "2025-08-10T16:20:00",
        "score": 245,
        "comments_count": 67,
        "subreddit": "stocks"
      }
    ]
  },
  "summary": {
    "total_items": 20,
    "time_range_days": 7,
    "newest_item": "2025-08-10T16:20:00",
    "oldest_item": "2025-08-03T09:15:00"
  }
}
```

#### `GET /news/{ticker}/news-only`
News articles only (faster response, no social media).

#### `GET /news/{ticker}/social-only`
Social media posts only.

---

### ETF Holdings

Temporarily unavailable. The ETF API is being redesigned. Previous endpoints under `/etf/*` have been removed and will return 404. See docs/ETF_API.md for historical reference only.

---

### Database & Metadata

#### `GET /database/stats`
Database statistics and data coverage information.

#### `GET /metadata/catalog`
Complete data field catalog with descriptions and types.

---

### Admin & Monitoring

#### `GET /admin/errors/logs`
Error log retrieval (admin access).

**Parameters:**
- `limit`: Number of logs (default: 100)
- `offset`: Pagination offset (default: 0)
- `min_level`: Minimum log level (ERROR, WARNING, INFO)

#### `GET /admin/errors/stats`
Error statistics and trends.

#### `POST /admin/migrate`
Database migration from another Stock Oracle instance.

---

## 🔧 Python Client Usage

### Installation
```bash
# Download the client from the repository
wget https://raw.githubusercontent.com/your-repo/stock-oracle/main/stock_oracle_client.py
```

### Basic Usage
```python
from stock_oracle_client import StockOracleClient

# Initialize client
client = StockOracleClient("http://localhost:18001")

# Check API health
health = client.get_health()
print("API Status:", health["status"])

# Get financial data
data = client.get_financial_data("AAPL", period="1y")
print(f"Found {len(data['financial_data'])} quarters of data")

# Get news and social media data
news = client.get_news_social_data("AAPL", days_back=7, max_articles=20)
print(f"Found {news['summary']['total_items']} news/social items")

# Get ETF holdings
etf = client.get_etf_holdings("QQQ", include_holdings=False)
print(f"QQQ has {etf['data']['holdings_count']} holdings")

# Bulk operations
bulk_data = client.get_bulk_financial_data(
    tickers=["AAPL", "MSFT", "GOOGL"],
    period="2y"
)
```

### Error Handling
```python
from stock_oracle_client import StockOracleAPIError, ETFDataNotAvailableError

try:
    data = client.get_financial_data("INVALID_TICKER")
except StockOracleAPIError as e:
    print(f"API Error: {e}")
    print(f"Status Code: {e.status_code}")

try:
    etf = client.get_etf_holdings("QQQ", as_of_date="1990-01-01")
except ETFDataNotAvailableError as e:
    print(f"ETF data not available: {e}")
    if e.availability_info:
        print(f"Available from: {e.availability_info['available_date_range']['start']}")
```

---

## 📈 Investment Analysis

### Stock Oracle Analyzer
The included analyzer provides comprehensive investment analysis combining all data sources:

```python
from stock_oracle_analyzer import StockOracleAnalyzer

# Initialize analyzer
analyzer = StockOracleAnalyzer("http://localhost:18001")

# Analyze single company
data = analyzer.get_company_data("AAPL", period="2y")
analysis = data['analysis_summary']

print(f"Investment Score: {analysis['investment_score']}/100")
print(f"Financial Grade: {analysis['financial_health']['grade']}")
print(f"Price Trend: {analysis['price_trends']['trend']}")
print(f"Sentiment: {analysis['sentiment_analysis']['sentiment_label']}")

# Compare multiple companies
tickers = ["AAPL", "MSFT", "GOOGL", "TSLA", "NVDA"]
results = analyzer.analyze_multiple_companies(tickers, period="1y")

# Generate comparison report
summary_df = analyzer.create_summary_report(results)
print(summary_df[['Ticker', 'Investment Score', 'Financial Grade', 'Sentiment']])
```

---

## ⚠️ Important Notes

### Data Availability
- **Period parameters** automatically use **yesterday** as end date to ensure data availability
- **SEC filings** may have delays - latest data is typically 1-3 months behind
- **ETF holdings** are updated quarterly via N-PORT filings
- **News data** is real-time but may have API rate limits

### Performance Tips
- Use **bulk endpoints** for multiple tickers to reduce latency
- Enable **caching** by avoiding `force_refresh=true` unless necessary
- Use **news-only** endpoints for faster sentiment analysis
- Implement **client-side caching** for frequently accessed data

### Error Codes
- `400`: Bad Request (invalid parameters)
- `404`: Data not found (ticker not found, no filings available)
- `429`: Rate limit exceeded
- `500`: Internal server error
- `503`: Service temporarily unavailable

---

## 🔗 Links

- **Interactive API Docs**: [/api/v1/docs](/api/v1/docs) (Swagger UI)
- **Alternative Docs**: [/api/v1/redoc](/api/v1/redoc) (ReDoc)
- **Health Check**: [/api/v1/health](/api/v1/health)
- **GitHub Repository**: [View on GitHub](https://github.com/your-repo/stock-oracle)

---

## 📞 Support

- **Issues**: Report bugs and feature requests on GitHub
- **Documentation**: This page is auto-generated from the latest API specification
- **Updates**: Check the changelog for latest features and improvements

---

*Stock Oracle - Empowering investment decisions with comprehensive data analysis* 🔮📈