# Stock Oracle Python Client Documentation

## Installation

### Option 1: Direct File Usage
```bash
# Copy the client file to your project
cp stock_oracle_client.py /path/to/your/project/

# Install dependencies
pip install requests python-dateutil
```

### Option 2: Install as Package (Future)
```bash
# Will be available on PyPI
pip install stock-oracle-client
```

### Option 3: Development Installation
```bash
# Clone the repository
git clone <repo-url>
cd stock-oracle

# Install client dependencies
pip install -r requirements-client.txt

# Run examples
python examples/client_usage.py
```

## Quick Start

```python
from stock_oracle_client import StockOracleClient

# Initialize client
client = StockOracleClient("http://localhost:18001")

# Get financial data
financial = client.get_financial_data("AAPL", period="1y")
print(f"Company: {financial['company']['name']}")

# Get price data
prices = client.get_price_data("MSFT", period="3m")
print(f"Latest close: ${prices['price_data'][-1]['close']:.2f}")

# Get ETF holdings
etf = client.get_etf_holdings("QQQ")
print(f"Holdings: {etf['data']['holdings_count']}")
```

## Core Features

### 1. Client Initialization

```python
from stock_oracle_client import StockOracleClient

# Basic initialization
client = StockOracleClient("http://localhost:18001")

# With options
client = StockOracleClient(
    base_url="http://localhost:18001",
    api_key=None,  # For future authentication
    timeout=30,     # Request timeout in seconds
    auto_retry=True,  # Automatic retry on failure
    max_retries=3    # Maximum retry attempts
)
```

### 2. Financial Data

```python
# Using period (recommended)
data = client.get_financial_data("AAPL", period="1y")

# Using date range
from datetime import date
data = client.get_financial_data(
    "AAPL",
    start_date=date(2023, 1, 1),
    end_date=date(2023, 12, 31),
    period_type=PeriodType.QUARTERLY
)

# Using quarters
data = client.get_financial_data(
    "AAPL",
    quarters=["2024Q1", "2024Q2"],
    include_metrics=True
)
```

### 3. Price Data

```python
from stock_oracle_client import PriceInterval

# Daily prices for last year
prices = client.get_price_data(
    "TSLA",
    period="1y",
    interval=PriceInterval.ONE_DAY
)

# Weekly prices for specific range
prices = client.get_price_data(
    "TSLA",
    start_date="2024-01-01",
    end_date="2024-06-30",
    interval=PriceInterval.ONE_WEEK
)
```

### 4. ETF Holdings

```python
# Current holdings
etf = client.get_etf_holdings("QQQ")

# Historical holdings
etf = client.get_etf_holdings("SPY", as_of_date="2023-12-31")

# Without detailed holdings (faster)
etf = client.get_etf_holdings("ARKK", include_holdings=False)

# With automatic fallback
try:
    etf = client.get_etf_holdings_with_fallback("QQQM", "2020-01-01")
except ETFDataNotAvailableError as e:
    print(f"Error: {e}")
    if e.availability_info:
        print(f"Available from: {e.availability_info['available_date_range']['start']}")
```

### 5. Bulk Operations

```python
# Bulk financial data
bulk_financial = client.get_bulk_financial_data(
    ["AAPL", "MSFT", "GOOGL"],
    period="6m",
    period_type=PeriodType.QUARTERLY
)

# Bulk price data
bulk_prices = client.get_bulk_price_data(
    ["TSLA", "NIO", "RIVN"],
    period="1m",
    interval=PriceInterval.ONE_DAY
)

# Bulk ETF holdings
bulk_etf = client.get_bulk_etf_holdings(
    ["QQQ", "SPY", "IWM"],
    include_holdings=False  # For performance
)
```

### 6. Alpaca Market Data

```python
import requests

BASE = "http://localhost:18001/api/v1"

# Check Alpaca connection
status = requests.get(f"{BASE}/alpaca/status").json()
print(f"Alpaca configured: {status['configured']}")

# ── 과거 분봉 (SIP 피드, DB 저장, 어제까지) ──────────────────────────
# 백테스트, 과거 OHLCV 분석용. 거래량 100% 정확.
hist = requests.get(f"{BASE}/alpaca/intraday", params={
    "tickers": "AAPL,MSFT,NVDA",
    "interval": "5m",
    "start_date": "2025-01-02",
    "end_date": "2025-01-31",
}).json()
print(f"Historical bars: {hist['count']} tickers")
for ticker, bars in hist["bars"].items():
    print(f"  {ticker}: {len(bars)} bars")

# ── 당일 실시간 분봉 (IEX 피드, 오늘만) ─────────────────────────────
# 당일 ORB 전략, 실시간 장 중 모니터링.
# 거래량은 실제의 2~5% (IEX 거래소 거래만 집계).
today = requests.get(f"{BASE}/alpaca/intraday/today", params={
    "tickers": "AAPL,MSFT,NVDA",
    "interval": "5m",
}).json()
for ticker, bars in today["bars"].items():
    if bars:
        latest = bars[-1]
        print(f"  {ticker}: last bar {latest['timestamp']} close={latest['close']}")

# ── 실시간 스냅샷 (IEX 피드, 캐시 없음) ─────────────────────────────
# 단일 종목도 tickers= 쿼리 파라미터로 조회 가능.
snap = requests.get(f"{BASE}/alpaca/snapshot", params={"tickers": "SPY,QQQ,AAPL"}).json()
for s in snap["snapshots"]:
    print(f"  {s['ticker']}: {s['price']} ({s['change_pct']:+.2f}%) bid={s['bid']} ask={s['ask']}")
```

### 7. FINRA Short Volume

```python
import requests

BASE = "http://localhost:18001/api/v1"

# Get short volume (auto-ingests if missing)
sv = requests.get(f"{BASE}/finra/short-volume/AAPL", params={"days": 30}).json()
print(f"AAPL short volume entries: {sv['total_count']}")
for entry in sv["entries"][:3]:
    print(f"  {entry['date']}: ratio={entry['short_ratio']:.2%}")

# Get short ratio history (aggregated across markets)
ratio = requests.get(f"{BASE}/finra/short-ratio/AAPL", params={"days": 60}).json()
print(f"Average short ratio: {ratio['avg_short_ratio']:.2%}")

# Manual ingest
ingest = requests.post(f"{BASE}/finra/admin/ingest", params={"date": "2025-03-10"}).json()
print(f"Ingested: {ingest['records_ingested']} records")
```

### 8. News v2 — Multi-source Headlines & Session Aggregates

Multi-source structured news ingest (Alpaca News, StockTwits, Finnhub) with
unified categorization and PIT-safe session aggregates. Distinct from the
on-demand `get_news_social_data()` aggregator above.

```python
from stock_oracle_client import StockOracleClient

client = StockOracleClient(base_url="http://localhost:18001")

# 1. Coverage probe — confirm archive depth before backtest window selection
coverage = client.get_news_coverage(source="alpaca_benzinga", symbol="AAPL")
print(f"AAPL Alpaca News: {coverage['ingested_count']} headlines, "
      f"{coverage['earliest']} → {coverage['latest']}")

# 2. Raw headlines — last 24 hours for two tickers
headlines = client.get_news_headlines(
    symbols=["AAPL", "MSFT"],
    start="2026-04-24T00:00:00Z",
    end="2026-04-25T00:00:00Z",
    sources=["alpaca_benzinga", "finnhub"],
    limit=100,
)
for h in headlines["items"][:3]:
    print(f"[{h['source']}] {h['published_at']} {h['ticker']}: {h['headline']}")
    print(f"   categories: {h['categories']}")

# 3. Single-ticker session aggregate (V49 use case)
agg = client.get_news_session_aggregate(
    symbol="AAPL",
    session_date="2026-04-25",
    window="premarket",  # premarket | intraday | post | full_session
)
print(f"AAPL premarket: {agg['headline_count']} headlines, "
      f"sentiment={agg['sentiment_recency_weighted']}, "
      f"categories={agg['category_counts']}")

# 4. Batch — fithia2's hot path (one call per session, then disk-cache)
batch = client.get_news_session_aggregate_batch(
    session_date="2026-04-25",
    window="premarket",
    symbols=["AAPL", "MSFT", "NVDA", "TSLA", "GOOGL"],
    sources=["alpaca_benzinga", "stocktwits"],
)
for ticker, item in batch["items"].items():
    soc = item["social"]
    print(f"{ticker}: count={item['headline_count']}, "
          f"social_msgs={soc['message_count']}, "
          f"bull_bear_ratio={soc['bull_bear_ratio']}")
```

**Method reference:**

| Method | Returns |
|---|---|
| `get_news_headlines(symbols=None, start=None, end=None, sources=None, limit=100, cursor=None)` | `{items: [...], next_cursor: str\|None}` |
| `get_news_session_aggregate(symbol, session_date, window="premarket", sources=None, force_refresh=False)` | session aggregate dict |
| `get_news_session_aggregate_batch(session_date, window, symbols, sources=None)` | `{items: {ticker: aggregate}}` |
| `get_news_coverage(source, symbol=None)` | `{source, symbol, earliest, latest, ingested_count}` |

`session_date` strings are ET dates (YYYY-MM-DD). Aggregates filter
`ingested_at <= window_end_utc` so backtests don't see lookahead headlines.
Tickers with no matching headlines are returned as zero-count rows (not omitted).

## Advanced Usage

### Error Handling

```python
from stock_oracle_client import (
    StockOracleClient,
    StockOracleAPIError,
    ETFDataNotAvailableError
)

client = StockOracleClient("http://localhost:18001")

try:
    # Try to get data
    data = client.get_financial_data("INVALID")
    
except StockOracleAPIError as e:
    print(f"API Error: {e}")
    print(f"Status Code: {e.status_code}")
    print(f"Response: {e.response_data}")
    
except ETFDataNotAvailableError as e:
    print(f"ETF Data Not Available: {e}")
    if e.availability_info:
        print(f"Launch Date: {e.availability_info.get('etf_launch_date')}")
        print(f"Available Range: {e.availability_info.get('available_date_range')}")
```

### Availability Checking

```python
# Check if ETF data is available
availability = client.check_etf_availability("QQQM", "2020-01-01")

if not availability.get('exists_for_date'):
    print(f"ETF didn't exist on requested date")
    print(f"Launch date: {availability.get('etf_launch_date')}")
    print(f"First available: {availability['available_date_range']['start']}")
```

### Using Enums

```python
from stock_oracle_client import PriceInterval, PeriodType

# Price intervals
intervals = [
    PriceInterval.ONE_MINUTE,
    PriceInterval.FIVE_MINUTES,
    PriceInterval.ONE_HOUR,
    PriceInterval.ONE_DAY,
    PriceInterval.ONE_WEEK,
    PriceInterval.ONE_MONTH
]

# Period types
period_types = [
    PeriodType.QUARTERLY,
    PeriodType.ANNUAL,
    PeriodType.ALL
]
```

### Utility Methods

```python
# Validate ticker
is_valid = client.validate_ticker("AAPL")  # Returns True
is_valid = client.validate_ticker("INVALID123")  # Returns False

# Get latest filing date
filing_date = client.get_latest_filing_date("MSFT")

# Search tickers (client-side)
results = client.search_tickers("AA")  # Returns ["AAPL", ...]

# Get supported ETFs
supported = client.get_supported_etfs()
print(f"Total ETFs: {supported['total_etfs']}")

# Get data catalog
catalog = client.get_data_catalog()
```

## Convenience Functions

For quick one-off requests without creating a client instance:

```python
from stock_oracle_client import get_financial_data, get_price_data, get_etf_holdings

# Quick financial data
financial = get_financial_data("AAPL", period="1y")

# Quick price data
prices = get_price_data("TSLA", period="3m")

# Quick ETF holdings
etf = get_etf_holdings("QQQ", as_of_date="2024-01-01")
```

## Best Practices

### 1. Use Period Strings
Period strings are the most convenient way to specify time ranges:
- `"1d"`, `"5d"`, `"1m"`, `"3m"`, `"6m"`, `"1y"`, `"2y"`, `"5y"`, `"10y"`, `"ytd"`, `"max"`

### 2. Batch Requests
Use bulk endpoints when fetching data for multiple tickers:
```python
# Good - single bulk request
bulk_data = client.get_bulk_financial_data(["AAPL", "MSFT", "GOOGL"])

# Bad - multiple individual requests
data1 = client.get_financial_data("AAPL")
data2 = client.get_financial_data("MSFT")
data3 = client.get_financial_data("GOOGL")
```

### 3. Handle ETF Availability
Always check availability when requesting historical ETF data:
```python
def get_etf_data_safe(client, ticker, date):
    try:
        return client.get_etf_holdings(ticker, as_of_date=date)
    except ETFDataNotAvailableError as e:
        if e.availability_info:
            # Try with earliest available date
            start_date = e.availability_info['available_date_range']['start']
            return client.get_etf_holdings(ticker, as_of_date=start_date)
        raise
```

### 4. Use Session Reuse
The client automatically reuses HTTP sessions for better performance:
```python
# Good - reuse client instance
client = StockOracleClient("http://localhost:18001")
for ticker in tickers:
    data = client.get_financial_data(ticker)

# Bad - create new client each time
for ticker in tickers:
    client = StockOracleClient("http://localhost:18001")
    data = client.get_financial_data(ticker)
```

### 5. Error Recovery
Enable auto-retry for better reliability:
```python
client = StockOracleClient(
    "http://localhost:18001",
    auto_retry=True,
    max_retries=3
)
```

## Examples

### Complete Example: Portfolio Analysis

```python
from stock_oracle_client import StockOracleClient, PeriodType
from datetime import datetime, timedelta

def analyze_portfolio(tickers, client):
    """Analyze a portfolio of stocks and ETFs"""
    
    results = {
        'stocks': {},
        'etfs': {},
        'errors': []
    }
    
    for ticker in tickers:
        try:
            # Try as ETF first
            etf_data = client.get_etf_holdings(ticker, include_holdings=False)
            if etf_data['success']:
                results['etfs'][ticker] = {
                    'type': 'ETF',
                    'holdings_count': etf_data['data']['holdings_count'],
                    'as_of_date': etf_data['as_of_date']
                }
                continue
        except:
            pass
        
        try:
            # Try as stock
            financial = client.get_financial_data(
                ticker, 
                period="1y",
                period_type=PeriodType.QUARTERLY
            )
            
            if financial['financial_data']:
                latest = financial['financial_data'][0]
                results['stocks'][ticker] = {
                    'type': 'Stock',
                    'company': financial['company']['name'],
                    'latest_filing': latest['date'],
                    'pe_ratio': latest.get('metrics', {}).get('pe_ratio'),
                    'market_cap': latest.get('market_cap')
                }
        except Exception as e:
            results['errors'].append({
                'ticker': ticker,
                'error': str(e)
            })
    
    return results

# Usage
client = StockOracleClient("http://localhost:18001")
portfolio = ["AAPL", "QQQ", "MSFT", "SPY", "TSLA", "ARKK"]
analysis = analyze_portfolio(portfolio, client)

print("Stocks:")
for ticker, data in analysis['stocks'].items():
    print(f"  {ticker}: {data['company']}")

print("\nETFs:")
for ticker, data in analysis['etfs'].items():
    print(f"  {ticker}: {data['holdings_count']} holdings")
```

### Data Export Example

```python
import pandas as pd
from stock_oracle_client import StockOracleClient

client = StockOracleClient("http://localhost:18001")

# Get data
financial = client.get_financial_data("AAPL", period="2y")

# Convert to DataFrame
df = pd.DataFrame(financial['financial_data'])

# Export to CSV
df.to_csv("aapl_financial_data.csv", index=False)

# Export to Excel
df.to_excel("aapl_financial_data.xlsx", index=False)

print(f"Exported {len(df)} records")
```

## Troubleshooting

### Common Issues

1. **Connection Error**
   ```python
   # Check if API is running
   try:
       health = client.get_health()
   except:
       print("API is not accessible")
   ```

2. **ETF Data Not Available**
   - Check availability first using `check_etf_availability()`
   - Use `get_etf_holdings_with_fallback()` for automatic fallback
   - Check the `availability_info` in the exception

3. **Rate Limiting**
   - Add delays between requests if needed
   - Use bulk endpoints for multiple tickers
   - Enable caching on the server side

4. **Timeout Issues**
   ```python
   # Increase timeout for large requests
   client = StockOracleClient("http://localhost:18001", timeout=60)
   ```

## API Response Structure

### Financial Data Response
```json
{
  "ticker": "AAPL",
  "company": {
    "name": "Apple Inc.",
    "sector": "Technology",
    "industry": "Consumer Electronics"
  },
  "financial_data": [
    {
      "date": "2024-03-31",
      "revenue": 119575000000,
      "net_income": 23636000000,
      "metrics": {
        "pe_ratio": 26.5,
        "return_on_equity": 1.47,
        "gross_margin": 0.455
      }
    }
  ]
}
```

### ETF Holdings Response
```json
{
  "ticker": "QQQ",
  "as_of_date": "2024-12-31",
  "success": true,
  "data": {
    "filing_info": {
      "filing_date": "2024-12-31",
      "cik": "1067839",
      "total_value": 250000000000,
      "total_holdings": 102
    },
    "holdings": [
      {
        "name": "MICROSOFT CORP",
        "cusip": "594918104",
        "value": 25000000000,
        "shares": 50000000,
        "percentage": 10.0
      }
    ],
    "holdings_count": 102
  },
  "availability": null
}
```

### Error Response with Availability
```json
{
  "ticker": "QQQM",
  "as_of_date": "2020-01-01",
  "success": false,
  "error": "ETF QQQM did not exist on 2020-01-01",
  "availability": {
    "exists_for_date": false,
    "etf_launch_date": "2020-10-13",
    "first_nport_date": "2021-01-31",
    "available_date_range": {
      "start": "2021-01-31",
      "end": "present"
    }
  }
}
```

## Version History

### v2.0.0 (Current)
- Added comprehensive ETF holdings support
- Improved error handling with custom exceptions
- Added availability checking for ETF data
- Bulk operations for all data types
- Enum support for intervals and period types
- Automatic retry logic
- Utility methods for validation and search

### v3.0.0
- Added Alpaca Market Data integration (bars, intraday, DB storage)
- Added FINRA Short Volume data (auto-ingest, short ratio history)
- New data sources: Alpaca (optional API key), FINRA (public CDN)

### v1.0.0
- Initial release
- Basic financial and price data retrieval
- Period string support
- Bulk financial data

## Support

For issues, questions, or contributions:
- GitHub Issues: [Report bugs or request features]
- Documentation: [API Documentation](../README.md)
- Examples: See `examples/` directory