docs: add project README with architecture, usage, and improvement tracking guide
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>main
I Luk Kim
5 months ago
parent
cb19afa87b
commit
764493dbe1
@ -0,0 +1,323 @@
|
||||
# ACE-F v1 — US Stock Event Swing Trading System
|
||||
|
||||
AI-powered event-driven swing trading system that uses SEC filings and free market data
|
||||
to identify 1-5 day continuation trades in US equities.
|
||||
|
||||
**Core principle:** AI/LLM is a document interpreter, not a price predictor.
|
||||
Entry signals come from official filings + price confirmation, never from social data alone.
|
||||
|
||||
```
|
||||
SEC Filings → Document Parser → Feature Builder → Signal Ranker
|
||||
↓ ↓
|
||||
Price/Volume Confirmation ←──── Backtest Engine ←── Risk Engine
|
||||
↓ ↓
|
||||
Attention Overlay (optional) ──→ Execution Engine → Post-trade Review
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
apps/ # Application entry points
|
||||
├── backtester/ # Event-driven backtest simulation & CLI
|
||||
├── pipeline/ # Multi-stage data processing
|
||||
│ ├── filing_poller/ # Poll SEC EDGAR for new filings
|
||||
│ ├── filing_fetcher/ # Download filing documents
|
||||
│ ├── event_parser/ # Parse events from filings (rule + LLM)
|
||||
│ ├── feature_builder/ # Generate scoring features
|
||||
│ ├── label_generator/ # Create forward-return labels
|
||||
│ └── dataset_export/ # Export Parquet snapshots for backtesting
|
||||
├── sync/ # Data synchronization
|
||||
│ ├── issuer_sync/ # Company metadata from Stock Oracle
|
||||
│ ├── macro_sync/ # FRED macro indicators
|
||||
│ └── short_volume_sync/ # FINRA short sale volume
|
||||
├── tracker/ # Strategy improvement tracking CLI
|
||||
├── tools/ # Analysis & utility scripts
|
||||
├── review/ # Manual review queue
|
||||
└── qa/ # Data quality checks
|
||||
|
||||
libs/ # Core libraries
|
||||
├── backtest/ # Backtesting engine
|
||||
│ ├── domain.py # Pydantic domain models (30+)
|
||||
│ ├── tracker.py # SQS scoring, journal I/O, leaderboard
|
||||
│ ├── execution.py # Entry/exit simulation
|
||||
│ ├── allocator.py # Position sizing & entry gates
|
||||
│ ├── scoring.py # Candidate scoring (PEAD, composite)
|
||||
│ ├── selector.py # Candidate filtering & ranking
|
||||
│ ├── metrics.py # 21-metric performance bundle + bootstrap CIs
|
||||
│ ├── artifacts.py # Run output writer (Parquet, JSON, CSV)
|
||||
│ ├── manifests.py # Experiment config resolution
|
||||
│ ├── snapshot_store.py # Parquet data loader
|
||||
│ ├── splits.py # Walk-forward window generation
|
||||
│ └── calendar.py # Trading day utilities
|
||||
├── common/ # Logging, config, time utils
|
||||
├── db/ # PostgreSQL models (async SQLAlchemy)
|
||||
├── oracle_client/ # Stock Oracle API client
|
||||
├── parser/ # Filing document parser
|
||||
├── features/ # Feature engineering
|
||||
├── labeler/ # Label generation
|
||||
├── schemas/ # Shared data schemas
|
||||
├── export/ # Snapshot export
|
||||
├── review/ # Review logic
|
||||
└── llm/ # LLM integration layer
|
||||
|
||||
configs/
|
||||
├── backtest/ # Base backtest configs (defaults.json)
|
||||
├── experiments/ # 66 experiment manifests
|
||||
├── app.yaml # Application settings
|
||||
└── symbols_*.yaml # Asset universe definitions
|
||||
|
||||
data/ # Data storage (parquet snapshots, cache)
|
||||
runs/ # Backtest execution outputs
|
||||
journal/ # Strategy improvement journal & leaderboard
|
||||
tests/
|
||||
├── unit/ # Unit tests (270+)
|
||||
├── integration/ # Integration tests (requires PostgreSQL)
|
||||
└── replay/ # Determinism replay tests
|
||||
```
|
||||
|
||||
## Setup
|
||||
|
||||
**Requirements:** Python 3.11+, PostgreSQL 16 (via Docker)
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -e ".[dev]"
|
||||
|
||||
# Start PostgreSQL
|
||||
docker compose up -d
|
||||
|
||||
# Run database migrations
|
||||
alembic upgrade head
|
||||
|
||||
# Copy and configure environment
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
**Key environment variables:**
|
||||
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `STOCK_ORACLE_URL` | `http://localhost:18001` | Stock Oracle API endpoint |
|
||||
| `POSTGRES_DSN` | `postgresql+asyncpg://acef:acef@localhost:5432/acef` | Database connection |
|
||||
| `DATA_ROOT` | `./data` | Data storage root |
|
||||
| `LOG_LEVEL` | `INFO` | Logging level |
|
||||
| `LLM_ENABLED` | `false` | Enable LLM document parsing |
|
||||
|
||||
## Usage
|
||||
|
||||
### Data Pipeline
|
||||
|
||||
```bash
|
||||
# Sync company metadata
|
||||
python -m apps.sync.issuer_sync.main
|
||||
|
||||
# Fetch and parse filings
|
||||
python -m apps.pipeline.filing_poller.main
|
||||
python -m apps.pipeline.filing_fetcher.main
|
||||
python -m apps.pipeline.event_parser.main
|
||||
|
||||
# Build features and labels
|
||||
python -m apps.pipeline.feature_builder.main
|
||||
python -m apps.pipeline.label_generator.main
|
||||
|
||||
# Export Parquet snapshot for backtesting
|
||||
python -m apps.pipeline.dataset_export.main
|
||||
```
|
||||
|
||||
### Backtesting
|
||||
|
||||
```bash
|
||||
# Single split backtest
|
||||
python -m apps.backtester.run \
|
||||
--manifest configs/experiments/pead_midcap_step1_fixedr.json \
|
||||
--split test --output-root runs/midcap_steps
|
||||
|
||||
# 3-split backtest (train/valid/test)
|
||||
for split in train valid test; do
|
||||
python -m apps.backtester.run \
|
||||
--manifest configs/experiments/pead_midcap_step1_fixedr.json \
|
||||
--split $split --output-root runs/midcap_steps
|
||||
done
|
||||
|
||||
# Walk-forward cross-validation
|
||||
python -m apps.backtester.run \
|
||||
--manifest configs/experiments/pead_midcap_step1_fixedr.json \
|
||||
--walk-forward --wf-train-days 252 --wf-test-days 63
|
||||
|
||||
# Output includes SQS score after each run:
|
||||
# Run complete: bt_baseline_swing_v1_...
|
||||
# Trades: 95
|
||||
# Total return: -0.46%
|
||||
# SQS: 39.7 (profitability=23.9, risk=52.4, consistency=24.8, robustness=80.7)
|
||||
```
|
||||
|
||||
### Experiment Configuration
|
||||
|
||||
Experiments are defined as JSON manifests in `configs/experiments/`:
|
||||
|
||||
```json
|
||||
{
|
||||
"experiment_name": "pead_midcap_step1_fixedr",
|
||||
"dataset_snapshot_id": "midcap-filtered",
|
||||
"base_config": "configs/backtest/defaults.json",
|
||||
"overrides": {
|
||||
"signal": { "scoring_model": "pead", "pead_reaction_threshold": 0.07 },
|
||||
"execution": { "target_model": "fixed_r", "target_1_r": 2.0 },
|
||||
"risk": { "max_positions": 8 }
|
||||
},
|
||||
"tags": ["pead", "midcap"]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Strategy Improvement Tracking System
|
||||
|
||||
A structured system to prevent duplicate experiments, enable data-driven decisions,
|
||||
and track the best strategy via a leaderboard.
|
||||
|
||||
### Strategy Quality Score (SQS)
|
||||
|
||||
Composite score (0-100) computed from **test split metrics only**. Higher is better.
|
||||
|
||||
| Category | Weight | Sub-metric | Weight | 0 pts | 100 pts |
|
||||
|----------|--------|------------|--------|-------|---------|
|
||||
| **Profitability** | 40% | profit_factor | 60% | ≤0.8 | ≥2.0 |
|
||||
| | | total_return_pct | 40% | ≤-5% | ≥+5% |
|
||||
| **Risk** | 25% | max_drawdown_pct (inv) | 50% | ≥10% | ≤1% |
|
||||
| | | sharpe_ratio | 50% | ≤-1.0 | ≥2.0 |
|
||||
| **Consistency** | 20% | win_rate | 50% | ≤0.35 | ≥0.65 |
|
||||
| | | monthly_win_rate | 50% | ≤0.30 | ≥0.70 |
|
||||
| **Robustness** | 15% | equity_curve_r_squared | 50% | ≤0.0 | ≥0.80 |
|
||||
| | | trade_count | 50% | ≤10 | ≥100 |
|
||||
|
||||
**Low-trade penalty:** If test trades < 20, SQS is halved.
|
||||
|
||||
| SQS Range | Interpretation |
|
||||
|-----------|----------------|
|
||||
| 0-20 | Losing strategy |
|
||||
| 20-40 | Near breakeven |
|
||||
| 40-55 | Promising, needs work |
|
||||
| 55-70 | Good, has OOS edge |
|
||||
| 70-85 | Strong, live candidate |
|
||||
| 85-100 | Exceptional (check for data issues) |
|
||||
|
||||
### Journal & Leaderboard
|
||||
|
||||
```
|
||||
journal/
|
||||
├── improvement_journal.jsonl ← Append-only improvement cycle log
|
||||
├── experiment_registry.json ← Leaderboard data (regenerated)
|
||||
└── LEADERBOARD.md ← Human-readable leaderboard (regenerated)
|
||||
```
|
||||
|
||||
Each journal entry records one improvement cycle:
|
||||
|
||||
```json
|
||||
{
|
||||
"entry_id": "IMP-0001",
|
||||
"timestamp": "2026-03-16T19:30:00",
|
||||
"experiment_name": "pead_midcap_step3_10pct",
|
||||
"hypothesis": "Raise reaction threshold to 10% for stronger signals",
|
||||
"results": {
|
||||
"train": { "run_id": "bt_...", "trade_count": 420, "profit_factor": 0.95, ... },
|
||||
"valid": { "run_id": "bt_...", ... },
|
||||
"test": { "run_id": "bt_...", ... }
|
||||
},
|
||||
"sqs_score": 38.5,
|
||||
"sqs_breakdown": { "profitability": 35.2, "risk": 45.0, "consistency": 30.0, "robustness": 42.0 },
|
||||
"verdict": "better",
|
||||
"verdict_reasoning": "Test PF 0.89 -> 1.00, breakeven achieved",
|
||||
"next_direction": "Combine 10% threshold + maxcand3"
|
||||
}
|
||||
```
|
||||
|
||||
### Tracker CLI
|
||||
|
||||
```bash
|
||||
# Record experiment results to journal
|
||||
python -m apps.tracker.cli record \
|
||||
--journal-dir journal/ \
|
||||
--runs-dir runs/midcap_steps/ \
|
||||
--experiment pead_midcap_step3_10pct \
|
||||
--hypothesis "Raise reaction threshold to 10%" \
|
||||
--baseline pead_7pct_midcap \
|
||||
--verdict better \
|
||||
--reasoning "Test PF improved from 0.89 to 1.00" \
|
||||
--next "Combine 10% threshold + maxcand3"
|
||||
|
||||
# View leaderboard
|
||||
python -m apps.tracker.cli leaderboard --journal-dir journal/
|
||||
|
||||
# # Experiment SQS PF Ret% Trades
|
||||
# ------------------------------------------------------------------
|
||||
# 1 pead_7pct_longshort_v2 60.7 1.31 +2.0 55
|
||||
# 2 pead_midcap_step3_10pct 38.5 1.00 +0.0 85
|
||||
|
||||
# Show entry details
|
||||
python -m apps.tracker.cli show --journal-dir journal/ IMP-0001
|
||||
|
||||
# Check for duplicate experiments
|
||||
python -m apps.tracker.cli check-duplicate \
|
||||
--journal-dir journal/ --experiment pead_midcap_step3_10pct
|
||||
```
|
||||
|
||||
### Improvement Workflow
|
||||
|
||||
```
|
||||
1. Create experiment config configs/experiments/my_experiment.json
|
||||
2. Run 3-split backtest for split in train valid test; do ... done
|
||||
3. Record to journal python -m apps.tracker.cli record ...
|
||||
4. Check leaderboard python -m apps.tracker.cli leaderboard ...
|
||||
5. Plan next experiment based on verdict + SQS breakdown
|
||||
6. Check for duplicates python -m apps.tracker.cli check-duplicate ...
|
||||
7. Repeat from step 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
# Unit tests (fast, no external deps)
|
||||
pytest tests/unit/ -v
|
||||
|
||||
# Backtest module tests only
|
||||
pytest tests/unit/backtest/ -v
|
||||
|
||||
# Integration tests (requires PostgreSQL)
|
||||
pytest tests/integration/ -v
|
||||
|
||||
# Full CI check (lint + typecheck + unit tests)
|
||||
make ci
|
||||
```
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Source | Role | Cost |
|
||||
|--------|------|------|
|
||||
| **SEC EDGAR** | Primary event source (8-K, 10-Q, 6-K filings) | Free |
|
||||
| **Stock Oracle API** | Market data (OHLCV bars, company info) | Internal |
|
||||
| **FRED** | Macro regime indicators (rates, spreads) | Free |
|
||||
| **FINRA** | Short sale volume (crowding signal) | Free |
|
||||
| Wikimedia | Retail attention via pageviews | Free |
|
||||
| YouTube | Channel-based attention tracking | Free (quota limited) |
|
||||
| Yahoo RSS | Headline burst detection | Free |
|
||||
|
||||
## Tech Stack
|
||||
|
||||
| Component | Technology |
|
||||
|-----------|-----------|
|
||||
| Language | Python 3.11+ |
|
||||
| Models | Pydantic v2 |
|
||||
| Database | PostgreSQL 16 + async SQLAlchemy |
|
||||
| Research data | DuckDB + Parquet |
|
||||
| Containers | Docker Compose |
|
||||
| Linting | Ruff |
|
||||
| Type checking | MyPy (strict) |
|
||||
| Testing | Pytest + asyncio |
|
||||
| Logging | structlog (JSON) |
|
||||
|
||||
## License
|
||||
|
||||
Private project. All rights reserved.
|
||||
Loading…
Reference in New Issue