Boosting inline/guidance engine per_trade_risk from 0.015-0.020 to 0.040.
These engines have 73-100% WR — bigger positions on best signals.
v6new.307: SQS 74.4 (#1), CW 350.2%, Test +46.8%, MaxDD 2.4%, PF 14.74
Full progression: 262.9% → 284.4% → 312.6% → 320.5% → 350.2%
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Additional tracker/leaderboard updates, overlay leaderboard, and
documentation improvements.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New modules for live/mock broker interface, SQLite session state,
auto-trading engine, and backtest result reporting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces per-experiment rglob with single-pass manifest/metrics indexing
and adds lru_cache. Removes rarely-used commands from help display.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds overlay strategy backtesting, flexible date parsing, --no-trades flag,
--rank range selection, session management improvements, circuit breaker
for screener failures, and bars_cache passthrough for 10x speed gain.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extends selector with new scoring model support, adds execution
enhancements, and improves snapshot store loading and split handling.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds earnings surprise extraction to parser/features/labeler pipeline,
improves filing fetcher robustness, and extends snapshot export with
new field support.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OME engine early_failure_no_progress_days=1 (from day 2 in v272) further
improves capital recovery speed. 198 trades, 312.6% CW return.
SQS 72.7 = new #1 on leaderboard (including overlays).
Test return +45.5%, Profit Factor 19.23.
Key: cutting OME losers at day 1 instead of day 2 frees capital faster
while losing only marginal OME winners that needed >1 day to show progress.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Trade analysis revealed OME engines had 54-60% WR with stop-heavy exits.
Adding early_failure (day 2, R=0.0) for OME engines frees capital faster,
enabling 6 more trades (194→200) and boosting CW from 293.2% to 310.8%.
Key insight: cutting low-quality engine losers early improves compounding
more than any scoring/feature/sizing change tested in this session.
v6new.272: SQS 72.4 (#6), CW 310.8%, 200 trades, test +44.9%
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Higher risk (0.069) + tighter stops (ATR 1.45) produce best CW return
but lower SQS than v6new.196 (72.6) due to WFV instability.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lowering market cap floors to 2B added 15 trades (194→209) but CW return
dropped from 293.2% to 278.5%. Small-cap PEAD events have lower average
quality — individual outliers like SEDG exist but don't compensate.
v6new.255 (293.2%) confirmed as optimal trade-quality balance.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bounce engine (buy negative reaction, bet on mean reversion) could not
execute: system architecture ties scoring to single model per backtest,
and selector/store indexes are optimized for positive-reaction PEAD.
Negative-reaction candidates get score=0 from PEAD scoring, blocking
engine selection regardless of engine-level threshold overrides.
Implementing bounce trades requires: dual scoring model support,
selector changes for negative-reaction candidate routing, and
store indexing changes. Deferred to future refactor.
Current best CW return: 293.2% (v6new.255)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
v6new.196 = v29 base + entropy scoring (v13e) + per_trade_risk 0.058
+ doc_quality 0.50. CW return 284.4% (+21.5pp over v122), 194 trades.
Key finding: v29 base with doc_quality 0.50 (not 0.55) is the optimal
quality gate when combined with entropy scoring.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Data analysis revealed OBV Q1 (distribution) has 56.4% WR vs Q5 51.2% —
contrarian signal confirmed. Previous OBV bonus was applied in wrong
direction. Corrected with v15 scoring models.
Best result: v6new.185 (entropy + risk 0.058) CW 274.4% but SQS 72.2,
still below v6new.122 (72.4). WFV/robustness offsets CW gains.
v6new.122 confirmed as optimal under current SQS v4 formula.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New engine: next_open_long_mean_reversion_high_vix
Targets: react < -7%, close 0.15-0.60, bearish/mixed/unknown direction
Signal: VIX>20 + big drop = 62.9% WR, +3.91% 5d mean (n=167)
VIX 25-30 sweet spot: 75% WR, +5.46% 5d mean
Test split: 3 MR trades, 67% WR, +5.59% total PnL
Paper BT: 51 trades vs 48 (v6new.30), return ~equal
Also fixes _rows_to_table to handle sparse feature columns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously only used rows[0].keys() — columns present in later rows
(like earnings_surprise_pct from sparse features) were silently dropped.
Now collects all unique keys across all rows.
YoY earnings surprise tested: WR spread only 2.5pp (55.2% vs 52.7%).
Not actionable — YoY growth != analyst consensus surprise.
v6new.30 remains the framework optimum.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New data source integration:
- EarningsSurpriseService: GET /api/v1/earnings/surprise/{symbol}
Returns actual vs estimated EPS with surprise_percentage
- Feature builder: creates earnings_surprise_v1 snapshots for earnings events
- Backfill script runs for existing 1,273 tickers (Alpha Vantage rate limited)
New scoring (v11):
- Small beat (0-3% surprise): +10% bonus (82.4% WR in sample)
- Medium beat (3-8%): +5% bonus
- Big beat (>8%): no bonus (already priced in)
- Miss (<=0%): -5% penalty
Signal validation (n=66 sample):
Small beat: 82.4% WR, +1.79% mean 5d return
Big beat: 54.8% WR, +0.47%
Miss: 55.6% WR, -0.10%
Backfill running (~4 hours). Experiment pending data completion.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Short Side (Direction 3):
Bearish events: 45-55% WR for short — no actionable edge
Mean reversion after large drops cancels short PEAD
Execution Model (Direction 2):
T+2 delayed entry: loses 54% of alpha (Day 1 = 54% of 5d return)
Wider stops (v6new.34): -0.68pp — smaller positions offset fewer stop-outs
Tighter trailing (v6new.35): -12.48pp — cuts winners too early
reaction_close >> next_open (86% vs 57% WR) but post_market can't use RC
Direction 1 (new data: Form 4, XBRL, Earnings Surprise) requires Oracle API
implementation. Free sources identified: SEC EDGAR, Alpha Vantage, FINRA.
v6new.30 is the confirmed framework optimum.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pipeline refreshed: 528 new filings, 240 new events processed.
Snapshot re-exported with start_date=2022-03-01: 9350 rows (+230 vs old).
Test split now extends to 2026-03-23 (was 03-13).
v6new.30 on fresh data:
SQS: 62.3 (down from 63.8 — OOT worst -4.61% from new period data)
Train: 117 trades, +59.31%, SQS 92.6
Valid: 33 trades, +41.05%
Test: 27 trades, +34.09%
WFV: 100% positive (worst +1.08%)
OOT: 90.0% positive (worst -4.61%)
Paper BT (1yr): +93.41%, 50 trades
Strategy holds up on fresh data with 100% WFV positive rate.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
v6new.25 (v6.29 base): OME risk halved → Paper BT +94.38% (vs +93.60%)
v6new.26 (v6new.25 on v6.100 snapshot): +93.75%
v6new.27 (v6.100 + further OME reduction): Paper BT +105.35%, MaxDD 3.35%, Sharpe 3.73
- other_material_unknown: 0.005→0.003
- unknown_ome: 0.012→0.008
All improvements validated by paper backtest (unified engine, same as backtester).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Based on paper backtest loss analysis (not backtester SQS optimization):
- OME engines contribute only +$1,405 on 8 trades but generate most losses
- Halving OME per_trade_risk reduces loss magnitude without removing trades
- Paper BT: +94.38% vs v6.29 +93.60%, MaxDD 4.07% vs 4.60%
- WFV: 100% positive fold rate (worst +0.02%)
- RM worst: -0.88%, OOT worst: -0.33%
- SQS: 62.9 (vs v6.29 62.8)
This is the first strategy improvement validated by BOTH backtester SQS
AND paper backtest return, using the now-unified engine systems.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FilledTrade now carries event_type and score from the Candidate.
These fields are written to trade_blotter.parquet and displayed in
paper backtest trade logs.
Previously score showed as 0.00 for all trades because the field
wasn't propagated from Candidate → FilledTrade → Parquet.
Score=0.00 is valid for trades from engines with score_threshold_override=0.0
(e.g. guidance_unknown_orderly) where engine gates, not score, determine entry.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 5 — Engine selection (both entry paths):
- Added residual_reserve_selected tracking between engines
- Added prelimit amplification (5x) for attention-requiring engines
- Added truncate_to parameter to select_candidates calls
Matches BacktestRunner._select_candidates_for_date() behavior.
Phase 6 — Macro data:
- Added FRED series fetch (VIXCLS, BAMLH0A0HYM2) to _fetch_macro()
- Matches SnapshotStore._fetch_macro() which loads from MacroObservation DB
- Enables VIX/HY regime sizing in live paper trading
All 6 phases of BacktestRunner ↔ PaperTradingEngine unification complete.
450 unit tests pass. Multi-strategy paper backtest verified.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1-4 of engine unification to eliminate research/live divergence.
Phase 1 — Scoring (event_detector.py):
EventDetector now uses config's scoring_model (v5/v9 etc.) when
event_v1 features are present (parse_confidence_overall not null).
Falls back to compute_entry_score only for incomplete events.
Phase 2 — Execution config (execution.py):
Extracted build_effective_execution_config() as shared function.
BacktestRunner delegates to it. PaperTradingEngine can now use
identical per-engine overrides, adaptive exit, tiered targets.
Phase 3 — Attention filtering (attention.py):
New AttentionFilterService class extracted from BacktestRunner.
Provides: engine_requires_attention, apply_filters, rescoring.
BacktestRunner now delegates to this service.
PaperTradingEngine can import and use the same service.
Phase 4 — Gap cap (execution.py):
check_next_open_gap_cap() shared function for next-open gap rejection.
All 450 unit tests pass. Paper backtest verified working.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Snapshots can be in data/parquet/ or data/datasets/snapshots/.
Now tries default parquet_dir first, falls back to data/datasets/snapshots/
if the snapshot exists there instead.
Fixes FileNotFoundError when running multi-strategy paper backtest with
configs that reference snapshots in the alternate directory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
run_backtest changed from async to sync function. Pipeline refresh
(async) runs via asyncio.run() before the sync BacktestRunner,
avoiding nested event loop when SnapshotStore.load() calls asyncio.run().
CLI updated to call run_backtest() directly (no asyncio.run wrapper).
Tested: `fithia2 paper backtest --config v6new.24 --start 2025-03-23 --end 2026-03-23` works.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When `fithia2 paper backtest --end <date>` requests a date beyond the
snapshot's latest event, automatically runs the pipeline:
1. Filing poller (discover new 8-Ks)
2. Filing fetcher (download exhibits)
3. Event parser (parse events)
4. Feature builder (compute features)
5. Label generator (compute labels)
6. Dataset export (re-generate Parquet snapshot)
Staleness check: snapshot is stale if its latest event_date is >14 days
before the requested end_date, or if the manifest is >7 days old.
If refresh fails, falls back to existing snapshot data gracefully.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Major refactor: `fithia2 paper backtest` now uses the exact same
BacktestRunner + SnapshotStore pipeline as `apps/backtester/run.py`.
Before: PaperTradingEngine + EventDetector + MockBroker
- Different scoring (compute_entry_score vs config scoring_model)
- Different data source (DB + Oracle vs Parquet snapshot)
- Different feature computation (real-time vs pipeline)
→ Config gate changes didn't take effect in paper backtest
After: BacktestRunner + SnapshotStore (Parquet)
- Identical scoring, engine matching, position sizing
- Same Parquet data as research backtester
- Config changes work identically in both systems
Trade output format preserved for reporter.py compatibility.
PaperTradingEngine still used for live Alpaca trading (unchanged).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The DB-first approach (prefer feature_json over Oracle recalculation) caused:
- LMND (+$782) and M (+$1,052) trades to disappear
- TEM loss to increase from -$321 to -$535
- Overall PnL drop from +$5,948 to +$3,078
Root cause: DB features were computed at a different time with different
Oracle data. When paper trader used DB values, the feature values didn't
match what the backtester's Parquet snapshot had, causing different
engine gate outcomes.
Paper trader must use Oracle real-time enrichment as primary source
(same as the original design). The volume_ratio_20d field name fix
is retained as that was a genuine bug.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The _compute_score → v5 dispatch caused v5's hard gates to reject almost
all events (v5 requires specific direction/guidance combos). This killed
all 2025 trades in paper backtest.
Root cause: BacktestRunner and PaperTradingEngine use different flows.
BacktestRunner applies scoring AFTER engine selection (engines have
score_threshold_override=0.0 that bypasses score gates). But EventDetector
applied scoring BEFORE engine matching, causing v5's hard gates to reject
events that engines would have accepted.
Fix: revert to compute_entry_score for EventDetector. Score is ranking-only
in paper trading; engine gates (reaction_min, close_min, etc.) handle filtering.
The volume_ratio_20d fix and DB-first feature fix remain in place.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
I Luk Kim