- enrich_tier2: prefetch price bars (parallel ThreadPool) and short ratio
(single batch DB query) instead of per-row HTTP/DB calls (~20min → ~2min)
- canonical_snapshots: add PYTHONUNBUFFERED=1 to enrichment subprocesses
so progress output is visible in real time
- backtest_sim: use incremental_update_canonical_snapshot when existing
snapshot is present, falling back to full rebuild only when needed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Live paper trader (engine.py):
- On first run_next_open per daemon session, call get_candidates_for_lookback()
to fetch events from [today - max_mhd*2, today) that are still active
- Skip gap-cap check for lookback entries (multi-day drift ≠ overnight gap)
- Initialize days_held to elapsed trading days when saving strategy state
EventDetector (event_detector.py):
- Extract shared enrichment logic into _enrich_raw_rows(raw_rows, bar_end_date, config)
- Add _fetch_events_for_date_range(start, end): single DB query with entry_date range
- Add get_candidates_for_lookback(today, start_date, config): annotates each row
with is_lookback_entry=True and lookback_days_elapsed=N
Mock broker (backtest_sim.py):
- Extend slice_by_date_range start backward when lookback_entry_enabled, mirroring
the same logic already present in apps/backtester/run.py main()
Verified: BX/EBAY/ENB all entered 2026-03-30 via lookback in both research
backtest and mock broker. Parking, idle_alpha, form4 sleeves unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a backtest starts mid-stream (via --start), events that fired
before the start date but are still within their max_holding_days
window can now be entered on the first simulation day.
- Add `lookback_entry_enabled: bool = False` to ExecutionConfig
- On first sim day, _collect_lookback_candidates() gathers pre-start
events, runs them through the same select_candidates() pipeline,
and injects them before normal candidates
- Entry fills at the first day's open price; gap-cap check is skipped
since the event is multi-days old
- days_held is initialized to the elapsed trading days so TIME exits
fire at the correct time relative to the original event date
- Store slice is extended backward by max_mhd calendar buffer so
pre-start rows survive slice_by_date_range when feature is enabled
- Enabled in return_max_long_v7.119 for testing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backtester (run.py):
- cash_available = (self._cash + parking_value) * multiplier caused trades to be
approved even when self._cash ≈ 0 (all money in SGOV/QQQ). Trades executed
by deducting from self._cash → negative cash (phantom money).
- Fix: after simulate_entry, if self._cash < actual trade cost and parking exists,
call _liquidate_parking_for_cash(shortfall) before deducting from cash.
- Verified: 2022-2026 backtest with qqqm_low_dd shows 0 cash_negative events.
Live engine (engine.py):
- Add _parking_liquidate_for_event(): frees parking cash to fund event entries.
SGOV (virtual) reduces entry_value in DB; QQQM/QQQ sells real shares via broker.
- Both entry loops (engines mode + flat/reaction_close mode) now attempt parking
liquidation when plan.skip_reason == "insufficient_cash" before giving up.
Also includes prior session work (accumulated since last commit):
- 6 novel parking gate signals: VRP, Market Temperature, Hurst exponent, Rolling
Kurtosis, Return Autocorrelation, SPY-QQQ Correlation (composite risk score v2)
- QQQM parking symbol support (lower expense ratio vs QQQ)
- Snapshot auto-refresh + bar extension cache (pickle) to avoid 10-min re-fetches
- Bar extension clamps to last market-closed date (ET 4PM check)
- fithia2 refresh command; --no-refresh flag for paper backtest
- Paper backtest macro extension beyond last event date (parking-only periods)
- parking_state DB schema: 7 new columns (peak_price, gate_in_sgov,
committed_target, pending_target, pending_days, sgov_entry_value, sold_today)
- Live engine: target confirmation (2-day), top-up drawdown gate, trailing stop,
SGOV interest accrual, full 6-signal gate evaluation
- New PARKING_PRESETS: qqqm_low_dd, composite_v2, vv_24_vrp8, vt_24_t13, etc.
- Web GUI / CLI result parity fix (Oracle URL via get_settings().stock_oracle_url)
- Force-close uses last_exec_date (has bar data); parking liquidates at last_date
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Builds a full synthetic market data pipeline to test strategies against
12 diverse market regimes (bull/bear/crash/chop/rotation/liquidity drought)
that may not exist in historical data. Computes Regime Robustness Score (RRS)
to detect overfitting and environment-specific fragility.
- libs/backtest/scenarios/: price_gen, macro_gen, event_gen, coupling,
store_builder, scenarios (12 pre-built), robustness (RRS)
- apps/scenario/cli.py: `fithia2 scenario-test` with Rich output
- apps/tracker/cli.py: scenario-test command routing
- tests/: 83 unit tests across 3 new test files
- docs/scenario_test.md: usage guide and result interpretation
- docs/research_workflow_and_handoff.md: Step 5.5 scenario test added
Fix: no_signal scenario uses drift=0% (was +10%) for fair signal integrity scoring.
Fix: synthetic candidates now carry macro_vix/macro_hy_spread from macro_by_date
to pass selector engine filters.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix log endpoint to serve .direct.log for direct mode tasks
- Fix _parse_dates: 4-digit start with no end now defaults to today
- Fix frontend year mode to send start=YYYY-01-01 instead of year param
- Replace DirectModePanel with DirectModeTaskView: live terminal log while
running, inline results (metric cards + equity chart + trade blotter) on
completion, collapsible log
- Add trade table sort/filter: symbol, engine, exit reason filters, Win/Loss
toggle, sortable columns (No., PnL, entry/exit price), stats bar
- Add No. column showing original trade order for sort restoration
- Add BacktestDirectResultsPage at /backtest/direct-results/:taskId
- Add Results button in task list for has_direct_result tasks
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Full paper trading page (sessions sidebar, 5-tab detail view)
- Auto daemon panel: status, schedule, start/stop, live log
- Auto daemon detection for terminal-started processes via psutil scan
- Log source detection: process stdout file → web GUI log file → TTY hint
- ANSI color rendering for paper task logs and auto daemon log
- Dark terminal theme (matching backtest log style) with macOS traffic lights
- Extracted ansiToHtml to shared lib/utils.ts (deduped from Backtest.tsx)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
conviction_v6new.307 (SQS 74.4 #1), core_boost_v6new.288 (72.8 #3),
entropy_safe_v6new.196 (72.6 #6) — now visible by name on leaderboard.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ReconciliationReport dataclass tracking orphaned/ghost positions and stale orders
- _cancel_stale_orders(): cancel leftover open orders at daily run start
- _reconcile_positions(): detect Alpaca vs local state mismatches; auto-close ghost positions with RECONCILED exit reason
- _verify_order_fill(): poll broker up to 2s to confirm market order fill before saving state
- _check_kill_switch(): activate and persist kill switch at 25% drawdown; blocks new entries
- run_daily() and _process_entries() wired with all safety checks
- 18 unit tests covering all reconciliation scenarios
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Boosting inline/guidance engine per_trade_risk from 0.015-0.020 to 0.040.
These engines have 73-100% WR — bigger positions on best signals.
v6new.307: SQS 74.4 (#1), CW 350.2%, Test +46.8%, MaxDD 2.4%, PF 14.74
Full progression: 262.9% → 284.4% → 312.6% → 320.5% → 350.2%
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Additional tracker/leaderboard updates, overlay leaderboard, and
documentation improvements.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New modules for live/mock broker interface, SQLite session state,
auto-trading engine, and backtest result reporting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces per-experiment rglob with single-pass manifest/metrics indexing
and adds lru_cache. Removes rarely-used commands from help display.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds overlay strategy backtesting, flexible date parsing, --no-trades flag,
--rank range selection, session management improvements, circuit breaker
for screener failures, and bars_cache passthrough for 10x speed gain.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extends selector with new scoring model support, adds execution
enhancements, and improves snapshot store loading and split handling.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds earnings surprise extraction to parser/features/labeler pipeline,
improves filing fetcher robustness, and extends snapshot export with
new field support.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OME engine early_failure_no_progress_days=1 (from day 2 in v272) further
improves capital recovery speed. 198 trades, 312.6% CW return.
SQS 72.7 = new #1 on leaderboard (including overlays).
Test return +45.5%, Profit Factor 19.23.
Key: cutting OME losers at day 1 instead of day 2 frees capital faster
while losing only marginal OME winners that needed >1 day to show progress.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Trade analysis revealed OME engines had 54-60% WR with stop-heavy exits.
Adding early_failure (day 2, R=0.0) for OME engines frees capital faster,
enabling 6 more trades (194→200) and boosting CW from 293.2% to 310.8%.
Key insight: cutting low-quality engine losers early improves compounding
more than any scoring/feature/sizing change tested in this session.
v6new.272: SQS 72.4 (#6), CW 310.8%, 200 trades, test +44.9%
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Higher risk (0.069) + tighter stops (ATR 1.45) produce best CW return
but lower SQS than v6new.196 (72.6) due to WFV instability.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lowering market cap floors to 2B added 15 trades (194→209) but CW return
dropped from 293.2% to 278.5%. Small-cap PEAD events have lower average
quality — individual outliers like SEDG exist but don't compensate.
v6new.255 (293.2%) confirmed as optimal trade-quality balance.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bounce engine (buy negative reaction, bet on mean reversion) could not
execute: system architecture ties scoring to single model per backtest,
and selector/store indexes are optimized for positive-reaction PEAD.
Negative-reaction candidates get score=0 from PEAD scoring, blocking
engine selection regardless of engine-level threshold overrides.
Implementing bounce trades requires: dual scoring model support,
selector changes for negative-reaction candidate routing, and
store indexing changes. Deferred to future refactor.
Current best CW return: 293.2% (v6new.255)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
v6new.196 = v29 base + entropy scoring (v13e) + per_trade_risk 0.058
+ doc_quality 0.50. CW return 284.4% (+21.5pp over v122), 194 trades.
Key finding: v29 base with doc_quality 0.50 (not 0.55) is the optimal
quality gate when combined with entropy scoring.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Data analysis revealed OBV Q1 (distribution) has 56.4% WR vs Q5 51.2% —
contrarian signal confirmed. Previous OBV bonus was applied in wrong
direction. Corrected with v15 scoring models.
Best result: v6new.185 (entropy + risk 0.058) CW 274.4% but SQS 72.2,
still below v6new.122 (72.4). WFV/robustness offsets CW gains.
v6new.122 confirmed as optimal under current SQS v4 formula.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New engine: next_open_long_mean_reversion_high_vix
Targets: react < -7%, close 0.15-0.60, bearish/mixed/unknown direction
Signal: VIX>20 + big drop = 62.9% WR, +3.91% 5d mean (n=167)
VIX 25-30 sweet spot: 75% WR, +5.46% 5d mean
Test split: 3 MR trades, 67% WR, +5.59% total PnL
Paper BT: 51 trades vs 48 (v6new.30), return ~equal
Also fixes _rows_to_table to handle sparse feature columns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously only used rows[0].keys() — columns present in later rows
(like earnings_surprise_pct from sparse features) were silently dropped.
Now collects all unique keys across all rows.
YoY earnings surprise tested: WR spread only 2.5pp (55.2% vs 52.7%).
Not actionable — YoY growth != analyst consensus surprise.
v6new.30 remains the framework optimum.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New data source integration:
- EarningsSurpriseService: GET /api/v1/earnings/surprise/{symbol}
Returns actual vs estimated EPS with surprise_percentage
- Feature builder: creates earnings_surprise_v1 snapshots for earnings events
- Backfill script runs for existing 1,273 tickers (Alpha Vantage rate limited)
New scoring (v11):
- Small beat (0-3% surprise): +10% bonus (82.4% WR in sample)
- Medium beat (3-8%): +5% bonus
- Big beat (>8%): no bonus (already priced in)
- Miss (<=0%): -5% penalty
Signal validation (n=66 sample):
Small beat: 82.4% WR, +1.79% mean 5d return
Big beat: 54.8% WR, +0.47%
Miss: 55.6% WR, -0.10%
Backfill running (~4 hours). Experiment pending data completion.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Short Side (Direction 3):
Bearish events: 45-55% WR for short — no actionable edge
Mean reversion after large drops cancels short PEAD
Execution Model (Direction 2):
T+2 delayed entry: loses 54% of alpha (Day 1 = 54% of 5d return)
Wider stops (v6new.34): -0.68pp — smaller positions offset fewer stop-outs
Tighter trailing (v6new.35): -12.48pp — cuts winners too early
reaction_close >> next_open (86% vs 57% WR) but post_market can't use RC
Direction 1 (new data: Form 4, XBRL, Earnings Surprise) requires Oracle API
implementation. Free sources identified: SEC EDGAR, Alpha Vantage, FINRA.
v6new.30 is the confirmed framework optimum.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
I Luk Kim