Live paper trader (engine.py):
- On first run_next_open per daemon session, call get_candidates_for_lookback()
to fetch events from [today - max_mhd*2, today) that are still active
- Skip gap-cap check for lookback entries (multi-day drift ≠ overnight gap)
- Initialize days_held to elapsed trading days when saving strategy state
EventDetector (event_detector.py):
- Extract shared enrichment logic into _enrich_raw_rows(raw_rows, bar_end_date, config)
- Add _fetch_events_for_date_range(start, end): single DB query with entry_date range
- Add get_candidates_for_lookback(today, start_date, config): annotates each row
with is_lookback_entry=True and lookback_days_elapsed=N
Mock broker (backtest_sim.py):
- Extend slice_by_date_range start backward when lookback_entry_enabled, mirroring
the same logic already present in apps/backtester/run.py main()
Verified: BX/EBAY/ENB all entered 2026-03-30 via lookback in both research
backtest and mock broker. Parking, idle_alpha, form4 sleeves unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backtester (run.py):
- cash_available = (self._cash + parking_value) * multiplier caused trades to be
approved even when self._cash ≈ 0 (all money in SGOV/QQQ). Trades executed
by deducting from self._cash → negative cash (phantom money).
- Fix: after simulate_entry, if self._cash < actual trade cost and parking exists,
call _liquidate_parking_for_cash(shortfall) before deducting from cash.
- Verified: 2022-2026 backtest with qqqm_low_dd shows 0 cash_negative events.
Live engine (engine.py):
- Add _parking_liquidate_for_event(): frees parking cash to fund event entries.
SGOV (virtual) reduces entry_value in DB; QQQM/QQQ sells real shares via broker.
- Both entry loops (engines mode + flat/reaction_close mode) now attempt parking
liquidation when plan.skip_reason == "insufficient_cash" before giving up.
Also includes prior session work (accumulated since last commit):
- 6 novel parking gate signals: VRP, Market Temperature, Hurst exponent, Rolling
Kurtosis, Return Autocorrelation, SPY-QQQ Correlation (composite risk score v2)
- QQQM parking symbol support (lower expense ratio vs QQQ)
- Snapshot auto-refresh + bar extension cache (pickle) to avoid 10-min re-fetches
- Bar extension clamps to last market-closed date (ET 4PM check)
- fithia2 refresh command; --no-refresh flag for paper backtest
- Paper backtest macro extension beyond last event date (parking-only periods)
- parking_state DB schema: 7 new columns (peak_price, gate_in_sgov,
committed_target, pending_target, pending_days, sgov_entry_value, sold_today)
- Live engine: target confirmation (2-day), top-up drawdown gate, trailing stop,
SGOV interest accrual, full 6-signal gate evaluation
- New PARKING_PRESETS: qqqm_low_dd, composite_v2, vv_24_vrp8, vt_24_t13, etc.
- Web GUI / CLI result parity fix (Oracle URL via get_settings().stock_oracle_url)
- Force-close uses last_exec_date (has bar data); parking liquidates at last_date
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ReconciliationReport dataclass tracking orphaned/ghost positions and stale orders
- _cancel_stale_orders(): cancel leftover open orders at daily run start
- _reconcile_positions(): detect Alpaca vs local state mismatches; auto-close ghost positions with RECONCILED exit reason
- _verify_order_fill(): poll broker up to 2s to confirm market order fill before saving state
- _check_kill_switch(): activate and persist kill switch at 25% drawdown; blocks new entries
- run_daily() and _process_entries() wired with all safety checks
- 18 unit tests covering all reconciliation scenarios
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New modules for live/mock broker interface, SQLite session state,
auto-trading engine, and backtest result reporting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds overlay strategy backtesting, flexible date parsing, --no-trades flag,
--rank range selection, session management improvements, circuit breaker
for screener failures, and bars_cache passthrough for 10x speed gain.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 5 — Engine selection (both entry paths):
- Added residual_reserve_selected tracking between engines
- Added prelimit amplification (5x) for attention-requiring engines
- Added truncate_to parameter to select_candidates calls
Matches BacktestRunner._select_candidates_for_date() behavior.
Phase 6 — Macro data:
- Added FRED series fetch (VIXCLS, BAMLH0A0HYM2) to _fetch_macro()
- Matches SnapshotStore._fetch_macro() which loads from MacroObservation DB
- Enables VIX/HY regime sizing in live paper trading
All 6 phases of BacktestRunner ↔ PaperTradingEngine unification complete.
450 unit tests pass. Multi-strategy paper backtest verified.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1-4 of engine unification to eliminate research/live divergence.
Phase 1 — Scoring (event_detector.py):
EventDetector now uses config's scoring_model (v5/v9 etc.) when
event_v1 features are present (parse_confidence_overall not null).
Falls back to compute_entry_score only for incomplete events.
Phase 2 — Execution config (execution.py):
Extracted build_effective_execution_config() as shared function.
BacktestRunner delegates to it. PaperTradingEngine can now use
identical per-engine overrides, adaptive exit, tiered targets.
Phase 3 — Attention filtering (attention.py):
New AttentionFilterService class extracted from BacktestRunner.
Provides: engine_requires_attention, apply_filters, rescoring.
BacktestRunner now delegates to this service.
PaperTradingEngine can import and use the same service.
Phase 4 — Gap cap (execution.py):
check_next_open_gap_cap() shared function for next-open gap rejection.
All 450 unit tests pass. Paper backtest verified working.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Snapshots can be in data/parquet/ or data/datasets/snapshots/.
Now tries default parquet_dir first, falls back to data/datasets/snapshots/
if the snapshot exists there instead.
Fixes FileNotFoundError when running multi-strategy paper backtest with
configs that reference snapshots in the alternate directory.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
run_backtest changed from async to sync function. Pipeline refresh
(async) runs via asyncio.run() before the sync BacktestRunner,
avoiding nested event loop when SnapshotStore.load() calls asyncio.run().
CLI updated to call run_backtest() directly (no asyncio.run wrapper).
Tested: `fithia2 paper backtest --config v6new.24 --start 2025-03-23 --end 2026-03-23` works.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When `fithia2 paper backtest --end <date>` requests a date beyond the
snapshot's latest event, automatically runs the pipeline:
1. Filing poller (discover new 8-Ks)
2. Filing fetcher (download exhibits)
3. Event parser (parse events)
4. Feature builder (compute features)
5. Label generator (compute labels)
6. Dataset export (re-generate Parquet snapshot)
Staleness check: snapshot is stale if its latest event_date is >14 days
before the requested end_date, or if the manifest is >7 days old.
If refresh fails, falls back to existing snapshot data gracefully.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Major refactor: `fithia2 paper backtest` now uses the exact same
BacktestRunner + SnapshotStore pipeline as `apps/backtester/run.py`.
Before: PaperTradingEngine + EventDetector + MockBroker
- Different scoring (compute_entry_score vs config scoring_model)
- Different data source (DB + Oracle vs Parquet snapshot)
- Different feature computation (real-time vs pipeline)
→ Config gate changes didn't take effect in paper backtest
After: BacktestRunner + SnapshotStore (Parquet)
- Identical scoring, engine matching, position sizing
- Same Parquet data as research backtester
- Config changes work identically in both systems
Trade output format preserved for reporter.py compatibility.
PaperTradingEngine still used for live Alpaca trading (unchanged).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The DB-first approach (prefer feature_json over Oracle recalculation) caused:
- LMND (+$782) and M (+$1,052) trades to disappear
- TEM loss to increase from -$321 to -$535
- Overall PnL drop from +$5,948 to +$3,078
Root cause: DB features were computed at a different time with different
Oracle data. When paper trader used DB values, the feature values didn't
match what the backtester's Parquet snapshot had, causing different
engine gate outcomes.
Paper trader must use Oracle real-time enrichment as primary source
(same as the original design). The volume_ratio_20d field name fix
is retained as that was a genuine bug.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The _compute_score → v5 dispatch caused v5's hard gates to reject almost
all events (v5 requires specific direction/guidance combos). This killed
all 2025 trades in paper backtest.
Root cause: BacktestRunner and PaperTradingEngine use different flows.
BacktestRunner applies scoring AFTER engine selection (engines have
score_threshold_override=0.0 that bypasses score gates). But EventDetector
applied scoring BEFORE engine matching, causing v5's hard gates to reject
events that engines would have accepted.
Fix: revert to compute_entry_score for EventDetector. Score is ranking-only
in paper trading; engine gates (reaction_min, close_min, etc.) handle filtering.
The volume_ratio_20d fix and DB-first feature fix remain in place.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three critical inconsistencies between BacktestRunner and PaperTradingEngine
that caused gate fixes to not work in paper trading:
1. DB feature values now take priority over Oracle recalculation
- Previously: Oracle bars always recomputed reaction_day_return etc.
- Now: if DB feature_json has the value, Oracle fallback is skipped
- Root cause of PII bug: DB had react=-5.3% but Oracle recomputed +13.9%
due to different date alignment, bypassing engine reaction_min gate
2. Scoring now uses config's scoring_model (v5/v8/v9/v10 etc.)
- Previously: always used compute_entry_score() regardless of config
- Now: _compute_score() dispatches to the correct scoring function
- Ensures hard gates and weights match between backtest and paper trading
3. volume_ratio_20d field name consistency (from prior commit)
These fixes ensure paper trading results match backtester behavior,
making engine gate changes (reaction_min, close_min, etc.) effective
in both systems.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
EventDetector computed volume_ratio as fallback but selector checks
volume_ratio_20d. When DB feature_json was missing this field, the
volume gate was silently bypassed in paper trading — allowing trades
like LKQ (vol=0.8) that the backtest correctly blocks.
Now sets both volume_ratio_20d and volume_ratio for consistency.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
I Luk Kim