20 Commits (24e88cb74fadf7aac41c60ec8662886b925d963e)

Author SHA1 Message Date
I Luk Kim 24e88cb74f Fix cash-capped entries + wire snapshot store into run daily
_process_entries silently let cash-capped plans through — e.g. a $1k
session with 0.55 risk sized NOW at 1 share, triggering parking
liquidation of only 2 TQQQ instead of the full balance needed for
the risk-based target. Broaden the _cash_limited trigger to include
plans where plan.shares < risk-based target, and size the liquidation
needed against that target instead of plan.shares. Mirrors the
engine_batches loop fix from the prior commit. cmd_run was also
constructing the engine without snapshot_store, so run daily's
lookback entry path never fired — route it through _make_engine.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 99ad043f14 Paper trader Phase 1.A.1: poll actual fill price on exits, kill silent drop
Previously close_position(..., fill_price=X) was silently swallowed by
**kwargs while the DB wrote the simulated exit_price — broker and ledger
drifted on every exit.

- AlpacaBroker.close_position drops the **kwargs sink; docstring documents
  that callers must poll get_order(order.id) for the actual filled_avg_price
- _poll_exit_fill(order_id, symbol, timeout=30s) added; 1s interval, returns
  actual Alpaca fill price, logs timeout/terminal status
- 3 exit sites now poll and write the actual fill + re-derive net_pnl
  with direction-aware sign (short-safe for future short configs):
    run_daily main exit path (simulate_exit)
    _monitor_close (intraday stop/target hits)
    _process_exits (scheduled & forced exits incl. partial T1)
- WARN-level "paper_engine_exit_fill_drift" when actual vs simulated
  diverges >0.5% so drift is visible in logs even when non-pathological

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim d83a666f30 Paper trader Phase 1.B.3 + 1.C.1: market clock via Alpaca + scoring dispatch log
- _is_market_open() now queries Alpaca's get_clock() so holidays, early
  closes, and halted markets no longer skip orders; falls back to weekday
  9:30–16:00 ET only on broker error (1.B.3)
- AlpacaBroker.get_clock() + MarketClock dataclass wrapping alpaca-py's
  TradingClient.get_clock()
- PaperTradingEngine logs scoring_model at session startup (WARNING level)
  so multi-session daemon makes the live-path scorer dispatch explicit
  in every boot log — verifies v7.356 config's return_max_long_v13e
  actually reaches _compute_score (1.C.1)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 2b6cea57b2 Paper trader Phase 1 fixes: multi-session isolation, pipeline halt, snapshot refresh unblock
- v7.356 config: swap dataset_snapshot_id from manual_only ftb_fix_v2 to
  auto_full_rebuild base canonical so paper trader can refresh snapshot
  (root cause of processed_events=0 for 30 days)
- Multi-session order isolation (1.A.2/1.A.3): tag client_order_id with
  pt-{session_id[:8]}-{uuid} prefix on all entry orders; _cancel_stale_orders
  filters by own session prefix so one session no longer ghost-cancels another's
  orders on shared Alpaca account
- Pipeline halt on failure (1.B.1): _run_pipeline returns bool and stops on
  first subprocess failure instead of silently progressing with stale data
- Daemon restart window skip (2.2): run_open/run_close only marked completed
  if processed_phases DB confirms prior execution — no more trading-less days
  after mid-day restart
- event_parser: periodic batch commits every 500 docs (hypothesis fix for
  3h hangs; unverified — may just be slow serial Oracle calls)
- Tests updated for _verify_order_fill tuple return + new cross-session
  isolation test; all 23 paper_trader unit tests green

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 2560556c72 Fix: NameError 'sym' undefined in parking overlay gate check
Should be actual_sym (defined in _parking_evaluate_gate scope),
not sym (which is only defined in _parking_buy).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 6489b22c89 Fix: paper trader missing TQQQ overlay evaluation in parking gate
_parking_evaluate_gate() was returning only QQQM or SGOV, never TQQQ,
because the low-vol overlay logic (_evaluate_low_vol_overlay_target in
backtester) was not implemented in the live engine. Add overlay check
at the end of the gate: vol threshold, temperature, entropy, and Hurst
R/S analysis — matching the backtester's conditions exactly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 41462c9e2c Fix: parking buy fill confirmation blocks event loop with 5s hard timeout
Make _parking_buy async (asyncio.sleep instead of time.sleep) and extend
confirmation from 5s to 30s per attempt with up to 3 retries on timeout.
On timeout, cancels the stale order before retrying with fresh price data.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 27b8a49d77 Fix: paper trader ignores per-engine overrides in build_planned_order
All five build_planned_order() calls in apps/paper_trader/engine.py were
missing execution_config, causing build_planned_order to fall back to
base config.execution and silently ignore per-engine target_1_r_override,
target_1_fraction_override, max_holding_days, tiered-target settings, etc.

The backtester has always passed execution_config=_build_effective_execution_config()
(run.py:2210). This divergence caused paper trading to compute wrong target
prices and partial-exit fractions — e.g. AVGO entered with target_r=1.5
(base) instead of 3.0 (engine override), triggering a premature partial
exit on 4/15 that the backtest never produced.

Fix: add execution_config=build_effective_execution_config(candidate, self._config)
to all five call sites and hoist the function to the module-level import.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim ebfada24cb Fix: don't permanently reject events when parking sell fails transiently
- When _parking_liquidate_for_event returns False but parking had shares
  (sell failed due to transient Alpaca error), skip without recording so
  event retries on next run_next_open instead of being permanently blocked

- Improve parking_sell_for_event_failed log: include exc_type, symbol, qty
  so we can see the actual exception (was silently swallowed with empty msg)

- Also deleted AVGO from processed_events so it retries tomorrow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim f93ed2cfb5 Fix paper engine: prevent permanent rejection on transient order failures
Three bugs fixed:

1. _verify_order_fill now distinguishes alpaca_rejected vs order_timeout:
   - alpaca_rejected → record processed_event permanently (real problem)
   - order_timeout → do NOT record, allows retry on next run_next_open

2. Add _is_market_open() guard before every market buy submission:
   skips without recording so event retries when market opens

3. _parking_liquidate_for_event: sleep 1s → 3s after SGOV sell to give
   Alpaca time to settle cash; if plan still shows insufficient_cash after
   parking freed (race condition), skip without recording instead of
   permanently rejecting the event

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 426de9038d Fix lookback entry bugs: current-price sizing + MHD expiration filter
Bug #2 (paper trader): lookback entries sized using historical entry_price_est
but filled at current market price, causing cash overdraft. Fix: override
entry_price_est with get_latest_bars() close before entering _process_entries.

Bug #3 (paper trader + backtester): paper trader was missing the per-candidate
MHD expiration check that the backtester already had. Also adds
lookback_min_remaining_days (default 3) to reject candidates with too little
holding time remaining — prevents entering a position the day before forced exit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 236148de2e Remove momentum breakout sleeve (overfitting, valid -31%) and revert related code
- Delete v7.360-v7.363 experiment configs (rotation/momentum tests)
- Remove _schedule_momentum_breakout_candidates() from backtester run.py
- Remove MomentumBreakoutConfig from domain.py
- Delete momentum_calendar.py, momentum_screener.py, build_momentum_calendar.py
- Delete data/momentum_calendar/ parquet data

Valid period performance was -31.36% vs +152.4% baseline — sleeve is not viable
without walk-forward validation. Abandoning for now.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim e415743444 Use SnapshotStore for paper trader next_open candidate selection
PaperTradingEngine now accepts an optional SnapshotStore and uses it
for run_next_open candidate fetching, ensuring live candidate selection
matches the backtester's pre-computed scores exactly. run_reaction_close
keeps EventDetector for real-time intraday event detection. Adds
load_snapshot_store_for_session() helper with auto-refresh logic.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 5cb2b9fcb8 Add non-core allocator v2 and Form4 freshness presets 4 months ago
I Luk Kim 4c5798913b Extend lookback entry to paper trader and mock broker
Live paper trader (engine.py):
- On first run_next_open per daemon session, call get_candidates_for_lookback()
  to fetch events from [today - max_mhd*2, today) that are still active
- Skip gap-cap check for lookback entries (multi-day drift ≠ overnight gap)
- Initialize days_held to elapsed trading days when saving strategy state

EventDetector (event_detector.py):
- Extract shared enrichment logic into _enrich_raw_rows(raw_rows, bar_end_date, config)
- Add _fetch_events_for_date_range(start, end): single DB query with entry_date range
- Add get_candidates_for_lookback(today, start_date, config): annotates each row
  with is_lookback_entry=True and lookback_days_elapsed=N

Mock broker (backtest_sim.py):
- Extend slice_by_date_range start backward when lookback_entry_enabled, mirroring
  the same logic already present in apps/backtester/run.py main()

Verified: BX/EBAY/ENB all entered 2026-03-30 via lookback in both research
backtest and mock broker. Parking, idle_alpha, form4 sleeves unaffected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 72681e69e5 Add Form4 residual-cash sleeve and UI support 4 months ago
I Luk Kim f2113b7e06 Fix cash parking phantom-money bug + live engine parking liquidation for events
Backtester (run.py):
- cash_available = (self._cash + parking_value) * multiplier caused trades to be
  approved even when self._cash ≈ 0 (all money in SGOV/QQQ).  Trades executed
  by deducting from self._cash → negative cash (phantom money).
- Fix: after simulate_entry, if self._cash < actual trade cost and parking exists,
  call _liquidate_parking_for_cash(shortfall) before deducting from cash.
- Verified: 2022-2026 backtest with qqqm_low_dd shows 0 cash_negative events.

Live engine (engine.py):
- Add _parking_liquidate_for_event(): frees parking cash to fund event entries.
  SGOV (virtual) reduces entry_value in DB; QQQM/QQQ sells real shares via broker.
- Both entry loops (engines mode + flat/reaction_close mode) now attempt parking
  liquidation when plan.skip_reason == "insufficient_cash" before giving up.

Also includes prior session work (accumulated since last commit):
- 6 novel parking gate signals: VRP, Market Temperature, Hurst exponent, Rolling
  Kurtosis, Return Autocorrelation, SPY-QQQ Correlation (composite risk score v2)
- QQQM parking symbol support (lower expense ratio vs QQQ)
- Snapshot auto-refresh + bar extension cache (pickle) to avoid 10-min re-fetches
- Bar extension clamps to last market-closed date (ET 4PM check)
- fithia2 refresh command; --no-refresh flag for paper backtest
- Paper backtest macro extension beyond last event date (parking-only periods)
- parking_state DB schema: 7 new columns (peak_price, gate_in_sgov,
  committed_target, pending_target, pending_days, sgov_entry_value, sold_today)
- Live engine: target confirmation (2-day), top-up drawdown gate, trailing stop,
  SGOV interest accrual, full 6-signal gate evaluation
- New PARKING_PRESETS: qqqm_low_dd, composite_v2, vv_24_vrp8, vt_24_t13, etc.
- Web GUI / CLI result parity fix (Oracle URL via get_settings().stock_oracle_url)
- Force-close uses last_exec_date (has bar data); parking liquidates at last_date

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
5 months ago
I Luk Kim 98923e8620 Add Phase 6 reconciliation: position integrity, kill switch, order fill verification
- ReconciliationReport dataclass tracking orphaned/ghost positions and stale orders
- _cancel_stale_orders(): cancel leftover open orders at daily run start
- _reconcile_positions(): detect Alpaca vs local state mismatches; auto-close ghost positions with RECONCILED exit reason
- _verify_order_fill(): poll broker up to 2s to confirm market order fill before saving state
- _check_kill_switch(): activate and persist kill switch at 25% drawdown; blocks new entries
- run_daily() and _process_entries() wired with all safety checks
- 18 unit tests covering all reconciliation scenarios

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim 07bcfbe51d Complete engine unification Phase 5-6: residual reserve + macro data
Phase 5 — Engine selection (both entry paths):
  - Added residual_reserve_selected tracking between engines
  - Added prelimit amplification (5x) for attention-requiring engines
  - Added truncate_to parameter to select_candidates calls
  Matches BacktestRunner._select_candidates_for_date() behavior.

Phase 6 — Macro data:
  - Added FRED series fetch (VIXCLS, BAMLH0A0HYM2) to _fetch_macro()
  - Matches SnapshotStore._fetch_macro() which loads from MacroObservation DB
  - Enables VIX/HY regime sizing in live paper trading

All 6 phases of BacktestRunner ↔ PaperTradingEngine unification complete.
450 unit tests pass. Multi-strategy paper backtest verified.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim 26ca89c058 Integrate shared trade logic into PaperTradingEngine (Phase 3-4 complete)
PaperTradingEngine now uses:
1. AttentionFilterService after select_candidates (2 entry paths)
   - run_reaction_close: attention filtering added
   - run_next_open: attention filtering added
2. build_effective_execution_config via _resolve_execution_config delegation
   - Replaces 60-line inline implementation with shared function
   - Adds adaptive exit support (was missing)
   - Adds event_type_profile max_holding_days (was missing)
3. check_next_open_gap_cap before order submission
   - Rejects excessive gap-up entries (was missing)

This eliminates divergences #2 (attention), #3 (execution config),
and #4 (gap cap) from the unification plan. All 450 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago