72 Commits (465e248cb4b9653b351c5a71df0c13bde084744a)

Author SHA1 Message Date
I Luk Kim 465e248cb4 Wire pct-trailing end-to-end for EarningsRunup; tune v2 risk profile
Fix EarningsRunup trailing config that was previously captured in
Candidate.features only. Add activation-gated pct trailing to
update_trailing_stop with entry-relative giveback semantics
("lock in peak − giveback% of entry"). Plumb through
Candidate → ExecutionConfig → simulator. Default None preserves
legacy pct_X behavior — no impact on engines that don't opt in.

Add earnings_runup_poc_v2_tuned config: per_trade_risk_pct 0.65→0.30,
max_positions 30→8, max_positions_per_sector 30→4, macro_vix_max=30,
trailing_warmup_days 7→0 (activation gate replaces warmup).

v2 backtest (4y, 121 trades) vs v1 baseline (119 trades):
  Total return:  +37.27%  →  +254.69%
  Max drawdown:  53.23%   →  23.97%
  SQS:           45.2     →  69.7
  Robustness:    50.1     →  100.0
  Risk score:    23.5     →  35.0
  STOP r-mult:   −0.41    →  +0.65 (trailing-locked winners)

Promotion thresholds met: MDD < 25%, return preserved, SQS > 55.
Recommend integration as PEAD sleeve adjunct (orthogonal entry timing:
pre-print attention runup vs post-print drift) rather than standalone.

7 new pct-trailing tests: activation gate, ratchet, no-ratchet-down,
reversal-stop, legacy compat, end-to-end engine wiring. 28 passed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
3 months ago
I Luk Kim 956cc78f1b Add 3 candidate engine classes beyond PEAD: EarningsRunup, PeerSympathy, VolBreakout52w
Adds three new synthetic-Candidate emitter engines parallel to the
existing leader_follower scheduler hook, plus look-ahead defenses
(LookaheadViolationError + per-engine assertions). Each engine is
covered by a standalone PoC config (no PEAD/parking/idle alpha) for
isolation backtests against the midlarge or broad snapshot.

Engines:

EarningsRunup (libs/backtest/earnings_runup.py)
 - Trigger: days_to_earnings ∈ [3,7] AND attention_zscore_20d ≥ 1.5
   AND dollar_volume_20d_zscore ≥ 1.0 (all evaluated at T-1 close)
 - Entry: T+1 next_open. Exit: -4% / +8% / max_holding_days =
   days_to_earnings - buffer (forced flat by close before announcement)
 - PIT calendar: PointInTimeEarningsCalendar adapter for backtest;
   oracle_surprise_prefetch fallback when parquet calendar absent
 - PoC verdict (configs/experiments/earnings_runup_poc_v1.json):
   119 trades over 1051 days, +37.27% total return, 44.46% MDD,
   SQS 45.2 (profitability=55.5, risk=23.5, robustness=50.1).
   VIABLE BUT NEEDS WORK — signal exists; standalone risk profile
   too aggressive for v7.356 baseline (8.8% MDD on v7.364). Path
   forward: per_trade_risk reduction, VIX gate, position cap, or
   integrate as PEAD sleeve adjunct (not as standalone replacement).

PeerSympathy (libs/backtest/peer_sympathy.py)
 - Trigger: leader passes PEAD filter (earnings_release / guidance_update
   / material_contract) AND leader reaction_close ≥ +5% AND peer 60d
   correlation ≥ 0.55 over [T-65, T-5]. Top-2 peers by correlation
   from leader_follower_extra_peer_symbols_by_sector + sector ETF
   holdings.
 - Entry: T+1 next_open on peer. Exit: -3.5% / +6% / max_holding=3 /
   peer-earnings blackout
 - PoC verdict (configs/experiments/peer_sympathy_poc_v1.json):
   256 trades over 1051 days, -52.92% total return, 54.47% MDD,
   SQS 19.6 (profitability=0.0, risk=5.4, robustness=100.0).
   DEAD. The leader's catalyst is already absorbed by T+1 next_open
   — peers gap up overnight before entry. robustness=100 confirms
   the negative result is not noise. Salvage paths (not implemented):
   reaction_close entry, raised-guidance-only restriction.
 - Note: initial run_id was 0 trades due to a select_candidates
   filter mismatch (engine.event_types=['peer_sympathy'] dropping
   real event_type='earnings_release' rows). The runner adapter
   was patched to bypass strategy_engine filtering for leader
   selection; the manual peer_sympathy_leader_event_types filter
   does the gating.

VolBreakout52w (libs/backtest/vol_breakout_52w.py)
 - Trigger: close_T-1 > max(high[T-252:T-2]) AND volume_T-1 ≥
   2 × median_volume_20d_T-2 AND ATR_14_T-1/close ∈ [0.015, 0.06].
   Entry T next_open, exit -3% / +5% / max_holding=2 / MOC.
 - Honest, look-ahead-safe descendant of the retired topgainer v1-v54
   family. Five layers of strict-before assertions guard the bar
   provider, candidate construction, trigger evaluation, and feature
   timestamps. A leaky-provider proof-by-contradiction test
   demonstrates the categorical catch.
 - PoC verdict (configs/experiments/vol_breakout_52w_poc_v1.json,
   broad-liquid universe): 1,332 trades, -87.28% total return,
   88.74% MDD, SQS 24.4 (profitability=0.0, robustness=100.0).
   DEAD AND HONEST. This is the most important finding of the three
   PoCs: the topgainer v1-v54 lineage's headline returns (+267%
   Sharpe 13.73 in best variants) were 100% lookahead bug. With
   the bug removed, the 52w-high + volume + ATR signal has no real
   alpha — the lookahead-corrected -4.3% from prior memory is
   confirmed and amplified to -87% on a fuller universe and longer
   horizon. Future "revive topgainer" proposals can cite this run
   (bt_return_max_long_v1_broad-liquid_20260509042903892342_3bb473d9)
   as definitive falsification.
 - Pre-open gap guard inactive (no premarket data in broad snapshot).
   skip_if_no_gap_data=true; the +4% gap-fade guard would not move
   the result given the magnitude.

Shared infrastructure additions:
 - libs/backtest/domain.py: LookaheadViolationError class +
   StrategyEngineConfig fields (11 EarningsRunup + 11 PeerSympathy
   + 13 VolBreakout52w = 35 new fields)
 - apps/backtester/run.py: _BacktestAttentionZscoreAdapter,
   _RunnerPeerResolver, _schedule_earnings_runup_candidates,
   _schedule_peer_sympathy_candidates,
   _schedule_vol_breakout_52w_candidates wired into the daily
   scheduler block. PeerSympathy adapter bypasses strategy_engine
   filtering on leader selection (manual filter handles gating).

Tests: 21 (EarningsRunup) + 27 (PeerSympathy) + 38 (VolBreakout52w)
= 86 new unit tests, all passing. Broader unit suite: 1392 passed,
2 pre-existing failures unrelated.

Net engine state: EarningsRunup is the only viable new engine class.
PeerSympathy and VolBreakout52w are kept in-tree as falsification
evidence, not as production engines.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
3 months ago
I Luk Kim f67c534ce3 Harden 8-K item-code classifier against dirty input strings
The _classify_event_type mapping (2.02 → earnings_release, 7.01 →
guidance_update, 1.01 → material_contract, 1.03 → other_material_event,
8.01 → other_material_event, 5.02 → management_change) was already in
place but used naive 'in items' string matching. Upstream extractors
sometimes deliver items as 'Item 2.02' or '2.02 - Results of Operations'
(full-description form), which silently slipped through to event_type
'unknown' and were rejected by all 12 v7.356 PEAD engines.

A reparse using the patched classifier touched 9,779 historical 'unknown'
rows; only 16 actually flipped (the rest are genuinely off-vocab 8-Ks
like 9.01-only, 3.01, 5.07). The fix is therefore small in retroactive
impact, but defends against future ingestion drift.

Changes:
 - libs/parser/rule_parser.py: rewrote _classify_event_type with
   _normalize_item_codes (regex \\b(\\d+\\.\\d+)\\b token extractor) and
   tuple-of-pairs _ITEM_TO_EVENT_TYPE mapping. Earnings_release wins
   priority over management_change when 2.02 + 5.02 co-occur, consistent
   with the strategy's vocabulary intent.
 - tests/unit/test_rule_parser.py: 7 new regression tests covering
   AMD/MNST-style 2.02+9.01, dirty 'Item 2.02' / '2.02 - Results...'
   forms, and negative cases (9.01-only, 2.03, 3.01 remain unknown).

Note: a follow-up vocabulary normalizer is still needed for the
Oracle-fallback path in apps/pipeline/event_parser/main.py:140, which
writes raw oracle_event.event_type values like 'earnings_result',
'shareholder_vote', 'regulation_fd' that don't match the strategy
vocabulary. Flagged for separate ticket.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
3 months ago
I Luk Kim 27de44c8d8 Fix label_price_unavailable: preserve future entry_dates as pending
Pre-market label_generator runs request future-dated price windows from
Stock Oracle, which correctly returns 404 because the data does not yet
exist. The labeler was swallowing this as label_status='unavailable' with
entry_date=None. Snapshot export then filtered these rows out, so live
PEAD trading silently lost candidates whose entry_dates fell on
later trading days (e.g., post-market 8-K filings late Friday → Monday
open entry). This explains today's missed RKLB/SNDK/AKAM/MNST/AMD/MRNA
even though their 8-Ks parsed correctly.

Changes:
 - libs/labeler/label_generator.py: in 404/empty-bars path, when
   entry_date >= today, preserve entry_date and mark label_status='pending'.
   New log event label_price_pending_future_window distinguishes from real
   data-unavailable failures (past dates still log label_price_unavailable).
 - libs/export/snapshot_export.py: include 'pending' in the
   label_status filter so today's not-yet-labeled events flow into the
   live snapshot.
 - apps/pipeline/label_generator/main.py: regeneration logic also
   retries existing 'unavailable' rows whose entry_date is null or future
   to recover events already mis-labeled in the DB.
 - tests/unit/test_labeler.py: regression test reproducing the
   RKLB/SNDK/AKAM failure mode and asserting label_status='pending' with
   entry_date preserved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
3 months ago
I Luk Kim bd26e7ab43 Fix incremental snapshot merge: coerce new row types to match existing schema
event_volume (and potentially other columns) can arrive as int64 from the
pipeline while the stored snapshot uses double, causing pa.concat_tables to
fail with "incompatible types" every run and silently fall back to a full
rebuild. _coerce_schema() casts new rows to the existing snapshot's types
before concatenation so incremental works without a full rebuild.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 004f1a5bce Fix parking entry/exit showing same price on last simulation day
On the last simulation day, parking was entered at EOD close price
(when _had_event_activity_today=True) and immediately liquidated at
close by end-of-backtest cleanup → entry == exit → PnL = 0.

Fix: force all six parking entry code paths to use "open" price when
date == last_simulation_date, so entry and cleanup-close are always
different prices.

Also adds _parking_cap logic in _extend_store_to_requested_window to
cap _requested_end_date at the last date where QQQM/TQQQ/SGOV all
have Oracle close-price data, preventing the simulation from including
days where macro is incomplete and the exit fallback would fire.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 9095b376d9 Clean up superseded configs and commit accumulated R&D infrastructure
Key changes:
- Delete superseded strategy configs: orb_gainers safe_v2-v9, orb_pullback, vwap_reclaim, hypergap, leader_safe
- Add V46 prior_event_types param to domain.py + run.py event type wiring
- Major simulator.py enhancements: sector thrust sleeve, sector proxy mapping, helper functions
- Improve screener.py with better scoring/filtering
- Add new test coverage: test_simulator.py (776 lines) + test_screener.py (313 lines)
- Add V24.1 research candidate configs (w002/w003/w004/entrycap/losscap010 variants)
- Add leader momentum research configs and sweep files
- Update configs/snapshots/registry.json with new strategy registrations
- Add docs/leader_intraday_momentum_workflow.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 08e41831bc Promote ORB Gainers V46: PEAD prior-event signal (D-7 lookback, w=0.12)
V24 → V46 via PEAD (Post-Earnings Announcement Drift) signal. Stocks with
earnings_release or guidance_update in prior 7 calendar days show +14.5pp
win rate improvement and +0.348R advantage on ORB breakouts.

Phase 1 diagnostic (291 V24 200d trades):
  Pearson=+0.135, Δ=+0.348R, WR gap=+14.5pp — all gates pass.

Phase 2 validation (w=0.12, Pareto-optimal from sweep):
  200d: V46 +114.60% / -11.83% / 3.21  vs  V24 +94.78% / -11.29% / 2.83
  400d: V46 +173.78% / -14.11% / 2.60  vs  V24 +162.1% / -13.70% / 2.471

Code changes:
- libs/intraday/domain.py: add prior_event_lookback_days: int = 0 param
- libs/intraday/orb_simulator.py: fix bug — weight_event_catalyst now wired
  for gainers_leader engine (was restricted to stocks_in_play_dual_regime only)
- apps/intraday_bt/run.py: _prefetch_prior_event_features_db() helper +
  DB routing in both catalyst trigger blocks when prior_event_lookback_days>0

V24 → status: superseded. V46 → status: live_champion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 16f49411cb Add V34-V44 ORB diagnostic scripts; wire obv_slope_5 + min_obv_slope_20d infra
Signal axes tested (V34-V44, all failed G2 ≥ 0.30R gate):
- V34 obv_slope_5 (5d): G2=0.083R (null)
- V35 obv_slope composite (5d+20d): regime artifact (200d +18pp, 400d -12pp)
- V36 RSI-14: G2=0.148R, G5a=0.742 (redundant with OBV)
- V37 BB %B / BB width: G2=0.186R
- V38 dollar_vol_trend / sleep_streak / prior_day / vol_trend: all fail G2
- V39 premarket acceleration + hold ratio: G2=0.013R (null)
- V40 prior-day market breadth: G2=0.008R (null)
- V41 min_obv_slope_20d=0.0 hard filter: -15pp (OBV as gate too aggressive)
- V42 52w-high proximity + range position: G2=0.200R (best near-miss, fails)
- V43 30-min ORB window: -14.79% (catastrophic)
- V44 trailing multiplier sweep 0.6-1.0: 0.80 confirmed global optimum

Infrastructure added (backward-compatible, V24 parity preserved):
- features.py: obv_slope_5 enrichment key
- domain.py: weight_obv_slope_5=0.0, min_obv_slope_20d=None
- orb_simulator.py: 5 wiring sites for obv_slope_5; min_obv_slope_20d gate

V24 remains live champion. 20 signal axes exhausted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 557d54921a V31 gap-zscore signal test: FAILED on both hard gate and negative weight
V31 research findings (2026-04-22):
- Hard gate (max_gap_zscore_20d=1.0): 45.2% vs V24 95.3% — catastrophically bad.
  All three terciles are profitable; hard rejection removes positive-EV trades.
- Negative weight (weight_gap_zscore=-0.05): 90.6% DD-12.33% Sh=2.649.
  Signal too weak (G2 failed at 0.181R < 0.30R threshold). G2 ≥ 0.30R
  validated as reliable promotion gate: OBV-slope (G2=0.394R) passed; all
  signals below 0.30R failed in backtest.

All 7 signal axes exhausted — V24 is the peak for current feature library.
domain.py: add max_gap_zscore_20d param (no-op at None default)
orb_simulator.py: add gainers_leader hard-gate (no-op at None default)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 4b8a157a67 Promote V24 ORB Gainers: add OBV-slope(20d) accumulation quality weight
Phase 1 diagnostic (diag_orb_quality_features.py) on V23 200d trade set found
obv_slope_20 passes all edge gates: Pearson=+0.2349 with r_multiple, top-tercile
WR 75% vs bottom 59.4% (+15.6pp), avg_R gap +0.394R. Hurst_60 and OU-θ_60 failed.

Weight sweep: 0.05 is Pareto-dominant (0.10/0.15 blow DD).

200d (same window): V24 +94.8% DD-11.3% Sharpe 2.83 vs V23 +85.0% DD-11.6% Sharpe 2.66
400d (same window): V24 +162.1% DD-13.7% Sharpe 2.47 vs V23 +149.4% DD-13.7% Sharpe 2.36
V24 Pareto-dominates V23 on both windows. V23 marked superseded.

Code changes:
- libs/intraday/features.py: add compute_obv_slope_approx() + enrich_daily_bars field
- libs/intraday/domain.py: add weight_obv_slope field to ORBStrategyParams
- libs/intraday/orb_simulator.py: wire obv_slope_20 read/store/score in gainers_leader branch
- configs: orb_gainers_v24_quality_overlay.yaml (new champion, live_readiness: experimental)
- configs: orb_gainers_v23.yaml status → superseded

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 9e622c6614 Investigate compound mode: V23 is absolute champion in all modes
- V23 pure compound (live-equivalent): 200d +148.13% DD-14.23%, 400d +209.41% DD-17.35%
- Hybrid V2 compound tested: 200d +175.41% looks promising but 400d +181.71% DD-23.72%
  loses to V23 by -27.7pp return AND -6.4pp worse DD → rejected
- Safe v9 compound 400d: +128.49% DD-14.43% — better DD but -81pp return vs V23 → rejected
- V23 tight governor compound 400d: +192.54% DD-17.32% — marginal gain, not worth config
- Live paper trader uses compound mode (engine.py:1816 session_equity = initial_equity + P&L)
- All improvement axes exhausted; V23 daily_reset declared TERMINAL

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 65afc95c7e Clean up: reduce Oracle timeout, fix company endpoint, archive old v7/v15/v16 experiments
- libs/oracle_client/alpaca.py: reduce bar-fetch timeout 90s→15s (fail fast on Oracle outage)
- libs/oracle_client/client.py: add health_check_fast() for cheap liveness probe; fix health path
- libs/oracle_client/company.py + financial.py: use /api/v1/company/{symbol} (newer endpoint)
- libs/oracle_client/__init__.py: re-export AlpacaSnapshot/get_snapshot/get_snapshots at package level
- configs/experiments: delete archived v15.x, v16.x, v7.119–v7.358 experiment configs (superseded)
- README.md: fix absolute path → relative path for ORB docs link

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim d7bfda039b Add expanded ORB simulator features and metrics
- orb_simulator.py: min_abs_gap_pct filter, premarket dollar vol filter,
  rolling_loss circuit breaker, drawdown_governor, streak_sizing,
  trailing_tighten_at_r, allow_doji/red_to_green breakout, abs_gap scoring
  for gainers_leader, ORBSimulationState, run_orb_simulation_with_state API
- metrics.py: loss_containment_score and related metrics
- features.py: enrich_daily_bars with gap_zscore, ATR ratio, range compression
- domain.py: extended ORBStrategyParams with new fields
- cache.py: DailyBarCache with merged parquet storage and coverage metadata
- simulator.py: base simulator updates for new entry/exit mechanics
- configs/intraday: updated orb_gainers_v23.yaml with canonical params
- Added BLD to midlarge symbol snapshot

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 8f0f99bdda Fix daily bar rebuild to use market-hours close; clean ORB simulator debug code
Two fixes:
1. _rebuild_daily_from_intraday_cache now filters to regular market hours
   (9:30–16:00 ET) before computing OHLCV. Previously used bars[-1] which
   included after-hours data, distorting prev_close for gap calculations.
   Root cause of V23 regression: HIMS Aug-4 after-hours drop to $54.81
   made it appear as a +0.89% gap on Aug 5 instead of the correct -12.85%
   gap (from $63.45 market close), causing it to fail min_abs_gap_pct filter.
   V23 with fix: +109.32%, WR 58.1%, Sharpe 3.01, DD -12.91%

2. Remove temporary debug instrumentation (HIMS/2025-08-05 trace blocks)
   that was left in orb_simulator.py during regression investigation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 0cae86aa87 Revert: remove daily_budget_reset from PEAD backtest
Feature was added to wrong system (PEAD backtester). Fully reverted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 189aa58343 Add daily_budget_reset mode for research backtesting
New mode (risk.daily_budget_reset=True) where cash_available and sizing
equity reset to initial_equity at the start of each day, regardless of
how many open positions or realized P&L exist. Unlike fixed_capital_sizing
(단리, sizing only), this also treats buying power as if no positions are
held — useful for evaluating signal quality independent of capital constraints.

- domain.py: daily_budget_reset field on RiskConfig
- run.py: _daily_budget_reset flag; _sizing_equity / _sleeve_equity_est /
  _build_portfolio_state all honor the new flag
- backtest_sim.py: daily_budget_reset param threaded through
- direct_runner.py: --daily-budget-reset CLI flag
- routers/backtest.py: BacktestRequest field + cmd arg
- client.ts: BacktestParams / BacktestTask types updated
- Backtest.tsx: checkbox in form + DBR badge in task list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 86419beeb0 Fix ORB intraday data pipeline and consolidate strategy configs
- screener: switch from non-existent single-ticker endpoint to multi-ticker
  /alpaca/intraday batch calls (grouped by date, chunk ≤ 75); fixes 0-trades
- cache: bump version 2→3 to invalidate stale IEX Parquet files
- oracle_client: add get_multi_intraday_bars_today() for IEX real-time feed
- paper_trader: use /alpaca/intraday/today for live sessions, /alpaca/intraday
  for historical (SIP)
- intraday.py: define _BUILTIN_STRATEGIES={} to fix /api/orb/strategies import
- delete orb_p1–p10_winner + variant configs; add strategies/orb_default.yaml
  (Phase 10 params) as the single registered web strategy

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 564bcba27c Add ORB pre-market screening and fix Oracle/Alpaca reliability bugs
- Add run_pre_screen() at 9:20 ET: fetch daily bars + enrichment + quality filter
  before market open, narrowing universe for faster orb_detect intraday fetch
- run_orb_detection() uses cached pre-screen data when available; falls back to
  full pipeline if pre_screen missed (late start, failure)
- Add _last_trading_day() helper to skip weekends/holidays for bars_end,
  preventing Alpaca 502 on Mondays (today-1 = Sunday was causing failures)
- Fix Oracle client chunk_size 300→75: Alpaca rejects 100+ ticker URL requests
- Add pre_screen event to build_schedule() at 9:20 ET and dispatch in _run_trading()
- run_session_now() runs pre_screen before orb_detect for efficiency
- Add ORB daemon, engine, models, state, screener, and intraday strategy configs
- Add intraday library (libs/intraday/) and web routes for ORB/intraday trading

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 658a741017 Route ORB bar data through Oracle API instead of calling Alpaca SDK directly
- libs/oracle_client/alpaca.py: Added get_multi_daily_bars() and
  get_multi_intraday_bars() helpers that call Oracle's /api/v1/price/data
  and /api/v1/alpaca/intraday endpoints respectively. Oracle handles
  symbol normalization (e.g. BF-B → BF.B) internally, so symbols like
  BF-B no longer crash the screening chunk.
- apps/paper_trader/alpaca_broker.py: get_bars() and get_intraday_bars()
  now use the new Oracle client helpers instead of the Alpaca SDK
  StockBarsRequest, eliminating direct Alpaca bar API calls from broker.
- apps/orb_trader/engine.py: Removed per-symbol BF-B workaround (now
  unnecessary since Oracle normalizes the symbol server-side); kept outer
  try/except for chunk-level resilience.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 426de9038d Fix lookback entry bugs: current-price sizing + MHD expiration filter
Bug #2 (paper trader): lookback entries sized using historical entry_price_est
but filled at current market price, causing cash overdraft. Fix: override
entry_price_est with get_latest_bars() close before entering _process_entries.

Bug #3 (paper trader + backtester): paper trader was missing the per-candidate
MHD expiration check that the backtester already had. Also adds
lookback_min_remaining_days (default 3) to reject candidates with too little
holding time remaining — prevents entering a position the day before forced exit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 236148de2e Remove momentum breakout sleeve (overfitting, valid -31%) and revert related code
- Delete v7.360-v7.363 experiment configs (rotation/momentum tests)
- Remove _schedule_momentum_breakout_candidates() from backtester run.py
- Remove MomentumBreakoutConfig from domain.py
- Delete momentum_calendar.py, momentum_screener.py, build_momentum_calendar.py
- Delete data/momentum_calendar/ parquet data

Valid period performance was -31.36% vs +152.4% baseline — sleeve is not viable
without walk-forward validation. Abandoning for now.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim b3ba914a8d Optimize v7 strategy: v7.356 achieves CW 2159% + SQS 90.7 (Pareto improvement over v7.314)
Key changes from v7.314 baseline (CW 2012%, SQS 90.0):
- max_position_value_pct 15→25, non_a_tier_target_1_fraction 0.2→0
- max_daily_new_risk_pct 30→50 (via v7.330, CW champion 2148%)
- bullish_raised_recovery per_trade_risk_pct 0.71→0.55 (DD improvement)
- bullish_raised_recovery max_holding_days 12→10 (sweet spot, +98pp CW)

Result: v7.356 CW 2159% (+147pp), SQS 90.7 (+0.7), risk 66.2 (+2.9), robustness 94.3 (+0.5)
All metrics improved simultaneously — return increase AND DD reduction achieved.

Also includes: web UI updates, pipeline scripts, v16/v17/v18/v19 experiment pruning,
Form4 preset additions, snapshot registry updates, domain.py enhancements.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 5496059b6c Add v7.120 composed GLD experiment updates 4 months ago
I Luk Kim f21caf23cb Prune Form4 experimental presets 4 months ago
I Luk Kim 4b1d9afde7 Prune unused sleeve presets and trim web preset lists 4 months ago
I Luk Kim 969dedc635 Speed up snapshot refresh: batch prefetch, unbuffered output, incremental-first
- enrich_tier2: prefetch price bars (parallel ThreadPool) and short ratio
  (single batch DB query) instead of per-row HTTP/DB calls (~20min → ~2min)
- canonical_snapshots: add PYTHONUNBUFFERED=1 to enrichment subprocesses
  so progress output is visible in real time
- backtest_sim: use incremental_update_canonical_snapshot when existing
  snapshot is present, falling back to full rebuild only when needed

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 5cb2b9fcb8 Add non-core allocator v2 and Form4 freshness presets 4 months ago
I Luk Kim e38c314a09 Add ownership/risk-off sleeves, v17-v19 experiments, and web app restructure
New features:
- Ownership 13D/13G residual-cash sleeve with PIT calendar and quality filters
- Risk-off alpha sleeve (GLD/DBC rotation on crisis regime signals)
- Crisis relay target in parking: evaluates before defensive relay
- Bearish symbol allocation split (bearish_alloc_pct + sgov remainder)
- Alternative defensive ETF candidate (cash_parking_defensive_alt_symbol)
- Composite eval and engine ablation tools
- experiment and overfit CLI apps

New experiments:
- v17.x series (v17.1 champion SQS 78.4; v17.5–v17.129 exhausted)
- v18.x and v19.x families from v12.8 OOT defense branch
- v7.119 composed variants (idle alpha + ownership + risk-off sleeves)
- parking_only configs: bufb, jepq, merix, regime_tiered
- empty_strategy baseline config

Web app:
- Restructured into routers/services modules (experiments, leaderboard, runs, sqs, docs)
- Ownership sleeve and risk-off sleeve controls in backtest UI
- Frontend: ComposeStrategy page, tradeSleeves lib, idle decomposition display

Research tools:
- Ownership 13D/13G probe and PIT cache builder
- Dividend capture probe and cache builder
- Insider Form4 idle alpha probe
- Alternative ETF parking probe, put-spread overlay probe
- Wikimedia low-attention and peer-relative idle alpha probes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim 5056295cb6 Add lookback entry feature for bounded backtests
When a backtest starts mid-stream (via --start), events that fired
before the start date but are still within their max_holding_days
window can now be entered on the first simulation day.

- Add `lookback_entry_enabled: bool = False` to ExecutionConfig
- On first sim day, _collect_lookback_candidates() gathers pre-start
  events, runs them through the same select_candidates() pipeline,
  and injects them before normal candidates
- Entry fills at the first day's open price; gap-cap check is skipped
  since the event is multi-days old
- days_held is initialized to the elapsed trading days so TIME exits
  fire at the correct time relative to the original event date
- Store slice is extended backward by max_mhd calendar buffer so
  pre-start rows survive slice_by_date_range when feature is enabled
- Enabled in return_max_long_v7.119 for testing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4 months ago
I Luk Kim ea9f156eeb Tune Form4 sleeve quality filters and reserve sizing 4 months ago
I Luk Kim 72681e69e5 Add Form4 residual-cash sleeve and UI support 4 months ago
I Luk Kim 86d55e01f9 Fix PIT snapshot store regressions for backtests 5 months ago
I Luk Kim f2113b7e06 Fix cash parking phantom-money bug + live engine parking liquidation for events
Backtester (run.py):
- cash_available = (self._cash + parking_value) * multiplier caused trades to be
  approved even when self._cash ≈ 0 (all money in SGOV/QQQ).  Trades executed
  by deducting from self._cash → negative cash (phantom money).
- Fix: after simulate_entry, if self._cash < actual trade cost and parking exists,
  call _liquidate_parking_for_cash(shortfall) before deducting from cash.
- Verified: 2022-2026 backtest with qqqm_low_dd shows 0 cash_negative events.

Live engine (engine.py):
- Add _parking_liquidate_for_event(): frees parking cash to fund event entries.
  SGOV (virtual) reduces entry_value in DB; QQQM/QQQ sells real shares via broker.
- Both entry loops (engines mode + flat/reaction_close mode) now attempt parking
  liquidation when plan.skip_reason == "insufficient_cash" before giving up.

Also includes prior session work (accumulated since last commit):
- 6 novel parking gate signals: VRP, Market Temperature, Hurst exponent, Rolling
  Kurtosis, Return Autocorrelation, SPY-QQQ Correlation (composite risk score v2)
- QQQM parking symbol support (lower expense ratio vs QQQ)
- Snapshot auto-refresh + bar extension cache (pickle) to avoid 10-min re-fetches
- Bar extension clamps to last market-closed date (ET 4PM check)
- fithia2 refresh command; --no-refresh flag for paper backtest
- Paper backtest macro extension beyond last event date (parking-only periods)
- parking_state DB schema: 7 new columns (peak_price, gate_in_sgov,
  committed_target, pending_target, pending_days, sgov_entry_value, sold_today)
- Live engine: target confirmation (2-day), top-up drawdown gate, trailing stop,
  SGOV interest accrual, full 6-signal gate evaluation
- New PARKING_PRESETS: qqqm_low_dd, composite_v2, vv_24_vrp8, vt_24_t13, etc.
- Web GUI / CLI result parity fix (Oracle URL via get_settings().stock_oracle_url)
- Force-close uses last_exec_date (has bar data); parking liquidates at last_date

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
5 months ago
I Luk Kim 9cb91ee846 Add synthetic scenario robustness testing system
Builds a full synthetic market data pipeline to test strategies against
12 diverse market regimes (bull/bear/crash/chop/rotation/liquidity drought)
that may not exist in historical data. Computes Regime Robustness Score (RRS)
to detect overfitting and environment-specific fragility.

- libs/backtest/scenarios/: price_gen, macro_gen, event_gen, coupling,
  store_builder, scenarios (12 pre-built), robustness (RRS)
- apps/scenario/cli.py: `fithia2 scenario-test` with Rich output
- apps/tracker/cli.py: scenario-test command routing
- tests/: 83 unit tests across 3 new test files
- docs/scenario_test.md: usage guide and result interpretation
- docs/research_workflow_and_handoff.md: Step 5.5 scenario test added

Fix: no_signal scenario uses drift=0% (was +10%) for fair signal integrity scoring.
Fix: synthetic candidates now carry macro_vix/macro_hy_spread from macro_by_date
     to pass selector engine filters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
5 months ago
I Luk Kim 76581ead04 Remove overlay backtesting and scoring 5 months ago
I Luk Kim 493b8a8d69 Add --overlay shorthand for lb command and gitignore *.db files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim a81b3a6ac4 Update tracker, leaderboard, docs, and overlay leaderboard
Additional tracker/leaderboard updates, overlay leaderboard, and
documentation improvements.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim 2aba6418e6 Add overlay engine, ranking models, snapshot pipelines, and research tools
New libs: overlay curve builder, ranking models, continuation/merged
snapshot export, intraday features. New tools: overlay evaluator,
ranking model builder, deep evaluation, fullsplit batch runner.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim ce2150789d Fix leaderboard performance regression (60min → 12s) and clean up CLI help
Replaces per-experiment rglob with single-pass manifest/metrics indexing
and adds lru_cache. Removes rarely-used commands from help display.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim 784c581f19 Enhance backtest engine: v11 scoring, selector expansion, snapshot store improvements
Extends selector with new scoring model support, adds execution
enhancements, and improves snapshot store loading and split handling.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim 57d38ecfe0 Add earnings surprise feature pipeline and snapshot export improvements
Adds earnings surprise extraction to parser/features/labeler pipeline,
improves filing fetcher robustness, and extends snapshot export with
new field support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim e2229646e6 Add oversold bounce engine experiments (v6new.259-264) — blocked by architecture
Bounce engine (buy negative reaction, bet on mean reversion) could not
execute: system architecture ties scoring to single model per backtest,
and selector/store indexes are optimized for positive-reaction PEAD.
Negative-reaction candidates get score=0 from PEAD scoring, blocking
engine selection regardless of engine-level threshold overrides.

Implementing bounce trades requires: dual scoring model support,
selector changes for negative-reaction candidate routing, and
store indexing changes. Deferred to future refactor.

Current best CW return: 293.2% (v6new.255)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim 090bfa8e36 Add contrarian feature analysis + v15 scoring (v6new.174-188)
Data analysis revealed OBV Q1 (distribution) has 56.4% WR vs Q5 51.2% —
contrarian signal confirmed. Previous OBV bonus was applied in wrong
direction. Corrected with v15 scoring models.

Best result: v6new.185 (entropy + risk 0.058) CW 274.4% but SQS 72.2,
still below v6new.122 (72.4). WFV/robustness offsets CW gains.

v6new.122 confirmed as optimal under current SQS v4 formula.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim 1e66b67c7a Add technical/scientific feature experiments (v6new.106-173) and v6new.122 SQS 72.4
Tier 1: Vol/RSI/BB/OBV features — sizing scalers hurt public SQS, scoring
adjustments ineffective on 28-30 trades. Only doc_quality gate lowering
(0.66→0.55) improved results (+2 trades, +0.8 SQS).

Tier 2: Hurst exponent, Shannon entropy, sector momentum — entropy bonus
CW +10.7pp but SQS equivalent (72.3 vs 72.4). Sector momentum hurt badly.

Tier 3: OU theta, gravitational pull, market temperature — all caused
large CW return drops (-80 to -103pp). Physics-based indicators don't
fit event-driven PEAD.

Best result: v6new.122 (SQS 72.4, #3 leaderboard) = v6new.29 + doc quality
gate 0.66→0.55. Single parameter change outperformed all feature engineering.

New code:
- libs/features/market_features.py: 9 new features (vol, RSI, BB, OBV,
  Hurst, entropy, OU theta, gravitational pull, market temperature)
- libs/backtest/scoring.py: v12-v14 scoring models with technical gates
- libs/backtest/allocator.py: volatility + conviction size scalers
- libs/backtest/domain.py: volatility_size_scaler + conviction_boost config
- scripts/enrich_*.py: snapshot enrichment scripts
- 68 experiment configs (v6new.106-173)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim 16613f1758 Fix _rows_to_table to collect keys from ALL rows, not just first
Previously only used rows[0].keys() — columns present in later rows
(like earnings_surprise_pct from sparse features) were silently dropped.
Now collects all unique keys across all rows.

YoY earnings surprise tested: WR spread only 2.5pp (55.2% vs 52.7%).
Not actionable — YoY growth != analyst consensus surprise.
v6new.30 remains the framework optimum.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim fecdc12007 Add earnings surprise feature pipeline and v11 scoring
New data source integration:
- EarningsSurpriseService: GET /api/v1/earnings/surprise/{symbol}
  Returns actual vs estimated EPS with surprise_percentage
- Feature builder: creates earnings_surprise_v1 snapshots for earnings events
- Backfill script runs for existing 1,273 tickers (Alpha Vantage rate limited)

New scoring (v11):
- Small beat (0-3% surprise): +10% bonus (82.4% WR in sample)
- Medium beat (3-8%): +5% bonus
- Big beat (>8%): no bonus (already priced in)
- Miss (<=0%): -5% penalty

Signal validation (n=66 sample):
  Small beat: 82.4% WR, +1.79% mean 5d return
  Big beat: 54.8% WR, +0.47%
  Miss: 55.6% WR, -0.10%

Backfill running (~4 hours). Experiment pending data completion.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim 529477727f Add score and event_type to trade blotter output
FilledTrade now carries event_type and score from the Candidate.
These fields are written to trade_blotter.parquet and displayed in
paper backtest trade logs.

Previously score showed as 0.00 for all trades because the field
wasn't propagated from Candidate → FilledTrade → Parquet.

Score=0.00 is valid for trades from engines with score_threshold_override=0.0
(e.g. guidance_unknown_orderly) where engine gates, not score, determine entry.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim fb4fec7dac Unify BacktestRunner and PaperTradingEngine trade decision logic
Phase 1-4 of engine unification to eliminate research/live divergence.

Phase 1 — Scoring (event_detector.py):
  EventDetector now uses config's scoring_model (v5/v9 etc.) when
  event_v1 features are present (parse_confidence_overall not null).
  Falls back to compute_entry_score only for incomplete events.

Phase 2 — Execution config (execution.py):
  Extracted build_effective_execution_config() as shared function.
  BacktestRunner delegates to it. PaperTradingEngine can now use
  identical per-engine overrides, adaptive exit, tiered targets.

Phase 3 — Attention filtering (attention.py):
  New AttentionFilterService class extracted from BacktestRunner.
  Provides: engine_requires_attention, apply_filters, rescoring.
  BacktestRunner now delegates to this service.
  PaperTradingEngine can import and use the same service.

Phase 4 — Gap cap (execution.py):
  check_next_open_gap_cap() shared function for next-open gap rejection.

All 450 unit tests pass. Paper backtest verified working.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago
I Luk Kim bb659cf7ae Add reaction-based position sizing to reduce mean-reversion loss risk
New feature: reaction_size_cap_threshold in RiskConfig
- When abs(reaction_day_return) > threshold, position size scales down
- Formula: scaler = threshold / abs(reaction) (linear inverse)
- E.g. threshold=8%: 8% reaction → 100%, 16% → 50%, 24% → 33%

Paper trading impact simulation (top 10 trades):
- PII (react +14%): loss $501 → $288 (saved $214)
- VSCO (react +18%): loss $792 → $350 (saved $442)
- RYTM (react +12%): loss $397 → $256 (saved $141)
- Winners (react <5%): unchanged (SSRM, FLS, LW, KGS all 100%)
- Total loss reduction: $2,610 → $1,813 (-30%)
- Net PnL improvement: +$797

v6new.24 backtest: Train SQS 92.5 (session best), risk=75.6.
Public SQS 52.5 (lower return due to sizing, but best risk profile).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5 months ago