Apply libs.parser.event_type_normalizer (added in commit 722e5cf) to
existing events whose parser_version LIKE 'oracle-fallback%' so historical
rows match the forward-going normalization wired into the parser.
Implementation:
- New renormalize_oracle_fallback_events() in apps/pipeline/event_parser/main.py
- SELECT filter Event.parser_version.like("oracle-fallback%") — broader
than a hardcoded IN list, so already-aligned values skip naturally and
future additions to _ORACLE_TO_STRATEGY get picked up automatically
- New --renormalize-oracle-fallback CLI flag, chainable with --reparse
- JobRun row written (job_name=event_parser_renormalize_oracle)
- Per-row renormalize_event_updated INFO log + final renormalize_done
summary with transition counters
Live DB run: seen=2937 / updated=1069 / skipped=1868 / errors=0.
Wall ~2 sec (pure DB UPDATEs, no Oracle calls).
Transitions:
earnings_result -> earnings_release : 412
shareholder_vote -> other_material_event : 409
regulation_fd -> guidance_update : 237
acquisition_disposition -> other_material_event : 7
other -> other_material_event : 4
Unmapped Oracle values (financial_obligation 225, articles_amendment 81,
contract_termination 48, etc.) preserved verbatim — honest filter-drop.
Stale-by-design (mirrors existing reparse_events convention):
- Event.event_id PK still embeds old raw event_type substring
- EventParse.output_json["event_type"] still carries raw Oracle value
Strategies read Event.event_type, not those fields. Avoids cascading
PK rewrites across event_parses/feature_snapshots/event_labels tables.
Integration test: tests/integration/test_renormalize_oracle_fallback.py
inserts 4 fixtures, drives _apply_oracle_renormalization() against the
rolled-back db_session, asserts updated/skipped/error counts and final
row state.
Snapshot rebuild not run — nightly auto-rebuild picks up normalized
values incrementally.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Backtester (run.py):
- cash_available = (self._cash + parking_value) * multiplier caused trades to be
approved even when self._cash ≈ 0 (all money in SGOV/QQQ). Trades executed
by deducting from self._cash → negative cash (phantom money).
- Fix: after simulate_entry, if self._cash < actual trade cost and parking exists,
call _liquidate_parking_for_cash(shortfall) before deducting from cash.
- Verified: 2022-2026 backtest with qqqm_low_dd shows 0 cash_negative events.
Live engine (engine.py):
- Add _parking_liquidate_for_event(): frees parking cash to fund event entries.
SGOV (virtual) reduces entry_value in DB; QQQM/QQQ sells real shares via broker.
- Both entry loops (engines mode + flat/reaction_close mode) now attempt parking
liquidation when plan.skip_reason == "insufficient_cash" before giving up.
Also includes prior session work (accumulated since last commit):
- 6 novel parking gate signals: VRP, Market Temperature, Hurst exponent, Rolling
Kurtosis, Return Autocorrelation, SPY-QQQ Correlation (composite risk score v2)
- QQQM parking symbol support (lower expense ratio vs QQQ)
- Snapshot auto-refresh + bar extension cache (pickle) to avoid 10-min re-fetches
- Bar extension clamps to last market-closed date (ET 4PM check)
- fithia2 refresh command; --no-refresh flag for paper backtest
- Paper backtest macro extension beyond last event date (parking-only periods)
- parking_state DB schema: 7 new columns (peak_price, gate_in_sgov,
committed_target, pending_target, pending_days, sgov_entry_value, sold_today)
- Live engine: target confirmation (2-day), top-up drawdown gate, trailing stop,
SGOV interest accrual, full 6-signal gate evaluation
- New PARKING_PRESETS: qqqm_low_dd, composite_v2, vv_24_vrp8, vt_24_t13, etc.
- Web GUI / CLI result parity fix (Oracle URL via get_settings().stock_oracle_url)
- Force-close uses last_exec_date (has bar data); parking liquidates at last_date
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- BacktestRunner.run() now iterates all NYSE trading days (not just candidate
days) via SnapshotStore.all_trading_days() so stop/target/time exits are
checked every day, not only on days with new candidates
- Record initial DailyPortfolioState before simulation loop starts so
total_return_pct is computed relative to the true initial equity (100k),
not the first post-entry equity snapshot
- SnapshotStore._fetch_event_metadata() now synthesises event_timestamp from
event_date + 21:00 UTC when filed_at_utc is NULL (transparent enrichment at
loader boundary, not silent substitution in selector)
- SnapshotStore._async_load() maps event_close → entry_price_est when the
column is absent, and derives score from abs(reaction_day_return) when the
Parquet snapshot has no score column
- Add --snapshot-dir CLI flag to BacktestRunner to override the default
parquet_dir base path (needed for non-standard snapshot locations)
- Fix integration test assertion: total_trading_days >= 2 (was == 2)
- Add configs/experiments/realdata_test_v1.json for real Phase 3 snapshot runs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Integration tests used ISSUER::0000320193 (Apple's real CIK) and
SYM::AAPL::XNYS as hardcoded IDs. After the real pipeline inserts
actual AAPL data, subsequent test runs fail with UniqueViolationError
since the db_session rollback only undoes intra-test writes.
Changed to ISSUER::TEST::0000320193 and SYM::AAPL::XNYS (distinct
from the real SYM::AAPL::US) while keeping ticker="AAPL" so the
Oracle price service returns real price data for label/feature tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Oracle 서비스 어댑터 5개를 실제 API 포맷에 맞게 수정
- 모든 경로에 /api/v1/ prefix 추가
- price: data[] → bars 매핑, volume float→int
- filings: accession_number→accession_no, total_count→total
- financial: financial_data[] → periods, period_date 파싱
- finra: entries[] → data 매핑
- fred: data.observations 언패킹, value string→float (버그 수정 포함)
- fixtures 6개를 실제 Oracle 응답 포맷으로 전면 교체
- 통합 테스트에서 httpx_mock 완전 제거 → 실제 Oracle 직접 호출
- 신규 단위 테스트 3개 파일 추가 (logging, fred_service, llm_parser_stub)
- test_retries.py에 exhaustion 테스트 추가
- test_oracle_client.py에 connection/timeout/no-ctx 테스트 추가
- Phase 1/2 testing_checklist.md 실제 구현 기준으로 전면 재작성
- 전체 114 tests pass (unit 100 + replay 5 + integration 9)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix all JSONB/text server_default values to use sa.text() wrapper to
prevent double-escaping in Alembic-generated SQL
- Replace testcontainers with direct docker-compose postgres connection in
integration conftest, removing asyncio.run() from async fixture context
- Change db_engine/db_session to function-scoped with explicit transaction
rollback for proper per-test isolation
- Flush IssuerMaster before Document insert to respect FK ordering
All 94 tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
I Luk Kim