When the rule parser can't classify an 8-K and falls back to Stock Oracle's
filing-events API, Oracle's vocabulary (e.g. earnings_result, shareholder_vote,
regulation_fd) was being written verbatim into events.event_type. The DB has
no CHECK constraint (libs/db/models.py:189), so 22 distinct Oracle values
silently leaked into a column the strategy's engine filters expect to be in
its 4-event vocabulary. Result: ~1,069 live rows silently dropped from
strategy candidate pool.
Files:
- NEW libs/parser/event_type_normalizer.py: normalize_oracle_event_type()
with conservative synonym map; normalize_oracle_event() additionally
uses _classify_event_type from rule_parser when an item_number is
present (item-code path is more reliable than Oracle's event taxonomy)
- MOD apps/pipeline/event_parser/main.py: oracle-fallback branch (~line
140) now calls normalize_oracle_event before writing to DB; emits
oracle_event_type_normalized log event when value changes
- NEW tests/unit/test_event_type_normalizer.py: 60 tests covering
identity, synonyms, case/separator insensitivity, None/empty,
non-string, item_number-precedence
Mapping highlights (justifications in test docstrings):
earnings_result/earnings_announcement/earnings -> earnings_release
guidance_revision/guidance_change/regulation_fd -> guidance_update
material_definitive_agreement/definitive_agreement -> material_contract
shareholder_vote/acquisition_disposition/bankruptcy/other -> other_material_event
Reg FD -> guidance_update mirrors rule_parser's Item 7.01 mapping for
internal consistency. Debatable but auditable.
Conservative pass-through for ambiguous values (financial_obligation,
articles_amendment, contract_termination, etc., 14 distinct values).
Visible filter-drop > silent re-tag.
Live DB counts that would reclassify on a future --reparse pass:
412 earnings_result -> earnings_release
409 shareholder_vote -> other_material_event
237 regulation_fd -> guidance_update
7 acquisition_disposition -> other_material_event
4 other -> other_material_event
TOTAL 1,069 rows currently in oracle-fallback dead-zone.
60/60 normalizer tests pass; combined parser+schema validator suite 86/86.
Follow-up flagged: run --reparse on historical oracle-fallback rows after
extending reparse_events() to also re-normalize known oracle-fallback
values (currently only re-parses event_type='unknown').
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The _classify_event_type mapping (2.02 → earnings_release, 7.01 →
guidance_update, 1.01 → material_contract, 1.03 → other_material_event,
8.01 → other_material_event, 5.02 → management_change) was already in
place but used naive 'in items' string matching. Upstream extractors
sometimes deliver items as 'Item 2.02' or '2.02 - Results of Operations'
(full-description form), which silently slipped through to event_type
'unknown' and were rejected by all 12 v7.356 PEAD engines.
A reparse using the patched classifier touched 9,779 historical 'unknown'
rows; only 16 actually flipped (the rest are genuinely off-vocab 8-Ks
like 9.01-only, 3.01, 5.07). The fix is therefore small in retroactive
impact, but defends against future ingestion drift.
Changes:
- libs/parser/rule_parser.py: rewrote _classify_event_type with
_normalize_item_codes (regex \\b(\\d+\\.\\d+)\\b token extractor) and
tuple-of-pairs _ITEM_TO_EVENT_TYPE mapping. Earnings_release wins
priority over management_change when 2.02 + 5.02 co-occur, consistent
with the strategy's vocabulary intent.
- tests/unit/test_rule_parser.py: 7 new regression tests covering
AMD/MNST-style 2.02+9.01, dirty 'Item 2.02' / '2.02 - Results...'
forms, and negative cases (9.01-only, 2.03, 3.01 remain unknown).
Note: a follow-up vocabulary normalizer is still needed for the
Oracle-fallback path in apps/pipeline/event_parser/main.py:140, which
writes raw oracle_event.event_type values like 'earnings_result',
'shareholder_vote', 'regulation_fd' that don't match the strategy
vocabulary. Flagged for separate ticket.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds earnings surprise extraction to parser/features/labeler pipeline,
improves filing fetcher robustness, and extends snapshot export with
new field support.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
I Luk Kim