fithia2/journal/attention_probe_20260317.md

Free Attention Probe

Date: 2026-03-17

Goal:

• Verify that free historical attention/news proxies can be fetched for old events.
• Run a short-sample sanity check before building anything into Stock Oracle.

Sources tested:

• Wikimedia pageviews for historical attention spikes
• GDELT Doc API for spot-check historical news article counts

Method:

• Start from data/datasets/snapshots/midcap-filtered/test.parquet
• Restrict to earnings_release
• Join ticker / issuer_name from local Postgres
• Keep only names that are not obvious {TICKER} Corporation placeholders
• Resolve a Wikipedia article title from issuer name
• Compute pageview_spike = event_day_views / median(last_10_pre_event_views)
• Compare against signed continuation:
  • signed_cont_3d = sign(reaction_day_return) * fwd_return_3d
  • signed_cont_5d = sign(reaction_day_return) * fwd_return_5d

Probe run:

• command:
  • python -m apps.tools.free_attention_probe --limit 30 --gdelt-limit 5
• output csv:
  • data/research/free_attention_probe_test_sample.csv

Results:

• Sampled 30 non-generic test-split earnings events
• Resolved 22 rows with usable Wikipedia pageviews
• Raw sample:
  • median pageview spike 1.226x
  • high-spike group signed 3D continuation mean +0.0672
  • low-spike group signed 3D continuation mean +0.0186
  • high-spike group signed 5D continuation mean +0.0693
  • low-spike group signed 5D continuation mean +0.0458
  • corr(pageview_spike, signed_cont_3d) +0.2646
  • corr(pageview_spike, signed_cont_5d) -0.1075
• After filtering to higher-confidence mappings and excluding obviously bad article matches, the broad signal became inconclusive.
• Negative-reaction subset looked more promising than the full sample on 5D continuation, but sample size was too small to trust.

GDELT spot-check:

• Historical fetch works.
• Exact-phrase matching is fragile without a better company-name resolver.
• In the small spot-check, valid 3-day article counts were observed for some names, but coverage was too patchy for immediate use as-is.

Conclusion:

• Free historical attention/news data is usable for short-window research.
• Wikimedia pageviews is immediately practical.
• GDELT is viable, but only after better issuer-name normalization and article/entity resolution.
• Current evidence does not justify adding raw pageview spike directly to strategy scoring yet.
• The most promising next test is a conditional filter on downside earnings reactions, not a global attention overlay.