stock-oracle/dev/phase5_deliverables/operations_and_monitoring.md

Operations and Monitoring

모니터링 목표

• source freshness
• quota consumption
• entity resolution precision drift
• overlay coverage ratio
• overlay contribution stability

핵심 메트릭

source health

• overlay_source_success_rate
• overlay_source_last_success_ts
• overlay_source_latency_seconds
• overlay_source_quota_remaining

pipeline health

• overlay_records_generated_count
• overlay_records_missing_source_ratio
• overlay_degraded_mode_count
• overlay_review_queue_size

quality

• entity_match_precision_sampled
• headline_dedupe_rate
• youtube_false_positive_rate_sampled
• wiki_map_missing_ratio

strategy impact

• overlay_on_vs_off_delta_return
• overlay_on_vs_off_delta_sharpe
• overlay_hold_extension_hit_rate
• overlay_add_on_hit_rate

알림 규칙

• source freshness SLA 초과
• YouTube quota 80% 초과
• review queue backlog threshold 초과
• overlay coverage ratio 급락
• overlay contribution delta 급변

degraded mode

trigger

• critical source 2개 이상 실패
• entity matcher precision alarm
• stale overlay snapshot

behavior

• core-only mode로 전환
• overlay-based add-on 금지
• hold extension 비활성화
• alert 발송

운영 점검 루틴

일간

• 실패한 source job 확인
• overlay coverage 확인
• review queue triage

주간

• source별 precision 샘플 검수
• weight calibration 검토
• channel registry 변경점 반영

월간

• source keep/drop review
• topic map refresh
• schema/version audit