# Production Hardening and Governance

## Objective
This document defines operational governance for live trading. It assumes trading can lose money quickly when operational discipline is weak. Governance must therefore be explicit, logged, and testable.

## Governance Domains
1. Runtime safety
2. Access control
3. Config integrity
4. Release approval
5. Incident response
6. Auditability
7. Reporting and review

## Runtime Safety Controls
Mandatory controls:
- kill switch with immediate new-order disable
- no-trade mode
- degraded mode classification by source and subsystem
- hard daily loss limit
- hard gross exposure limit
- hard net exposure limit
- per-symbol and per-sector caps
- stale-data rejection for mandatory inputs
- duplicate order prevention
- restart-safe state recovery

## Access Control
Minimum roles:
- Operator: may start/stop jobs and review health, but may not change strategy logic.
- Approver: may approve live rollout stage changes and exceptional trades.
- Engineer: may deploy code through approved process, but not bypass approvals.
- Auditor: read-only access to logs, configs, incidents, and trade history.

## Configuration Governance
- All live configs must be versioned and immutable after activation.
- Any manual override must create an audit log.
- Runtime should record exact config hash used for each trade.
- Config changes may not be applied mid-session unless emergency procedure is used.

## Release Gates
A release may move to live only if:
- code passed all required tests
- migration steps were rehearsed
- rollback steps are documented
- a release note exists with risk classification
- dry-run or replay evidence is attached
- approval is recorded

## Emergency Procedures
Emergency actions include:
- disable new orders
- cancel open non-protective orders
- force no-trade mode
- disable specific adapters or overlays
- revert to previous known-good config
- pause stage promotion

Each emergency action must be logged with who, when, why, and expected blast radius.

## Incident Governance
Incidents are classified by severity:
- Sev-1: material financial or safety risk; immediate intervention required.
- Sev-2: significant malfunction but contained.
- Sev-3: degraded behavior with workaround.
- Sev-4: cosmetic or low-risk issue.

For every incident, capture:
- detection time
- impacted systems
- first symptom
- trades affected
- capital at risk
- mitigation taken
- root cause
- corrective action
- prevention action

## Audit Requirements
The system must retain for each trade:
- strategy id
- config version/hash
- code version/build id
- parser version
- feature version
- approval state
- order lifecycle events
- reconciliation outcome
- post-trade review id

## Governance Meetings
- Daily ops review: incidents, fills, reconciliation, drift alerts.
- Weekly strategy ops review: attribution, execution drift, parser quality, config exceptions.
- Monthly governance review: stage eligibility, risk policy adherence, release quality, recurring issues.