Observability and human review
Production controls, continued
Observability and human review
Operations measure business outcomes, system health, model behavior, and cost, and people own material actions.
Observability
Observability covers business outcomes, system health, model behavior, tool behavior, security signals, and cost. Core metrics include workflow volume, state duration, exception rate, human-review rate, tool success, tool latency, model latency, grounded-response rate, guardrail interventions, token usage, cost per case, and final disposition.
Human review
Review triggers: low confidence, ambiguous interpretation, conflicting source facts, material correction, and unknown pattern. Reviews have approved reviewer roles, service levels, disposition codes, and escalation paths. Explanations show answer, evidence, rules, confidence, and next action, and never expose chain-of-thought.
