Observability and human review

Production controls, continued

Observability and human review

Operations measure business outcomes, system health, model behavior, and cost, and people own material actions.

Observability

Observability covers business outcomes, system health, model behavior, tool behavior, security signals, and cost. Core metrics include workflow volume, state duration, exception rate, human-review rate, tool success, tool latency, model latency, grounded-response rate, guardrail interventions, token usage, cost per case, and final disposition.

Human review

Review triggers: low confidence, ambiguous interpretation, conflicting source facts, material correction, and unknown pattern. Reviews have approved reviewer roles, service levels, disposition codes, and escalation paths. Explanations show answer, evidence, rules, confidence, and next action, and never expose chain-of-thought.

Slide 37 of 48: Observability and human review