Six patterns we see across teams that run agents in production — and how the learning loop turns each into a system improvement.
A refund agent approves a return it should have escalated. LonsLabs captures the trace, clusters the 40 similar tickets, adds a harness case, and routes a prompt-and-policy change to Support Ops. The lesson lives in prompts/refund-v14 and policies/gifts.md, not in a retro.
Two annotators tag overlapping speech differently and the agent inherits both readings. LonsLabs flags the 14 disagreements as a defect in clause 4.2, drafts a precise threshold, and routes the clause edit for two approvals. Three new evaluation cases now guard it.
A scheduling agent double-books rooms when attendees span time zones. The fix touches the calendar tool schema and a context file. Both are owned by IT Platform, so the change waits for their approval and ships with 28 harness cases passing.
A triage agent misroutes claims that mention a prior incident. Every change to the routing policy requires Compliance sign-off; LonsLabs enforces that, attaches the evidence, and keeps the review history for audit.
An internal assistant calls a deployment tool with a malformed argument. LonsLabs updates the tool guard so the call cannot be issued again and adds the exact bad call as a permanent test.
Answers cite a pricing page that changed months ago. The stale context is replaced where the agent actually reads it, and a freshness check is added to the evaluation suite so the drift is caught next time before customers do.
Different teams, different artifacts, the same loop.
Not a message in a thread — a captured trace with input, outcome, and the person who flagged it.
Prompt, context, tool, policy, evaluation, or environment. The exact artifact the next reviewer will open.
Owner, permissions, review history, and production measurement travel with it.
A working session starts from a real trace and ends with a proposed, reviewable change.
Request access