A support agent mishandles an address change, so the optimization system repairs its identity check. The test turns green. Two weeks later, legitimate customers with recently changed phone numbers can no longer update anything.
The original failure was fixed. The product still got worse.
Kore.ai’s new Autoloop product continuously evaluates, diagnoses, changes, and re-verifies enterprise agents. It can reject a proposed change when another goal regresses, or send an uncertain fix to a person. That is a sensible release loop. It also exposes a product question that will follow every self-improving system: what evidence remains outside the loop?
Reusing every failure creates a study guide
Production failures are valuable test cases. They are also unusually easy to overfit.
Once a failure triggers a repair, becomes an evaluation, and helps judge the repair, the system has seen both the question and the answer. Passing that case proves the bug stayed fixed. It does not prove the change will work for the next customer, accent, account state, or tool response.
This is familiar machine-learning hygiene, but agent products make the boundary messy. Tests arrive from live conversations. Goals change. A single interaction can cross several agents, tools, and policies. The evaluation set slowly becomes a study guide assembled from yesterday’s mistakes.
A green score can then describe excellent memory instead of better behavior.
Protect evidence the optimizer cannot study
Product teams need a holdout set that the optimization loop cannot use to propose changes. It should contain realistic variations of important journeys. It should also include cases that have never failed in production.
The holdout does not need to be enormous. It needs to be protected, refreshed, and representative of the risks that matter. For an address-change agent, that could include new devices, stale phone numbers, delegated users, shared households, translation differences, and unavailable identity services.
Rotate part of the set on a schedule, and restrict access to the team or service that controls releases. Otherwise yesterday’s secret exam eventually becomes another worksheet in the optimizer’s folder.
Run the familiar regression suite first. Then evaluate the surviving candidate against the holdout. A change that fixes the known failure but weakens unseen variations should not advance.
This extends the feedback loop described in AI Agent Evals: Your Test Passed. Production Found the Bug.. Production evidence should improve the test set, but some evidence must remain reserved long enough to measure whether the improvement generalizes.
Separate repair confidence from release confidence
Self-improving products should report two different results.
Repair confidence asks whether the proposed change resolves the diagnosed problem. Release confidence asks whether the changed agent remains acceptable across untouched journeys and business goals. Combining them into one score hides the trade.
The distinction matters because agent changes can be precise and still have wide effects. Kore.ai’s Agent Blueprint Language makes the agent’s identity, tools, memory, routes, constraints, and guardrails versioned and diffable. A routing edit may be small in the artifact while changing which agent receives thousands of future sessions.
The release view should therefore show the changed construct, the failure it repairs, results for the ordinary regression suite, results for the protected holdout, and the rollout boundary. Product owners can then choose an automatic release, limited exposure, human approval, or rejection.
Improvement needs fresh evidence
Continuous optimization sounds like a machine that becomes more competent every night. Without reserved evidence, it can become a machine that gets increasingly good at its own exam.
Keep some cases out of the lesson plan. If each release still works there, the product has evidence of improvement rather than another green dashboard.
Comments
Comments are powered by GitHub Discussions through Giscus.