Producing a good recommendation used to be the finish line for a software product.
Now it is closer to the starting gun.
AWS’s new Well-Architected Agent illustrates the shift. The preview analyzes cloud environments across cost, security, performance, and resilience, then prioritizes recommendations against business goals. More interestingly, it can package findings with updated infrastructure-as-code, command-line steps, console walkthroughs, and, in some cases, automation runbooks.
The recommendation is no longer just an answer. It is the beginning of a change.
That changes the product requirements considerably.
Advice needs an execution path
Most organizations don’t suffer from a total shortage of recommendations. Cloud consoles, security tools, consultants, architecture reviews, and that one engineer who comments on every pull request are already producing them.
The scarce resource is turning the right recommendation into a safe, completed change.
That path usually requires somebody to recover the context, estimate the blast radius, find the owner, translate prose into code, obtain approval, schedule the work, and verify the result. A recommendation can be technically correct and still die somewhere in that sequence.
AWS is trying to remove part of that translation work. Its agent generates recommendations at the resource, application, and architecture levels. It also shows cross-pillar effects and trade-offs. A resilience improvement, for example, may increase cost or alter performance. That context travels with the proposed fix instead of living in a separate meeting.
This is where the product value sits: less reconstruction between insight and action.
A proposed fix is a different product object
A finding and a proposed change shouldn’t share the same acceptance criteria.
For a finding, I care about relevance, accuracy, severity, and prioritization. For a proposed change, I also want to know:
- What exactly will change?
- Which resources and customers could be affected?
- Who can approve it?
- How will we know it worked?
- How do we reverse it?
Those questions sound operational because they are. Once a product emits deployable code or commands, it has crossed from analysis into change management.
AWS appropriately warns that the generated recommendations may contain errors or incomplete information and should be reviewed. That review should not be a ceremonial click. The interface has to make the evidence, assumptions, trade-offs, permissions, and proposed diff easy to inspect.
Otherwise the product has made execution faster while leaving judgment exactly where it was, only with more momentum behind it.
Measure completed improvements, not generated findings
An AI optimization product can produce impressive activity metrics: environments scanned, recommendations generated, potential savings identified, or runbooks created.
None proves the system improved.
I would follow the recommendation through its full lifecycle. Measure the percentage accepted, time to decision, time to verified remediation, rollback rate, recurrence rate, and realized improvement against the original goal. Also measure rejection reasons. A high rejection rate may indicate weak context, poor prioritization, unacceptable trade-offs, or recommendations aimed at teams that lack authority to act.
The denominator matters too. Ten completed improvements from twelve useful recommendations may be better than fifty completed improvements buried inside five hundred low-value findings.
This connects to my earlier argument that AI agent permissions are a product decision. Generating a fix, approving it, and executing it are separate authority levels. Keeping those boundaries visible is part of the product, not friction to be designed away.
The best AI recommendation system won’t merely tell a team what should change.
It will preserve enough context for the team to decide, make the safe path cheaper, and prove whether the change actually helped.
The answer is useful. The completed improvement is the outcome.
Comments
Comments are powered by GitHub Discussions through Giscus.