Companies are rushing AI agents into production — and many of them will fail. But the reason has nothing to do with their AI models.

On day two of VB Transform 2025, industry leaders shared hard-won lessons from deploying AI agents at scale. A panel moderated by Joanne Chen, general partner at Foundation Capital, included Shawn Malhotra, CTO at Rocket Companies, which uses agents across the home ownership journey from mortgage underwriting to customer chat; Shailesh Nalawadi, head of product at Sendbird, which builds agentic customer service experiences for companies across multiple verticals; and Thys Waanders, SVP of AI transformation at Cognigy, whose platform automates customer experiences for large enterprise contact centers.

Their shared discovery: Companies that build evaluation and orchestration infrastructure first are successful, while those rushing to production with powerful models fail at scale.

>>See all our Transform 2025 coverage here<<

The ROI reality: Beyond simple cost cutting

A key part of engineering AI agent for success is understanding the return on investment (ROI). Early AI agent deployments focused on cost reduction. While that remains a key component, enterprise leaders now report more complex ROI patterns that demand different technical architectures.

Cost reduction wins

Malhotra shared the most dramatic cost example from Rocket Companies. “We had an engineer [who] in about two days of work was able to build a simple agent to handle a very niche problem called ‘transfer tax calculations’ in the mortgage underwriting part of the process. And that two days of effort saved us a million dollars a year in expense,” he said.

For Cognigy, Waanders noted that cost per call is a key metric. He said that if AI agents are used to automate parts of those calls, it’s possible to reduce the average handling time per call.

Revenue generation methods

Saving is one thing; making more revenue is another. Malhotra reported that his team has seen conversion improvements: As clients get the answers to their questions faster and have a good experience, they are converting at higher rates.

Proactive revenue opportunities

Nalawadi highlighted entirely new revenue capabilities through proactive outreach. His team enables proactive customer service, reaching out before customers even realize they have a problem.

A food delivery example illustrates this perfectly. “They already know when an order is going to be late, and rather than waiting for the customer to get upset and call them, they realize that there was an opportunity to get ahead of it,” he said.

Why AI agents break in production

While there are solid ROI opportunities for enterprises that deploy agentic AI, there are also some challenges in production deployments.

Nalawadi identified the core technical failure: Companies build AI agents without evaluation infrastructure.

“Before you even start building it, you should have an eval infrastructure in place,” Nalawadi said. “All of us used to be software engineers. No one deploys to production without running unit tests. And I think a very simplistic way of thinking about eval is that it’s the unit test for your AI agent system.”

Traditional software testing approaches don’t work for AI agents. He noted that it’s just not possible to  predict every possible input or write comprehensive test cases for natural language interactions. Nalawadi’s team learned this through customer service deployments across retail, food delivery and financial services. Standard quality assurance approaches missed edge cases that emerged in production.

AI testing AI: The new quality assurance paradigm

Given the complexity of AI testing, what should organizations do? Waanders solved the testing problem through simulation.

“We have a feature that we’re releasing soon that is about simulating potential conversations,” Waanders explained. “So it’s essentially AI agents testing AI agents.”

The testing isn’t just conversation quality testing, it’s behavioral analysis at scale. Can it help to understand how an agent responds to angry customers? How does it handle multiple languages? What happens when customers use slang?

“The biggest challenge is you don’t know what you don’t know,” Waanders said. “How does it react to anything that anyone could come up with? You only find it out by simulating conversations, by really pushing it under thousands of different scenarios.”

The approach tests demographic variations, emotional states and edge cases that human QA teams can’t cover comprehensively.

The coming complexity explosion

Current AI agents handle single tasks independently. Enterprise leaders need to prepare for a different reality: Hundreds of agents per organization learning from each other.

The infrastructure implications are massive. When agents share data and collaborate, failure modes multiply exponentially. Traditional monitoring systems can’t track these interactions.

Companies must architect for this complexity now. Retrofitting infrastructure for multi-agent systems costs significantly more than building it correctly from the start.

“If you fast forward in what’s theoretically possible, there could be hundreds of them in an organization, and perhaps they are learning from each other,”Chen said. “The number of things that could happen just explodes. The complexity explodes.”