Enterprises are navigating agentic transformation, searching for productivity wins and cost savings despite high failure rates among AI pilots.
Companies that learn to build trust, measure impact, show their results and maintain governance as projects progress find more success during initial AI efforts, said Mani Gill, SVP of product at Boomi. Gill and colleague Patricia Bradby Moore, AI field CTO and innovation lead for the application integration platform, spoke at the AWS Summit in New York City on Wednesday about their clients’ case studies that have had the most success.
“It's not just a matter of having a connection to the data itself, it's a matter of the agents knowing what that data is about and how to utilize it,” Bradby Moore said.
Teams need a strong data foundation and proper introduction to the AI tools it is expected to use to grow trust, Gill said. He recommended teams start with lower stakes AI use cases to test tools, because it’s easier for leaders to set a foundation, identify changes and scale up that way.
“Everybody goes for the [use case] that seems to be the coolest, the one that seems to have the biggest impact,” Gill said. “The reality is the coolest and the one that has the biggest impact — that one is the most complex.”
After teams establish trust, they can begin to measure impact and prove ROI. Although departments within an organization may define ROI differently, it can’t be done at all without defining the business impact of the specific application, Gill said. Teams should also decide what exactly they need to measure.
ROI can’t be determined without also considering risk, Gill said. Teams should consider if the productivity gains they make are worth the potential risk they could be introducing.
Spotting AI value
Leaders can guide adoption plans toward success by making it clear for employees that experimentation is encouraged.
Showing the results of AI projects across an organization helps everyone learn what is and isn't working, Gill said. He’s observed that some teams feel shame around using AI outputs because they didn’t complete tasks or code something themselves.
“We need to flip that mindset and say, 'No, we're using the tools not to create AI slop, but to create AI value,’ and we need to be able to demonstrate those,” Gill said.
Maintaining guardrails and governance with the AI projects that make it past the pilot phase is essential, Gill said. This means deciding the rules and training needed for employees to use and monitor the outputs, but businesses should keep track of how many agents are running and what they have access to, he said.
Human-in-the-loop protocols are imperative as teams scale their use cases up from the most simple, such as automation, to more complex endeavors, such as agentic workflows that are contributing to decision-making within an organization.
“We're trying to drive not just technology change, but also process and culture change,” Gill said.
