ORLANDO, Fla. — Enterprises are overflowing with potential AI use cases, but CIOs are working to weed out the duds and accelerate time to value for the standouts.
The process isn’t easy. Nearly half of AI decision-makers admit their organization struggles to estimate and demonstrate the value of AI, according to Gartner research. IT leaders who fail to connect AI projects to business impact risk falling into AI hype traps, such as resource overextension or project bloat.
Businesses are wary of AI costs and the accompanying implementation challenges. Disruption abounds, pressing CIOs to make a business case for projects, set clear expectations and, only then, follow through.
“Know what value metric you’re gunning for when you choose a use case,” Arun Chandrasekaran, distinguished VP analyst at Gartner, said during a presentation at the firm’s IT Symposium/Xpo last week.
Companies are not all aiming for the same benefits when adopting AI, but there are some common targets, such as customer experience improvements or revenue growth.
Food and beverage giant Kraft Heinz created a quantitative and qualitative assessment to measure its AI project success. PepsiCo keeps a close eye on assigned KPIs for its AI pursuits, too.
Knowing the end goal of a project helps organizations identify areas for improvement if adoption goes astray. Lowe’s was able to iterate on its employee-facing AI tools this summer after tracking usage metrics that showed workers were more likely to use voice-to-text rather than typing words into the program. Backed by data-driven insights, Lowe’s then directed efforts to beef up the voice-to-text functionality, leading to improved employee and customer experiences.
“One of the underrated reasons why AI fails is because AI does not deliver value, so employees are not using it,” Chandrasekaran said. “We need to be very clear of what the value is going to be.”
No matter the use case, defining success early on and identifying metrics that will help reach the goal is critical.
“As CIOs and technology leaders, you have an opportunity to not just translate value in technical metrics, but to really translate those technical metrics into business arguments,” said Chandrasekaran. “Our ability to cross that chasm from inward focused technical metrics into more business value metrics is what’s going to make us very successful.”
Here are the top 10 metrics CIOs are tracking to measure AI success, according to Chandrasekaran:
1. Average labor cost per worker
Often used by businesses wanting to reduce costs, a labor cost formula can help measure operational efficiency.
2. Days working capital and average revolving debt utilization
Days working capital measures a business’ cash conversion cycle. Average revolving debt utilization looks at how much of the available credit a business is using.
3. Supplier spend
Businesses can track total reduction in supplier spend to determine savings yielded by an AI tool over a specific period.
4. Enforceable contract penalty credit or upside
This metric shows up as clauses in supplier contracts that provide performance-based incentives and penalties for exceeding or falling short of agreed-upon targets. Businesses tracking this measure typically aim to reduce costs and increase revenue.
5. Median time to deliver value
This metric helps determine how quickly a user reaps the benefits of adoption, identifying areas of friction. CIOs track this metric when aiming to reduce costs and grow revenue.
6. Reduction in low-experience workers or increased innovation and strategic output
CIOs looking to reduce costs and grow revenue often look to reduce the number of low-experienced workers on staff or track the amount of innovation and strategic output to gauge success against the goals.
7. Collection efficiency index
CIOs can track progress toward revenue growth by seeing if there's an improvement in their company's ability to collect payments from customers and how they manage credit.
8. Sales conversion rate
Businesses track the percentage of potential customers that turn into buyers to gauge progress on revenue growth goals.
9. Average contract value
When looking to grow revenue, businesses measure the typical size of their transactions. Use cases could include AI adoption in sales functions or indicate AI-enhanced services or experiences that customers pay a premium for.
10. Employee net promoter score
CIOs who are pursuing better employee experiences can track AI’s impact on the workforce via satisfaction scores. Higher scores would show effective implementation among other desired outcomes.
