CIOs are tracking the AI vendor market as new models and updates are released, such as OpenAI’s highly anticipated GPT-5 last week and Anthropic's expanded Sonnet 4 earlier this week.
“GPT-5 was on my radar, not that it will be immediately put to use in some business use case,” Feroz Merchhiya, CIO for the City of Santa Monica, said in an email. “I had it on my radar to explore how well we could solve AI challenges with GPT-5 that GPT-4 and others couldn’t solve.”
Enterprises are tying AI closer to the overall success of the business, using the technology as a lever to drive growth, add a competitive edge and save costs. Newly released AI models with better capabilities represent a chance to improve ongoing efforts.
“My approach to evaluating an AI model … is based on data and algorithms,” Merchhiya said. “Taking a deeper look, my approach also includes evaluating ethical considerations, such as bias and fairness, data privacy, security and transparency.”
CIOs, like Merchhiya, are also looking to evaluate compute requirements, hallucination rates and the overall enterprise readiness of additional guardrails, monitoring and other controls.
First impressions of GPT-5 weren’t all positive; even OpenAI CEO Sam Altman acknowledged the bumpy rollout in a series of social media posts following the model’s release. Analysts said part of the cold reaction was due to the less flashy nature of upgrades.
“I gave the GPT-5 launch video a few minutes of my attention — underwhelming,” Brian Hopkins, VP emerging technology and principal analyst at Forrester, said in a blog post last week. “The upgrades — integrated routing, a rebuilt multimodal core, and adaptive inference — aren’t crowd-pleasing upgrades, but they directly address operational pain points that enterprises face today with generative AI applications.”
Some industry watchers say GPT-5’s anticlimactic arrival is a symptom of an overhyped, elongated marketing rollout.
“For enterprises and watchers who are expecting AGI, it might have been underwhelming,” Chirag Dekate, VP analyst at Gartner, told CIO Dive. “But at the end of the day, AGI is a marketing term, just like super intelligence. I have no idea what those elements actually mean.”
Pace of innovation
Longer waits and more iterative updates signal the end of equating significantly bigger investments in compute and data with better model results, according to some industry experts. Conversations around diminishing returns have picked up momentum following GPT-5’s release.
Other analysts say: Not so fast.
“We are nowhere close to the peak of what Gen AI can do,” Dekate said. “We are now discovering the next phase will require us to connect the dots between compute, data, model and that is clearly starting to take shape.”
OpenAI kick-started the AI vendor shakeup nearly two years ago, commanding 50% of the enterprise LLM market, but that lead has since eroded, according to a July Menlo Ventures report. The early-stage venture capital firm surveyed 150-plus technical leaders across startups and enterprises. While OpenAI has now halved its market share from two years ago, Anthropic commands 32% and Google comes in third at 20%.
“The two most dominant patterns we've seen in the enterprise LLM market have been value accrual to the most state-of-the-art models, and the rise of code as AI's first killer use case,” Derek Xiao, investor at early-stage venture capital firm Menlo Ventures, said in an email. “Anthropic has capitalized on both trends at OpenAI's expense.”
Experts have characterized industry dynamics as a race for a reason.
“The consumer side of the house is likely going to be a winner-takes-all market,” Dekate said, pointing to the smartphone sector as a similar phenomenon. “The frontier model labs … are aggressively trying to deliver model innovations faster because they know falling behind is not an option. The market is designed to capture consumer attention, and now the enterprise to a lesser extent, because they’re not necessarily leading adopters.”
The pressure to keep up with the pace of innovation haunts technology leaders even after they’ve closed the computer at the end of the workday.
“Discussions about AI are happening everywhere,” Merchhiya said, pointing to traditional media sources as well as social media.
CIOs are finding ways to sift through the noise and push back on outsized claims.
“What we recommend enterprises to do is go beyond the headlines into testing and validating,” said Dekate. CIOs with mature AI strategies can make time to experiment in sandboxed environments to get a better understanding of a new model’s capabilities and explore potential use cases.
“Fast followers are waiting on the sidelines, and they are approaching it differently,” Dekate said. “They look at some of these developments with a bit of fear and uncertainty because … they’re not as experiential with AI and they’re trying to figure out when and where to latch on … and they’re finding themselves to be increasingly behind.”
No matter where an enterprise lands on the AI maturity spectrum, its technology leaders are likely to notice AI vendors bringing less shiny new tricks to the table and more iterative benefits.
“The simplistic brute force era is behind us,” Dekate said. “It’s much more complex. Some would term it as slowing down. I would not. It’s a different scale and experience set, essentially translating to more compute, more data.”
