Editor’s note: Welcome to Tech Bytes, a regular column from CIO Dive that rounds up relevant product and industry news for CIOs and other senior IT leaders. Have a tip for future editions? Email our team.
AI systems are quickly eating up compute capacity, putting pressure on existing infrastructure and driving large deals to support expansions. For CIOs, the downstream effect is showing up in the shape of rising costs and memory shortages.
A cohort of financial institutions including Bank of America, Citi, Morgan Stanley and Blackstone laid out a plan this week to pour $63 billion into financing the expansion of AI compute capacity for frontier model developer Anthropic. The financial partners will buy Broadcom XPU AI accelerators and networking products and lease them to Anthropic starting in 2027.
“Advancing AI responsibly requires turning ambitious technology plans into tangible compute infrastructure,” Jim DeMare, co-president of Bank of America, said in a press release.
The rush to finance more AI compute is boosting global IT spend, which is set to reach $7.35 trillion in 2027 as organizations seek to “industrialize AI deployment,” according to Gartner projections. Currently, compute capacity is exceeding enterprise demand, and vendors are offering deals on models and token pricing to gain users.
However, the bill will come due for CIOs, John-David Lovelock, distinguished VP analyst at Gartner, told CIO Dive in a previous interview.
“As much as CIOs are worried about the cost of AI today, it’s not going to be this low again,” he said.
With compute costs set to rise, enterprises are starting to think more strategically about AI model use. That includes the financial establishments funding the massive Anthropic deal. Bank of America’s AI routing layer Orchestra uses a small, classifier model to send queries to the most cost-effective option, helping the bank avoid the use of costlier frontier models for basic queries.
Another financial challenge facing CIOs lies in memory chip shortages and the resulting price increases driven by the AI infrastructure buildout. Makers of PCs, laptops, tablets and mobile phones are passing on surging memory prices to end devices, which Lovelock said will continue into at least 2030.
Here are a few other notable updates for CIOs:
Nvidia, Meta expand Muse partnership
Nvidia and Meta are working to improve the inference and training performance of Meta’s Muse models and agents, which are built on Nvidia’s infrastructure. As part of the collaboration, unveiled Tuesday, Nvidia also said it plans to invest $3 billion in Meta’s equity offering in support of Meta’s AI infrastructure ambitions, according to a press release.
The collaborative effort to improve Muse, a personal AI agent, comes as financial institutions contend with security risks associated with allowing third-party AI agents into their environments. Multiple banks, including JPMorgan Chase, have decided to block agents in recent weeks, citing security concerns.
OpenAI adds Oracle apps to ChatGPT
Frontier model provider OpenAI is bringing Oracle applications and data into ChatGPT, according to a Wednesday announcement. The development builds on the companies’ existing relationship, as OpenAI models power new Oracle products and services. The companies aim to “use AI to run core operations,” according to a press release.
“Our goal is to make Oracle software easier to discover, use, and manage, with simpler administration, unified sign-in, and access governed by each organization’s permissions and approvals,” the release said.
IBM bolsters agent controls
On Wednesday, IBM unveiled a set of changes to its AI operating model in a bid to provide better control, authority and accountability over AI agents to IT and security teams.
The company extended HashiCorp Vault — a data protection tool that manages secrets and credentials used by applications to access business systems — to AI agents. IBM also extended HashiCorp Vault's agent identity and credential controls, enabling IT teams to stop agents in real-time if an unauthorized action is attempted.
Slack Code API goes live
Slack, owned by Salesforce, launched its Slack Code service in August, letting partners including GitHub develop agents without leaving the Slack channel. This week, Slack made its Slack Code API generally available, allowing developers to have agents start a channel in Slack, add relevant people and post updates.
“We’re building the GitHub Copilot interactions you’re already familiar with into the same space you use for all of your other work,” Tim Hunter, staff engineer at GitHub, said in a press release. “Being able to see what the agent has worked on and steer it right from inside Slack is a game changer.”
