Dive Brief:
- Latency issues are on the rise for enterprises as systems cope with agentic workloads, according to a survey of 528 IT decision-makers conducted by Forrester Consulting on behalf of cloud provider Akamai. Nearly all respondents said they had taken at least one action in the last year to mitigate lagging infrastructure performance.
- After an application failed to meet its response time goal, 58% of respondents increased spending on infrastructure to try and improve the metric, the survey found. Other IT decision-makers made structural changes, with 38% reducing the scope of a product’s features, 34% delaying product launches and 30% restricting capabilities.
- Nearly half of respondents accepted degraded performance capabilities for some users. “Loads of organizations are struggling to hit the performance they have said they want to, and [some] said ‘we just kind of deal with it,’ and that’s not great,” Danielle Cook, director of product marketing at Akamai, told CIO Dive. “It’s definitely not great when you have agents.”
Dive Insight:
Shortening the distance between compute and applications is becoming a bigger consideration for CIOs as they support agentic workloads and manage performance issues.
Agentic workloads are placing greater strain on operations and becoming more costly for enterprises, running up the bill for data processing, increasing demand for low latency and hiking infrastructure costs, according to a report from Ocient and Propeller Insights. Roughly 25% of tech leaders said their AI demand has outgrown the infrastructure to support it, the report found.
Global IT spend is expected to reach $7.35 trillion next year as organizations pump money into AI infrastructure, seeking to “industrialize AI deployment,” according to analyst firm Gartner.
As response times become more crucial, only 16% of companies say they consistently met response time goals for business critical applications, according to the Akamai survey. One of the biggest contributors to latency issues was physical distance separating end users and compute, according to 38% of respondents. Another 41% said where data was stored versus where it was processed was a contributing factor.
Real-time workloads are becoming more prevalent, with three-quarters of respondents saying that over 25% of their applications are interactive. However, traditional workloads including batch processing and large-scale data analytics continue to account for the biggest shares of enterprise compute.
“Centralized infrastructure is great, there’s a lot of good, solid reasons you need that,” Cook told CIO Dive. “But when you have agents that are competing for information or trying to get a loop finished fast, you need to have distributed compute.”
For companies creating new applications or looking to access open weight models, Cook said IT leaders can assess what compute can move and how to measure performance instead of re-architecting existing environments.