Generative AI

Chief AI, technology and data executives will gain greater C-suite prominence over the next five years as tech adoption plans advance. The trend reflects ongoing generative AI interest in the enterprise.

Two in 5 respondents to a West Monroe survey published in June said the chief AI officer role would increase in importance over the next five years. It was the most popular option among the 1,000 consulted professionals holding director, VP, SVP roles and above. 

The second most favored role was chief data or information officer, highlighted by 11% of respondents as the C-suite role most likely to gain greater traction.

Companies align C-suite tech roles with their business strategy. But the rise of specific roles, like chief digital or AI officer, often tracks with organizational or technological maturity, according to Casey Foss, chief commercial officer at West Monroe.

"We're seeing organizations that, five years ago, named a chief digital officer in preparation to respond to this physical and digital world and the commingling of the two to get their organizations ready," Foss said. "Now, digital permeates every function and every component, and so that role is going away but the need isn't."

As AI interest deepens, appointing an officer to guide AI strategy signals to the market and employees that the company plans to invest heavily in the technology and its adoption, Foss said.

CIOs still lead on AI

Despite the popularity of the title, the majority of enterprises still task their CIOs with leading AI initiatives.

One-quarter of businesses entrust a CIO with the technology, though more than half of businesses have a head of AI or AI leader to shepherd adoption, according to Gartner survey of 1,800 executives published June 26.

As the technology permeates the enterprise, organizations will adjust their leadership strategies, Gartner expects.

“AI and GenAI are complex and far-reaching and touch every job, activity and strategic conversation in the organization,” Frances Karamouzis, distinguished VP analyst at Gartner, said in the study announcement. “However, this does not mean that the people or team responsible for orchestrating AI at an organization have to have a title at the altitude of the C-suite.”

The next five years will probably bring a different title into the mix, in order to let businesses address new kinds of emerging technology, according to Foss.

"I don't think AI is going anywhere," Foss said. "I think it's going to become more like digital, in that it's going to become infused in every aspect of the business."

IT leaders expect AI to deliver operational improvements this year, according to a Celigo report published earlier this month.  The company surveyed more than 1,200 IT and operations leaders. 

Nearly three-quarters of businesses are currently using AI within the IT department and half have implemented the technology in general operations, according to the survey.

Executives already using AI said nearly half of teams experienced improved productivity and optimized operations. More than one-third pointed to customer experience gains and reduced costs as a result of adopting the technology. 

AI adoption in the workplace is rapidly expanding, but leaders have different goals when deploying the technology. 

“While both departments are eager to adopt AI, they have different priorities,” Celigo said in a blog post. “IT leaders are focusing on automating and reducing manual processes, whereas operations leaders are looking to cut costs.”

As IT teams tackle more projects and face more stakeholders, CIOs are trying to prevent burnout and reduce backlogs to support workers. 

Critical skills gaps within most enterprises are already adding to tech workers’ burden. The majority of technologists said workloads have increased and projects have been abandoned due to a dearth of talent, according to a Pluralsight survey.

Some enterprises are turning to citizen developers to lighten the load. More than half of IT leaders said their company allows business employees to implement AI solutions on their own, according to Celigo.

Sponsored content

By Grammarly

Artificial intelligence is rapidly transforming our society by improving efficiency in our daily tasks and advancing new technological frontiers. However, AI’s quick adoption raises essential discussions about its impacts and safety.

To approach AI responsibly, consider some parallels between AI and the automobile.

What makes cars safe? It’s not just seatbelts, traffic laws, or crash tests—they all contribute to their safety. A constellation of manufacturing processes, features, testing, governance, education, and societal norms allows billions of people to safely use cars daily.

Cars and AI are similar. At Grammarly, we think about responsible AI as a series of checks and balances throughout the AI pipeline, from conception to development to deployment. No single factor or control makes AI responsible, but standards and practices adopted across an organization can establish a comprehensive approach to responsible AI.

What is responsible AI?

Responsible AI is creating and utilizing artificial intelligence in a manner that is mindful, morally sound, and aligned with human values. It’s about reining in AI in a way that prioritizes the intended impact while decreasing unwanted behavior or outcomes. This requires being fully aware of the capabilities of the AI technology at our disposal, identifying the potential pitfalls, selecting the right use cases, and instituting protections against risks.

Responsible AI takes different forms at different stages within the AI pipeline. Deploying AI responsibly may call for different principles than those needed to implement an existing AI model or to build AI-based technology from the ground up. Setting clear expectations and establishing guideposts for your AI to operate within at every stage of the AI pipeline is essential.

With that in mind, how can companies ensure they are on the right track when implementing responsible AI?

Crafting responsible AI frameworks

The journey toward responsible AI involves understanding the technology, considering its intended impact, and mitigating potential risks, such as unintended behaviors, hallucinations, or generating hazardous content. These steps ensure that AI behavior aligns with your company’s values.

Companies considering embedding AI into their businesses should consider how AI might affect their brand, users, or decision outcomes. Establishing a framework for responsible AI at your organization can help guide decisions around building or adopting AI.

The AI Risk Management Framework, published by the National Institute of Standards and Technology, is a valuable resource in this endeavor. This framework helps organizations recognize and manage the risks associated with generative AI and guides companies as they develop their principles for responsible AI.

Grammarly’s responsible AI standards

At Grammarly, we create and consume AI-based solutions every day. Responsible AI is a cornerstone of our product development and operational excellence. We have a dedicated Responsible AI team comprised of researchers, analytical linguists, machine learning engineers, and security experts who think critically about what we are trying to achieve for our company, our users, and our product.

As our company has evolved, we’ve developed our own responsible AI standards:

• Transparency: Users should be able to tell when they are interacting with AI. This includes identifying AI-generated content and providing details about AI training methods. This will help users understand how AI makes decisions. Knowing AI’s limitations and abilities enables users to make more informed decisions about its application.
• Fairness: AI fairness isn’t merely a buzzword at Grammarly; it’s a guiding principle. Through tools that evaluate AI outputs and rigorous sensitivity risk assessments, Grammarly proactively mitigates biases and offensive content. This commitment to guaranteeing respect, inclusivity, and fairness drives every user interaction.
• User Agency: True control rests in the hands of the user. Grammarly empowers its users with the ability to shape their interactions with AI. Users have the final say—whether they choose to accept writing suggestions or decide whether or not their content trains models. This ensures that AI amplifies, rather than overrides, their voice.
• Accountability: Recognizing the potential for misuse, Grammarly directly confronts the challenges of AI. Grammarly ensures accountability for its AI outputs through comprehensive testing for biases and by employing our Responsible AI team throughout the development process. Responsible AI is part of the company’s fabric, ensuring that AI is a tool for empowerment, not a source of error or harm.
• Privacy and Security: Grammarly’s approach to responsible AI is strongly committed to user privacy and security. We do not sell user data or allow third parties to access user data for advertising or training. Strict adherence to legal, regulatory, and internal standards supports this promise, ensuring that all AI development and training maintain the highest privacy and security measures.

Toward a more responsible future

Fostering a responsible environment for AI technology requires a collaborative effort from external stakeholders—from the technology industry to regulators to nation-states. To use AI responsibly, we must acknowledge and address inherent biases, strive for transparency in AI’s decision-making processes, and ensure that users have the necessary knowledge to make informed decisions about its use.

Embracing these principles is crucial for unlocking AI's full potential while mitigating its potential risks. This collective effort will pave the way for a future where AI technology is innovative, fair, and reliable.

Emerging software tools regularly challenge CIOs to weigh the desires of the business against the pitfalls of hasty adoption. Generative AI is just the latest technology that's almost too tempting for organizations to resist.

Companies have already seen a surge in employees using off-the-books generative AI tools, according to Productiv. 

The software portfolio management company crowned ChatGPT the king of shadow IT in April, based on an analysis of billions of app usage data points across millions of SaaS licenses dating back to 2021.

“Shadow IT exists for a reason,” Chris Bedi, chief digital information officer at ServiceNow, said last week at the MIT Sloan CIO Symposium. “Somebody needs to drive the business forward and they don't have the tech to do it.”

Mojgan Lefebvre, EVP and chief technology and operations officer at Travelers, agreed, speaking on the panel. “You’ve got to be aware of what’s going on before you can get in front of it,” she said.

Travelers keeps shadow IT in check with an open yet prudent procurement process, ensuring that “as demand arises there is a path for it to get to the right people,” Lefebvre said.

Telegraphing a clear tech procurement policy across the enterprise is a step in the right direction. But a robust vetting process for new technologies is a must. 

“We've set up a partnership between procurement, finance and our technology team,” Lefebvre said, adding that approval begins with a compelling business case and progresses through a vendor evaluation and cyber, compliance and financial reviews.

Cause for caution

CIOs are motivated to put guardrails in place. For all the impressive capabilities the technology exhibits, data-hungry large language models remain vulnerable to jailbreaks, according to a recent analysis by the U.K. AI Safety Institute.

Major tech providers, including Amazon, Google and Microsoft, agreed this week to a set of responsible AI guidelines, but are still in the early stages of defining a model safety risk threshold.

As the fog of uncertainty around the technology lifts, CIOs will have to grapple with adoption cost, workforce AI training and a chaotic vendor ecosphere rife with startups.

“I don't mean to diminish them, but the majority of companies getting funded right now are a UI on top of OpenAI,” Bedi said. “We’re just getting started on small language models and domain-specific language models that provide unique differentiation.”

A level of risk tolerance is part of the adoption process.

“With cloud, there’s so much maturity you can play with fewer vendors,” Lefebvre said. “With Gen AI, we don’t know who the winners are and right now I’m not ready [to] say we should just deal with one or two vendors.”

CIOs are carefully monitoring the potential advantages of deploying generative AI copilots within their technology stacks. Providers promise a smooth plug-and-play experience followed by quick productivity wins.

But the new technology is not guaranteed to fit in seamlessly. CIOs must rely on targeted tests to learn how tools can perform — and where potential issues lie.

“There’s been so much hype over the last 12 to 18 months around AI," said Kevin Miller, chief technology officer of IFS, a software solution provider. "It’s been pitched as this magic box where you just open it and it fixes everything. Realistically you have to have some goals for and some examples or use cases.”

A 30-day test rollout of an AI copilot can equip CIOs with critical information to support wider adoption plans.

To give a copilot the best chance of working, CIOs should do some basic data hygiene before the rollout, said J.P. Gownder, VP and principal analyst at Forrester. His primary experience is with Microsoft Copilot for 365, which he says does a good job training the permissioning of documents, “so if I don’t have permission to hear the answer, Copilot should not tell me the answer.”

However, if a company doesn’t have good data hygiene, documents aren’t permissioned in advance, which means AI could be getting into — and giving answers about — things not every employee should have. 

CIOs should also be thinking about which workers are best suited to try out an AI copilot. Employees should know how to use effective prompts, and be aware that copilots, which are largely based on LLMs, can hallucinate and generate nonsense. “If you are taking the word of AI as sacrosanct, you are not going to be using it effectively,” Gownder said.

CIOs should not simply take someone’s word that they know how to use AI efficiently, he added, since “most companies are underinvesting in training and skill development.” It’s not uncommon for businesses to deploy a copilot with one hour of training per user, when best practices call for 10 to 15 hours, according to Gownder. 

Getting feedback from early users

As with any other IT project, data can help CIOs tailor their approach as employees leverage copilots during the first 30 days.

Metrics help show how employees are using the tool, said Varun Singh, president and co-founder of Moveworks, makers of an enterprise copilot platform. Specific metrics will vary per company based on what they are hoping a copilot can help achieve. Companies may want to check if a copilot is helping to reduce the number of support tickets filed, for example.

CIOs and technology leaders can also ask users if the AI assistant’s response was helpful, but Singh noted that people are more likely to rate something as not helpful than helpful, much like someone is more likely to go online to say something about a negative experience than write a positive review. Singh suggests tracking to see if a user proceeded to file a ticket for a live agent, he said. “If they’re doing that, it means the copilot is not effective.”

In addition to training workers before launching an AI copilot, companies should offer continuous training opportunities, and support, throughout the first 30 days — and beyond. There can be synchronous training sessions, but also “virtual training on demand,” Gownder added. 

Companies can also set up weekly office hours, or weekly sessions where “people can bring their successes and failures,” he said. Dedicated Teams or Slack channels for discussion can also enable peer-to-peer learning. Employee testimonial videos are helpful too, so they can see “someone at their organization who is using copilot effectively,” and will more likely trust a video of one of their colleagues at their own company instead of a vendor representative.

A copilot can also be used to gather feedback, whether it’s from chat logs or interactions, and summarize next action steps based directly on that information, said Gownder. That feedback “shouldn’t just go into this broad data lake that no one’s going to look after.”

 As part of their training, testers should also be encouraged to play around with the copilot, even try to “break it somewhat,” Gownder said. “Ask it questions where it might not have the answer, or don’t provide context it’s looking at.” That can help determine if a copilot is hitting the mark, or not accessing the right information for an answer. 

Taking action on user feedback

While companies should certainly conduct surveys, they should also make sure to talk to testers about their experiences, Singh said. 

“Surveys often have pockets of data where we might be missing something important that doesn’t reveal itself in the big picture … with other forms of ratings,” he said. Talking to people can help CIOs and leadership teams “see how they’re feeling about the copilot and see if it’s helping or just making more noise. That’s also a way to tell “if they’re using it right or are they making bread recipes with it,” he joked.

This is often best done with intentional conversations outside of the IT setting. 

“Technologists are always on the bleeding edge, and they can feel [if] a technology is not good enough or smart enough,” he said. A regular user may appreciate small changes that don’t seem cool enough for a technologist, but end up helping a company improve and expand upon its copilot use.

Generative AI use in the workplace has nearly doubled in the last six months, but governance isn’t keeping up, according to a Microsoft and LinkedIn report published Wednesday.  

Three in 5 leaders are worried about their organization’s lack of plan and vision for AI implementation, according to the survey of 31,000 people across 31 countries. More than three-quarters of employees using AI admit to bringing their own AI tools to work, straining enterprise cybersecurity and data privacy standards.  

Employees are using AI tools not provided by their organization to assist with some of their most important tasks as the volume of work outpaces employees’ ability to keep up, the report found.

Reigning in shadow IT has been an ongoing crusade for CIOs, but the risks accompanying AI tools have upped the ante. 

Without guidance and enterprise-grade guardrails, employees can expose critical data or intellectual property, introduce additional legal liability or violate copyright restrictions. 

The stakes are high — and expected to escalate as ungoverned AI systems present more opportunities for threat actors to attack. 

CIOs and IT teams are responsible for finding enterprise-grade frameworks for employees who want AI assistance — if they haven’t already. The CIO is the C-suite member most frequently tasked with overseeing AI governance in companies with ongoing generative AI investments, an Info-Tech Research Group survey in October found. 

CIOs who set the business up for successful adoption will communicate with employees and provide resources related to best practices and usage guidelines, according to experts.

Generative AI is poised to disrupt software development, but CIOs can help ease the transition for workers as they infuse the technology into their daily tasks. 

Executives should explicitly define goals and metrics, provide training and keep the lines of communication open throughout the adoption process, Forrester recommends.  

Businesses that rush deployments face risks that extend beyond a worsening developer experience, including security issues, performance problems, litigation and disjointed teams.   

“The first thing is to make it clear that it is a support for developers and not a replacement,” Andrew Cornwall, senior analyst at Forrester, told CIO Dive. Some organizations are expected to buck the trend and try to trim their developer headcount in response to AI-driven productivity gains, but Cornwall said “I think they’re going to crash and burn.” 

In the eyes of CIOs and tech teams, software development is a prime target for generative AI implementation. 

“Developers for a couple of years now have started looking at things like cogeneration,” Cornwall said. “Organizations that don’t provide some sort of that mechanism have developers that are bringing their own.”

Ungoverned AI usage can put businesses at risk. Traditional shadow IT has long been a bane for enterprises, but the accessibility of popular models has caused organizations to lose ground in the ongoing battle. More than three-quarters of employees admit to using AI tools that were not approved by their employer, according to a Microsoft and LinkedIn report published in May.

“That presents a significant challenge, not least in terms of data protection and data integrity,” Thomas Humphreys, compliance expert and content manager at third-party risk management company Prevalent, told CIO Dive. “The data protection risks can be potentially quite severe.”

The shadow AI effect

CIOs can combat shadow AI by updating procurement and vetting processes, installing guardrails and promoting acceptable use policies. Organizations that get ahead of the curve will set employees up for success later down the line. 

AI-powered coding tools are expected to become fairly ubiquitous within enterprises in the next four years, according to Gartner research. The analyst firm predicts around three-quarters of software engineers will add AI coding assistants to their workflows by 2028, a considerable jump from the 1 in 10 enterprise developers leveraging the tools early last year. 

Generative AI will impact more than coding, though, Cornwall said. “Writing code is a chunk of developers' time, but it’s not a huge chunk.”

Developers can use generative AI to interpret code bases, assist in debugging, enhance onboarding processes, translate programming languages and expedite testing processes, according to Forrester’s research. 

As AI tools gain traction in developer communities, there are concerns around employee dependency and skill level. Junior back-end and full-stack developers are on average nearly twice as reliant on AI tools compared to more experienced engineers, a Docker report in April found. 

“We are going to see possibly a disruption in the pipeline of new developers, because if I am as productive as I can be with a couple of senior developers and a couple mid-level developers, why would I hire junior developers who can write crummy code when I can have a generic AI do it for me,” Cornwall said. 

On the other hand, Cornwall said, some organizations are pairing junior developers with AI for boilerplate tasks. Either way, there could be a disconnect between senior development and junior development on the horizon, according to Cornwall.

Generative AI use soared in the enterprise since last year, according to a study from consulting firm Altman Solon. Nearly two-thirds of businesses say they’ve adopted the technology, compared to just 11% in the first quarter of 2023.

Software development remains a key focus of adoption as more than three-quarters of developers use generative AI tools in the workplace, the survey of 400 senior-level executives found.  

Over half of enterprises are using more than one generative AI tool, while 1 in 10 businesses said teams deployed four or more. Text-based tools were most prominent, followed by images and audio. 

Executives are motivated to adopt generative AI in search of expedited processes, lower costs and improved workflows. But industry experts urge CIOs to move forward with a healthy dose of caution. 

“If the benefits of generative AI are becoming more visible, so are the risks,” Altman Solon said in the report.

The primary adoption challenges enterprises cited were concerns around data security, a lack of tool history, unclear value and policy limitations, according to the study. Despite concerns, leaders report a pressing sense of urgency to adopt generative AI as the clock ticks to capitalize on market differentiation and innovation.

“I think the biggest risk with regard to [AI technologies] would be if you decided not to adopt it,” Jeff Harmening, CEO at General Mills, said Wednesday during a conference in response to a question about how the technology would impact the industry. “We’ve adopted AI and machine learning, now generative AI.”

General Mills rolled out a generative AI tool called MillsChat to employees in its U.S, Canada, U.K. and India non-plant locations in February. Chief Digital and Technology Officer Jaime Montemayor said the tool’s introduction was the result of technology modernization efforts, a process household brands have prioritized to move beyond generative AI hype.

Across industries, leaders are working to get systems and employees ready for more AI usage. 

JPMorgan Chase is in the process of migrating its data to the cloud, a key part of the banking giant’s strategy to facilitate AI progress. 

The company has migrated around 70% of its data to the cloud, though CFO Jeremy Barnum said the platform shift “is not the same as having it be modernized and usable for AI and ML, so there’s work to do there still,” during an investor call last week.

As organizations continue to embark on AI initiatives, “navigating the balance between innovation and security will be paramount, profoundly shaping the future of work,” Altman Solon said in the report.