Vendors of all shapes and sizes are rushing to implement generative AI, a race begun by the release of OpenAI's ChatGPT and its newest large language model GPT-4. 

The newest tech vendor strategy is akin to a race for a participation trophy — if a vendor can give a tool or application generative AI capabilities, it will, from CRMs and ERPs to email, documents and presentations. 

Microsoft unveiled a “new era of Teams”, featuring AI-powered filters for video conferencing, an AI-powered selling experience called Microsoft Viva Sales and AI-generated meeting recaps.

The Teams changes followed an earlier announcement teasing the integration of Copilot technology into Microsoft 365 apps, including Word, Excel, PowerPoint, Outlook and Teams. Google has followed suit, embedding new generative AI capabilities in its workplace tool suite, cloud services and search engine. 

These high-profile vendors aren’t alone. Zoom is incorporating OpenAI’s technology to generate Zoom Team Chat and email responses based on input from existing chat, email, phone and meeting threads, the company announced. Cisco announced new AI capabilities for its Webex Contact Center and cloud platform, Webex Connect.

From end-to-end providers to pure-play specialists, technology companies vendors are racing to include AI in workplace tools, injecting intelligence into daily communication. In order for enterprises to get any value from these additions, business leaders have to put in the work. 

AI-based tools have the potential to boost employee efficiency, productivity and creativity, but tech leaders need to communicate how employees can leverage tools so the technology benefits aren’t lost in the hype. 

Prepping for updates

One area of preparation that IT teams need to think about is storage capacity, according to Bill Wong, principal research director at Info-Tech Research Group.

“Generative AI tools tend to be more memory and compute-intensive, [and] it is quite possible that with an environment that is more AI-enabled, employees may find that they generate more documents, presentations and analysis, which would increase the need for more storage,” Wong said in an email. “To prepare for this environment, the best practice is to build a data platform that is optimized for AI and advanced analytics.”

Cost is a huge barrier to entry for most enterprises when it comes to developing and implementing large language models internally. While accessing generative AI capabilities through vendors can be used as a workaround, tech leaders need to be mindful of smaller or niche vendors passing the costs onto customers, according to Wong.

Tech leaders are also worried about data privacy, a concern recently amplified after a bug in an open source library allowed some users to see other users' chat histories in ChatGPT.

Cyberhaven Labs research found there were more than 5,000 attempts in a single day to paste corporate data into ChatGPT per 100,000 employees.

“Since the large language models are in the cloud, all input is used as training data and available for any future query,” Wong said. “For this reason, several companies have totally restricted the use of technologies such as ChatGPT.”

Microsoft is placing parameters around its GPT-4 security offering to ease concerns, and does not plan to use customer data to enrich or train AI models in its Security Copilot, a tool announced recently. 

ChatGPT bans at large enterprises and organizations have made headlines as companies try to limit risk and protect sensitive data. Nearly half of HR leaders said they are in the process of formulating guidance on employee’s use of OpenAI’s ChatGPT, according to Gartner data.  

“Before training employees on how to maximize the use of generative AI, it would be useful to communicate the company’s policy on the use of generative AI technology, its responsible AI guiding principles and how it aligns with the organization’s mission and strategy,” Wong said.

Maximizing value

Once policies are established, IT teams need to have the expertise and experience to assist workers throughout the enterprise as more generative AI capabilities are embedded into tools. 

To maximize value, Adam Preset, VP analyst for digital workplace at Gartner, suggests companies should lean into the changing tech landscape.

“Get ahead of this wave,” Preset said in an email. “Understanding generative AI within common workplace tools should be a priority.”  

Businesses can get a head start by getting involved with vendor beta testing previews, Preset said. IT teams that can use generative AI in daily work and develop skills will have a greater chance of adequately supporting other workers who will need help and best practices.

A large part of making generative AI tools useful is becoming good at creating prompts. If IT teams already have ample experience with this, it will be easier to pass that knowledge on during company training or one-on-one support sessions.

“The skill [that] workers will need to develop is prompting the generative AI so you can direct it to create a business proposal, an informal email, content for your website or something else fit to their goals,” Preset said. “It will be less about the beginning of filling in a blank page and more about clarifying what you would like as an outcome.”

“Then it’s about checking and double-checking that generative AI is giving you something trustworthy, true and useful,” Preset said.

How IT leaders make use of the array of AI tools at their disposal will determine success in customer experience, staff retention and even revenue. 

As the technology advances, AI will increasingly become "an artificer of different key elements of your work and the work of the people whom you support," said Whit Andrews, distinguished VP analyst at Gartner, speaking at the Gartner IT Symposium/Xpo, in Orlando, Florida.

And leaders are investing in this long-term trend. The forecasted spending on AI software is set to double by 2025, reaching $64 billion and growing faster than the general software category, according to Forrester projections.

With AI's full potential still under development, here are three specific actions to start taking — today:

Build a right-sized portfolio of projects

Businesses have long targeted repeatable, time-consuming tasks with AI projects, a strategy to address time and labor constraints. But modern businesses should have a right-sized scope when it comes to AI experimentation — not too much, not too little.

"It breaks my heart when I hear from an organization that they're doing this one big thing," said Andrews. Instead of a single big AI project, technology leaders should ensure their organizations can develop a diversified portfolio of ongoing projects. 

On the flip side, too many initiatives can also lead to wasted efforts. Organizations doing 20% more AI projects than their counterparts were more likely to report failure than organizations doing fewer projects.

Focus AI governance toward scale

AI has continually spread throughout organizations, more and more frequently as a component of other technology rather than the core. This expansion can be difficult for leaders to effectively manage its use.

"Governance is probably the most important investment we can make today," said Andrews.

Though 1 in 3 CIOs are currently using AI, just 1 in 10 are doing so strategically, in multiple business units for purposes. As executives shape governance, they should avoid constraining their organization's use of the technology — instead using AI to scale. 

It's up to leaders to ask how and where AI can spark creativity, whether that means designing networks or setting up new pricing strategies, according to Andrews.

 "What is it that you can turn to artificial intelligence to start to meaningfully help you jumpstart or kickstart?" he said.

Solve for a new talent landscape

Talent is becoming less of a need toward AI implementation, most leaders said in a recent Gartner survey. The trend was a reversal of the long-held narrative around AI talent scarcity. 

This shift, Andrews said, signals a change in how AI is generated, opening up access to a greater number of people within the enterprise. 

The new framework for connecting talent to AI means potentially improving employee satisfaction.

"Help people invest their interest in AI work to discover what they're passionate about, in order to discover what your organization is passionate about," Andrews said.

Generative AI and the modern dog have a lot in common.

Humans began domesticating wolves by using them as hunting companions, an innovation that allowed mankind to hunt at scale, according to Whit Andrews, distinguished VP analyst at Gartner. From there, dogs were used as surveillance, invader protection and to help corral and shepherd other animals.

In other words, dogs — much like AI — have specialized use cases based on the task at hand. For a shepherd, having dogs enabled better flock protection and easier flock movement from point A to point B. Shepherds could manage bigger flocks, too.

“You have to ask yourself, though, does that mean that being a shepherd is something fundamentally different? Yes, it does,” Andrews said. “[If] I have multiple assistants in herding, my work as a shepherd is essentially strategic.”

A common fear associated with AI is that it will take jobs away from people. In reality, experts say generative AI will level up work for most jobs, from repetitive tasks to more strategic ones. Some job loss or disruption is likely, but AI could create new roles, too, a process that has already started. 

Generative AI could substitute up to one-quarter of current work in the U.S. and Europe, Goldman Sachs economics research published last week shows. Around two-thirds of current jobs are exposed to some degree of AI and automation in the U.S. and Europe, according to the report.

Job disruption due to automation

Most industries could see a level of automation, from education to technology. 

For finance-related jobs, such as accountants, financial advisors and traders, automation and generative AI could generate financial reports and substitute customer service. Duties like personalized financial planning and personalized legal and tax advice would be difficult to automate, according to Bill Wong, principal research director at Info-Tech Research Group. 

Generative AI is expected to raise annual U.S. labor productivity growth by just under 1.5 percentage points over the 10 year period following widespread adoption, according to Goldman Sachs research. This boost could result in an increase of 7% — or almost $7 trillion — in annual global GDP over a 10 year period, the report said.

“The boost to labor productivity growth could be much smaller or larger depending on the difficulty level of tasks AI will be able to perform and how many jobs are ultimately automated,” the report said.

Some programmers have experience with AI-assisted coding tools or generators already. Within the first month of GitHub Copilot’s public release, the platform’s AI-powered programming tool added 400,000 subscribers. The autocompletion tool uses generative AI technology made with OpenAI’s Codex model. In February, GitHub Copilot was responsible for generating an average of 46% of code when using the tool, up from 27% of code in June. 

Programmers and data analysts could automate code creation and generation as well as unit testing. However, the way technologists understand user requirements and architect complex solutions would be difficult to automate, according to Wong. 

“Programmers do much more than just code,” Wong said. “[They] spend time on designing, brainstorming, collaborating and assessing business requirements to improve the quality of the code.”

But that doesn’t mean generative AI can’t help support programmers. Leading organizations are using generative AI to augment programmers, not replace them, Wong said. 

“When ATMs came out, they thought tellers would disappear, but tellers are doing much more sophisticated things now,” Wong said. “You may need less tellers, but each one does more and that’s how we believe we will see programmers.”

Creation of new jobs

While generative AI is likely to disrupt jobs, there will also be new roles created. And this isn’t a new phenomenon. 

Three in 5 workers today are employed in occupations that did not exist in 1940, “implying that over 85% of employment growth over the last 80 years is explained by the technology-driven creation of new positions,” research from a Goldman Sachs report shows.

One of those new roles created from the advancement of generative AI and large language models is a prompt engineer. 

Prompt engineers develop and optimize prompts to efficiently use language models for a variety of applications and research topics, according to the Prompt Engineering Guide by DAIR.AI posted on GitHub.

Job postings for prompt engineers are popping up across sectors. Anthropic, a Google partner and selected participant in Salesforce’s generative AI fund, posted a job description for a prompt engineer and librarian. The candidate would build a library of high-quality prompts or prompt chains with an easy guide to help users search for the one they need.

Boston Children’s Hospital posted a job listing for an AI prompt engineer, which required candidates to have five years of engineering, developing or coding experience in Python or similar languages and at least two years of experience with AI and natural language processing. One of the goals is to design and develop AI prompts using LLMs for healthcare research studies and clinical practice. 

“The combination of significant labor cost savings, new job creation and a productivity boost for non-displaced workers raises the possibility of a labor productivity boom like those that followed the emergence of earlier general-purpose technologies like the electric motor and personal computer,” the Goldman Sachs report said.

But job creation is unlikely to come at a one-for-one transition that every worker is comfortable with, Andrews said. 

“It’s very important that we respect the profound impact of this displacement,” Andrews said. “I’m sure that any institution would not disagree with the fact that when we close a job, we don’t necessarily provide an easy path for the person to transition to the next job.”

Cloud migration has pushed data mining beyond the current analytics capabilities of most companies, according to a Coleman Parkes global survey of 1,300 CIOs and senior IT practitioners commissioned by software company Dynatrace.

Nearly three-quarters said they were struggling with data management. Almost all executives struggle to extend observability and security across cloud-native and multicloud environments.

Platform proliferation in tech stacks, and difficulty attracting and retaining IT operations and DevOps talent are hindering efforts to remediate data management issues. Three in five CIOs said automation would help overburdened IT teams deal with complexity of cloud-enabled tech stacks.

More data facilitates better business decisions, improves operational efficiency and drives growth. But too much of a good thing creates problems.

Cloud implementation allows businesses to rapidly onboard new and better technologies and intake a lot more data — so much data that analytics teams can be overwhelmed. When companies can't process, map and mine incoming data while also monitoring for security anomalies, value is lost.

Most companies discard a majority of data collected. Up to 90% of data mined never makes it to the analytics stage, and most companies’ tech stacks offer end-to-end observability of just 9% of data, according to the survey.

“The systems that exist today are just too slow and expensive,” said Mike Maciag, CMO at Dynatrace. “So, you have to make a hypothesis ahead of time about what kind of data might even be usable because you’re going to throw away 90% of it, which is insane.”

With IT staffing and retention woes impeding modernization efforts, companies are turning to automation and AI deployments to take up the slack. A majority of executives said they were bullish on automation technologies in an August report from Gartner, and seven in 10 reported relative ease sourcing AI talent.

In contrast, a scarcity of data science talent is hindering analytics projects. To exacerbate the gap, demand for data professionals showed significant year-over-year growth in the first half of 2022, according to a mid-year analysis of three million job postings by Dice. Data analysts, data engineers and data scientists all placed in the top ten of most in-demand tech positions.

Even without a talent shortage, data operations would be stressed.

Cloud is an adrenaline boost for enterprise IT — a blessing and a curse for data teams. As the volume, velocity and variety of data has increased, so have the number of tools needed to monitor and track that data.

With companies pushing for real-time insights across all business functions, simply managing data, preventing dreaded siloing and providing adequate access is a challenge. 

“Observability logging and analytics at scale goes from something you might be able to do on your own to something you wouldn’t be able to do manually even if you could hire the people you need,” Maciag said.

AI tools are the top three fastest growing software products across software marketplace and review website G2 this year, the company said, comparing more than 2,000 categories. Jasper, an AI writing assistant, topped the list, according to a spokesperson.

Within AI, chatbot software is the fastest growing subcategory, according to G2 data. The other 14 subcategories on the website include virtual assistants, data science and machine learning.  

There was a 261% increase for chatbot software traffic from February 2022 to February 2023 on the website, according to G2 data. 

AI tools, including chatbots, have quickly become a hot topic and shifted enterprise focus since the launch of ChatGPT in late November. 

The chatbots category saw a 202% increase in search traffic on G2 since November 2022, according to data shared with CIO Dive.

The AI race initiated by Microsoft and Google has created a sense of enterprise urgency to adopt and implement conversational generative AI models.

In late March, OpenAI teased plugin capabilities for ChatGPT. Companies such as Expedia, Instacart, Klarna, Shopify, Slack and OpenTable were among the first to create plugins powered by ChatGPT.

Broader access to data across the enterprise correlates to higher ROI on analytics technologies, according to an IDC study which surveyed 1,501 IT leaders on behalf of data analytics software company Alteryx.

Data governance and optimal deployment of analytics capabilities remain a challenge for many companies. Nine of 10 respondents said their company has not fully leveraged its data technologies, reporting that less than half of business decisions are analytics-based.

Companies with improved financial metrics utilized workforce analytics skills more effectively, provided workers with easy access to data and had at least a quarter of knowledge workers active in using analytics technologies.

Companies invested in data technologies have not fully optimized that spend, according to the survey.

Nine in 10 respondents said their company has not made full use of existing workforce analytics skills and only one in five reported adequate spending on upskilling. 

“They're buying technology, but they’re not really teaching people how to use it or how to optimize their use with it,” said Jason Klein, product marketing director at Alteryx.

Data can drive revenue, but only if companies commit to workforce data literacy training, analytics upskilling and governance policies that spread data wealth. Companies that limit data access to a “few small pockets and don’t democratize data technologies face ROI challenges,” Klein said.

There are a number of roadblocks to data democratization, including talent shortages, data integration problems, security concerns and misalignment between the business and technology functions within an organization, according to Klein. The latter can lead to a “shadow IT dynamic,” Klein said.

As talent is onboarded and workforces are upskilled, companies should look to governance policies that protect information while also providing data access to as much of the workforce as possible.

“It doesn't necessarily mean that everyone is crunching the numbers,” Klein said. “It means that everyone is contributing to and getting value from the analytics process.”

More than half of professionals say insufficient data science talent is a top barrier to the technology's adoption, according to Anaconda's State of Data Science 2022 report.

Nine in 10 professionals say their organizations are concerned about the impact of the current talent shortage. Anaconda, a data science and machine learning software company, surveyed nearly 3,500 individuals globally, a mix of students, academics and other professionals. 

Companies are addressing talent shortages by training existing employees, enabling remote work and partnering with academic institutions for talent development, according to the report. 

Data science roles remain particularly challenging to fill thanks, in part, to appeal across industries. 

Finding data scientists is difficult because the position is more a form of literacy than a specific skill, according to Peter Wang, co-founder and CEO of Anaconda.

"There are some people we're going to specialize in being data scientists, but if you're a business, you're going to want to get ready for everyone in the business being able to do some better levels of data reasoning," Wang said.

Six in 10 hiring managers said data science and analytics roles will be the hardest to fill in the next six months, according to a separate study from Upwork.

Employers are responding to the need with dollars, salary trends show. Glassdoor placed the role of data scientist in the No. 3 slot on its Best Jobs in America list for 2022, commanding a median base salary of $120,000. Software engineers, ranked eighth, earn a median salary of $116,000.

An inability to extend data science literacy throughout the organization can lead to higher levels of attrition, Wang said. 

"There's a center of gravity that pulls towards this future [data] capability, and if the rest of the organization doesn't understand the value of that, they're just going to put up blockers," Wang said. "That slows the data science team down and then guess what happens? Those data scientists get frustrated and they leave for some faster company."

When Ally Financial wanted to enhance the way it used data to its advantage, the key change was one of perspective. To borrow a term from supply chain lore, data strategy had to be the first mile — not the last.

"Data strategy is the beginning of your data journey, then you go towards data execution," said Sathish Muthukrishnan, Ally Financial's chief information, data and digital officer, speaking at the Forbes CIO Next Summit. "What I thought was the last mile problem is actually the first mile problem."

In other words, the critical challenge to address is enabling the organization to use data as a stepping stone to make decisions, improve operations or launch products.

Most organizations struggle with this, as one Coleman Parkes survey found: three-quarters of organizations grapple with data management. Data migration landed too much data at the hands of organizations, surpassing their ability to reap insights from it.

Ally Financial launched three groups within the company to change its outlook on data:

• Product group: The company stood up a product group unaffiliated with a specific function or business to focus on treating data as an asset and launching data products. "Their goal is to deliver use cases that will add value to the bottom line of the company," said Muthukrishnan.
• Data steering group: Ally Financial then set up a group to consume data, helping shape how it is presented and what tools are relevant. "They are actually defining the future of how they will use it, and in a way refining my strategy," he said.
• Data enablement group: Muthukrishnan hired a business data analyst to run this group, whose purpose is to ensure that after data tools are launched internally they track usage.

At your average organization, talent shortages hamper ambitious analytics goals. Over half of data pros say insufficient data workers is a top barrier to the technology’s adoption, Anaconda’s State of Data Science report found.

But leadership, and support from the top, can help. 

"I asked my CEO to send a note around how much data has been progressing in certain groups," said Muthukrishnan. "And he used that in his message to the organization."

When the tone is set by leadership, businesses can use data to add value to the company, he said.

"Then it becomes everybody's asset," he said.