CIOs have been flooded with generative AI use cases, tools and foundational models over the last 11 months. Amid the hype, executives and vendors alike were quick to spot value in use cases geared at software development.
Vendors have released updated versions of models that can act as coding assistants and tools that can aid throughout the development process. Reports suggest the technology is good at it.
Software development jobs are the most likely job to become augmented by generative AI, according to an Indeed study in September. The research found that the technology was “good” or “excellent” at performing 95% of skills in software development job postings.
Developers do more than just code, but having a tool that can accelerate the process could let IT workers spend more time connecting with end users and identifying problems and solutions. Left unchecked, a bad tool could lead to faulty or insecure code, exposing the business to a host of risks.
Executives are tracking improvements in tools and taking a cautious approach to implementation. Experimentation is already helping identify where the technology's value lies within the software development life cycle.
Investment management company Vanguard is experimenting “rapidly and safely” with generative AI tools, incorporating human oversight and expertise to enable productivity boosts for employees, according to CIO and managing director Nitin Tandon.
“Initial results from our code generation pilot are encouraging, with our developers reporting that tasks are easier to complete, helping them stay in their flow longer,” Tandon said in an email. “We are looking into training LLMs on our code base to unlock further productivity boosts. With our large developer community at Vanguard, even a moderate increase in productivity can lead to meaningful value for our clients.”
Individual enterprises will need to assess their risk appetite and provide safe spaces for workers to experiment. Testing outside tools could help find the solutions that work best, but employees will need guidance on acceptable use, what data sets to use and what information to input.
Beyond the smoke and mirrors
There are risks to adopting generative AI in the software development lifecycle, especially without proper security policies. More than two-thirds of developers admit to pushing code to production without testing, and more than 3 in 5 admitted to using untested ChatGPT-generated code, according to a survey of 500 U.S. full-time developers by Sauce Labs.
Creating a policy that empowers employees to use tools in a safe and effective way, while underlining the importance of guardrails and security practices is a challenge.
“It’s the hardest part of the job,” Sharon Mandell, CIO at Juniper Networks, told CIO Dive. “Because the end users, they’re just trying to accomplish something.”
Tech leaders can move forward through experimentation to find valuable use cases within the software development realm.
Among software developers who are using AI and machine learning this year, nearly two-thirds said they use AI/ML to check code, half said they use bots for testing and more than one-third use AI for code review, according to a GitLab survey of 5,000 individual contributors and leaders in IT operations and security.
But generative AI adoption in software development is not widespread yet. A recent report from Google Cloud found the technology’s impact on software delivery has been relatively minimal thus far.
“I think it’s easy to brush off we’re in the early days … but the rate at which improvements are happening, and the rate at which change is happening, is faster than we’ve seen in technology in a really long time,” CircleCI CTO Rob Zuber said. “I would be aware of where you are in the lifecycle of all this and know that it’s going to take some time to get up to speed.”
Generative AI implementation also requires adequate budgets, infrastructure and employee training to bring value to the enterprise. CIOs need to set expectations in the process as the enthusiasm to adopt generative AI in workflows may not be coming from senior tech leaders, but from other executives and CEOs as the technology is expected to forgo traditional business lines.
CEOs are prioritizing generative AI investments, but they're not expecting a quick return on their investment. The majority of CEOs are planning for ROI on the technology to take three to five years, according to a KPMG survey. Fewer — less than one-third — expect a faster ROI of one to three years, according to the survey.
CircleCI has internally “dabbled” with different tools that had varying levels of preference among developers. “Very senior engineers are saying 'this thing is slower than I am' or 'it's designed for the wrong thing,'” Zuber said. “We’re in the early stages of exploration as an industry, not just at CircleCI.”
Vanguard also found junior engineers experienced the biggest productivity boosts, Tandon told CIO Dive.
Enterprise appetite for generative AI in software development exists, even as adoption lags behind. Businesses such as Papa Johns, EY and General Motors are exploring how generative AI might help team members working in software development.
