Dive Brief:
- Meta is positioning its newest large language model as a specialized coding tool free for research and commercial use, according to a blog post Thursday.
- The large language model, called Code Llama, was built on Meta’s Llama 2 model and uses text prompts to generate code. It is intended to help with code completion and debugging.
- Meta trained Code Llama on code-specific databases and it supports popular languages, including Python, C++ and Java. The model, however, is not designed to perform general natural language tasks and isn’t appropriate as a foundational model for other tasks, Meta said in the blog post.
Dive Insight:
Enterprises are looking for generative AI tools and LLMs to bring tangible value to the enterprise. In IT operations, the allure of more efficient coding draws CIOs, engineers and developers in.
Meta was met with developer interest when it launched its free Llama 2 model in July. The hype surrounded Meta’s open-source approach and the model’s capabilities, but Llama 2 wasn’t so great at coding.
In a test that measures an AI system’s accuracy on human-written programming questions, GPT-3.5 scored a 48, GPT-4 scored a 67 and Llama 2 scored a 30, according to Meta’s research. The programming questions assess language comprehension, reasoning, algorithms and simple mathematics.
With Code Llama, the AI system scored a 49, just above GPT-3.5’s score, Meta’s research found.
Code Llama is available in three models, including the foundational code model, a model specialized for Python and a model fine-tuned to understand natural language instructions. Code Llama - Instruct scored a 62 on the HumanEval, while Code Llama - Python scored a 54.
The models are also available in three sizes, running from 7 billion parameters to 34 billion parameters each trained with 500 billion tokens of code and code-related data.
