Code Llama, CodeWhisperer, and AlphaCode: What Software Developers are Using to Write, Check, and Update Code?

Code Llama, Meta said, can create strings of code from prompts or complete and debug code when pointed to a specific code string. The Code Llama model was one of several versions released by Meta, which can understand instructions in natural language. According to Meta, each specific version of Code Llama is not interchangeable, and the company does not recommend the base Code Llama or Code Llama-Python for natural language instructions.

Meta said in a post that programmers are already using LLMs to assist in a variety of tasks. “The goal is to make developer workflows more efficient so they can focus on the most human-centric aspects of their jobs.”

Code generators help developers with their work. Copilot is powered by OpenAI and is meant to be used to quickly write and check code. GitHub Copilot can also rewrite old code to update it. CodeWhisperer is a service which writes, checks, and updates code. While AlphaCode does have a code-writing tool, it isn’t out yet.

Microsoft is being sued by a group of people who claim that the company violated copyright law by using the tool to make unlicensed copies of licensed code.

The Copilot project: a toolbox for automating open-source programming using Meta’s regular language model and open AI’s GPT

The release of the weights to the community is exciting according to a PhD student in Artificial Intelligence, who has studied the parameters of the neural network at the core of the model.

Kumar says that the release of Meta’s regular language model led to the creation of communities dedicated to discussing how it behaves and how it can be modified. It gives us more flexibility in how we use the system compared to what we can get from open-sourced models.

Kumar said that developers are likely to build new applications with Code Llama. Kumar has studied how artificial intelligence can lead to less secure code and he believes that a programming assistant that performs additional safety checks may be able to create a program that recommends more secure code. Kumar thinks the release could prompt the creation of specialized assistants for coding. “You can build all sorts of tooling on top of the model,” he says.

Copilot is a plug-in for coding programs that auto- completes sections of code if the first line or a comment is typed by the user. Copilot uses a version of Open AI’s GPT, the large language model behind ChatGPT. A model is trained even further using code that can be found in GitHub, as well as contractors who are paid to make their own annotations.

Meta is likely to have limited the training data to avoid such problems, as it faces litigation for using some open source code in its training data. Copilot costs $10 per month for individuals and $19 per month, per user, for businesses.

Llama 2: A Free, Open Source, Accessible Artificial Intelligence Platform for Learning and Deploying Scalable Models

ChatGPT made it possible for anyone to play with powerful artificial intelligence, but the inner workings of the world-famous chatbot remain a closely guarded secret.

The open source approach, which has democratized access to software, provided transparency, and improved security, might have a similar impact on the field of Artificial Intelligence.

Llama 2 is free to download, modify, and deploy, but it is not covered by a conventional open source license. Meta’s license prohibits using Llama 2 to train other language models, and it requires a special license if a developer deploys it in an app or service with more than 700 million daily users.

This level of control allows Llama 2 to offer significant technical and strategic benefits to Meta, such as allowing it to benefit from useful tweaks made by outside developers when it uses the model in its own apps.

Models that are released under normal open source licenses, like GPT Neo from the nonprofit EleutherAI, are more fully open, the researchers say. It’s difficult for projects of that nature to be treated equally.

Data is kept secret when training advanced models. Second, software frameworks required to build such models are often controlled by large corporations. The two most popular ones, TensorFlow and Pytorch, are maintained by Google and Meta, respectively. Third, computer power required to train a large model is also beyond the reach of any normal developer or company, typically requiring tens or hundreds of millions of dollars for a single training run. And finally, the human labor required to finesse and improve these models is also a resource that is mostly only available to big companies with deep pockets.