GitHub Copilot Launch: Codex, Editor Suggestions, and Preview

Suggestions appeared where developers were already working

GitHub launched Copilot as a limited technical preview on June 29, 2021. It used the code a developer was working on as context and suggested whole lines or functions. GitHub highlighted Python, JavaScript, TypeScript, Ruby and Go among the languages the preview handled especially well. [1]

Atlas interpretation: For example, a developer could write a comment asking for duplicate items to be removed while preserving order. The comment and nearby code give the model a pattern to complete. The developer still needs to check that the suggested implementation preserves order and handles the inputs their application actually uses. [1]

Atlas interpretation: The immediate change was in the workflow: a possible implementation arrived inside the editor, ready to accept, modify or reject. The developer could compare it with the surrounding code without first searching for an example and adapting it by hand. [1]

A language model specialized for code

Copilot was powered by Codex, developed with OpenAI. Its later July 7 paper described fine-tuning a GPT language model on public GitHub code. The connection to GPT-3 is the prompt-to-completion approach: use preceding text to predict what comes next. Training on code made that approach more useful for programming. [1][3]

The paper's HumanEval benchmark tested standalone Python functions against unit tests. A 12-billion-parameter Codex model solved 28.8% of problems with one generated sample. The paper also explicitly distinguished the production model powering Copilot from the evaluated research system. Those numbers were not a measured success rate for suggestions in a developer's project. [3]

Atlas interpretation: Passing tests for a small function says something useful about functional correctness. It says less about whether a suggestion fits an application's undocumented rules, handles its real inputs or avoids security mistakes. The paper warned that plausible-looking code could be wrong or insecure. Review and testing remain part of completing the task. [3]

Copilot's preview preceded the Codex API

OpenAI announced an improved Codex API in private beta on August 10. That opened another way to build interfaces around natural-language-to-code generation. Copilot had already packaged the capability around a specific activity: writing code in an editor. [4]

Atlas interpretation: An underlying model and a product built around it are different things. The June preview supplied suggestions; the later API let developers explore other applications. Neither milestone, by itself, established that a system could take responsibility for a repository change from requirements through deployment. [1][4]