Assessing GitHub Copilot
Understand LLM Security Risks and Safe Practices
Technology continues to become more innovative and efficient for development teams, and tools like GitHub Copilot are gaining attention for their ability to enhance productivity through the power of Artificial Intelligence (AI). GitHub Copilot, developed by GitHub in collaboration with OpenAI, offers an AI-powered code completion feature that suggests code snippets and entire functions in real-time as developers write code. This innovative tool leverages OpenAi’s GPT-4 model, primarily trained on public code repositories from platforms like GitHub, GitLab, and other publicly available sources.
While the potential for increased efficiency and creativity is significant, it is still important to review GitHub Copilot from a security perspective. Ensuring the safe implementation of these tools is essential for maintaining the integrity and security of software development projects. In this article, I will cover what GitHub Copilot offers, the security concerns that are involved with any Large Language Model (LLM), and my stance on if GitHub Copilot is safe and secure to use.
Why GitHub Copilot Stands Out
GitHub Copilot’s popularity among developers can be attributed to its advanced features that significantly improve coding efficiency and problem-solving capabilities. The following features of GitHub Copilot are just a glimpse of why it is growing in popularity today:
Code Completions
GitHub Copilot’s core functionality lies in suggesting relevant code snippets based on the context of what’s being written in the developer’s editor. This not only speeds up the coding process, but also introduces developers to new methods and potential best practices.
GitHub Copilot Chat
One of Copilot’s newer features is the ability to interact with it directly within your Integrated Development Environment (IDE) through natural language prompts, like ChatGPT. This means you can ask for specific code generation, generate documentation, create unit tests, or inquire about the functionality of existing code, just as if you were discussing with another developer.
CLI Assistance
GitHub Copilot extends its capabilities beyond a developer’s IDE with its Command Line Interface (CLI) assistance, providing code suggestions and completions even when developers are working directly in the command line.
These features collectively make GitHub Copilot not just a tool for auto-completing code, but a comprehensive assistant that enhances the coding experience across various platforms and scenarios for a developer.
Security and Privacy Concerns
As an application security expert, it is essential to ensure development teams can safely leverage the benefits of AI without compromising the security or integrity of their projects. Some of the main security concerns regarding LLMs like GitHub Copilot include:
Sensitive Information Disclosures
When asking natural language questions or providing input, you don’t want to leak sensitive information in any way. This includes things like company references, secrets, proprietary code, etc.
A great example of this: The Samsung ChatGPT Leak, after employees leaked trade secrets to the public platform of OpenAI’s ChatGPT
Overreliance
This is exactly what it sounds like – “over relying” on Large Language Models. This can lead to accepting insecure code patterns or misinformation they produce, which can cause a multitude of security issues. LLM-generated code can introduce vulnerabilities, so it is important to have strict reviews and validation mechanisms in place to prevent this.
Supply Chain Vulnerabilities and Model Theft
LLMs can inadvertently become a part of the software supply chain, which further requires the need to carefully consider their potential vulnerabilities. This includes both the risk of AI-assisted tools being manipulated to insert malicious code into projects and the theft of AI models themselves. The former can happen when developers trust the code generated by AI without proper scrutiny (like overreliance), while the latter refers to the unauthorized copying or reverse-engineering of AI models. To safeguard against these risks, it is crucial to monitor the dependencies introduced by LLMs and to secure intellectual property rights for AI models.
Introducing: OWASP Top 10 for LLM Applications
This overall security review on GitHub Copilot was based on the OWASP Top 10 Framework for LLM Applications. This framework was primarily designed for developers, data scientists, and security experts and provides practical security guidelines to help make risk-based decisions on LLM technology that is available today. The following is an outline for this framework, including my professional opinion for how GitHub Copilot ranks against each vulnerability.
LLM01: Prompt Injection
Prompt Injections allow attackers to manipulate an LLM using crafted inputs, executing their intentions. This can lead to data leaks, social engineering, and influencing decisions. In advanced scenarios, the LLM might mimic harmful personas, interact with user plugins, or aid the attacker by bypassing safeguards, all while keeping the user unaware of the attack.
♦ GitHub Copilot shows sources for its external content and allows users to identify untrusted content. It also adheres to Microsoft Security Standards and AI Use.
LLM02: Insecure Output Handling
Insecure Output Handling involves poor validation of LLM outputs before they’re sent to other systems. It differs from Overreliance, which is about too much trust in LLM outputs. Exploiting this can lead to web and backend attacks, especially if the LLM has undue privileges.
♦ GitHub Copilot Chat, available to all GitHub Copilot for Business subscribers, offers an AI tool for developers, supporting tasks like code explanations in Visual Studio and VSCode. Contextual prompts and suggestions aren’t stored for business users. Organizations can also block suggestions matching public code in the organization policy settings.
Data handling details depend on whether you are using GitHub Copilot Chat as a GitHub Copilot for Business or for Individuals user. If opted-in, an individual user’s interactions with GitHub Copilot’s Chat, including contextual prompts and suggestions, are stored, potentially posing a risk to their code’s privacy.
LLM03: Training Data Poisoning
Machine learning relies on diverse training data. LLMs use this data to produce outputs. Training data poisoning harms this data, introducing vulnerabilities and biases, which can lead to risks like brand damage. External data sources increase the risk of data poisoning due to potential uncontrollable biases or false information.
♦ GitHub Copilot is built upon a very large dataset from publicly available code, which inherently has a wide variety of sources, reducing the risk of concentrated data poisoning. GitHub’s development and review process, combined with OpenAI’s expertise in training LLMs, ensure a robust defense against potential data poisoning. The model’s continuous updates and refinements further minimize the chances of retaining biases or vulnerabilities from tampered data sources.
LLM04: Model Denial of Service
Attackers can overload an LLM, degrading its service quality and incurring costs. There’s also concern about attackers manipulating an LLM’s context window, which determines the text length the model can handle. This vulnerability is growing due to increased LLM usage, their resource intensity, unpredictable outputs, and developer unawareness.
♦ GitHub Copilot is designed with robust infrastructure and optimization in mind, ensuring it can handle many requests without being easily overwhelmed. Built-in rate limits and safeguards prevent excessive resource consumption from a single source.
LLM05: Supply Chain Vulnerabilities
LLMs’ supply chain can be compromised, affecting the data, models, and platforms, leading to biases, breaches, or system failures. While traditional vulnerabilities target software, in LLMs, pre-trained models and data from third parties can be tampered with.
♦ GitHub Copilot is built on an LLM developed by OpenAI. Given its foundation on an LLM, it is theoretically susceptible to the general risks associated above.
LLM06: Sensitive Information Disclosure
LLM applications can unintentionally expose sensitive data. Users should interact cautiously, and LLMs should sanitize inputs and have clear policies. Despite safety measures, LLMs’ unpredictability can still present risks.
♦ GitHub Copilot for Business have subscriptions for an organization’s account and can customize user access based on roles. Organizations can define policies for code completions and security protocols (telemetry controls).
LLM07: Insecure Plugin Design
LLM plugins automatically execute during user interactions and often lack input validation, exposing them to malicious attacks, including remote code execution. These risks are heightened by weak access controls. While this focuses on self-created LLM plugins, third-party plugin vulnerabilities are categorized separately.
♦ GitHub Copilot does not rely on third-party LLM plugins, avoiding the potential vulnerabilities associated with them.
LLM08: Excessive Agency
LLM-based systems can autonomously interact with other systems. The “Excessive Agency” vulnerability arises when these systems take harmful actions due to unclear outputs, often resulting from too much autonomy or permissions.
♦ GitHub Copilot is designed with limited autonomy and specifically tailored outputs, primarily providing code suggestions and completions. It does not initiate interactions with external systems or take actions beyond the scope of code generation. Its functionality is confined to the context of the developer’s IDE, which mitigates the risk of “Excessive Agency.” Also, Copilot does not have permissions to make changes autonomously, ensuring that the end user always has the final say in accepting or rejecting its suggestions.
LLM09: Overreliance
Reliance on LLMs can lead to accepting erroneous information they produce, causing security issues and misinformation. LLM-generated code can introduce vulnerabilities. It is essential to have strict reviews, validation mechanisms, and risk disclaimers.
♦ GitHub Copilot generates suggestions based on public code repositories, which can sometimes contain insecure or outdated coding patterns. To address this, GitHub introduced an AI system to block insecure patterns, focusing on common vulnerabilities like SQL injections. Despite these advancements, users are advised to use GitHub Copilot in conjunction with thorough testing, code reviews, security tools, and personal judgment.
LLM10: Model Theft
LLM applications can inadvertently disclose sensitive information or proprietary details through their outputs, leading to potential security breaches, including unauthorized access to data and privacy violations.
♦ User engagement data is shared with Microsoft, but GitHub Copilot for Business does not store prompts or code snippets. Transmitted data is encrypted, and private code is not suggested to other users.
Okay, So Is It Secure Or Not?
Of course, my answer is: It depends.
First, I do not recommend GitHub Copilot for individual users if they are using it at work for a company, mainly because they would have to manage their own policies and certain policies are only available for Business accounts.
Second, although Copilot does have the ability to block suggestions matching public code and GitHub does offer a vulnerability filtering system to prevent vulnerabilities from being suggested in code, it is still susceptible to producing vulnerabilities in code.
Therefore, I recommend developers also use a security code scanning tool along side GitHub Copilot. Snyk, a developer centric platform, offers a free IDE extension that can be used to scan all output generated. This tool, utilized in parallel with GitHub Copilot will assist in preventing vulnerable output provided in any developer’s code base.
Conclusion
The integration of AI tools like GitHub Copilot into software development processes presents an exciting frontier for enhancing efficiency and innovation. However, it is essential to approach these tools with a security-first mindset, addressing potential vulnerabilities and ensuring the safe handling of sensitive information.
My examination of GitHub Copilot, grounded by the OWASP framework, confirms its possibility as a secure tool for developers. As the technology landscape continues to progress and evolve, maintaining an open dialogue about security best practices will remain crucial. So, what do you think?