The objective of the OWASP Top 10 for Large Language Model Applications initiative is to enhance the understanding of developers, designers, architects, managers, and organizations concerning security risks associated with the implementation and management of Large Language Models (LLMs). This endeavor offers a compilation of the ten most significant vulnerabilities frequently encountered in LLM applications, underlining their potential consequences, susceptibility to exploitation, and prevalence in practical scenarios. Notable vulnerabilities encompass prompt injections, data leakage, insufficient sandboxing, and unauthorized code execution, among others. The primary aim is to heighten awareness about these vulnerabilities, propose effective mitigation techniques, and ultimately elevate the security posture of LLM applications.
OWASP Top 10 for LLM
To highlight the distinct nature of Large Language Model Applications security, OWASP has released a dedicated API Top 10 for Large Language Model Applications document alongside the Web Application Security Top 10 and API Security Top 10. The OWASP Top 10 for Large Language Model Applications Project centers on providing strategies and solutions to comprehend and address the specific vulnerabilities and security risks associated with Large Language Model Applications (LLMs).
- LLM01:Prompt Injection: This manipulates a large language model (LLM) through crafty inputs, causing unintended actions by the LLM. Direct injections overwrite system prompts, while indirect ones manipulate inputs from external sources.
- LLM02:Insecure Output Handling: This vulnerability occurs when an LLM output is accepted without scrutiny, exposing backend systems. Misuse may lead to severe consequences like XSS, CSRF, SSRF, privilege escalation, or remote code execution.
- LLM03:Training Data Poisoning: This occurs when LLM training data is tampered, introducing vulnerabilities or biases that compromise security, effectiveness, or ethical behavior. Sources include Common Crawl, WebText, OpenWebText, & books.
- LLM04:Model Denial of Service: Attackers cause resource-heavy operations on LLMs, leading to service degradation or high costs. The vulnerability is magnified due to the resource-intensive nature of LLMs and unpredictability of user inputs.
- LLM05:Supply Chain Vulnerabilities: LLM application lifecycle can be compromised by vulnerable components or services, leading to security attacks. Using third-party datasets, pre-trained models, and plugins can add vulnerabilities.
- LLM06:Sensitive Information Disclosure: LLM's may inadvertently reveal confidential data in its responses, leading to unauthorized data access, privacy violations, and security breaches. It's crucial to implement data sanitization and strict user policies to mitigate this.
- LLM07:Insecure Plugin Design: LLM plugins can have insecure inputs and insufficient access control. This lack of application control makes them easier to exploit and can result in consequences like remote code execution.
- LLM08:Excessive Agency: LLM-based systems may undertake actions leading to unintended consequences. The issue arises from excessive functionality, permissions, or autonomy granted to the LLM-based systems.
- LLM09:Overreliance: Systems or people overly depending on LLMs without oversight may face misinformation, miscommunication, legal issues, and security vulnerabilities due to incorrect or inappropriate content generated by LLMs.
- LLM10:Model Theft: This involves unauthorized access, copying, or exfiltration of proprietary LLM models. The impact includes economic losses, compromised competitive advantage, and potential access to sensitive information.
Download OWASP Top 10 for Large Language Model Applications
» OWASP Top 10 for LMM V1.0 (PDF)
» OWASP Top 10 for LMM V1.0 Presentation (PDF)
Reference:
https://owasp.org/www-project-top-10-for-large-language-model-applications