Prompt Injection Defenses
The tldrsec/prompt-injection-defenses repository centralizes and summarizes practical and proposed defenses against prompt injection. It aims to reduce the impact of successful prompt injections through various defensive designs and methodologies. Here are some key features and highlights:
Key Features
- Blast Radius Reduction: Strategies to minimize the impact of prompt injections.
- Input Pre-processing: Techniques like paraphrasing and retokenization to transform inputs and make adversarial prompts harder to create.
- Guardrails & Overseers: Monitoring inputs and outputs to detect prompt injections and their impacts.
- Taint Tracking: Categorizing input to mitigate risks based on trust levels.
- Secure Threads / Dual LLM: Using multiple models with different permission levels for safer data handling.
- Ensemble Decisions: Combining multiple models for added resilience against prompt injections.
- Prompt Engineering: Utilizing query structures to complicate prompt injections.
Benefits
- Enhances security for applications using large language models (LLMs).
- Provides a comprehensive overview of existing defenses and research proposals.
- Encourages collaboration and feedback from the community to improve defenses.
Highlights
- Centralized resource for understanding and mitigating prompt injection risks.
- Includes references to academic papers and critiques of existing controls.
- Actively maintained with contributions from the community.