Introduction
LettuceDetect is a lightweight and efficient tool designed for detecting hallucinations in Retrieval-Augmented Generation (RAG) systems. It identifies unsupported parts of an answer by comparing it to the provided context, ensuring that AI-generated responses are accurate and reliable.
Key Features
- Token-Level Precision: Detects exact hallucinated spans in generated text.
- Optimized for Inference: Smaller model size and faster inference compared to traditional methods.
- 4K Context Window: Utilizes ModernBERT for processing extensive context windows, enhancing performance in complex queries.
- Easy Integration: Simple installation via pip and straightforward API for quick implementation in RAG systems.
Benefits
- High Performance: Outperforms existing models on the RAGTruth dataset, achieving an F1 score of 79.22%.
- Lightweight: Designed to be efficient, making it suitable for real-time applications.
- Open Source: Released under the MIT license, allowing for community contributions and improvements.
Highlights
- Trained and evaluated on the RAGTruth dataset.
- Supports both span-level and token-level evaluations.
- Provides a web API for easy access and integration into applications.
