LettuceDetect

Introduction

LettuceDetect is a lightweight and efficient tool designed for detecting hallucinations in Retrieval-Augmented Generation (RAG) systems. It identifies unsupported parts of an answer by comparing it to the provided context, ensuring that AI-generated responses are accurate and reliable.

Key Features

• Token-Level Precision: Detects exact hallucinated spans in generated text.
• Optimized for Inference: Smaller model size and faster inference compared to traditional methods.
• 4K Context Window: Utilizes ModernBERT for processing extensive context windows, enhancing performance in complex queries.
• Easy Integration: Simple installation via pip and straightforward API for quick implementation in RAG systems.

Benefits

• High Performance: Outperforms existing models on the RAGTruth dataset, achieving an F1 score of 79.22%.
• Lightweight: Designed to be efficient, making it suitable for real-time applications.
• Open Source: Released under the MIT license, allowing for community contributions and improvements.

Highlights

• Trained and evaluated on the RAGTruth dataset.
• Supports both span-level and token-level evaluations.
• Provides a web API for easy access and integration into applications.