COcodelion·Mar 7, 2025
I built an open-source hallucination detector that identifies when LLM outputs contain information not present in the source context. The tool is particularly useful for RAG systems where ensuring factual accuracy is critical.
Unlike most hallucination detection approaches that require separate LLM calls (which add cost and latency), this is a lightweight classifier built on HuggingFace transformers. It's adaptive, meaning it continuously improves as it processes more examples.
Technical approach:
- Uses a prototype memory system that maintains class examples for quick adaptation
- Combines transformer embeddings with an adaptive neural layer
- Trained on the RAGTruth benchmark dataset across QA, summarization, and data-to-text tasks
- Achieves 80.7% recall overall (51.5% F1), with strongest performance on data-to-text generation
Example
Read the rest
usage:
from adaptive_classifier import AdaptiveClassifier
# Load pre-trained detector
detector = AdaptiveClassifier.from_pretrained("adaptive-classifier/llm-hallucination-detector")
# Format input with context, query and response
input_text = f"Context: {your_context}\nQuestion: {your_question}\nAnswer: {llm_response}"
# Get prediction
prediction = detector.predict(input_text)
# Returns: [('HALLUCINATED', 0.72), ('NOT_HALLUCINATED', 0.28)]
Current limitations:
- Performance varies by task type (stronger on data-to-text, weaker on summarization precision)
- Initial version focuses on binary classification; token-level detection is planned
- The model is relatively small, so it won't catch subtle nuanced hallucinations that require deep domain knowledge
The library's wider goal is to enable adaptive classification for use cases where models need to continuously learn from new examples. We've also built LLM routers and configuration optimizers with it.
Would love feedback from anyone working on RAG systems or LLM evaluation. What metrics or capabilities would be most useful to you in a hallucination detector?
Project: https://github.com/codelion/adaptive-classifier
Docs: https://github.com/codelion/adaptive-classifier#hallucinatio...