adaptive-classifier

usage: from adaptive_classifier import AdaptiveClassifier # Load pre-trained detector detector = AdaptiveClassifier.from_pretrained("adaptive-classifier/llm-hallucination-detector") # Format input with context, query and response input_text = f"Context: {your_context}\nQuestion: {your_question}\nAnswer: {llm_response}" # Get prediction prediction = detector.predict(input_text) # Returns: [('HALLUCINATED', 0.72), ('NOT_HALLUCINATED', 0.28)] Current limitations: - Performance varies by task type (stronger on data-to-text, weaker on summarization precision) - Initial version focuses on binary classification; token-level detection is planned - The model is relatively small, so it won't catch subtle nuanced hallucinations that require deep domain knowledge The library's wider goal is to enable adaptive classification for use cases where models need to continuously learn from new examples. We've also built LLM routers and configuration optimizers with it. Would love feedback from anyone working on RAG systems or LLM evaluation. What metrics or capabilities would be most useful to you in a hallucination detector? Project: https://github.com/codelion/adaptive-classifier Docs: https://github.com/codelion/adaptive-classifier#hallucinatio...

und-Trip Consistency (RTC) testing: generate a response, create an alternate query targeting similar content, generate second response, measure semantic similarity. This provides an automated way to evaluate temperature effectiveness without human intervention. The classifier learns continuously from query-response patterns. In production, this reduced "hallucinations" by 42% for factual queries while improving creativity scores by 35% for open-ended tasks. Technical details and implementation: https://github.com/codelion/adaptive-classifier We're particularly interested in feedback from others who've dealt with temperature optimization at scale.

ophic forgetting when learning new classes - Combines prototype memory with neural adaptation for efficient few-shot learning - Memory-efficient: stores only representative examples using k-means clustering - Fully deterministic results through careful seed management - Built on PyTorch/HuggingFace, works with any transformer model The key insight was combining EWC (usually used in reinforcement learning) with a prototype memory system. This lets the model learn new patterns while preserving its knowledge of existing categories. Real-world use cases I've tested: - Customer feedback classification where new issue types emerge over time - Content moderation systems that need to handle new violation types - Technical support routing where new product categories get added pip install adaptive-classifier GitHub: https://github.com/codelion/adaptive-classifier Would love feedback from anyone who's dealt with similar challenges in production ML systems!

abels = ["positive", "negative", "neutral"] classifier.add_examples(texts, labels) # Make predictions print(classifier.predict("This is amazing!")) # [('positive', 0.85), ('neutral', 0.10), ('negative', 0.05)] # Add a completely new class later classifier.add_examples( ["Error 404", "System crashed"], ["technical", "technical"] ) The library came out of my work on building a model router for optillm where approaches and model types keep changing. Traditional classifiers require full retraining when adding new classes, which becomes impractical with large datasets. Source code: https://github.com/codelion/adaptive-classifier I'd love to hear your thoughts and feedback! Happy to answer any questions about the implementation details, use cases, or future plans.