rellm

I also released a hosted version of my open-source libraries ReLLM and ParserLLM that already supports APIs for * Regex completion for LLMs * Context-free Grammar completion for LLMs https://thiggle.com/ [0] https://github.com/r2d4/rellm [1] https://github.com/r2d4/parserllm [2] https://github.com/thiggle/api There's also another API on Thiggle that I've build that supports classification via a similar logit-based strategy.

When you're building applications on top of LLMs, there are a number of central problems that you're trying to solve and this is one of them. Solutions are numerous and widely variable, everything from basic regex parsing to fine-tuning validator models to new programming/modeling languages. Here's some examples: - https://github.com/microsoft/guidance - https://github.com/NVIDIA/NeMo-Guardrails/ - https://github.com/r2d4/rellm - https://shreyar.github.io/guardrails/ - https://lmql.ai/ - https://github.com/jbrukh/gpt-jargon

I agree, it looks great. A couple similar projects you might find interesting: - https://github.com/newhouseb/clownfish - https://github.com/r2d4/rellm The first one is JSON only and the second one uses regular expressions, but they both take the same "logit masking" approach as the project GP linked to.

There's probably a better API that wraps generate, but there's a bit more work than the logit mask. You have to go one token at a time, otherwise the masking becomes combinatoric rather than linear (two tokens at a time -- need to generate all two token pairs, etc.). But otherwise, that's what the code does! https://github.com/r2d4/rellm/blob/main/rellm/rellm.py#L21