guardrails

Use one of these structured output libraries: https://github.com/outlines-dev/outlines https://github.com/jxnl/instructor https://github.com/guardrails-ai/guardrails https://www.askmarvin.ai/docs/text/transformation/ Some of them allow a JSON schema, others a Pydantic model (which you can transform to/from JSON).

The general concept is called output guardrails. See https://github.com/guardrails-ai/guardrails for an open source implementation.

Post-LLM validation. We're currently working on this at https://github.com/guardrails-ai/guardrails

Oh nice! I built a similar system a few weeks ago: https://github.com/newhouseb/clownfish I think the main differentiating factor here is that this is better if you have a simpler JSON schema without enums or oneOf constraints. If you do have these constraints, i.e. let's say you wanted an array of different types that represented a items on a menu { kind: pizza, toppings: [pepperoni] } or { kind: ice_cream, flavor: vanilla | strawberry } then you would need something more sophisticated like clownfish that can ask the LLM to pick specific properties (and an ability to do some backtracking so you can do proper beam search). For completeness, another common approach can be found here: https://github.com/ShreyaR/guardrails which essentially boils down to "provide the schema in the prompt and ask the LLM to correct things if it fails to get t

> I still haven't found a 100% reliable way of getting a LLM to always produce results in JSON for example. Have you tried Guardrails? https://github.com/ShreyaR/guardrails

There are a couple different approaches: - Use multi-shot prompting with something like guardrails to try prompting a commercial model until it works. [1] - Use a local model with a final layer that steers token selection towards syntactically valid tokens [2] [1] https://github.com/ShreyaR/guardrails [2] "Structural Alignment: Modifying Transformers (like GPT) to Follow a JSON Schema" @ https://github.com/newhouseb/clownfish (full disclosure: this is my work)

https://shreyar.github.io/guardrails/ https://github.com/shreyar/guardrails