HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
guidance-ai
guidance
Jupyter Notebook2023–2026
guidance-ai / guidance

guidance

A guidance language for controlling large language models.

64Distinct people
72Mentions
0Explicit recommenders
21.8KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

A guidance language for controlling large language models.

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 5:04:15 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year72 dated mentions

Observed mentions: 2023-05-09 19:03:22 UTC to 2026-09-15 20:56:14 UTC.

02040602023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 52 mentions; 8 submissions, 44 comments20232024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 12 mentions; 0 submissions, 12 comments20242025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 4 mentions; 0 submissions, 4 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 4 mentions; 1 submissions, 3 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC52844
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC12012
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC404
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC413

From the project README

An excerpt from the repository, fetched Nov 12, 2026.

Read the project’s introduction
Guidance is an efficient programming paradigm for steering language models. With Guidance, you can control how output is structured and get high-quality output for your use case—while reducing latency and cost vs. conventional prompting or fine-tuning. It allows users to constrain generation (e.g. with regex and CFGs) as well as to interleave control (conditionals, loops, tool use) and generation seamlessly. Install Features Install Guidance is available through PyPI and supports a variety of backends (Transformers, llama.cpp, OpenAI, etc.). If you already have the backend required for your model, you can simply run pip install guidance Features A Pythonic interface for language models When using Guidance, you can work with large language models using common Python idioms: from guidance import system, user, assistant, gen from guidance.models import Transformers # Could also do LlamaCpp or many other models phi_lm = Transformers("microsoft/Phi-4-mini-instruct") # Model objects are immutable, so this is a copy lm = phi_lm with system(): lm += "You are a helpful assistant" with user(): lm += "Hello. What is your name?" with assistant(): lm += gen(max_tokens=20) print(lm) If run at the command line, this will produce output like: <|system|>You are a helpful assistant<|end|><|user|>Hello. What is your name?<|end|><|assistant|>I am Phi, an AI developed by Microsoft. How can I help you today? However, if running in a Jupyter notebook, then Guidance provides a widget for a richer user experience: With Guidance, it's really easy to capture generated text: # Get a new copy of the Model lm = phi_lm with system(): lm += "You are a helpful assistant" with user(): lm += "Hello. What is your name?" with assistant(): lm += gen(name="lm_response", max_tokens=20) print(f"{lm['lm_response']=}") lm['lm_response']='I am Phi, an AI developed by Microsoft. How can I help you today?' Guarantee output syntax with constrained generation Guidance provides an easy to use, yet immensely powerful syntax for constraining the output of a language model. For example, a gen() call can be constrained to match a regular expression: lm = phi_lm with system(): lm += "You are a teenager" with user(): lm += "How old are you?" with assistant(): lm += gen("lm_age", regex=r"\d+", temperature=0.8) print(f"The language model is {lm['lm_age']} years old") The language model is 13 years old Often, we know that the output has to be an item from a list we know in advance. Guidance provides a select() function for this scenario: from guidance import select lm = phi_lm with system(): lm += "You are a geography expert" with user(): lm += """What is the capital of Sweden? Answer with the correct letter. A) Helsinki B) Reykjavík C) Stockholm D) Oslo """ with assistant(): lm += select(["A", "B", "C", "D"], name="model_selection") print(f"The model selected {lm['model_selection']}") The model selected C The constraint system offered by Guidance is extremely powerful. It can ensure that the output conforms to any context free grammar (so long as the backend LLM has full support for Guidance). More on this below. Debug grammars offline (no model API calls) When iterating on constraints, you can validate candidate strings locally and test a full run with the Mock model. from guidance import gen from guidance.models import Mock grammar = "expr=" + gen(regex=r"\d+([+*]\d+)*", name="expr") # 1) Validate strings directly against the grammar assert grammar.match("expr=12+7*3") is not None assert grammar.match("expr=12+*3") is None # 2) Run the same grammar with a local mock model lm = Mock(b"<s>expr=12+7*3") lm += grammar print(lm["expr"]) # 12+7*3 Create your own Guidance functions With Guidance, you can create your own Guidance functions which can interact with language models. These are marked using the @guidance decorator. Suppose we wanted to answer lots of multiple choice questions. We could do something like the following: import guidance from guidance.models import Model ASCII_OFFSET = ord("a") @guidance def zero_shot_multiple_choice( language_model: Model, question: str, choices: list[str], ): with user(): language_model += question + "\n" for i, choice in enumerate(choices): language_model += f"{chr(i+ASCII_OFFSET)} : {choice}\n" with assistant(): language_model += select( [chr(i + ASCII_OFFSET) for i in range(len(choices))], name="string_choice" ) return language_model Now, define some questions: questions = [ { "question" : "Which state has the northernmost capital?", "choices" : [ "New South Wales", "Northern Territory", "Queensland", "South Australia", "Tasmania", "Victoria", "Western Australia", ], "answer" : 1, }, { "question" : "Which of the following is venomous?", "choices" : [ "Kangaroo", "Koala Bear", "Platypus", ], "answer" : 2, } ] We can use our decorated function like gen() or select(). The language_model argument will be filled in for us automatically: lm = phi_lm with system(): lm += "You are a student taking a multiple choice test." for mcq in questions: lm_temp = lm + zero_shot_multiple_choice(question=mcq["question"], choices=mcq["choices"]) converted_answer = ord(lm_temp["string_choice"]) - ASCII_OFFSET print(lm_temp) print(f"LM Answer: {converted_answer}, Correct Answer: {mcq['answer']}") <|system|>You are a student taking a multiple choice test.<|end|><|user|>Which state has the northernmost capital? a : New South Wales b : Northern Territory c : Queensland d : South Australia e : Tasmania f : Victoria g : Western Australia <|end|><|assistant|>b LM Answer: 1, Correct Answer: 1 <|system|>You are a student taking a mu
Read the full README on GitHub

What Hacker News says

52 sources
Thanks for your comment. I did not know about "Betteridge's law of headlines", quite interesting. Thanks for sharing :) You raise some interesting points. 1) Safety: It is true that LVMs and LLMs have unknown biases and could potentially create unsafe content. However, this is not necessarily unique to them, for example, Google had the same problem with their supervised learning model https://www.theverge.com/2018/1/12/16882408/google-racist-go.... It all depends on the original data. I believe we need systems on top of our models to ensure safety. It is also possible to restrict the output domain of our models (https://github.com/guidance-ai/guidance). Instead of allowing our LVMs to output any words, we could restrict it to only being able to answer "red, green, blue..." when giving the color of a car. 2) Cost: You are right right now L
Read the rest
VMs are quite expensive to run. As you said are a great way to go to market faster but they cannot run on low-cost hardware for the moment. However, they could help with training those smaller models. Indeed, with see in the NLP domain that a lot of smaller models are trained on data created with GPT models. You can still distill the knowledge of your LVMs into a custom smaller model that can run on embedded devices. The advantage is that you can use your LVMs to generate data when it is scarce and use it as a fallback when your smaller device is uncertain of the answer. 3) Labelling data: I don't think labeling data is necessarily cheap. First, you have to collect the data, depending on the frequency of your events could take months of monitoring if you want to build a large-scale dataset. Lastly, not all labeling is necessarily cheap. I worked at a semiconductor company and labeled data was scarce as it required expert knowledge and could only be done by experienced employees. Indeed not all labelling can be done externally. However, both approaches are indeed complementary and I think systems that will work the best will rely on both. Thanks again for the thought-provoking discussion. I hope this answer some of the concerns you raised
In Is supervised learning dead for computer vision?
I've had a bit of trouble getting function calling to work with cases that aren't just extracting some data from the input. The format is correct but it was harder to get the correct data if it wasn't a simple extraction. Hopefully OpenAI and others will offer something like https://github.com/guidance-ai/guidance at some point to guarantee overall output structure. Failed validations will retry, but from what I've seen JSONSchema + generated JSON examples are decently reliable in practice for gpt-3.5-turbo and extremely reliable on gpt-4.
In Show HN: Elelem – TypeScript LLMs with tracing, retries, and type safety
Right now it just works with OpenAI chat models (gpt-3.5-turbo, gpt-4) but if there's interest I plan to extend it to have several backends. These would probably each be an existing library that implements generating structured output like https://github.com/outlines-dev/outlines or https://github.com/guidance-ai/guidance. If you have ideas how this should be done let me know - on a github issue would be great to make it visible to others.
In Show HN: Magentic – Use LLMs as simple Python functions
I dug into this a while back, iirc, :handwavy: it comes down to "pausing" template rendering and calling the LLM with all content generated so far. https://github.com/guidance-ai/guidance/blob/main/guidance/l... This is how we implemented it anyhow, with some more parameters to control how that all works (and the LLM params) at each "pause" point. The _neat_ part for us was that a template helper could make use of the partially generated content. Hadn't thought about that before for a templating engine, but was trivial to implement in the end
In Guidance: A guidance language for controlling large language models
ukuina·Sep 7, 2023Submission

Show HN: HackYourNews – AI summaries of the top HN stories

Hey there HN! I wanted to share a pet project of mine. I built HackYourNews [1] to scratch a personal itch: Knowing which stories to focus on while browsing aimlessly (though there is a certain joy in that, as well!) HackYourNews uses OpenAI's gpt-3.5-turbo to summarize the destination article as well as the comments section. Summarization of the article is always cached, while summaries of the comments are regenerated if the comments count is >10% (or >10 comments) different. While I styled the homepage to welcome HNers, my preferred view is the Mobile view, accessed from the navbar. This no-frills view honors OS-level dark mode and is easy to skim on any device. Tried to keep the site minimal. The only JS is Cloudflare's privacy-preserving analytics [2], just to gauge interest. This is the first time I'm releasing something to the wild.
Read the rest
Hope you find this useful! The frontend is pure HTML+CSS. The backend is NodeJS (Puppeteer) + Python with the excellent Microsoft Guidance [3] library to interface to OpenAI's API. [1] https://hackyournews.com/ [2] https://www.cloudflare.com/web-analytics/ [3] https://github.com/guidance-ai/guidance
335 points · Open discussion
I'm not 100% sure they're talking about this specifically but logit control/manipulation is often used to conform to a specific schema. https://github.com/guidance-ai/guidance I'm going to butcher this explanation - after you've generated your selection of logits but before you sample from them, you check which ones conform to your schema. If you want the only two options to be "true" or "false", then you take any of the logits that would provide invalid answers and lower their probabilities manually. Another example is structures like JSON can be validated so when your sample is "{'name':'Carl'" you lower the probability of "{" since that would invalidate the json. In fact the only valid ones you'd likely have left would be ",", " ", and "}"
In Llama 2 is about as factually accurate as GPT-4 for summaries and is 30X cheaper

Showing the 15 most recent of 52 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.