TypeChat

Really cool project! I'm doing something similar at a very small scale for my personal project using TypeChat with Zod (https://github.com/microsoft/Typechat) and Unstructured (https://unstructured.io/)

Semantic Memory (renamed to Kernel Memory - https://github.com/microsoft/kernel-memory) complements SK. Guidance's features are being absorbed into SK, following the departure of that team from Microsoft. Additionally, we have TypeChat (https://github.com/microsoft/TypeChat), which aims to ensure type-safe responses from LLMs. Most features of Autogen are also being integrated into SK, along with Assistants. SK serves as the orchestration engine powering Microsoft Copilots.

I'd say its zod, but it does not have validation in the same way instructor does. Typechat has some options tho! https://github.com/microsoft/typechat/

That re-prompting on error trick is what this new Microsoft library does, too: https://github.com/microsoft/TypeChat Here's their prompt for that: https://github.com/microsoft/TypeChat/blob/c45460f4030938da3... I think the approach using grammars (seen here, but also in things like https://github.com/ggerganov/llama.cpp/pull/1773 ) is a much more elegant solution.

How does TypeChat tie you to OpenAI more than zod-gpt does? The interface required of a chat completion model is as simple as it gets, and you can provide your own easily (as the linked post makes clear) https://github.com/microsoft/TypeChat/blob/4d34a5005c67bc494...

I'm very surprised that they're not using `guidance` [0] here. It not only would allow them to suggest that required fields be completed (avoiding the need for validation [1]) and probably save them GPU time in the end. There must be a reason and I'm dying to know what it is! :) Side-note, I was in the process of building this very thing and good ol' Misrocoft just swung in and ate my lunch.. :/ [0] https://github.com/microsoft/guidance [1] https://github.com/microsoft/TypeChat/blob/main/src/typechat...

Yes. https://github.com/microsoft/TypeChat/blob/main/src/typechat...

The value is in: 1. Running the typescript type checker against what is returned by the LLM. 2. If there are type errors, combining those into a "repair prompt" that will (it is assumed) have a higher likelihood of eliciting an LLM output that type checks. 3. Gracefully handling the cases where the heuristic in #2 fails. https://github.com/microsoft/TypeChat/blob/main/src/typechat... In my experience experimenting with the same basic idea, the heuristic in #2 works surprisingly well for relatively simple types (i.e. records and arrays not nested too deeply, limited use of type variables). It turns out that prompting LLMs to return values inhabiting relatively simple types can be used to create useful applications. Since that is valuable, this library is valuable inasmuch as it eliminates the need to hand roll this request pattern, and pro

maybe this function? https://github.com/microsoft/TypeChat/blob/4d34a5005c67bc494...

I don't see the value add here. Here's the core of the message sent to the LLM: https://github.com/microsoft/TypeChat/blob/main/src/typechat... You are basically getting a fixed prompt to return structured data with a small amount of automation and vendor lockin. All these LLM libraries are just crappy APIs to the underlying API. It is trivial to write a script that does the same and will be much more flexible as models and user needs evolve. As an example, think about how you could change the prompt or use python classes instead. How much work would this be using a library like this versus something that lifts the API calls and text templating to the user like: https://github.com/hofstadter-io/hof/blob/_dev/flow/chat/llm...

Someone should just get this working on Llama 2 instead of O̶p̶e̶n̶AI.com [0] All this is it's just talking to a AI model sitting on someone else's server. [0] https://github.com/microsoft/TypeChat/blob/main/src/model.ts...