textgrad

And out of that 73%, 99% of them don't even do the obvious step of trying to actually optimize/engineer their damn prompts! https://github.com/zou-group/textgrad and bonus, my rant about this circa 2023 in the context of Stable Diffusion models: https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...

Stumbled upon this in another thread: https://ai-analytics.wharton.upenn.edu/generative-ai-labs/re... My point still holds that it is optimizable though (https://github.com/zou-group/textgrad, https://arxiv.org/abs/2501.16673) >Subjects develop elaborate "rain dances" in the belief that they can influence the outcome. Not unlike sports fans superstitions. Anybody tuning neural weights by hand would feel like this.

>People really don't understand how much better LLM swarms get with more agents. I never hit a point of diminishing returns on text quality Could you elaborate please ? One use for swarms is to use multiple agents/prompts in place of one single agent with one long prompt in order to increase performance by splitting one big task into many. It is very time consuming though, as it requires experimenting to determine how best to divide one task into subtasks, including writing code to parse and sanitize each task output and plug it back into the rest of the agent graph. Dspy [1] seems to target this problem space but last time I checked it only focused on single prompt optimization (by selecting which few shots examples lead to the best prompt performance for instance), but even though I have seen papers on the subject, I have yet to find a

At least some of it comes from "hype" too. The author of Dspy (the writer) (https://github.com/stanfordnlp/dspy) should know this, given that Dspy is nothing more than fancy prompts optimizing prompts to be fancier according to prompt chains described in papers (i.e. Chain of thought, Tree of thought, etc). Textgrad (https://github.com/zou-group/textgrad) is an even worse example of this, as it makes people think that it's not just a prompt optimizing another prompt Dspy has 17k stars, meanwhile PyReft (https://github.com/stanfordnlp/pyreft) isn't even at 1200 yet and it has Christopher Manning (head of AI at stanford) working on it (see their paper: https://arxiv.org/abs/2404.03592). Sometimes what the world deems "impactful" in the short-medium term is wrong. Think long term. PyReft is likely the beginning of an explosion in demand for

We are broader. We have essential building blocks for RAG, Agents. But also made whatever you build possible to auto-optimize. You can think of us as the library to do in-context learning. Just like PyTorch is for model-training. Our benchmark has compared with Dspy and Text-grad(https://github.com/zou-group/textgrad) We have better accuracy, more token-efficient, and faster convergence speed. We are publishing three research papers to explain this better to researchers. https://adalflow.sylph.ai/use_cases/question_answering.html We will compare with these optimization libraries but wont compare with libraries like LangChain or LlamaIndex. As they simply dont have optimization and it is pain to build on them. Hope this make sense

Agreed 100%. DSPy along with libraries inspired by it (i.e. https://github.com/zou-group/textgrad) are nothing more than fancy prompt chains under the hood. These libraries mostly exist as "cope" for the fact that we don't have good fine-tuning (i.e. lora) capabilities for ChatGPT et al, so we try to instead optimize the prompt.