Trace

n't be to just tell you LLMs are fixed and can't learn but will also add this isn't a limitation since RAG pipelines are better suited for this task than fine-tuning [1]. Also RAG isn't a monolithic solutions, there are many, many variations. For your use-case, I'd consider more elaborate solutions than just baseline RAG, such as GraphRAG [2]. For the box problem above, you might want to consider integrating symbolic reasoning tools such as prolog, or consider using reasoning models and developing your own reinforcement learning environments. Needless to say, all of these aspects need to be carefully balanced and optimized to work together, and you need to follow a benchmark/dataset centric-approach to developing your solution. For this task consider frameworks that were designed to optimize llm/agentic workflows as a whole [3][4]. Shit is complex really. [1] https://arxiv.org/abs/2505.24832 tells us generalization happens in LLM once their capacity for remembering things is saturated, and this might explain why fine-tuning has been less efficient than RAG so far. [2] https://microsoft.github.io/graphrag/ [3] https://github.com/microsoft/Trace [4] https://github.com/SylphAI-Inc/AdalFlow