deep-searcher

Yep, I haven't tried this particular project but that's my overall experience with similar projects as well. Smaller models that can be ran locally in compute-poor environments really need structured outputs and just prompting them to "you can ONLY return a python list of str, WITHOUT any other additional content" (a piece of prompt from this project) is nowhere sufficient for any resemblance of reliability. If you're feeling adventurous, you can probably refactor the prompt functions in https://github.com/zilliztech/deep-searcher/blob/master/deep... to return additional metadata (required output structure) together with the prompt itself, update all `llm.chat()` calls throughout the codebase to account for this (probably changing the `chat` method API by adding an extra `format` argument and not just `messages`) and implement a custom Ol

Looking at the code (https://github.com/zilliztech/deep-searcher/blob/master/deep...), I think it probably may work at least with Ollama without any additional tweaks if you run it with `OPENAI_BASE_URL=http://localhost:11434/v1` or define `provide_settings.llm.base_url` in `config.yaml` (https://github.com/zilliztech/deep-searcher/blob/6c77b1e5597...) and tweak the model appropriately. From a quick glance, this project doesn't seem to use any tool/function calling or streaming or format enforcement or any other "fancy" API features, so all chances are that it may just work, although I have some reservations about the quality, especially with smaller models.

github https://github.com/zilliztech/deep-searcher