basaran

Found a couple others that do something like this. Turns out I had bookmarked them a while ago. https://github.com/r2d4/openlm https://github.com/hyperonym/basaran

Disclaimer: I am curating LLM-tools on github [1] A few thoughts: * allow for custom endpoint URLs, this way people can use open source LLMs with a fake openAI API backend like basaran[2] or llama-api-server[3] * look into better embedding methods for info-retrieval like InstructorEmbeddings or Document Summary Index * Don't use a single embedding per content item, use multiple to increase retrieval quality 1 https://github.com/underlines/awesome-marketing-datascience/... 2 https://github.com/hyperonym/basaran 3 https://github.com/iaalm/llama-api-server

Guanaco-65B[0] using Basaran[1] for your OpenAI compatible API. (You can use any ChatGPT front-end which lets you change the OpenAI endpoint URL.) [0] https://huggingface.co/TheBloke/guanaco-65B-HF A QLoRA finetune of LLaMA-65B by Tim Dettmers from the paper here: https://arxiv.org/abs/2305.14314 [1] https://github.com/hyperonym/basaran

Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models. The open source community will eventually witness the Stable Diffusion moment for large language models (LLMs), and Basaran is committed to becoming the Stable Diffusion web UI for LLMs. Basaran allows you to replace OpenAI's service with the latest open-source model to power your application without modifying a single line of code.