sidekick

g your own internal search product. Now, with libraries like LangChain (https://github.com/hwchase17/langchain), LlamaIndex (https://github.com/jerryjliu/llama_index), and open sources tools like Sidekick (https://github.com/ai-sidekick/sidekick), it’s possible to build a product that works just as well as most workplace search vendors in a matter of days, with the added benefit of being free + fully customizable. (Disclaimer: I am one of the cofounders at Sidekick) It’s also not hard to find developers who would love to get their hands dirty with emerging technologies like vector databases, LLMs, and GPT agents. The economic incentive structure for internal search products (and maybe all SaaS product) has been flipped on its head. Why spend $100k+ year on a vendor that comes with 1-3 month implementation when you can spend half that time building something in-house that plugs in to where your team already works like Teams or Slack? Examples: - Supabase built GPT powered search right into their docs ([https://supabase.com/docs](https://supabase.com/docs)) - PostHog built and deployed their own slackbot to answer questions about PostHog ([https://posthog.com/blog/aruba-hackathon](https://posthog.com/blog/aruba-hackathon)) Would you consider building internal semantic search in-house at your company?

eveloper docs, process it into chunks of <500 tokens, and add those chunks to a vector store, you can build a prototype with Langchain fairly quickly. However, if you want to deploy it to customers like we did for BentoML ([https://www.bentoml.com/](https://www.bentoml.com/)) you’ll quickly realize that a naive chunking method that splits by character/token leads to poor results, and that “delete and re-vectorize everything” when the source docs change doesn’t scale as a data synchronization strategy. We took the code we used to build chatbots for our early customers and turned it into an open source framework to rapidly build new data Connectors and Chunkers. This way developers can use community built Connectors and Chunkers to start running vector searches on data from any source in a matter of minutes, or write their own in a matter of hours. Here’s a video demo: [https://youtu.be/I2V3Cu8L6wk](https://youtu.be/I2V3Cu8L6wk) The repo has instructions on how to get started and set up API endpoints to load, chunk, and vectorize data quickly. Right now it only works with websites and Github repos, but we’ll be adding Zendesk, Google Drive, and Confluence integrations soon too.

for companies because they need to hire skilled engineers to do it. We’ve seen community support channels with a 2000:1 ratio of developers to support engineers - there’s no way every question will get answered. We built Sidekick to make this much easier. It’s particularly helpful for open-source companies/projects because many OSS communities have a lot of people asking questions, but hardly anyone helping troubleshoot. We integrate with Slack and Discord, since that’s where developer support is already happening. On the backend, we use Weaviate to index the data and OpenAI’s text-davinci-3 model to generate responses. In addition to answering questions, Sidekick can also update .md files automatically with new information. When someone reacts to a message in Slack with the emoji, Sidekick will use Weaviate to find the part in your documentation that’s most related to the message, then use GPT3 to merge the new info into the documentation. Finally, it will submit a pull request on Github with the changes. We saw that devrel teams are already making product announcements and helping users troubleshoot common issues in the community, so we built this feature to save them even more time. We use GPT for generating the responses and new documentation, but are relying less and less on it after learning that you hit a ceiling on answer quality very quickly by using only GPT and prompt engineering techniques. Here’s some of what we learned trying to prevent hallucinations in our answers: https://medium.com/@jfan001/how-we-cut-the-rate-of-gpt-hallu... What we found makes a much bigger difference is the breadth and quality of the content you can search through, which is why we now rely a lot more on cleaning and annotating data, which yields far better results when combined with prompt chaining. For example, instead of naively chunking data into 1000 token blocks, we parse the markdown into semantically meaningful sections (e.g. paragraphs, lists, code blocks) and tag the content with the header name and document name so it’s more likely to surface for searches that are match for the section it’s from, even if it doesn’t exactly match the content in that chunk. One fun thing we also learned is that when Sidekick gets added to a #help channel, people who otherwise wouldn’t ask questions start using it. It turns out, there are a lot of “lurkers” who come to these channels to find answers, but don’t want to bother anyone with their issue. Adding a tool that they can get answers from instantly brings these people out into the community, giving founders and community managers an opportunity to reach out to them. To summarize, Sidekick 1) saves support engineers time, 2) keeps the docs up to date and 3) helps engage developers in the community. Long-term we want to provide an analytics product on top of Sidekick so companies can understand how their product is being used and where there are opportunities to add more value to their customers (and charge more money for it). We’d love to hear from the HN community about this product! Do you think using a tool to search through and update developer docs from Slack would save you time?

orials, and past issues/threads. Feeding it readmes barely scratches the surface, so if you have some docs you want to ask questions over put up a PR with the data and we’ll ingest it and make it queryable through this site! Source: [https://github.com/ai-sidekick/sidekick/tree/main/client/sid...