graphrag

Cool project! How do you compare with Microsoft's graphrag? https://github.com/microsoft/graphrag

Faiss is for similarity search over vectors via k-NN. GraphRAG is, well, a graph. More precisely, GraphRAG has more in common with old school knowledge graph techniques involving named entity extraction and the various forms of black magic used to identify relationships between entities. If you remember RDF and the semantic web it's sort of along those lines. One of the uses of Faiss is in a k-NN graph but the edges between nodes in that graph are (similarity) distance based. Looking at an example prompt from GraphRAG will make things clear https://github.com/microsoft/graphrag/blob/main/graphrag/pro... especially these lines: Return output in English as a single list of all the entities and relationships identified in steps 1 and 2. Format each relationship as a JSON entry with the following format: {{"source": <source_entity>, "target":

I find it interesting that their entity extraction method for building a knowledge graph does not use or require one of the 'in-vogue' extraction libraries like instructor, Marvin, or Guardrails (all of which build off of pydantic). They just tell the llm to list graph nodes and edges in a list, and do some basic delimiter parsing, and load the result right into a networkx graph [1]. Is this because GPT-4 and the like have become very reliable at following specific formatting instructions, like a certain .json schema? It looks like they just provide in the prompt a number of examples that follow the schema they want [2]. [1] https://github.com/microsoft/graphrag/blob/main/graphrag/ind... [2] https://github.com/microsoft/graphrag/blob/main/graphrag/ind...

The team at Microsoft is pleased to announce that GraphRAG is now available in open-source! Check out the announcement video: https://youtu.be/dsesHoTXyk0 GraphRAG is a research project from Microsoft exploring the use of knowledge graphs and large language models for enhanced retrieval augmented generation. It is an end-to-end system for richly understanding text-heavy datasets by combining text extraction, network analysis, LLM prompting, and summarization. For more details on GraphRAG check out aka.ms/graphrag Try out the Python code on your own machine: https://github.com/microsoft/graphrag Easily deploy GraphRAG in Azure: https://github.com/Azure-Samples/graphrag-accelerator Leave a comment below for what you want to build with GraphRAG!