semantic-grep

Some small tips from superficially reading the code: https://github.com/arunsupe/semantic-grep/blob/b7dcc82a7cbab... You can read the vector all at once. See e.g.: https://github.com/danieldk/go2vec/blob/ee0e8720a8f518315f35... --- https://github.com/arunsupe/semantic-grep/blob/b7dcc82a7cbab... You can compute the similarity much faster by using BLAS. Good BLAS libraries have SIMD-optimized implementations. Or if you do multiple tokens as once, you can do a matrix-vector multiplication (sgemv), which will be even faster in many implementations. Alternatively, there is probably also a SIMD implementation in Go using assembly (it has been 7 years since I looked at anything in the Go ecosystem). You could also normalize the vectors while loading. Then during runtime the cosine similarity is just the dot product of the vectors (whether it pay

Much improved new version. Search for words similar to the query. For example, "death" will find "death", "dying", "dead", "killing"... Incredibly useful for exploring large text datasets where exact matches are too restrictive.

I've built a command-line tool called semantic-grep for searching free-text English content using natural language queries. Some key features: Uses word2vec embeddings to find semantically similar text Written in Go, using only the standard library Familiar CLI UI Can process data streams piped as input Finds similarity in a sliding window of text It's designed for developers and researchers who need to search large text datasets more intelligently than simple keyword matching. The tool is still in early stages, so I'd appreciate any feedback or feature suggestions. GitHub: https://github.com/arunsupe/semantic-grep