clip-retrieval

relevant: https://rom1504.github.io/clip-retrieval/ https://github.com/rom1504/clip-retrieval Ironically, the stableattribution.com authors didn't give any attribution or credit to clip-retrieval and its developer

This is a company that allows you to search for images from a training dataset that have a high cosine similarity with a given image. It appears to be the same as the open source software published by LAION. https://github.com/rom1504/clip-retrieval It does not appear to actually show you how images were used to train a generative AI.

Hehe, well you know, PR welcome, the front end is 500 lines https://github.com/rom1504/clip-retrieval/blob/main/front/sr... Other people have done a few alternate front ends already This one is meant to be functional, but could sure be made prettier

Below is an OSS alternative, originating from same engineering as stable diffusion. There is other awesome work in rom1504's Github I recommend exploring as well. https://rom1504.github.io/clip-retrieval/ (usually performant) https://github.com/rom1504/clip-retrieval

Done https://github.com/rom1504/clip-retrieval/commit/53e3383f58b... Using clip for searching is better than direct text indexing for a variety of reasons but here for example because it matches better what stable diffusion sees Still interesting to have a different view over the dataset! If you want to scale this out, you could use elastic search

Based on OpenAI's CLIP and the clip-retrieval library (https://github.com/rom1504/clip-retrieval), I've built an end-to-end demo for a semantic and similarity image search engine. It's incredibly powerful for finding similar images amongst large image datasets, or just submitting text/natural language queries and finding the most relevant images in your dataset. Really useful tool for introspection into large datasets before annotation or ML work begins. This could potentially be used to filter or downsize your datasets by several orders of magnitude and make annotation and ML work easier and less costly. Checkout the demo here: http://ec2-52-39-251-116.us-west-2.compute.amazonaws.com/ And you can checkout our website or email me for updates and email list, etc.: https://machineperception.co ameer@machineperception.co

This gh repo makes it pretty easy to create similar tech by first embedding any images you have using the released "CLIP" model from Open AI and then creating a Faiss index over these embeds for quick retrieval/decode. You can then do text->image, and image->image semantic search. https://github.com/rom1504/clip-retrieval

Assuming you have an NVIDIA GPU, you can build a semantic image search engine by indexing CLIP embeds (image or text). https://github.com/rom1504/clip-retrieval

Would the embeds from openai's CLIP be suitably "dense" for this to work? The usage requires calculating the cosine similarity between the embeds and there are many instances where a lookup table approach is already being applied e.g. github.com/rom1504/clip-retrieval.