Ask HN: Is there any good semantic search GUI for images or documents?

3 pointskklemon2y ago2 comments

When OpenAI's CLIP model was released in 2021 and other major advances in representation learning were made around that time, I expected, or rather hoped, that these models would soon be integrated into file browsers to improve semantic search capabilities, especially for images and documents.

Now, a few years later, the train still doesn't seem to have arrived. Vector databases are now a thing, but despite hours of research, I couldn't find a single GUI application that allows embedding and semantic search of (local) files.

Was I just unlucky in my search or is there really no such tool or integration, e.g. for Gnome Search or file browser such as Nautilus?

2 comments

jn2clark2y ago

Take a look here https://github.com/marqo-ai/local-image-search-demo. It is based on https://github.com/marqo-ai/marqo. We do a lot of image search applications. Feel free to reach out if you have other questions (email in profile).

kklemonOP2y ago

That looks indeed pretty interesting. But I still feel that it's not very convenient for usage in a desktop environment with local files. This is of course not to blame on the project itself, since I assume that it simply targets different use-cases and audiences.

I also researched in the meantime whether such a functionality could be implemented at all for the Gnome Shell and, more specifically, for its file browser. But the search and extension APIs would not even allow it or require many hacks.

j / k navigate · click thread line to collapse