> So “dog” vs image of dog would both translate to a primordial signal : identity representation and in the domain of frequency do the comparison and project a coordinate in the spatial sense and eventually those two nodes would more likely be triggered at the same time due to the likelihood of “dog” being next to image of dog when parsing information across future events.
That is how CLIP embeddings work and were trained to work.
Hugging Face transformers now has a get_image_features() and get_text_features() function for CLIP models to make getting the embeddings for different modalities easy: https://huggingface.co/docs/transformers/model_doc/clip#tran...
