Can you save on LLM tokens using images instead of text? (opens in new tab)

(pagewatch.ai)

48 pointslpellis4mo ago19 comments

19 comments

bikeshaving4mo ago

Does this mean we’ll finally get empirical proof for the aphorism “a picture is worth a thousand words”?

https://en.wikipedia.org/wiki/A_picture_is_worth_a_thousand_...

heltale4mo ago

I suppose it’s only worth 256 words at a time right now. ;)

https://arxiv.org/abs/2010.11929

estebarb4mo ago

The CALM paper https://shaochenze.github.io/blog/2025/CALM/ says it is possible to compress 4 tokens in a single embedding, so... image = 4×256=1024 words > 1000 words. QED

2 more replies

floodfx4mo ago

Why are completion tokens more with image prompts yet the text output was about the same?

cma4mo ago

Some multimodal models may have a hidden captioning step that may take completion tokens, others work on a fully native representation, and some do both I think.

Garlef4mo ago

"Thinking" Mode

nunodonato4mo ago

it doesn't say that anywhere.

ashed964mo ago

In my experience, LLMs tend to take noticeably longer to process images than text.

weird-eye-issue4mo ago

It has to get the image data first, basically just IO time before processing it

ashed964mo ago

IIRC there's pre-processing (embedding/tokenization?) before feeding images to LLMs?

Hit this issue optimizing LLM request times. Ending up lowering image resolution. Lost some accuracy but could bear that.

psadri4mo ago

I wonder if these stay in the prefix cache?

j / k navigate · click thread line to collapse

19 comments

bikeshaving4mo ago

Does this mean we’ll finally get empirical proof for the aphorism “a picture is worth a thousand words”?

https://en.wikipedia.org/wiki/A_picture_is_worth_a_thousand_...

heltale4mo ago

I suppose it’s only worth 256 words at a time right now. ;)

https://arxiv.org/abs/2010.11929

estebarb4mo ago

The CALM paper https://shaochenze.github.io/blog/2025/CALM/ says it is possible to compress 4 tokens in a single embedding, so... image = 4×256=1024 words > 1000 words. QED

2 more replies

floodfx4mo ago

Why are completion tokens more with image prompts yet the text output was about the same?

cma4mo ago

Some multimodal models may have a hidden captioning step that may take completion tokens, others work on a fully native representation, and some do both I think.

Garlef4mo ago

"Thinking" Mode

nunodonato4mo ago

it doesn't say that anywhere.

ashed964mo ago

In my experience, LLMs tend to take noticeably longer to process images than text.

weird-eye-issue4mo ago

It has to get the image data first, basically just IO time before processing it

ashed964mo ago

IIRC there's pre-processing (embedding/tokenization?) before feeding images to LLMs?

Hit this issue optimizing LLM request times. Ending up lowering image resolution. Lost some accuracy but could bear that.

psadri4mo ago

I wonder if these stay in the prefix cache?

j / k navigate · click thread line to collapse