undefined | Better HN

0 pointskadushka1y ago0 comments

I don’t know which text diffusion models you’re talking about, the latest and greatest is this one: https://arxiv.org/abs/2502.09992 and it’s extremely slow – couple of orders of magnitude slower than a regular LLM, mainly because it does not support KV caching, and requires many full sequence processing steps per token.

0 comments

janalsncm1y ago

I’m not familiar with that paper but it would probably be best to compare speeds with an unoptimized transformer decoder. The Vaswani paper came out 8 years ago so implementations will be pretty highly optimized at this point.

On the other hand if there was a theoretical reason why text diffusion models could never be faster than autoregressive transformers it would be notable.

kadushkaOP1y ago

There’s not enough improvement over regular LLMs to motivate optimization effort. Recall that the original transformer was well received because it was fast and scalable compared to RNNs.

j / k navigate · click thread line to collapse

0 pointskadushka1y ago0 comments

0 comments

janalsncm1y ago

On the other hand if there was a theoretical reason why text diffusion models could never be faster than autoregressive transformers it would be notable.

kadushkaOP1y ago

There’s not enough improvement over regular LLMs to motivate optimization effort. Recall that the original transformer was well received because it was fast and scalable compared to RNNs.

j / k navigate · click thread line to collapse