danielEM

If you're a software developer and using coding agents "all the time" for your work please share here your bills and APIs you're working with.

I was easily able to burn $1 a minute with deepinfra. But I have just tried it only a couple of times.

Can't find "apple to apple" comparison on performance on QWQ 32b (4bit), can anyone help me with decision on which solution to pick?

From what I dig so far it looks like dual Arc A770 is supported by llama.cpp. And saw some reports that llama.cpp on top of IPEX-LLM is fastest way for inference on intel card.

On the other end there is more expensive 7900 XTX on which AMD claims (Jan '25) that inference is faster than on 4090.

So - what is the state of the art as of today, how does one compare to another (apple to apple)? What is tokens/s diff?