undefined | Better HN

0 pointsterramex1y ago0 comments

Lvl 3 is supposed to support other models and providers in the future too. I hope it will support every server with simple, standard API so I can run self-hosted LLama 3 (or whatever will be released in next 6-12 months).

0 comments

hmottestad1y ago

Or Groq. They can do 1250 tokens/s with Llama 3 8B.

j / k navigate · click thread line to collapse

0 pointsterramex1y ago0 comments

Lvl 3 is supposed to support other models and providers in the future too. I hope it will support every server with simple, standard API so I can run self-hosted LLama 3 (or whatever will be released in next 6-12 months).

0 comments

hmottestad1y ago

Or Groq. They can do 1250 tokens/s with Llama 3 8B.

j / k navigate · click thread line to collapse