4k

This is more like a thought experiment and I am hoping to learn the other developments in the LLM inference space that are not strictly GPUs.

Conditions:

1. You want a solution for LLM inference and LLM inference only. You don't care about any other general or special purpose computing

2. The solution can use any kind of hardware you want

3. Your only goal is to maximize the (inference speed) X (model size) for 70b+ models

4. You're allowed to build this with tech mostly likely available by end of 2025.

How do you do it?

I am talking about the 3 providers I know of which claim super fast inference: Groq, Cerebras and Sambanova. Every one of those claim extremely fast multi hundred tokens per second inference speeds on reasonably large models. Every one of those also have a chat demo on their website which seems to confirm their proposed numbers

However, for many months now, each of those providers have literally the same API page where only the Free option with low rates is available. Everything else is "Coming Soon". No updates, no dates, no estimates, nothing.

Come to think of it, there is not a single good inference provider in the whole open source models space that offers a paid API without throttle in over 50 tps consistently. There's money to be made here and surprisingly nobody is doing it aggressively

Read about this approach at a few places: Have only a REST API as a backend and do everything else in the frontend JS framework.

I don't know enough about web architectures to evaluate this approach. Is it possible to develop a site/app with this approach regardless of complexity of the app, or is this only suitable for certain scenarios?

Any help would be greatly appreciated.

I am a backend dev and know little of web dev.

Assuming you are designing a web app, and corresponding mobile apps in 2018 from scratch, how would you go about structuring it? What would be the best practices? What tools you'd choose?

Since it's hard to discuss it without a concrete example, let's say you're designing a Quora or a microblogging site.

P.S. I am personally fond of golang.

There was a proper discussion on it about two years ago. Recently a similar thread was posted, but not to many response.

I do not work at any of the above. I work at a fintech company and make 80k + Bonus (Europe) with no equity (12 years experience, senior dev).

I have used gnuplot for basic tasks in the past. This time I am trying to generate graphs that'll be sent to the customers and it's important that they look neat and aesthetically pleasant.

Spent last 3 hours on different ways to do that. Checked out some scripts etc to make gnuplot output nicer looking, but it just doesn't seem to compare with the output of some of the js libraries (D3/charts). I wonder if there is a command line way to generate slick graphs that are visually every bit as appealing as using libraries?

TL;DR: Anyone know a way to get stylish looking graphs generated from command line on linux?

I am trying to validate if hosting people's personal email servers would be a viable business.

In short, you just buy a domain and point it to the service's name server. Everything else is taken care of including updates etc. What you get is your private email server, not a mailbox on a server, but a server that's 100% yours. Would you pay for something like this?