undefined | Better HN

0 pointsjulianeon2y ago0 comments

What I want is even simpler: just an API that you make requests to and receive answers back. Surprisingly hard to find, outside OpenAI that is.

0 comments

xanderatallah2y ago

We’re trying to do this at https://openrouter.ai

ingridpan2y ago

https://gradient.ai/ is doing that with llama2

worldsayshi2y ago

Looks really promising. I wonder if the similar pricing to OpenAI means that Gradient is also(?) bleeding money even if they get a good customer base. Or are these prices sustainable over time?

ingridpan2y ago

Good question, esp as Gradient fine-tuning is so much cheaper than Open AI's

1 more reply

BoorishBears2y ago

I'm not into open source LLMs in the slightest, and yet even I've trivially found tools to do what both you and the poster above you wanted

lmstudio actually does what both of you want: provides an easy GUI and serves up your model over a local endpoint that mirrors the OpenAI API.

There's just too much noise in terms of the tooling for LLMs, the solution is fewer higher quality solutions, not more solutions

MaKey2y ago

Oobabooga exposes an API.

j / k navigate · click thread line to collapse

0 comments

xanderatallah2y ago

We’re trying to do this at https://openrouter.ai

ingridpan2y ago

https://gradient.ai/ is doing that with llama2

worldsayshi2y ago

Looks really promising. I wonder if the similar pricing to OpenAI means that Gradient is also(?) bleeding money even if they get a good customer base. Or are these prices sustainable over time?

ingridpan2y ago

Good question, esp as Gradient fine-tuning is so much cheaper than Open AI's

1 more reply

BoorishBears2y ago

I'm not into open source LLMs in the slightest, and yet even I've trivially found tools to do what both you and the poster above you wanted

lmstudio actually does what both of you want: provides an easy GUI and serves up your model over a local endpoint that mirrors the OpenAI API.

There's just too much noise in terms of the tooling for LLMs, the solution is fewer higher quality solutions, not more solutions

MaKey2y ago

Oobabooga exposes an API.

j / k navigate · click thread line to collapse