undefined | Better HN

0 pointssailingparrot4y ago0 comments

OpenAI offering is bringing model quality and lantecy that you are not going to get elsewhere.

GPT-J is 6B, the biggest version of GPT-3 available with the API is 175B, those two models are nothing alike in term of quality. Even the 6B version of GPT-3 (curie) is better quality than GPT-J IIRC.

So if you need better quality than GPT-J there are basically no alternatives.

And even if 6B is enough for you, but you care about latency, OpenAI has the best inference runtime by far, and you are not going to replicate that on your cloud/bare-metal. Unless your scenario specifically benefits from your API and your other servives to be colocated.

Edit: I forgot about finetuning. OpenAI gives you the ability to finetune all of their variants. Maybe you already have the knowledge to finetune something like GPT-J yourself, but I would guess that most potential users of the API do not have it.

0 comments

Filligree4y ago

Yeah, that’s great, but they won’t let me use it as co-writer for my fiction.

It turns out that this is by far what these models are best at. I am, without exaggeration, ten times faster at writing with AI assistance than without. I’m also learning faster; getting instant tips on how something might be phrased is invaluable, even if I go on to rewrite it.

NovelAI allows this, and provides an easy mechanism for fine-tuning as well as a number of excellent fine-tuned models I can choose between.

OpenAI thinks I can’t be trusted with the technology, because I might… what? Cause them bad PR? Well, I’m sorry my SF has a little violence in it sometimes! Good luck finding a book that doesn’t.

So I’m not going to use them, and I’ll take every opportunity to recommend against anyone else doing so. You’re going to regret it.

truthsayer714y ago

any chance on how one might one get a glimpse of what you mean or get started in this bit : "It turns out that this is by far what these models are best at. I am, without exaggeration, ten times faster at writing with AI assistance than without. I’m also learning faster; getting instant tips on how something might be phrased is invaluable, even if I go on to rewrite it."

harpersealtako4y ago

I know that at least on most common performance benchmarks these claims are measurably false (gpt-j has a number of key performance improvements to the equivalently sized models), and in particular code generation for 6B is very clearly a strength of GPT-J even above the 275B GPT-3. None of that is very controvertial as far as I can tell.

But even just subjectively, having used GPT-3 based AI Dungeon for fiction writing in the past until OpenAI forced them to censor outputs, effectively smothering it in its sleep, and now using NovelAI, which is a GPT-J-6B based alternative, EleutherAI's model is clearly a step above GPT-3 in most practical applications. And this isn't even getting into OpenAI's privacy/control issues.

sailingparrotOP4y ago

> I know that at least on most common performance benchmarks these claims are measurably false

What "these claims" are you referring to? It seems you are taking issue with only one specific claim of my comment, namely than GPT-3 6B is better quality than GPT-J 6B. Evaluations run by Eulether folks are available here [1] and I have the opposite subjective experience from you.

But even assuming I'm wrong, that doesn't change at all the substance of what I am saying: If you need better quality than GPT-J, then GPT-3 (DaVicing, 175B) is your only option.

And if you care about latency, last time I checked (6 months ago) OpenAI was miles ahead.

> in particular code generation for 6B is very clearly a strength of GPT-J even above the 275B GPT-3.

Note on that: ~8% of GPT-J training data is GitHub code, that's not the case for GPT-3 hence the difference. But OpenAI has a separate model avaiable in their API called Codex that is specifically tailored for code generation (also the model behind GH copilot) and that is much much better than GPT-J: Even the 300M params version of Codex outperforms it [2], and the API gives you access to a 12B version.

I'm not trying to sell you OpenAI's API though, it has indeed pretty severe limitations, I'm only saying there are real reasons that people might want to use it contrary to what the comment I was replying to was saying, and just replicating what they do isn't exactly a walk in the park.

[1]: https://arankomatsuzaki.wordpress.com/2021/06/04/gpt-j/ [2]: https://arxiv.org/pdf/2107.03374.pdf

j / k navigate · click thread line to collapse