undefined | Better HN

0 pointscelestialcheese3y ago0 comments

32k context is absolutely huge. There's all sorts of techniques for summarizing large documents down to get into 4k right now with 3.5, but it's incredibly lossy.

But boy, not cheap at all - $2 per api call on a 32k token document + whatever the output.

gpt-3.5-turbo is going to be around for a long time. At this price, your use case is going to need to be replacing a large cost center. Which based on their released results on common benchmarks, is absolutely going to happen.

0 comments

ren_engineer3y ago

3.5 might be their loss leader to keep people in their ecosystem for most use cases and to create a unique wall in terms of the training dataset they made via ChatGPT, GPT-4 they must be confident enough that nobody can compete that they can charge much more. Plus the use cases it can be used to replace cost centers like you said

j / k navigate · click thread line to collapse

0 pointscelestialcheese3y ago0 comments

32k context is absolutely huge. There's all sorts of techniques for summarizing large documents down to get into 4k right now with 3.5, but it's incredibly lossy.

But boy, not cheap at all - $2 per api call on a 32k token document + whatever the output.

0 comments

ren_engineer3y ago

j / k navigate · click thread line to collapse