undefined | Better HN

basically in my testing really felt that gpt5 was "using tools to think" rather than just "using tools". it gets very powerful when coding long horizon tasks (a separate post i'm publishing later).

to give one substantive example, in my developer beta (they will release the video in a bit) i put it to a task that claude code had been stuck on for the last week - same prompts - and it just added logging to instrument some of the failures that we were seeing and - from the logs that it added and asked me to rerun - figured out the solve.

diggan7mo ago

Was just skimming along that review, while watching the live-stream, where they just mentioned how much better at writing prose GPT-5 is, while I skimmed across:

> It’s actually worse at writing than GPT-4.5

Sounds like we need to wait a bit for the dust to settle before one can trust anything one hears/reads :)

dudeinhawaii7mo ago

This is why you need to have your own set of personal benchmarks. I have a few short stories that I have models continue (ones I wrote ages ago in my youth) or refactor. Some models are fantastic at writing but miss key details or enmesh them (Claude). Some are terrible writers at higher reasoning (o3). Some are decent writers but tend to provide very short outputs (gpt-4o). For my personal benchmarks Gemini 2.5 Pro has always generated the most compelling writing that _also_ sticks to the world/script -- and sometimes surprises me by having characters react in ways that I hadn't considered but are consistent with their "worldview" as presented by the context (usually a world guide).

I find coding to be harder to benchmark because there are so many ways to write the same solution. A "correct" solution may be terrible in another context due to loss of speed, security, etc.

2 more replies

tough7mo ago

well it's difficult to trust the people selling it in the first place. They're too biased to not lie

It's hard to make a man understand something standing between them and their salary

1 more reply

yen2237mo ago

It has been pointed out, but GPT-5 is really five different models with wildly different capabilities under the hood. Which model get picked for the task at hand is, for now, not deterministic.

swyxOP7mo ago

better than 4o but worse than 4.5 is internally consistent. and ofc writing is extremely multidimensional.

1 more reply

billmalarky7mo ago

Hi Swyx I always appreciate your insights, something you wrote really resonated with a personal theory I've been developing:

>"While I never use AI for personal writing (because I have a strong belief in writing to think)"

The optimal AI productivity process is starting to look like:

AI Generates > Human Validates > Loop

Yet cognitive generation is how humans learn and develop cognitive strength, as well as how they maintain such strength.

Similar to how physical activity is how muscles/bone density/etc grow, and how body tissues maintain.

Physical technology freed us from hard physical labor that kept our bodies in shape -- at a cost of physical atrophy.

AI seems to have a similar effect for our minds. AI will accelerate our cognitive productivity, and allow for cognitive convenience -- at a cost of cognitive atrophy.

At present we must be intentional about building/maintaining physical strength (dedicated strength training, cardio, etc).

Soon we will need to be intentional about building/maintaining cognitive strength.

I suspect the workday/week of the future will be split on AI-on-a-leash work for optimal productivity, with carve-outs for dedicated AI-enhanced-learning solely for building/maintaining cognitive health (where productivity is not the goal, building/maintaining cognition is). Similar to how we carve out time for working out.

What are your thoughts on this? Based on what you wrote above, it seems you have similar feelings?

Is there a name for this theory?

If not can you coin one? You're great at that :)

tailspin20197mo ago

This is very interesting - I like the way you’ve explained this.

The parallel with “intentionally working out to maintain physical strength” is extremely helpful as an analogy to communicate this concept.

That’s exactly what we might be faced with… cognitive atrophy…

It’s arguably already started, and is accelerating!

swyxOP7mo ago

thanks very much :)

problem with your theory is it bundles 2-3 steps which each could be their own theses

suggest you nail those down before building up to a general bundle (or mental model/framework)

2 more replies

lyxell7mo ago

”I think GPT-5 is the closest to AGI we’ve ever been”

Sorry, but this sounds like overly sensational marketing speak and just leaves a bad taste in the mouth for me.

Aurornis7mo ago

I found a Hacker News thread via Google a few days ago. One of the top comments was from someone describing their RAG architecture and a certain technique (my search term). The comment boasted that their system was so good it that their team thought they created something close to AGI.

Then I noticed the date on the comment: 2023.

Technically, every advancement in the space is “the closest to AGI that we’ve ever been”. It’s technically correct, since we’re not moving backward. It’s just not a very meaningful statement.

2 more replies

Fargren7mo ago

AGI, like AI before it, has been coopted into a marketing term. Most of the time, outside of sci-fi, what people mean when they say AGI is "a profitable LLM".

In the words OpenAI: “AGI is defined as highly autonomous systems that outperform humans at most economically valuable work”

2 more replies

martin_a7mo ago

Same marketing BS like "the best iPhone ever!". Well, duh, if your new version (of hardware/software) isn't better, what the deal then?

thierrydamiba7mo ago

Great writeup. I particularly like the idea of splitting your tools into the four buckets.

1)Internal Retrieval

2)Web Search

3)Code Interpreter

4)Actions

How did you come up with this idea?

swyxOP7mo ago

ive been pushing the idea of "the Big 3" tools (https://news.smol.ai/issues/25-05-27-mistral-agents and https://www.latent.space/p/agent) and Ben added a 4th

yen2237mo ago

Re the substantive example, doesn't Claude Code already do this? When I'm vibe-coding scripts or even mobile apps cc can be pretty aggressive about adding targeted logging to solve specific issues.

swyxOP7mo ago

well idk for me it didnt and gpt5 did haha

elros7mo ago

https://www.youtube.com/watch?v=0Uu_VJeVVfo

slowmovintarget7mo ago

The coding examples link returns a 404.

owlninja7mo ago

https://github.com/openai/gpt-5-coding-examples

perching_aix7mo ago

Aaand hugged to death.

edit:

livestream here: https://www.youtube.com/live/0Uu_VJeVVfo

j / k navigate · click thread line to collapse

0 comments

nabla97mo ago

Holy misleading graph Batman/Altman!

Academic benchmark score improves only 5% but they make the bar 50% higher.

ambicapter7mo ago

Which graph? There are dozens of graph on the page.

swyxOP7mo ago

they had a bad graph on stream which they have since fixed. wouldnt get too upset about a simple error

swyxOP7mo ago

our hands on review: https://www.latent.space/p/gpt-5-review

basically in my testing really felt that gpt5 was "using tools to think" rather than just "using tools". it gets very powerful when coding long horizon tasks (a separate post i'm publishing later).

diggan7mo ago

Was just skimming along that review, while watching the live-stream, where they just mentioned how much better at writing prose GPT-5 is, while I skimmed across:

> It’s actually worse at writing than GPT-4.5

Sounds like we need to wait a bit for the dust to settle before one can trust anything one hears/reads :)

dudeinhawaii7mo ago

I find coding to be harder to benchmark because there are so many ways to write the same solution. A "correct" solution may be terrible in another context due to loss of speed, security, etc.

2 more replies

tough7mo ago

well it's difficult to trust the people selling it in the first place. They're too biased to not lie

It's hard to make a man understand something standing between them and their salary

1 more reply

yen2237mo ago

It has been pointed out, but GPT-5 is really five different models with wildly different capabilities under the hood. Which model get picked for the task at hand is, for now, not deterministic.

swyxOP7mo ago

better than 4o but worse than 4.5 is internally consistent. and ofc writing is extremely multidimensional.

1 more reply

billmalarky7mo ago

Hi Swyx I always appreciate your insights, something you wrote really resonated with a personal theory I've been developing:

>"While I never use AI for personal writing (because I have a strong belief in writing to think)"

The optimal AI productivity process is starting to look like:

AI Generates > Human Validates > Loop

Yet cognitive generation is how humans learn and develop cognitive strength, as well as how they maintain such strength.

Similar to how physical activity is how muscles/bone density/etc grow, and how body tissues maintain.

Physical technology freed us from hard physical labor that kept our bodies in shape -- at a cost of physical atrophy.

AI seems to have a similar effect for our minds. AI will accelerate our cognitive productivity, and allow for cognitive convenience -- at a cost of cognitive atrophy.

At present we must be intentional about building/maintaining physical strength (dedicated strength training, cardio, etc).

Soon we will need to be intentional about building/maintaining cognitive strength.

What are your thoughts on this? Based on what you wrote above, it seems you have similar feelings?

Is there a name for this theory?

If not can you coin one? You're great at that :)

tailspin20197mo ago

This is very interesting - I like the way you’ve explained this.

The parallel with “intentionally working out to maintain physical strength” is extremely helpful as an analogy to communicate this concept.

That’s exactly what we might be faced with… cognitive atrophy…

It’s arguably already started, and is accelerating!

swyxOP7mo ago

thanks very much :)

problem with your theory is it bundles 2-3 steps which each could be their own theses

suggest you nail those down before building up to a general bundle (or mental model/framework)

2 more replies

lyxell7mo ago

”I think GPT-5 is the closest to AGI we’ve ever been”

Sorry, but this sounds like overly sensational marketing speak and just leaves a bad taste in the mouth for me.

Aurornis7mo ago

Then I noticed the date on the comment: 2023.

2 more replies

Fargren7mo ago

AGI, like AI before it, has been coopted into a marketing term. Most of the time, outside of sci-fi, what people mean when they say AGI is "a profitable LLM".

In the words OpenAI: “AGI is defined as highly autonomous systems that outperform humans at most economically valuable work”

2 more replies

martin_a7mo ago

Same marketing BS like "the best iPhone ever!". Well, duh, if your new version (of hardware/software) isn't better, what the deal then?

thierrydamiba7mo ago

Great writeup. I particularly like the idea of splitting your tools into the four buckets.

1)Internal Retrieval

2)Web Search

3)Code Interpreter

4)Actions

How did you come up with this idea?

swyxOP7mo ago

ive been pushing the idea of "the Big 3" tools (https://news.smol.ai/issues/25-05-27-mistral-agents and https://www.latent.space/p/agent) and Ben added a 4th

yen2237mo ago

Re the substantive example, doesn't Claude Code already do this? When I'm vibe-coding scripts or even mobile apps cc can be pretty aggressive about adding targeted logging to solve specific issues.

swyxOP7mo ago

well idk for me it didnt and gpt5 did haha

elros7mo ago

https://www.youtube.com/watch?v=0Uu_VJeVVfo

slowmovintarget7mo ago

The coding examples link returns a 404.

owlninja7mo ago

https://github.com/openai/gpt-5-coding-examples

perching_aix7mo ago

Aaand hugged to death.

edit:

livestream here: https://www.youtube.com/live/0Uu_VJeVVfo

j / k navigate · click thread line to collapse