undefined | Better HN

0 pointsdns_snek3y ago0 comments

I've done all of those things. It wasn't even operating on code but a problem described in plain English language with clear bullet points describing constraints of the problem. There were no context window limitations at play.

When it broke requirement or constraint "A", I reminded it about it. It apologized and formulated a new "solution" that broke constraint "B", when reminded about that, it apologized again and proceeded to break constraint "A" again.

The conversation went on for 20 odd turns where I tried to iterate and arrive at a solution - a custom algorithm to solve a problem it's likely never encountered before (at least in that form). It wasn't a particularly difficult problem, it just had many logical branches and steps it would have to reason about holistically. Instead it kept breaking one or more requirements that were clearly explained just 1 or 2 turns ago.

Your response is needlessly rude, perhaps the reality is that you haven't pushed it enough to discover severe limitations of GPT's logical reasoning capabilities.

> If this is truly your judgement then GPT4 already has better reasoning capabilities than you.

Wouldn't it stand to reason that someone with poor reasoning capabilities would be easily impressed by something or someone with better reasoning capabilities? ;)

0 comments

ilaksh3y ago

"The reason for that is simple - GPT lacks logical reasoning capabilities. It can 'fake' them on the surface level, it might even fake them really well for extremely common problems, but as soon as you prod deeper or tack on an extra requirement or two, it starts spinning in circles indefinitely."

Those unqualified statements are false. GPT4 may not have been able to pass your particularly complex reasoning task, but that does not mean that it can't reason.

My tone is really out of extreme frustration because misjudgement like you are displaying literally puts the fate of the human race at risk.

imtringued3y ago

Is this supposed to be some kind of joke? If someone disagrees with you, you whack them with rhetoric about existential risk? What are you? A cult member who has heard a prophecy from tech priests?

spaceman_20203y ago

Did you use GPT-3.5 or 4?

dns_snekOP3y ago

GPT-3.5 for this specific example, but I've been experimenting with GPT-4 via Bing Chat and it hasn't really changed my impression. I'm going to reevaluate once OpenAI releases ChatGPT + GPT-4 for free users.

spaceman_20203y ago

3.5 and 4 are really miles apart. The difference isn’t 10-20%. It’s 90%.

And I don’t know what they’re doing with Bing, but Bing Chat has substantially poorer results than GPT-4 via chatGPT.

If you’re in tech, you should shell out the 20 bucks to give it a try. Basic curiosity demands that at least I reckon.

j / k navigate · click thread line to collapse

0 comments

ilaksh3y ago

Those unqualified statements are false. GPT4 may not have been able to pass your particularly complex reasoning task, but that does not mean that it can't reason.

My tone is really out of extreme frustration because misjudgement like you are displaying literally puts the fate of the human race at risk.

imtringued3y ago

Is this supposed to be some kind of joke? If someone disagrees with you, you whack them with rhetoric about existential risk? What are you? A cult member who has heard a prophecy from tech priests?

spaceman_20203y ago

Did you use GPT-3.5 or 4?

dns_snekOP3y ago

spaceman_20203y ago

3.5 and 4 are really miles apart. The difference isn’t 10-20%. It’s 90%.

And I don’t know what they’re doing with Bing, but Bing Chat has substantially poorer results than GPT-4 via chatGPT.

If you’re in tech, you should shell out the 20 bucks to give it a try. Basic curiosity demands that at least I reckon.

j / k navigate · click thread line to collapse