undefined | Better HN

0 pointsboxed2y ago0 comments

If you are shown only the title of a coding problem and the site name where it's from, and you manage to solve it you are showing that you either cheated or knew the answer.

0 comments

Terretta2y ago

On the contrary, it could mean you were, to some percentage of success, able to guess what problem is, and then, to some multiplier percentage of success, solve it.

The key is, can you guess the problem from the title and the function name? I'd argue, sure, at least half the time?, why not...

famouswaffles2y ago

I mean sure, it memorized some of the answers. I'm not denying that. Clearly, it didn't memorize all of them.

boxedOP2y ago

When people say "oh look how amazing, it can solve programming problems!" when in fact it has only seen the models CHEAT, is an enormous problem.

For cases where finding the answer it's perfectly fine, but it's not fine for claims that it can code. There's a huge difference.

notnaut2y ago

It can generate never-before-seen strings of comprehensible language. It can react to the inherent logic embedded in words and text and provide a brute forced version of what a human could. That it can “solve” a problem only through “cheating” is an anthropomorphism that betrays the magic that is evident to anyone who has used these things.

FeepingCreature2y ago

I've seen it code on completely novel tasks, so I'm not sure what you're suggesting here. The model can unquestionably code.

raincole2y ago

Almost 2024 and people still can't accept that LLM can code...

1 more reply

famouswaffles2y ago

Okay... Funny how forcing it to not CHEAT did not increase apparent ability.

It can code and it has memorized some coding questions are not mutually exclusive.

2 more replies

j / k navigate · click thread line to collapse

0 comments

Terretta2y ago

On the contrary, it could mean you were, to some percentage of success, able to guess what problem is, and then, to some multiplier percentage of success, solve it.

The key is, can you guess the problem from the title and the function name? I'd argue, sure, at least half the time?, why not...

famouswaffles2y ago

I mean sure, it memorized some of the answers. I'm not denying that. Clearly, it didn't memorize all of them.

boxedOP2y ago

When people say "oh look how amazing, it can solve programming problems!" when in fact it has only seen the models CHEAT, is an enormous problem.

For cases where finding the answer it's perfectly fine, but it's not fine for claims that it can code. There's a huge difference.

notnaut2y ago

FeepingCreature2y ago

I've seen it code on completely novel tasks, so I'm not sure what you're suggesting here. The model can unquestionably code.

raincole2y ago

Almost 2024 and people still can't accept that LLM can code...

1 more reply

famouswaffles2y ago

Okay... Funny how forcing it to not CHEAT did not increase apparent ability.

It can code and it has memorized some coding questions are not mutually exclusive.

2 more replies

j / k navigate · click thread line to collapse