undefined | Better HN

0 pointsbehnamoh21d ago0 comments

> which includes hard evidence that you’re wrong?

You should already know what to ask to extract the answer OpenAI claims gpt-5.2-pro gave them.

Then you should be lucky to get an answer that makes sense.

Then you should already know how to verify the model's response.

Only after all these steps should you cherry-pick the one-in-a-million successful response to feature on your website.

And finally, you should prove that the answer didn't already exist in the training data. It's highly likely that the problem was solved before and the model picked that up. I have yet to see a genuinely novel discovery these models can produce.

* I'm an LLM researcher, but that doesn't mean I should close my eyes to the unjustified hype around language models.

0 comments

MajimasEyepatch21d ago

According to the post, this result was first derived for gluons in a previous paper. That paper was provided to the model as context, and then the model was asked to derive an analogous result for gravitons, which presumably has not been done before. The authors claim it would have taken "considerable time" for human experts to derive the graviton result.

I don't see any reason to believe that this exact problem was solved before in the training data, but it's definitely an incremental result based on a very similar problem that the model had seen before.

gus_massa20d ago

Gluons have spin 1 and Gravitons [1] have spin 2. This adds a lot of subtle difference in the calculations. Some are obvious like replacing a 1 with a 2 (or a 1^2 with a 2^2). Some are super tricky:

For example if you have two gluons, you apply the rules of sum of spin and get

  1 × 1 = 2 + 1 + 0

(They can be coupled in the same direction and get 2, or the oposite direction and get 0, or something in between and get 1.)

But for gravitons, the rules are

  2 × 2 = 4 + 3 + 2 + 1 + 0

(They can be coupled in the same direction and get 4, or the oposite direction and get 0, or something in between and get more cases in between like 2, but also 1 or 3.)

If you want to make give physicist nightmares and make mathematician cry, a tiny part of the details are in https://en.wikipedia.org/wiki/Table_of_Clebsch%E2%80%93Gorda... and https://en.wikipedia.org/wiki/Table_of_Clebsch%E2%80%93Gorda...

In conclusion, I'm not sure how difficult is to do the conversion from gluons to gravitons, but I'd recommend to run away.

[1] Assuming they exist.

JProthero21d ago

>It's highly likely that the problem was solved before and the model picked that up.

If you can demonstrate that, I would put it to Strominger and his colleagues, and I imagine they would be obligated to cite your contribution in the peer-reviewed publication.

behnamohOP21d ago

> If you can demonstrate that, I would put it to Strominger and his colleagues, and I imagine they would be obligated to cite your contribution in the peer-reviewed publication.

There's one little problem: OpenAI isn't actually open and doesn't reveal which dataset they used for training.

ChrisClark21d ago

Or at least, please explain the results found, and how they are already known. Any papers of your own in the field?

JProthero21d ago

This shouldn't prevent anyone from finding and reporting a similar pre-existing result in the literature.

ChrisClark21d ago

But you yourself said they found the answer already online, so go find it, prove it

1 more reply

j / k navigate · click thread line to collapse

0 pointsbehnamoh21d ago0 comments

> which includes hard evidence that you’re wrong?

You should already know what to ask to extract the answer OpenAI claims gpt-5.2-pro gave them.

Then you should be lucky to get an answer that makes sense.

Then you should already know how to verify the model's response.

Only after all these steps should you cherry-pick the one-in-a-million successful response to feature on your website.

* I'm an LLM researcher, but that doesn't mean I should close my eyes to the unjustified hype around language models.

0 comments

MajimasEyepatch21d ago

gus_massa20d ago

Gluons have spin 1 and Gravitons [1] have spin 2. This adds a lot of subtle difference in the calculations. Some are obvious like replacing a 1 with a 2 (or a 1^2 with a 2^2). Some are super tricky:

For example if you have two gluons, you apply the rules of sum of spin and get

  1 × 1 = 2 + 1 + 0

(They can be coupled in the same direction and get 2, or the oposite direction and get 0, or something in between and get 1.)

But for gravitons, the rules are

  2 × 2 = 4 + 3 + 2 + 1 + 0

(They can be coupled in the same direction and get 4, or the oposite direction and get 0, or something in between and get more cases in between like 2, but also 1 or 3.)

In conclusion, I'm not sure how difficult is to do the conversion from gluons to gravitons, but I'd recommend to run away.

[1] Assuming they exist.

JProthero21d ago

>It's highly likely that the problem was solved before and the model picked that up.

If you can demonstrate that, I would put it to Strominger and his colleagues, and I imagine they would be obligated to cite your contribution in the peer-reviewed publication.

behnamohOP21d ago

> If you can demonstrate that, I would put it to Strominger and his colleagues, and I imagine they would be obligated to cite your contribution in the peer-reviewed publication.

There's one little problem: OpenAI isn't actually open and doesn't reveal which dataset they used for training.

ChrisClark21d ago

Or at least, please explain the results found, and how they are already known. Any papers of your own in the field?

JProthero21d ago

This shouldn't prevent anyone from finding and reporting a similar pre-existing result in the literature.

ChrisClark21d ago

But you yourself said they found the answer already online, so go find it, prove it

1 more reply

j / k navigate · click thread line to collapse