undefined | Better HN

0 pointsfamouswaffles1y ago0 comments

>This model was trained to pass this test, it was trained heavily on the example questions, so it was a narrow technique.

You are allowed to train on the train set. That's the entire point of the test.

>We even have proof that it isn't AGI, since it scores horribly on ARC-AGI 2. It overfitted for this test.

Arc 2 does not even exist yet. All we have are "early signs", not that that would be proof of anything. Whether I believe the models are generally intelligent or not doesn't depend on ARC

0 comments

Jensson1y ago

> You are allowed to train on the train test. That's the entire point of the test.

Right, but by training on those test cases you are creating a narrow model. The whole point of training questions is to create narrow models, like all the models we did before.

famouswafflesOP1y ago

That doesn't make any sense. Training on the train set does not make the models capabilities narrow. Models are narrow when you can't train them to do anything else even if you wanted to.

You are not narrow for undergoing training and it's honestly kind of ridiculous to think so. Not even the ARC maintainers believe so.

Jensson1y ago

> Training on the train set does not make the models capabilities narrow

Humans didn't need to see the training set to pass this, the AI needing it means it is narrower than the humans, at least on these kind of tasks.

The system might be more general than previous models, but still not as general as humans, and the G in AGI typically means being as general as humans. We are moving towards more general models, but still not at the level where we call them AGI.

j / k navigate · click thread line to collapse

0 comments

Jensson1y ago

> You are allowed to train on the train test. That's the entire point of the test.

Right, but by training on those test cases you are creating a narrow model. The whole point of training questions is to create narrow models, like all the models we did before.

famouswafflesOP1y ago

That doesn't make any sense. Training on the train set does not make the models capabilities narrow. Models are narrow when you can't train them to do anything else even if you wanted to.

You are not narrow for undergoing training and it's honestly kind of ridiculous to think so. Not even the ARC maintainers believe so.

Jensson1y ago

> Training on the train set does not make the models capabilities narrow

Humans didn't need to see the training set to pass this, the AI needing it means it is narrower than the humans, at least on these kind of tasks.

j / k navigate · click thread line to collapse