undefined | Better HN

0 pointsCamperBob21y ago0 comments

Can you point me towards a citation for the 25% figure? I'm seeing numbers like 96% ( https://paperswithcode.com/sota/common-sense-reasoning-on-ar... ) but I'm guessing that's just for a subset of the larger class of questions.

Also, are you familiar with this study? What are your thoughts on it? https://www.esmo.org/newsroom/press-and-media-hub/esmo-media... Seems like a valid case where AI is competitive with skilled humans at object/image recognition.

0 comments

andoando1y ago

It seems theres multiple things by the name ARC. There is one by AI2 which is a text based science questions/word problems. The one Im referring to is this https://lab42.global/arc/

https://lab42.global/arcathon/leaderboard/

https://openreview.net/forum?id=E8m8oySvPJ

As to the study, I have the same objection as the radiology one. This isnt about object recognition and certainly not spatial reasoning, its the ability to predict cancer based on presence of visual features.

The "object recognition" part of this is super simple. Its a single, mostly 2D object in more or less the same angle, and the AI is trained on detecting just this.

CamperBob2OP1y ago

The "object recognition" part of this is super simple. Its a single, mostly 2D object in more or less the same angle, and the AI is trained on detecting just this.

And yet it outperforms human dermatologists.

j / k navigate · click thread line to collapse