Part of the concern might be that visual reasoning problems are overrepresented in ARC in the space of all abstract reasoning problems.
It’s similar to how chess problems are technically reasoning problems but they are not representative of general reasoning.