undefined | Better HN

0 pointsnecovek4d ago0 comments

You missed the core of my point: humans operate, including in the real world, on much less training data. Give a human a shopping cart and ask them to push it backwards, and they'll figure it out in a few minutes even if they've never done it before.

This is the bit that's missing that LLMs do approximate amazingly well through sheer training set size, but in my opinion, it puts a cap on what novel things they can achieve in comparison with humans.

To me, I've thought about a related "invention space" before: with us creating software to solve many problems people are facing, why are there not any perfect solutions for any problem (running a cafe? a CNC machine? ...), and we always need more software built to cover one small (novel?) change for a particular owner?

The world space is just so large that you need whatever this intelligence is humans (and animals) have to navigate it successfully — but LLMs do not intrinsically.

Whether they can be so large that it does not matter in 99.99% of cases is to be seen.

0 comments

virgildotcodes4d ago

> You missed the core of my point: humans operate, including in the real world, on much less training data.

I very specifically addressed this in my response to you. How much training data is contained in 16 waking hours of navigating the world fusing all sensory data, never mind data being simultaneously generated within the mind while this is all going on, from birth til death? From birth til pushing that shopping cart?

Far, far more than in all the training datasets being used for AI.

I also addressed this again in my reply to the sibling comment.

People tend to discount how much data humans have passing through their minds 24/7.

A human isn’t born in a vacuum as a fully formed adult and dropped into the shopping cart navigation problem.

A human has had far, far more training data fed into it that contains all the pieces necessary to translate to pushing a shopping cart when first seeing it, than a machine learning model which has been fed 1 million videos of a robot pushing a shopping cart.

j / k navigate · click thread line to collapse