undefined | Better HN

0 pointscyanydeez1y ago0 comments

Theres never going to be quality data at tge scale they need it.

At best, theres a slow march to incremental improvements that look exactly like how human culture developed knowledge.

And all the downsides will remain, the same way people, despite hundreds of good sourxes of info still prefer garbage.

0 comments

Video data is still very much untapped and likely to unlock a step function worth of data. Current image-language models are trained mostly on {image, caption} pairs with a bit of extra fine tuning

KPGv21y ago

Do you think it matters that there's orders of magnitude less video (and audio) data than text data?

jerpint1y ago

I’m not sure I agree, text is full of compressed information but lacks all of the visual cues we all use to navigate and understand our world. Video data also has temporal components which text is really bad at.

mike_hearn1y ago

OpenAI has been training on YouTube for years.

luma1y ago

Nobody close the the edge of this tech seems to believe that this is true, and the 4o release suggests synthetic approaches work well.

cyanydeezOP1y ago

Beliefs and outcomes are different things.

j / k navigate · click thread line to collapse

0 pointscyanydeez1y ago0 comments

Theres never going to be quality data at tge scale they need it.

At best, theres a slow march to incremental improvements that look exactly like how human culture developed knowledge.

And all the downsides will remain, the same way people, despite hundreds of good sourxes of info still prefer garbage.

0 comments

jerpint1y ago

Video data is still very much untapped and likely to unlock a step function worth of data. Current image-language models are trained mostly on {image, caption} pairs with a bit of extra fine tuning

KPGv21y ago

Do you think it matters that there's orders of magnitude less video (and audio) data than text data?

jerpint1y ago

mike_hearn1y ago

OpenAI has been training on YouTube for years.

luma1y ago

Nobody close the the edge of this tech seems to believe that this is true, and the 4o release suggests synthetic approaches work well.

cyanydeezOP1y ago

Beliefs and outcomes are different things.

j / k navigate · click thread line to collapse