undefined | Better HN

0 pointstimschmidt10mo ago0 comments

The weights seem much more like a binary to me, the training pipeline the compiler, and the training dataset the source.

0 comments

jumski10mo ago

Come here to write this - perfect analogy!

reedciccio10mo ago

It's very imperfect analogy though these things can't be rebuilt "from scratch" like a program, the training process doesn't seem to be replicable anyway. Nonetheless, full data disclosure is necessary, according to the result of the years-long consultation led by the Open Source Initiative https://opensource.org/ai

timschmidtOP10mo ago

> the training process doesn't seem to be replicable anyway

The training process is fully deterministic. It's just an algorithm. Feed the same data in and you'll get the same weights out.

If you're speaking about the computational cost, it used to be that way for compilers too. Give it 20 years and you'll be able to train one of today's models on your phone.

5 more replies

j / k navigate · click thread line to collapse