> By your logic, many video games have been “open source” for decades because tools were accessible to modify the binary files in certain ways. We lacked the source code, but that’s just “part of the development process”, and maybe parts like comments were lost during the compiling process, but really, why isn’t it open source?

Because the binaries were not licensed under a FOSS license?

Also, as I note in another comment [0], source code is the preferred form of a piece of software for making modifications to it. The same cannot be said about the training data, because getting from that to weights costs hundreds of millions of dollars in compute. Even the original companies prefer to fine-tune their existing foundation models for as long as possible, rather than starting over from training data alone.

> In short, your argument doesn’t work because source code is to binaries as training data is to MLMs.

I disagree. Training data does not allow me to recreate an LLM. It might allow Jeff Bezos to recreate an LLM, but not me. But weights allow me to modify it, embed it, and fine tune it.

The weights are all that really matters for practical modification in the real world, because in the real world people don't want to spend hundreds of millions to "recompile" Llama when someone already did that, any more than people want to rewrite the Linux kernel from scratch based on whiteboard sketches and mailing list discussions.

[0] https://news.ycombinator.com/item?id=41951945