undefined | Better HN

0 pointsbmcahren17d ago0 comments

LLMs do not encode nor encrypt their training data. The fact they can recite training data is a defect not a default. You can understand this more simply by calculating the model size as an inverse of a fantasy compression algorithm that is 50% better than SOTA. You'll find you'd still be missing 80-90% of the training data even if it were as much of a stochastic parrot as you may be implying. The outputs of AI are not derivative just because they saw training data including the original library.

Then onto prompting: 'He fed only the API and (his) test suite to Claude'

This is Google v Oracle all over again - are APIs copyrightable?

0 comments

thunderfork17d ago

I find the "compression" argument not very strong, both because copyright still applies to (very) lossy codecs (e.g. your 16kbps Opus file of Thriller infringes, even if the original 192khz/32bit wav file was 12,000kbps), and because copyright still applies to transformed derivative works (a tiny midi file of Thriller might still be enough for the Jackson's label to get you)

satvikpendem17d ago

> This is Google v Oracle all over again - are APIs copyrightable?

Yes this is the best way to ask the question. If I take a public facing API and reimplement everything, whether it's by human or machine, it should be sufficient. After all, that's what Google did, and it's not like their engineers never read a single line of the Java source code. Even in "clean room" implementations, a human might still have remembered or recalled a previous implementation of some function they had encountered before.

azakai17d ago

> LLMs do not encode nor encrypt their training data. The fact they can recite training data is a defect not a default.

About this specific point, it is unclear how much of a defect memorization actually is - there are also reasons to see it as necessary for effective learning. This link explains it well:

https://infinitefaculty.substack.com/p/memorization-vs-gener...

tw198417d ago

> This is Google v Oracle all over again - are APIs copyrightable?

No, it is completely different.

Claude was trained on chardet, anything built by Claude would fail the clean-room reimplementation test.

LegionMammal97817d ago

"The clean-room reimplementation test" isn't a legal standard, it's a particular strategy used by would-be defendants to clearly meet the standard of "is the new work free of copyrightable expression from the original work".

j / k navigate · click thread line to collapse