undefined | Better HN

0 pointshomebrewer1y ago0 comments

Not really, there' a world of difference between how every model I've tried¹ handles English vs my own native language². It's especially noticeable with smaller models, which produce coherent English, but completely break down in other languages: spewing out incorrect grammar, mixing words from other languages where they make zero sense, etc.

¹: which is dozens of them at this point, open and not.

²: with lots of speakers and training data.

0 comments

gunian1y ago

Would finetuning fix this? If they are statstical parrots it would seem like it wouldn't

ioulaum1y ago

Probably tough to fix small models with fine-tuning... You'd likely need to train them with enough data from languages (and maybe translations between them) that they need to support, so that they can connect ideas in different languages properly.

Separately, the fact that they can translate between languages where they have never seen any translation data in their inputs shows that they have internal models of the world and language, that go beyond what one might expect from a statistical parrot.

They do have world understanding - perhaps limited some by the fact that their input data may not cover a lot of everyday things.

j / k navigate · click thread line to collapse