That's more or less how it works in English too... is it really that bad? Spanish and French also have special words for the teens, it's quite common in Europe and I've never heard of it being that difficult to learn.
Some languages have it much worse... I think Hindi is pretty irregular up to 100 .