> Your approach leads to calling them all sticks.
No, it leads to calling them all xylophones. Or calling the metal ones metal xylophones (which I do only if I remember that "xylophone" is the wrong term) and the wooden ones wooden xylophones (which I don't want to do but am stupid enough to do).
> I knew both words when I was about 5, as we had both instruments at the back of the classroom.
Even so, remembering that different words can exist for different instruments with the same general shape but differing other characteristics (wood vs. metal bar material, in this case) is a waste of mental effort for people who have reliable eyes (like me) and don't practice those instruments (also me).
I call the piano the "piano", even though the more functionally accurate term (also more historically accurate) is pianoforte. Even though I know this, I'm still going to call the piano a piano, because I don't want to bother to remember that the word "pianoforte" exists until someone asks me "why is it called piano?" As for tomato, I've learned to accept that a tomato can be a "fruit" in some contexts and a "vegetable" in some contexts (not mutually exclusive).