One problem with your explanation is that the muon and the tau (and the pion as a decay product of the tau) all decay into electrons, neutrinos and photons, which would suggest that neither muon or tau are fundamental.

This would put the fundamental leptons being only the electron (and its antiparticle) with the neutrino and the photon.

Such an idea would upset the "symmetry" model.