The big philosophical problem with much of this is that people assume that the smallest things are the most fundamental. So people think that stuff, whatever that stuff is, is fundamentally made up of much smaller stuff, and that stuff is fundamentally made of yet smaller stuff, and so the smaller you get, the more fundamental you get. And so (they think) if you want to work out what is really going on at any layer of reality, you need to figure out what the smallest possible things are.
Yet this is ultimately a philosophical posit -- it's not empirically-informed. There's no good reason for thinking it.
To be clear, none of this is about physicists doing physics. It's about the philosophy that many people bring into, and therefore take away from, these kinds of discussions.