The reason most physicists don't think about the philosophical underpinnings has more to do with the way quantum mechanics is taught. Many of the founders were interested in the philosophical questions but after WWII you can see a sharp decrease in that aspect and more of the hardcore calculation and the philosophy kind of disappeared. Here [1] is a great talk that discusses this very point. Unfortunate for quantum mechanics because understanding the philosophical underpinnings is critical to understanding what quantum mechanics says and doesn't say. But, thanks to things like quantum computing and better experiments, interpretations is becoming more mainstream since there are experiments being planned (at least when I was heavily involved in the area in 2010) that would be able to test some of the claims made by different interpretations.
>We are all waiting for a better theory of physics. Of course rewriting the quantum theory in different forms, like this attempt, might help in getting there.
To me this is a contradiction. If we are all waiting for a better theory, then why are we re-writing the theory? A critical part of all the interpretations I've encountered [2] is making different ontological claims about quantum mechanics. Take Bohmian, which posits a guiding wave. Sure we can make some of the mathematics nicer, but we can't go very far without explaining what this 'guiding wave' is, which itself is rather ad hoc. Plus, how do we rectify it with Bell's theorem on hidden variables? To me, we can't really begin re-writing quantum mechanics without inevitably running into ontological statements.
But I still agree with your premise. I believe that by pushing the limits of quantum mechanics both scientifically and philosophically we will get a better grasp of what things to look for / where to look, for a deeper theory. Case in point, wavefunction collapse. Some might consider it a huge problem of quantum mechanics, but if you take a different ontological view, wavefunction collapse isn't that interesting or important. So maybe wavefunction collapse isn't as important and trying to develop a theory that doesn't have it is the wrong idea.