Messing around in lightroom etc is just not worth it for hundreds of shots I take per day sometimes.
I absolutely can get better shots with my non-phone cameras, but they take more work.
They both fit in your pocket and startup as fast as you can activate the camera on your phone. Out of habit I usually reach the power button inconsciently when grabbing it from my pocket and the lense is usually extending while my arm is moving in position so it is usually immediately ready to snap a picture.
If you are in a setting where you are almost certain to take picture, a micro four third format also works great and depending on the lense are usually small enough if you are the kind of guy who wears small shoulder or sling bags.