Two days of battery life was while using Sailfish rather than Android. I'd imagine part of that is due to the fact that Sailfish doesn't have drivers for the GPS chip in the Nexus 4, and therefore would never power it up.
Aside from that, I still get decent battery life out of it even in Android by carefully policing what runs on it. I have very few applications on my phone that run in the background, and I don't have the Google apps installed - I use microG instead, which is a FOSS implementation of (a subset of) Play Services.
I'm aware that the vast majority of users probably wouldn't like using their phone the way I do. That said, this should make it clear that battery life is very much affected by the software you run.