I wonder to what degree these complex problems are self inflicted due to more than a decade of flipflopping between a myriad packaging and environment management solution. In such an environment, you are bound to have so many different approaches between important projects that trying to bring them all under one umbrella becomes next to impossible. What is a python package is relatively well defined, but how you build one is completely up to you.
Edit: I've just read another comment which I think pointed out the most pertinent fact - that python has often served as mere glue code for stuff written in lower level languages. This then results in an explosion of complexity as a python package not only has to package the python part, but has to be able to build and package the lower level stuff.
