It's not just energy and CO2. Roads and spread out human development disrupts ecosystems and natural water flows. They result in the death of species, plants, animals, insects. Mining the material required to build cars and infrastructure also has significant ecological impact.
Please note that less than 18% (likely around 10%) of the world's population own cars [1]. 5-10x the number of the current cars in the world, even if they are EVs, will have more than a 5-10x impact on the environment since ecological systems are non-linear in nature.
[1] https://www.motorbiscuit.com/number-car-owners-world-less-th...