Yes and this is exactly the problem with buses! You and everyone else rides the bus on the core routes at the peak times, but the buses run for long hours and the routes cover more areas than the core, so the average utilisation is very low and pollution per passenger-km very high - although the average passenger experiences crowded buses.

Deciding to switch a solo peak hour journey from a SUV to bus might help a little, but the bus will still be driving empty outside the peak times & routes. Working from home takes a passenger out of the peak times entirely, and with a reduction in peak time travel then demand-responsive transport becomes more attractive than huge buses blindly following rigid routes, and eventually maybe we can dream that sustainable transport like walking and cycling might be deserve a safe place on the street.