Calculus and chemistry are very relevant to physics!
With less time used taking classes like Psych 101 (which is generally a waste of time for engineering and science majors) you have more time to study and complete assignments in your core classes. The university says it wants you to take these classes to be "well rounded", but honestly, they just want more money from students.
Regarding Calculus and Chemistry (all important to physics of course);There were several students I knew that clipped out of all of the freshman/sophomore chemistry and calculus classes and just started with organic chemistry and differential equations. This meant freshman year was a little rougher, but they were done with all of their non-major courses by sophomore year, so they were able to take a lower course load in their next three years and better absorb the core classes. I majored in electrical engineering, and still really struggled with all my calculus classes despite taking AP Calculus in high school. So instead of just focusing on circuits, coding, control systems...etc, I had to split my time with stuff like chemistry/calculus/random classes like Psych, so there was a lot less time to use for homework, projects, and harder subjects (assembly and microprocessors aren't too bad by themselves, but when you're mentally sharing that same space with a bunch of other classes it is brutal). Of course, most colleges are like this, but I think I'd rather have less required subjects, but time to go into them in additional depth.
In my experience, every STEM class in college is just paint over a math class. The stronger your math skills are, the more you can concentrate on the paint rather than struggling with the math.
I agree it's a good plan, however, to take AP courses outside of your major so in college you can take more stem classes. My college offered so many exciting stem classes I was a bit frustrated to have to take some non-stem classes to satisfy the degree requirements. No, I don't think that requirement was to make more money, it was genuinely believed that those were needed for a well-rounded education. That idea isn't entirely without merit.
For example, I really wish I could've only taken EE classes relevant to power systems + public speaking + several comp sci electives (programming, databases, etc) and as much statistics as I possibly can. The standard major was still good though and set me up to learn on my on.