How much are physicist or chemists really controlling in the lab setting? There could be plenty of confounding variables in their experiments too. Maybe "RT" in this lab for that publication for that experiment is actually 75*F and its 71*F in your lab, or you are at different elevations. Maybe no one calibrated the instruments for years. Maybe the reagent wasn't fresh and absorbed too much moisture or oxygen from the room. Maybe an undergrad dropped the balance on the floor and was afraid to tell anyone.
To overcome those potential confounding variables and other biases, chemists and physicists often turn to the exact same statistical tests being employed by people in the social sciences. Technical replicates are the norm in hard scientific experimental design because of how many biases could be present in the laboratory. It's a chaotic environment. Good experimental design builds robustness no matter what your topic is.
A lot more than can possibly be controlled when you're gathering data from events in the real world instead of in a controlled lab environment.
> To overcome those potential confounding variables and other biases, chemists and physicists often turn to the exact same statistical tests being employed by people in the social sciences.
No, not "often"--"when they have no other choice". The preferred method of dealing with such variables is to measure them, develop predictive models for how they affect the desired outputs, and test those models in further experiments. For ewxample, if "nominal" temperature is 75 F but temperature in labs can vary, physicists or chemists will want to do experiments over a range of temperatures, develop a predictive model for how temperature affects the results, and test the model. They won't just throw up their hands and do "statistical tests" and call it a day--unless it's impossible to do anything else. Which it almost never is in physics and chemistry.
> Good experimental design builds robustness no matter what your topic is.
And good science is aware of the limitations inherent in each specific field no matter what the field is. Good science does not make claims that are not justified in the light of the limitations of the field. "Statistical tests" simply cannot give the same level of confidence as predictive models that have been tested against further experimental data and have passed the tests. And good scientists should not pretend otherwise.
