undefined | Better HN

0 pointslukego2y ago0 comments

... and one such correction is the (simple, conservative, underused) Bonferroni Correction.

0 comments

Super helpful - looked it up, will aim to apply next time!

Curious how the bonferroni correction applies in cases where the overlap is partial - IE, experiment A ran from Day 1 to 14, and experiment B ran (on the same group) from days 8 to 21. Do you just apply the correction as if there was full overlap?

lukegoOP2y ago

I believe you would apply the correction for every comparison you make regardless of the conditions. It's a conservative default to avoid accidentally p-hacking.

There might be other more specific corrections that give you power in a specific case. I don't know about that, I went Bayesian somewhere around this point myself.

RandomLensman2y ago

There are a bunch of procedures under the label Family-wise Error Correction, some have issues in situations with non-independence (Bonferoni can handle any dependency structure, I think).

If there are a lot of tests/comparisons could also look at controlling for the False Discovery Rate (usually increases power at the expense of more type I errors).

j / k navigate · click thread line to collapse