undefined | Better HN

0 pointspdonis2y ago0 comments

> when humans are on both side of the equation, how do you get rid of all the social and psychological effects that imply, including placebo and the desire to see the study bend in some direction, be it at some unconscious level?

You don't. But again, in the scenario under discussion, these are stipulated to be the same in both cases. (Or more precisely, the underlying unconscious factors involved have the same distribution in both cases.) So again, these kinds of "intentions" don't make the distributions different in the two cases.

Another comment is relevant here. The whole point of things like double blind studies in medicine is to make it the case that, whatever unconscious factors are involved along the lines you describe, they don't change the underlying distribution from which the sampled results are drawn. In the scenario as described in the article, it was assumed that all of these precautions were taken. That is part of the reason for the article's statement that the experimenters' intentions about the stopping criterion can't affect the results.

Of course if you know that in a particular case, those precautions were not taken, that changes how you view the results. But Bayesian analysis can cover this case too: you just expand your hypothesis space and your prior to include things like "the experimenters unconsciously influenced the results in different ways because of their different stopping criterion". The article excludes this possibility in the scenario it describes, but in the real world, yes, we know it is possible to have study designs that don't eliminate this failure mode, and our analysis should allow for that in cases where the study design was such that it might have happened.

0 comments

psychoslave2y ago

> But again, in the scenario under discussion, these are stipulated to be the same in both cases.

That's not what I read. The two studies imply that they are conducted under two very different mindsets, which will most likely also mean people will receive a different human treatment. At this point, the statistics you will get out of it is almost ornamental. Sometime the most significant information to extract from a description is not the one that is the most obvious that is pointed at as the thing you can quantify and draw comparisons.

pdonisOP2y ago

> The two studies imply that they are conducted under two very different mindsets, which will most likely also mean people will receive a different human treatment.

Even if the studies are conducted using the standard double blind protocol for medical and social science research, where the experimenters who have the different stopping criteria are not involved in any of the actual experimental activities? They don't do the random assignment of patients to treatment and control groups; they don't administer any of the actual treatments or placebos; they don't interact with the patients or the treatment personnel in any way during the experiment; and they don't inform anyone who is actually involved in the experimental activities of their intentions, regarding stopping criterion or anything else.

You're saying that even if all this is done--and, as I've said, this is standard procedure in medical and social science research--it is still impossible to prevent the experimenters' intentions--which aren't known to anyone else involved--from affecting the results? Remember that, as I've already said, the whole point of these double blind protocols is to prevent the experimenters' intentions from affecting the results. You're saying that's a fool's errand?

j / k navigate · click thread line to collapse