From the paper:
> To do so, I leverage a unique setting arising from the largest gas leak in United States history, whereby the offending gas company installed air filters in every classroom, office and common area for all schools within five miles of the leak (but not beyond). This variation allows me to compare student achievement in schools receiving air filters relative to those that did not using a spatial regression discontinuity design.
In other words, the paper looked at test scores at different schools in different areas on different years and assumed that the only change was the air filters. Anyone who has worked with school kids knows that the variations between classes from year to year can be extreme, as can differences produced by different teachers or even school policies.
Again, I think air filtration is great indoors, but expecting test scores to improve dramatically like this is not realistic. This feels like another extremely exaggerated health claim, like past claims made about fish oil supplements. Fish oil was briefly thought to have extreme positive health benefits from a number of very small studies like this, but as sample sizes became larger and studies became higher quality, most of the beneficial effects disappeared.
To me its like looking at schools that buy newer buses and trying to show new buses improve test scores. When in practice the only schools that are buying any significant number of new buses have far more money coming in than in the past and have a lot more to spend on students compared to other schools, which is way more relevant than what year a kid's bus is made. Maybe better buses would improve scores too, but there is no way to tell if 95% of an improvement is due to other unrelated factors based on funding.
This is what the study looked at.
The problem was that it wasn’t randomized within schools or across teachers. They also looked at a very limited time window. They also note that some teachers weren’t using the air filters. They also found that the VOCs they were trying to filter weren’t even detected before the filters were used. They also used some questionable regressions to imply larger trends.
The list of problems goes on and on. It’s fascinating how easily people are tricked into pivoting around this one study, though, simply because it’s the one introduced by the headline.
The smoking gun is really in Table 3 and Table 4, where you can see that the effects that were observed are compatible with a population effect of 0, or alternatively you can look at Figure 2 and note that you could draw a straight line (no effect) within the confidence bands. Doesn't mean the effect is not there, but that there's insufficient evidence that it is, and that we should indeed be very careful about taking the estimates at face value.
All research is met on HN by people who know better and will tell you why it's flawed. There isn't a greater collection of expertise in the history of the world than on HN.
Edit: I meant to add: What value can we find in this research? It wasn't published as scripture, the perfect answer to all our problems. It's one study of some interesting events and data; what can we get out of it?
The reality of any underpowered study could always be “larger as well as smaller”. This statement doesn’t add anything to the conversation.
The mistake is pivoting around poorly structured and underpowered research.
> All research is met on HN by people who know better and will tell you why it's flawed.
This is a misunderstanding. People who know how to read studies will always be aware of the limitations.
There’s a difference between saying “everything is flawed” and pointing out the limitations. Most early research comes with significant limitations like small sample sizes or large cofounders. You have to understand these in conjunction with the results to know how to interpret it.
There’s a cynical approach where people see discussion of limitations, don’t understand it, and instead go into a mode where they think it’s smarter to ignore all criticisms equally because every paper attracts criticisms.
This is just lazy cynicism, though. There are different degrees of criticisms and you have to be able to see the difference between something like a slightly underpowered study, and something like this paper where the authors threw a lot of regressions at a lot of numbers and kind of sort of claimed to have found a trend.
Eg if you assume there is a real effect plus a lot of noise, given the study has been published etc the noise will have more likely acted in the favourable direction.
IMHO given the relatively large size of the effect it seems quite likely that the noise part is in fact potentially large (this is much more subjective) which makes is less clear that there is measurable signal at all here. I’d have to see a lot of replication or a very strong explanation of the underlying mechanism to believe the magnitude of the effect, but will very easily believe the sign (with a small magnitude).
It is almost certainly flawed, and it is probably wrong: https://journals.plos.org/plosmedicine/article?id=10.1371/jo...
If you are discussing research at all it is important to discuss the flaws too. The alternative I can see would be to take every published paper as proven true even though we know this is not the case.
It seemed like a pretty valid criticism. These studies should be taken with a massive pinch of salt because they're fairly uncontrolled.
Yes, but since we know that there's a huge bias to publish and publicise larger results, you know what way to bet.
1. Why this blog post study is flawed and dumb
2. America is bad and shameful.
Your local utility should send you water that doesn't need further filtering.
As for painting over asbestos--that's actually considered acceptable. Asbestos sitting there isn't going to hurt you. Asbestos only hurts you when it's disturbed. Removing intact asbestos is likely to increase your exposure, not decrease it.
As for cancer--why are you so sure it's chemicals and not lifestyle?
Maybe the tests results are better because the children are more rested on the day of the test. Maybe the hum of the machine creates some kind of meditative noise that helps children concentrate. Or maybe none of that is true..
However, I don't see that it proves pollution is the cause. What about infection? Air filtration can reduce the spread of pathogens. Schools throw together a large pool of people, the bug of the day will go around. Less if there are good filters.
there already is further research. and the results do seem to be holding up.
the study you're quoting from is the one linked in the 2nd paragraph of the article. this is from the 3rd paragraph:
> But it’s consistent with a growing literature on the cognitive impact of air pollution, which finds that everyone from chess players to baseball umpires to workers in a pear-packing factory suffer deteriorations in performance when the air is more polluted.
that paragraph links to an earlier Vox article [0] which goes into more detail, and well as linking to all of the various studies:
> A wide range of studies about the impact of pollution on cognitive functioning have been published in recent years, showing impacts across a strikingly wide range of endeavors. Stripe CEO Patrick Collison has taken an interest in this subject and compiled much of the key research on his personal blog. Among the findings he’s highlighted include:
> - Exposure to fine particulates over the long term leads to increased incidences of dementia and Alzheimer’s disease in the elderly (a second study confirms this).
> - A study of 20,000 older women found that 10 micrograms of additional long-term particulate exposure is equivalent, across the board, to about two additional years of aging.
> - The impacts are not limited to the elderly, however, nor are they exclusively long-term. A range of specialized professionals also seem to suffer short-term impairment due to air pollution. Skilled chess players, for example, make more mistakes on more polluted days. Baseball umpires are also more likely to make erroneous calls on days with poor air quality. Politicians’ statements become less verbally complex on high-pollution days, too.
> - Ordinary office workers also exhibit these impacts, showing higher scores on cognitive tests when working in low-pollution ( or “green”) office environments. Individual stock traders become less productive on high-pollution days.
> - The same also appears to be true for blue collar work. A study of a pear-packing factory found that higher levels of outdoor particulate pollution “leads to a statistically and economically significant decrease in packing speeds inside the factory, with effects arising at levels well below current air quality standards.”
> - Last but by no means least, the cognitive impacts appear to be present in children, with a Georgia study that looked at retrofits of school buses showing large increases in English test scores and smaller ones in math driven by reduced exposure to diesel emissions.
0: https://www.vox.com/future-perfect/2019/12/11/20996968/air-p...
You’re conflating different results.
The results of the headline study are dramatic in ways that aren’t holding up. The test score increase happened after a single year of putting air filters in class rooms. That’s minimal exposure to purified air for a fraction of the day, 5 days a week, for less than a year of classes.
The other studies are much longe term and look at things like decades of exposure at city scale.
Do you see the difference? The study tried to claim that purifying air immediately improved test scores by dramatic amounts.
There are other serious limitations in the study, like the fact that they can’t even identify which air purifiers were installed or how effective they were. There’s a footnote that says many weren’t even used. There’s a section on air quality monitoring that says they didn’t even detect the VOCs they were trying to filter before they started filtering.
This is the type of study that people implicitly believe because it makes logical sense, but when you read the details you realize that there isn’t much substance in it.
If parents would be allowed to contribute towards it like a 10year bond it would pay for itself…
Would you support a small study saying a medicine or a vaccine produces a 20 year life expectancy increase, all that to end up 20 years later with no improvement, everyone on that medicine, and the anti-everything yelling on every platform that the big pharma lobby poisoned our children ?
Even when the studies are on large samples, double-blind, long time range with a clear explanation as to why there's an effect, we have people trying to kill the resulting health campaigns. Don't encourage fake ones !
IQ has about a 0.2 correlation with income. The paradox arises when you zoom out for a more macro view. National IQ has about a 0.6-0.8 correlation with GDP per capita.
Performance in class rooms is definitely an IQ thing and different view points will likely generate different sets of data.
I'm interested in seeing controlled trials on individual performance, not just observing real world scenarios.
Imagine if some schools installed air conditioning in their gym one year. Running times around an indoor track would improve considerably, but mostly because conditions at the point of testing improved. Not necessarily because the air conditioning made the students actually improve their stamina or speed.
That seems like a problem for the reader, not a problem with the text. Why would the reader expect this? Is it the use of present tense in the title rather than past tense?
They hedge by saying “could” but like most articles in this vein it goes on to pivot around the outlier study.
> The whole thing is driven by one data point and a linear trend which makes no theoretical sense in the context of the paper (from the abstract: “Air testing conducted inside schools during the leak (but before air filters were installed) showed no presence of natural gas pollutants, implying that the effectiveness of air filters came from removing common air pollutants”) but does serve to create a background trend to allow a big discontinuity with some statistical significance.
I’m reminded of the walkback of scientific studies showing massive benefits from giving kids in third world countries deworming medications: https://www.theguardian.com/society/2015/jul/23/research-glo....
My beef with Matt Y.’s worldview of “scientifically driven public policy” is that the costs and benefits of public policy interventions are so devilishly difficult to study that you can’t meaningfully use them on realistic time scales to drive policy. This is an exceedingly simple hypothesis—filtering air improves test scores—that can easily be tested while controlling for confounding factors. But even then it’s hard!
A thread on Gelman's article is here: https://news.ycombinator.com/item?id=22006595.
We seem to know:
- Elevated CO2 in rooms impairs cognitive performance.
- Elevated CO2 in submarines, at levels far higher than you would see in a normal room does not appear to impact cognitive performance.
- Installing carbon filters (what this study actually looked at) might improve classroom performance.
- People don’t like stuffy rooms.
All this is consistent with multiple hypotheses. It could be that we just don’t know anything about it. Or maybe there is some gas or gasses emitted by people that isn’t CO2 that makes people mildly uncomfortable and have worse cognitive performance.
CO2 is certainly a good proxy for ventilation quality in a space where air is exchanged with outdoors but where the gasses in the air are not otherwise changed. Carbon-filtered classrooms and submarines are not examples of this.
Elevated CO2 in submarines absolutely impairs performance. One example: there was a guy on my boat who got migraines when CO2 got too high - he was useless. Luckily the fix is simple - just turn on another CO2 scrubber.
There's nothing special about a submarine that makes CO2 somehow different than anywhere else.
https://pubmed.ncbi.nlm.nih.gov/29789085/
I’m definitely not an expert.
I wonder if sorbent quantity correlates with performance
Many of those gasses do impact cognitive performance. It's not obvious to me why CO2 would, but if CO2 is going up, so is everything else we breathe out. CO2 where I am is somewhere in the ≈400ppm-1000ppm range -- 0.04% or 0.1% -- and it's pretty inert. I'm not sure what harm it does.
If it does to harm, rising CO2 levels should be much more concerning than "just" climate change.
But I suspect it's other gasses.
And HEPA filters don’t really scrub CO2 that effectively, even with a carbon membrane, so can we really expect lower CO2 levels?
Considering this, what’s the actual takeaway here? Cleaner air (dust/virus free?) is better for productivity?
I am asking because I want to buy the same filter for work, but I am doubtful that the $700 HEPA filter sounds like the same filter they used, even though the article mentions they used readily accessible 5 stage filters.
Most of the stuff for sale as “carbon” filters has too little carbon to do much. You want quite a lot, and, for some gasses, you can get special impregnated carbon or other media.
I wasn't sure what .22 std deviations meant, so I looked stuff up a bit. For a normal distribution, going from the average to 1 standard deviation above is going from the 50th percentile to the 84th percentile. Going up .22 standard deviations from the average is going from the 50th percentile to about the 55th percentile.
However, if these results were observed in grades 3 or higher, it could suggest a more substantial phenomenon. I randomly picked the third grade, but perhaps there’s a specific age after which the medical community considers a child’s immunity to be significantly enhanced.
I exaggerate a bit, but I found that during covid, where the mask was mandatory in my place, I was never sick. The only few years in my life where I was actually healthy continuously for YEARS, I and my friends could not believe the impact of the mask. But then we were stuck at home, living in constant misery and stress.
Once the masks disappeared, finally we could live again, and got extremely sick the first few years... Maybe a more normal balance would have been better ? Sick a bit continuously ? I think trying to avoid sickness is like trying to swim against the current, nature just works that way.
As a result, the daycare got a grant to get N-95 air filters installed and those UV lights that Chinese restaurants tend to have in the bathrooms. One per room.
What. A. Difference.
The infants and kids coming up are not nearly as sick, and when they do get sick, it's not nearly as terrible. The RSV vaccine has also been a godsend.
I can't really tell/feel what what the silver bullet here, but the combinations have been amazing. So much so that we got them for the house.
Rigorously: no way to tell now, the time has passed.
Likely: you're right. But what little staff is there from pre-covid says that they have helped a lot too.
Reality: here's $500/wk and my kid, in return I get really sick for a year no matter what.
The only disinfecting UV lights I've heard of for use in occupied areas are hanging things that only point up where nobody's going to be exposed. I have heard of some research for safe UV lights that are high up in the UV, they actually still burn but without enough penetration to get through the dead skin layer.
Ref: https://en.wikipedia.org/wiki/Respiratory_syncytial_virus_va...
Normally, I use this clinic in Thailand to compare prices for vaccines: https://www.thaitravelclinic.com/cost.html
The RSV vaccine is crazy expensive, even more than the HPV one. Over 200 USD is incredibly expensive in a developing country (like Thailand).
> those UV lights that Chinese restaurants tend to have in the bathrooms
In studies of pollution and impacts on health, the confounding factors often have a larger impact on the health outcome than the pollutant, such as particulate level, and therefore significant control of confounders is required to estimate any impact on the health outcome. The strong effect in this study is highly suggestive of a confounder rather than a real effect from particles or other pollutants and therefore would require a much better study design to support tacking action at a policy level with an expectation of a huge impact.
Fine to put filters into improve overall air quality but just not with the benefit rationale suggested in this study.
Imo we're way overdue standards and controls for clean indoor air that are on par with standards for drinking water and food. Like this article shows, we have the tech to provide clean air today. All we're missing is policy to uniformly deploy it.
[1]: https://www.sciencedirect.com/science/article/abs/pii/S01966...
[2]: SARS-CoV-19: ~75% of sick days, Influenza: ~10% of sick days
[1]: https://www.ekathimerini.com/economy/1255154/greek-work-ethi...
You could also add potentially get a HEPA with a carbon filter which will get rid of volatile organic compounds which can also be damaging (but also just be the smell of food) but they don't tend to be as effective and depending on the mix of VOCs verses particles one filter may run out before the other and carbon really doesn't capture all that much or well. It is a good way to get rid of smell at least for a while.
There is a whole world of different standards for filtration for industrial and hazard chemicals which the FFP2/3 and N95 standards for Personal Protective Equipment respirators will lead you into if you want to go into that rabbit hole, but for a household typically its mostlt about Particulate matter, Volatile organic compounds and CO2. CO2 is about bringing in fresh air from outside.
Then when outside of the household N95/99 or FFP2/3 respitators do the same job in unclean air environments which is basically everywhere, outside or indoors in public places pretty much never meet the World Health Organisations levels for PM2.5 and often exceed CO2 (a proxy for re-breathing and a high change of viral infection spread) standards too.
I didn't see the model specified. They also said some schools got carbon filters, which is a different type of filter.
There's footnote saying many filters weren't even installed because some teachers thought they made the air "too dry", which is major placebo effect at work (air purifiers don't extract moisture from the air).
The entire paper is really not good quality, to be honest.
You can get a small HEPA purifier for a single room to remove particulates. The size of the filter, noise level, and amount of air moved are things to look for. Stepping up to activated carbon would remove VOCs, but cost significantly more (see IQAir, Austin Air, but ignore the cheap models that don't have 10-20lbs or more of activated carbon).
A review of the effects of installing air filters in classrooms - https://news.ycombinator.com/item?id=22006595 - Jan 2020 (26 comments)
Installing air filters in classrooms has large educational benefits? - https://news.ycombinator.com/item?id=22006033 - Jan 2020 (48 comments)
Places with clean outdoor air probably won't see much benefit.