A month ago: https://news.ycombinator.com/item?id=31828509
Potential fabrication in research threatens the amyloid theory of Alzheimer’s - https://news.ycombinator.com/item?id=32183302 - July 2022 (236 comments)
Alzheimer’s amyloid hypothesis ‘cabal’ thwarted progress toward a cure (2019) - https://news.ycombinator.com/item?id=31828509 - June 2022 (307 comments)
How an Alzheimer’s ‘cabal’ thwarted progress toward a cure - https://news.ycombinator.com/item?id=21911225 - Dec 2019 (382 comments)
The amyloid hypothesis on trial - https://news.ycombinator.com/item?id=17618027 - July 2018 (43 comments)
Is the Alzheimer's “Amyloid Hypothesis” Wrong? (2017) - https://news.ycombinator.com/item?id=17444214 - July 2018 (109 comments)
These comments are written by real Alzheimer's researchers. They all disagree with the notion that Lesné's papers have been important to the field, and therefore undermine the idea that this has any bearing on "two decades of Alzheimer's research". (Karen Ashe, co-author of the main paper referenced here, also stops by the thread.)
[1] https://www.alzforum.org/news/community-news/sylvain-lesne-w...
We can blame regulatory capture of the FDA for approval of failed drugs, rather than the scientific establishment. And we can blame Sylvain Lesné for Sylvain Lesné's fraud.
I would encourage those of us whose only knowledge of this topic is the word "amyloid" (I admit I am one of them) to read the scientists' comments and appreciate that there is more to this than we know. There are complexities, nuances, diverse perspectives and healthy disagreements. It's not just a political battlefield. Projecting culture war into it would be harmful to the scientific progress we all value and to the millions who suffer from Alzheimer's.
They have an interest in limiting the scope of the perception of fraud in the field.
Alzheimer’s amyloid hypothesis ‘cabal’ thwarted progress toward a cure (2019) https://news.ycombinator.com/item?id=31828509
How an Alzheimer’s ‘cabal’ thwarted progress toward a cure https://news.ycombinator.com/item?id=21911225
The amyloid hypothesis on trial https://news.ycombinator.com/item?id=17618027
Is the Alzheimer's “Amyloid Hypothesis” Wrong? (2017) https://news.ycombinator.com/item?id=17444214
Why, at this point should we believe any one scientist writing in that forum wasn't already sunk far deep into amyloid research in their career?
Not to mention that after a quick glance on the comments section, I fail to see where you get the idea that "They all disagree with the notion that Lesné's papers have been important to the field". Apart from the very first comment from karen Ashe (who will obviously be defending her research) and a few other who's working on related topics, other commenters seems to be keeping their suspicion at amyloid hypothesis.
It's also plenty obvious that there is no single, monolithic "current research direction" or even that this researcher's work was of fundamental impact when it was published - not to mention the number of people that were highly skeptical from the beginning.
"Alzheimer’s amyloid hypothesis ‘cabal’ thwarted progress toward a cure (2019)"
https://news.ycombinator.com/item?id=31828509
https://news.ycombinator.com/item?id=21911225
(also see dang's comment who lists 2 more thread https://news.ycombinator.com/item?id=32213973)
Parent comment is downvoted but it seems like most alzheimer's researchers have vested interest in amyloid hypothesis one way or the other.
[1] https://www.dailykos.com/stories/2022/7/22/2111914/-Two-deca...
[1] https://www.science.org/content/article/potential-fabricatio...
https://www.science.org/content/article/potential-fabricatio...
The problem is that somehow researchers managed to get hundreds of millions of dollars, entire labs, and 15+ years of research on the back of a study that presumably nobody ever tried to replicate, nor did they notice that the claims weren't true even when building new research on top of it. That's the problem and it is frankly very hard to understand how it's possible.
This was (mostly) true back then, but it is definitely not true today. People tempted to commit fraud now have to be worried about people like Elisabeth Bik exposing them and ruining their careers. In my experience, the type of people known lie in papers overlap strongly with those that are career-minded/money-driven. So having a few journalists with the skills to detect fraud is an obvious win. Some of the frauds will just get better at, that's just how it does, but it's not like there are no imaging experts in the field.
Academia and research needs a new broom. Presently incentives are peverse. Impact factors, publisher corruption, grant applications and funding are a blight on science.
Also remember that Kickstarter found it necessary to write a blog post “Kickstarter is not a store“, because people were expecting projects to return fast, reliable, tangible results. By its nature, science will be even further from that revealed preference of funders
Neither solutions are perfect as most things in life, I guess the question is, will it be better enough compared to what we have now ?
My opinion: capitalism has corrupted literally every single thing it has touched.
> "The suspicion that something was more than a little wrong with the model that is getting almost all Alzheimer’s research funding ($1.6 billion in the last year alone) began with a fight over the drug Simufilam. The drug was being pushed into trials by its manufacturer, Cassava Sciences, but a group of scientists who reviewed the drug maker’s claims about Simufilam believed that it was exaggerating the potential. So they did what any reasonable person would do: They purchased short sell positions in Cassava Sciences stock, filed a letter with the FDA calling for a review before allowing the drug to go to trial, and hired an investigator to provide some support for this position."
However, the desire to gain a profit by pushing a questionable drug through trials was also involved. I'd note however, that in the Soviet Union, the likes of Lysenko also pushed fraudulent research in order to improve their standing in the Soviet heirarchy, which came with various rewards.
People like wealth and power, and some will do anything to get it, regardless of the nature of the society they live in.
Funding for these projects is often disconnected from the market.
They are usually dispersed by massive bureaucratic agents with incentive structures that have nothing to do with profit (more internal politics and prestige).
Imagine if the human body didn’t have a dozen different organs each producing a specialized product for your body and instead had one organ that tried to do everything?
I think capitalism facilitates specialization - and that quite effectively at scale. Specialization can occur without capitalism, too. There are other systems to coordinate specialists.
Just because capitalism has generated good things doesn’t mean it’s perfect. We need to acknowledge the damage it’s causing and think of ways to move beyond it.
Unless you want this to be the way things are for the rest of humanity.
Blaming “capitalism” is just the “old man yelling at cloud” if you are 20.
Some people latch onto whatever they can to gain power to exploit people.
Using failed dictatorships to prove 'communism is worse than capitalism' (which in the West we heavily curtail, but only to the point where we avoid uprisings against those holding power) is exactly the sort of shallow, illogical thinking you seem to rail against.
Communism fails because it doesn't account for human greed; it assumes everyone is onboard with improving society. Capitalism succeeds inasmuch as it panders to human greed to the point of evil and ignores the vast majority whom it fails.
Mind you, anarcho-syndicalism is where it's at, you can expect to wield supreme executive power just because some water tart gives you a sword ... /montypython
I don't think capitalism corrupts everything it touches - but I'm pretty sure it corrupts saturated markets, and these days that's an awful lot of things.
Clearly institutional competition for grant money has increased and changed what it means to be a successful researcher. Incentives are displacing virtues. But it's hard to find incentives, especially financial ones, that are fully aligned with honest research. Maybe we are trying too hard to find those.
https://physicstoday.scitation.org/doi/full/10.1063/1.324848...
A key point, from the above review, that I think explains a lot of this behavior:
> "Reich points out that fraudsters like Schön could get credit for “first discovery” if, before they are caught, their false claims are confirmed by others on the basis of genuine data."
It did shake up the field of organic semiconductor device research in physics by increasing scrutiny and changing some requirements (for example, electron microscope imagery of claimed devices is now a requirement for publication). However, as the top post at present notes, the incentives are backwards in academic science these days, and the role of funding organizations and high-profile journals is as problematic as that of the originating fraudsters.
Maybe this instance of fraud will do the same for the biomedical field, by forcing researchers to release their raw data and full-resolution images as a condition of publication, although that would require a major shift in behavior in today's patent-driven startup-centered heavily-corporatized biomedical research world.
Personally, I'll note that during the years I worked in academia, of the three PIs I worked with, I discovered two engaging in fraudulent research to greater or lesser extent. The main differences between them and the one who wasn't were (1) lab notebook discipline and recording and storing data securely, (2) in-house replication was required, (3) no toleration for BS and shady behavior. The others broke all those rules. (Unfortunately I picked the wrong PI to work with, and ended up leaving academia in a fit of contemptuous disgust.)
A good rule of thumb: If some research claim hasn't been replicated, and if the data and methods aren't transparently available, then it's as likely to be fraudulent garbage as not, and it's not worthy of further examination.
Fraud by false representation
(1) A person is in breach of this section if he—
(a) dishonestly makes a false representation, and
(b) intends, by making the representation—
(i) to make a gain for himself or another
It really lays open how easy it is to mislead everyone. All these siloed scientists won't have a clue anything is wrong. This is how conspiracies would work... if there is advantage to someone somewhere and they have the means to alter the model in their favour, why wouldn't they?
I suspect there was deliberate fraud, but this article doesn't provide any more evidence of that than previous articles.
> Since that 2006 publication, the presence or absence of this specific amyloid has often been treated as diagnostic of Alzheimer’s. Meaning that patients who did die from Alzheimer’s may have been misdiagnosed as having something else. Those whose dementia came from other causes may have falsely been dragged under the Alzheimer’s umbrella.
I think the author is confused about the controversy he is reporting on. Nobody is suggesting that there aren't elevated levels of Aβ in Alzheimer's brains. The controversy is only about the presence of Aβ56, and as far as I know Aβ56 was never used to diagnose Alzheimer's disease. It should also be noted that this is only relevant to postmortem diagnosis, so even if they were testing for Aβ*56 it wouldn't have affected the diagnosis of living patients.
At the bottom of the article is a note, "Article written by Mark Sumner via Daily Kos". This explains a lot. Daily Kos is a site that got its start with sensationalized political articles. Now they've apparently expanded to subjects where they can do more damage.
---
As I post the comment, the title of the linked article is: "Two decades of Alzheimer’s research was based on deliberate fraud by 2 scientists that has cost billions of dollars and millions of lives"
[0] https://www.science.org/content/article/potential-fabricatio... [1] https://news.ycombinator.com/item?id=32183302
People do this all the time. In fact I’d say it’s the usual reaction when confronted with data contradicting one’s beliefs.
Scientists are supposed to learn to go past that but I wonder how many actually do, especially when there is both social and economic pressure to conform to a school of thought.
Looking at reality in an unbiased way and trying to draw rational conclusions is incredibly rare and requires effort. Ego and social group pressure are the enemy.
Cackling evildoers are also very rare. Most evil is a product of how we normally behave and of normal social and economic incentives.
And so the very people who are supposed to call out and fight the irrationality are actually blind to it. You do get people calling out the irrationality, but mostly they are driven by social incentives rather than duty and so they turn their eyes away from realms that are deemed pure. They have no incentive to look inwards to the scientific institutions because that way they lose the social esteem they gain from association with the scientific establishment.
These people need to be formally immediately banned from any NIH activity, and criminally charged. We have known for years that their crap work was useless.
Though the effect is from "antiherpetic medication", not vaccines.
„I sincerely doubt that the absence of this particular paper and AB*56 from historical scientific record would have significantly changed the last 20 years of AD drug development. That is because there is strong genetic and other evidence for the role of amyloid in disease.“
Want to see amyloid defenders? Read comments on Derek Lowe's blog, and there's quite a few commenters there who are arguing that a better amyloid drug that actually clears out the plaques is needed (apparently not realizing that's what Aduhelm, and it still didn't work).
Evidence is mounting that amyloid is the downstream result of whatever actually causes these diseases.
Highly recommend this book to discuss why: https://www.amazon.com/Rigor-Mortis-Science-Worthless-Billio...
It all comes down to incentives.
(1) As a researcher you lose funding m if you don’t produce
(2) funds aren’t allocated to reproduce
(3) Researchers who publish will block research that disagrees with their work (as they’re also reviewers) (will lose future funding / have more competition)
(4) Researchers wont rescind their work if later findings warrant it (no incentive to)
(5) N numbers are way too low (higher N is more money)
Maybe that's the case in your area of research. In mine (math, physics) it definitely isn't. So I would be a bit more careful about the wording here. ("80% of science" – what science?)
P < 0.05 is outdated at this point, for anything truly ground breaking p < 0.005 or < 0.0005 is probably a better choice, and even then I would ask "Where did you get your dataset from and did you combine (!!!) datasets from multiple orgs."
The methodology is deeply flawed, given that you haven't stopped accidental p-hacking (publication bias) and deliberate p-hacking (researcher data mining, e.g adding endpoints after looking at results), which compounds to create fake results with astronomically tiny p-values.
You can only trust single, extremely large, canonical RCTs which were announced in advance, and in which you are confident there is no survivorship bias in terms of the possibility that this RCT would have been cancelled had the results been thought to be negative halfway through.
Epidemiology (victim of researcher p-hacking and impossible to deal with confounders) or meta-analysis of RCTs or single small RCTs or RCTs that weren't announced in advance (victim of publication bias) should be taken with a grain of salt. If you accept conclusions drawn from these things at face value, be prepared to accept anything, because the methodology you've accepted is proven to be easily capable of demonstrating any fake phenomenon as true.
One factor contributing to this. In natural sciences, you take other people's papers as truth and build on that. In theoretical physics on the other hand the _first step_ is you reproduce their results.
As for physics, well it depends. Laboratory physics has produced the finest predictions in any science by several orders of magnitude. Quantum Electrodynamics is freakishly accurate. On the other hand it’s hard for me to see cosmology, to gently pick on an easy target, as more than extremely well researched and plausible science fiction. Then you have particle physics which has excellent laboratory equipment and produced fantastic results, but which has, in the opinion of at least one elite particle physicist personally known to me, perhaps painted itself into a corner. The Standard Model is good enough for government work, but nobody believes it’s the best possible theory.
For physics, it depends.. is it just "applied math" (so, verified easily), is it a CERN-type (LHC,...) experiment (hard to replicate, unless well.. you work at CERN), where many many people process the data, or is it something that is done only in your "lab", and hard for others to replicate.
On the other hand, finding thousands of patients and running a study is practically always hard and expensive.
But after becoming a scientist, coming across so many fraudulent papers, so much non-reproducible and poorly done research. And seeing how it's affected my own, the thing that's dearest to me, having to build on top of those results and work in that global environment. It's been heart breaking, I feel no love for science anymore.
I spend most of my career reanalyzing others' data and combining it together into larger datasets, in molecular biology and genomics. The only time I encountered mistakes it was from my own labeling errors or bugs, pre-publication. And in pre-publications datasets I would sometimes detect accidental swaps on the labels of samples as part of QC checks.
“Sokal Squared” is a great example of the problems in the field.
The more money is involved, the more fraud to expect. This should not surprise: fraud goes where the money is. And where there is money, the stakes are higher.
But I have seen reprehensible behavior even in opponents of the Younger Dryas Impact Hypothesis, where practically no money seemed to be at stake.
To that end, does anyone know of any list of techniques for spotting or identifying fraud in scientific works? This is a subject that I wish somebody would write a book on, so I am curious as to what techniques people use for this purpose.
But: science isn't really self correcting. Someone has to step up and fight to correct it, every single time. Spend some time learning about this problem - as I have - and you will find the overriding attitude is one of despair. Science isn't self correcting, it's arguably self corrupting:
1. There's no incentive to pick fights with colleagues so it almost never happens
2. Even when someone does pick that fight the institutions go to ground and defend their people, so there's no outcome.
3. The first response of journals and universities on being informed of fraud is to turn around and tell the fraudster everything, so there's no way to keep fraud detection techniques secret.
4. And usually after that they ask the fraudster to submit a "correction" (i.e. higher quality fake). The idea that you should maybe NOT let a fraudster have a second try once caught, does not seem to occur to the brightest of sparks that run our scientific institutions. There are actually cases where this has happened and then the original complainants spotted that the newly submitted correction was also fraudulent.
How do these papers get spotted? Spend time on PubPeer. It's a site where people compare notes on dodgy papers and how they're being detected. Also read the blogs of people like Elizabeth Bik, Smut Clyde, read the old blog posts by Joe Hilgard when he was still in academia, follow https://twitter.com/steamtraen
tl;dr There's a range of techniques used for different fields, often by looking for re-used images or data across papers that shouldn't be re-using them, or by spotting internal inconsistencies in reported data tables.
There is no question that a very significant portion is wrong and some of it is fraud, but I feel like this is the wrong summary takeaway. Science isn’t the reason that science is messed up, people are the reason. All fields of human endeavor suffer from the same problems due to emotional and political and selfish people. It’s more like, welcome to humanity. Science is actually the best thing we’ve got, there is no alternative that has less BS and more truth. The 10% or so of science that’s right has transformed the earth in the last century.
I worry about framing this the right way, about the subtleties of how you say it, because there is currently a war being waged on public trust in science, and to some degree that anti-science war is being won. It’s potentially damaging to say “most of science is wrong” and just stop there. That’s a misleading framing in my opinion. In order to fix the funding problems, society as a whole needs to have trust in science, to believe that the majority of people doing science are politically impartial and also not wasting money or lining their own pockets, to believe that scientific progress is human progress.
It’s important to note that the incentive problems you cite mostly aren’t caused my any malicious intent. Disagreeing with someone else’s research, in my experience, isn’t often done with the primary goal of holding back good research, it’s done because the researchers actually disagree on the science, and the reviewer actually believes the proposed paper isn’t complete or correct or up to publication standards. Moreover, for science it’s very important for researchers to be critical of each other. That is part of why we need more replication study.
A lot of things look like science and use the word “science” in them but don’t use the process of science to make their claims to truth. This is useful because how can you argue with “the science”?
Yeah, totally agreed. It doesn’t even make sense to suggest that testing to check whether things work or not is the wrong process. The scientific method is tautologically true, right? This is one of the reasons I cringe at the yard signs in my neighborhood that declare “science is real”, as if it needed an affirmation. Somehow that seems to give more credibility to the idea that science might not be real, whatever that means. Of course science is real. We don’t have modern houses or cars or computers without, sillies.
Trying to manipulate people like this is guaranteed to backfire long term. People will realize that it's being done and trust things coming from the world of science even less. Instead we need to acknowledge that there are real problems in how we practice and communicate about science and take visible steps to fix them.
I agree completely we should fix the problems, and I have no problem openly discussing what those problems are. In order to do that there needs to be hope that fixing the problems is a viable and likely-to-succeed activity. Summarizing all of science activity as broken is neither accurate nor helpful in terms of fixing the problems, right? That really is a framing problem because it is not broadly true, the parent poster implied that the practice of science is the problem, when it isn’t the problem. The problem is that people are involved, and science is actually one of the least problematic things we do. It’s incorrect to call out science as broken without comparing it to all other fields.
If everyone is convinced that science is a complete and total waste, and people are opposed to spending tax money to fund it, we will not be able to fix the existing problems. If we believe something about science specifically is broken, and not with the rest of the world, then we will come to the wrong conclusion about what is broken and be unable to fix the existing problems.
You mustn't forget that universities are only one player of many in the science system. And the main task of some universities rather is education and not research.
There is a lot of funding going to universities, and a lot going to specific projects. I think maybe the main model for research funding generally is that funding doesn’t just appear, people have to go ask for it in the form of a grant proposal. Those grants typically once granted go 40% to the university anyway. Universities, for their part, have been absorbing money for larger and more expensive administrations for the last 30-40 years, and it’s out of hand. Most people I know practicing science feel the opposite of what you said - that less funding should go to the university and more directly toward funding research.
Science is losing the battle for public trust because it wants to simultaneously be an infallible source of truth and this messy, chaotic discipline where we tumble towards an approximate answer. It gets defined as a one or the other when it's convenient.
In the first breath: Oh, X% of all published papers are wrong? No big deal, that's just how science works. Can we have another 100 billion of taxpayer dollars please?
In the second breath: The Science Says vaccines are safe and effective. Take it or get fired.
The US currently somewhat operates on the principle that companies are allowed to make and enforce their own rules, as long as they’re legal. And there’s a long list of things you must do if you don’t want to get fired, including work forty hours a week, and not be a jerk to your co-workers. Do you think companies should not be allowed to fire people who choose to do things that can hurt the company’s bottom line? Do you want the government to monitor businesses more than it does today?
Science funding is complicated and political, but the high level summary of the situation is that the public funds science right now because that’s what the public wants, because a good chunk of society understands that doing science has proven to be a great investment. It would be a bummer if the public decided not to fund science or education, that sounds like the fastest way for the US to become irrelevant globally.
Anyway, what is the alternative to doing science? Do you like your computer/car/cell phone? Do you want to see technological progress stop? Can you name a better approach to seeking truth than the scientific method? What has ever proven to be more effective than science?
Even $5 to someone with a long history of finding scientific fraud puts you on the right side of history: https://www.patreon.com/elisabethbik
I’ve been donating for years and I’ve never been more proud of that decision.
That way the only possible competition will be number of citations. The pressure will be for quality, not quantity.
Well, popularity, anyway. While popularity is sometimes a useful proxy for quality, that's not always the case.
It’s probably about the worst in any field dominated by studies and statistics. Harder sciences are harder to fake as bad results can be more conclusively falsified.
It’s quite useful to listen to Buffet and Munger. They’ve accumulated lots of wisdom.
“You get what you reward for” https://youtu.be/hJYLJRr3hEY
This hour of Munger is well worth it:
Here’s the transcript:
I'm not sure if it is a good idea to try to optimize the scientific process for fraud resilience. In the end, we still rely on people doing the right thing most of the time.
He said something exactly the same, and his conscious wouldn't allow him to work in the field of science.
He left the field many years ago. He delivers mails now.
I don’t think it’s remotely accurate to describe GPS as “socialist”. It was a military project. There’s a long list of military projects with more funding than GPS that have been invented and implemented since then.
Then you also need a realignment of the entire academic system as well. A new professor spending 50% of their time redoing the experiments of others, especially on capital expenditures and grad student time is going to be at a severe disadvantage against others who spent 100% of their time building their own research agenda.
We'd probably need a whole new degree that's like a PhD, but whose dissertation isn't a novel extension of the field, but reproduction of other studies. Because as it is, a grad student can spend up to a decade on a Ph.D. just doing novel research. Now you want them to spend 50% of their time on reproduction? So is a Ph.D. supposed to take up to 20 years with your proposal? Or does a dissertation contain 50% less novel research?
Certainly we should have funding for reproduction studies, but it's not enough to just have a grant; you have to have a valid career path doing such work. That doesn't exist today.
I don't think that would work - those positions would go empty. People apply for grad school because they want to do original research. On the other hand, once people are already in the academic pipeline and they run into struggles advancing from postdoc to assistant prof or maintaining funding as an assistant prof, I think a lot of people would take a 50/50 grant if lets them keep one foot in the door for original research. They aren't going to turn their noses up at the only grants they are competitive for.
You do raise a good point that labs with 50/50 grants wouldn't have as good of a value proposition for prospective grad students as labs doing 100% original research. I'm not sure what the solution is there.
Humans are not incentive-following automata, though we have incentive-following tendencies.
This is the biggest problem in my view. My work in R&D taught me that most of the time we don't produce anything. It's high risk. But it's high in rewards, often in adjacent areas not primarily the focus of the initial brief.
Surely all serious investors understand this. Research is something we do for marginal returns. It's not an "innovation factory". With things currently stacked against risk, research can only yield tepid results.
Not necessarily when the investor is an academic institution or a government.
It would appear that this isn’t really “Science” if the ideal of research is being veered away from so much.
Vested interests suffocating the process.