Show HN: A Platform For Self-Experimentation (opens in new tab)

That's a valid concern, and I agree that it's alarming. We will disclose where we are obtaining the L-Theanine from and provide a Certificate of Analysis.

Also, since this is a double blinded trial, if there are significant differences between placebo and real powders then that affirms that the powder was real.

I kinda wish the FDA had some certification to the effect that "this contains what it says it contains and its contents are no more likely to kill you than a peanut butter and jelly sandwich."

For day to day things, this is what I consult.

https://labdoor.com/

In the first link, all of those stores are places I'd never buy supplements from. Many stores sell supplements from brands I never see when I research what to take to fix some problem.

When I realized that, that was the end of ever considering buying supplements from almost all stores.

Whole Foods and similar types of stores are reliable. However, I still prefer researching and buying online. Analyzing comments, both good and bad, are very revealing and can sometimes give you an idea if it will work for you.

But now I just try to eat my veggies and so on. Diversification of food choices gives you a more solid spectrum of nutrients and some foods/nutrients amplify the effects of other nutrients.

The first link is bogus; it turned out to be a failure of testing and they had to abandon all the accusations and settled for nothing: https://www.reddit.com/r/Nootropics/comments/3ice6t/on_suppl...

There are some in existence already:

- quantifiedmind.com for self-testing your mind, which has a few group experiments as well. I'm looking into integrating with them as part of doubleblinded.com.

- PACO (by Google) https://www.pacoapp.com/ Pretty cool and flexible platform, although not the most user friendly or easy to figure out. It is open source and has an app, which is cool.

- A few sites like 23andme and ubiome have a lengthy list of survey questions they ask you, which I only imagine they would pass through pattern recognition algorthms to detect correlations between whatever information you give them and your Genome (or gut genome in the case of 23andme).

So yeah, there is stuff out there. We want to do this for nootropics and supplements (and eventually much more).

IIRC, https://experiment.com/ is funded by by YC

Yes, independent self experimentation is the way to go. You are not hoping that you can trust the people doing the study. You don't need to hope they did things the way they were reportedly done. You don't need to hope someone did not get paid off to tweak the numbers at the end. You just need to educate yourself on how to do proper studies and how to avoid false conclusions and techniques to bypass your own biases. But you also need to realize the results only apply to you at this part of your life. You can't promote your discoveries as being an absolute cure for other peoples problems. Even your own body can change over time and what used to work for you may no longer work.

I, too, have thought about conducting a pseudo-double-blind experiment on myself. I think I could pull it off, but I'd also be willing to pay a third party to help me. For example, DoubleBlinded could partner with a respect supplement maker, and offer personal experiments in any supplement made by their partner. The consumer wouldn't have to worry about setting up the experiment correctly, and they would get their own results immediately upon completion. Aggregate studies can wait until enough customers have self-experimented on the same supplement, and DoubleBlinded can afford to wait that long, because the partnership model keeps their costs low.

If you can pill it, you could add unique markers (maybe apply pseudo-random patterns to pill blanks) and take pictures before randomization and immediately before consumption. Or build something that dispenses a random choice from multiple boxes and keeps a log.

If you're willing to involve a friend for 30 seconds, get them to label pillboxes and make a note of which pillboxes were labelled.

As far as I can tell, gwern.net does exactly that (and it's well documented in his website - under the nootropics category).

I'm not sure I see that as a fatal flaw... ultimately, the user is sitting in front of a digital box that can guide and prompt them in all the same ways a researcher can--the only limit is in performing tests that require a medical professional to assess biomarkers. If the pace of medical device development continues, it's even reasonable to think something like a Theranos-that-works could commoditize the process while being intrinsically tamper-resistant.

Regardless, users can be prompted to perform any software action (knowingly or unknowingly, to affect bias) and that action can be measured by the system. It may so happen that every critical measurement occurs unbeknownst to the user, before they self-report anything (if at all). As we are currently undergoing a period of sensor-proliferation (fitness/health devices, wearables, internet of things, etc...) it's not unrealistic to think we will soon be able to instantly correlate data from a smartphone camera, blood/tissue, and the cloud.

Now there's always the problem of intentional fraud/deception, but I think the aggregate nature solves that problem. A small percentage will try to "break" the system, and that small percentage will never surpass a critical threshold with enough volume. In terms of ML/SVM's, we're now very good about filtering outliers or "misrepresented data"... while the responsibility is on you to develop a reliable classifier (for data-consistency more than arbitrary measurement), I imagine at scale you could infer trends with the same relative accuracy of traditional academia and research.

It's a really fascinating new direction--even if only an adjunct to traditional research--and I'll definitely be keeping an eye on the project.

Just read through that. I agree with the points about self-reporting being unreliable.

Even with the bias of self reporting, differences between real and placebo will still be meaningful. You can't entirely deceive yourself if you don't know what you just took.

Also, there is some degree of honesty required for this experiment. If someone really wanted to, they could open up the pills and taste them to see if it's placebo or not. They could also intentionally lie. But the goal here is to learn about yourself through experimentation, so I expect most people that participate to have some level of desire for truth and honesty. I would also argue that while these biases do exist, even large randomized clinical trials have the same pitfalls, so we are not necessarily worse off than what's already out there.

AFAICT, there isn't a control group. You have 4 weeks of pills and each week it could be real or fake. So you are comparing against yourself.

Are you mainly referring to it being a CYP3A4 inhibitor?

And it's been banned in EU in 2011.

Hey, thanks for the feedback. I totally agree about educating people on good practices. This is a big part of what I want to do over the next few years - I think educating people on the scientific method, especially in regards to self experimentation is important. We want to work to create best practices and protocols in the self-experimentation/quantified self world. We are planning on putting together some content (videos and such) on the topic and incorporating that educational aspect in our platform.

As far as the supplements go, I agree with that as well. We're planning on adding more once the L-Theanine trial is sorted out. This is just our practice round to get the protocol right. What other supplements would you be curious to see?

Thank you for your informative and critical feedback - skepticism is at the heart of science! Very true, the system could potentially be used to promote a specific agenda. However, we very much want to avoid a system that allows something such as this as much as possible. Ideally through implementing open-source / 3rd party / community-input based protocols. If you have suggestions in this regard we'd love to hear them!

And yes, we plan on including food at a later date!

We picked L-Theanine because of its safety profile and mild psychactive properties, meaning you can tell if you've taking it or not but it's not an overbearing feeling. We wanted the first experiment to be fairly predictable while we work through all the kinks of launching this.

This won't be an issue because we are supplying the supplements which are going to come from an established and reputable supplier. This is not a platform where anyone can sell their own supplements.