'Comically bad' datasets used to train clinical models for stroke and diabetes (opens in new tab)

As a clown, I can confirm.

If you hand me a clean, well-labeled, representative dataset, I can make the model do a respectable little dance by lunch.

If you hand me a Kaggle CSV with duplicated rows, target leakage, mislabeled outcomes, and columns named final_final_v2_REAL, suddenly I’m not doing ML anymore. I’m doing archaeology with a red nose on.

The model is the balloon animal. The dataset is the elephant you had to drag into the tent.

This holds in software as well. I encounter people trying to build solutions for problems that might not even exist, even in the context of addressing a specific bug. The act of measuring and collecting data is hard work, pretty boring sometimes, and often prescriptive in ways that aren't appealing. It's like we'd rather guess and use the ambiguity to allow ourselves to explore solutions we're more interested in. The alternative is manually profiling and poring through logs, so, I kind of get it.

For a lot of clinical decision support use cases you don't even need fancy AI models to get accurate results. If you have good quality cleansed data you can literally just import it into Excel and run a simple linear regression analysis. But unfortunately that won't get you a reputation as an "AI thought leader".

So true and it's been like that for ages. It's why I called these people rogue data scientists five years ago:

https://ianreppel.org/how-to-spot-a-rogue-data-scientist/