I'd be very interested in your thoughts on that position, because if it's mistaken, I shouldn't be saying it. It represents whatever small contribution I can make to fellow new ML researchers, which is roughly: "watch out."

In short, for two years, I kept trying to implement stated claims -- to reproduce them in exactly the way you say here -- and they simply didn't work as stated.

It might sound confusing that the claims were "simply wrong" or "didn't work." But every time I tried, achieving anything remotely close to "success" was the exception, not the norm.

And I don't think it was because I failed to implement what they were saying in the paper. I agree that that's the most likely thing. But I was careful. It's very easy to make mistakes, and I tried to make none, as both someone with over a decade of experience (https://shawnpresser.blogspot.com/) and someone who cares deeply about the things I'm talking about here.

It takes hard work to reproduce the technique the way you're saying. I put all my heart and soul into trying to. And I kept getting dismayed, because people kept trying to convince me of things that either I couldn't verify (because verification is extremely hard, as you well know) or were simply wrong.

So if I sound entitled, I agree. When I got into this job, as an ML researcher, I thought I was entitled to the scientific method. Or anything vaguely resembling "careful, distilled, correct knowledge that I can build on."