Not that I disagree, but I don't think that's a reason to not publish. There's another way to rephrase what you've said
many ideas that work well at small scales do not trivially work at large scales
> Other ideas that work at super specialized settings don’t transfer or don’t generalize.
It is also hard to tell if these are hyper-parameter settings. Not that I disagree with you, but it is hard to tell.
> Correlations in huge multimodal datasets are way more complicated than most humans can grasp and we will not get to AGI before we can have a large enough group of people dealing with such data routinely.
I'm not sure I understand your argument here. The people I know that work at scale often have the worst understanding of large data. Not understanding the differences between density in a normal distribution and a uniform. Thinking that LERPing in a normal yields representative data. Or cosine simularity and orthogonality. IME people that work at scale benefit from being able to throw compute at problems.
> we don’t do anybody a favor by increasing the entropy of the publications in the huge ML conferences
You and I have very different ideas as to what constitutes information gain. I would say a majority of people studying two models (LLMs and diffusion) results in lower gain, not more.
And as I've said above, I don't care about novelty. It's a meaningless term. (and I wish to god people would read the fucking conference reviewer guidelines as they constantly violate them when discussing novelty)