SO, at its best, is numerous highly-experienced and intelligent humans trying to demonstrate how clever they are. A bit like HN, you learn from watching the back and forth. I don't think this is something that LLMs can ever replicate. They don't have the egos and they certainly don't have the experience.
Whatever people's gripes about the site, I learned a hell of a lot from it. I still find solutions there, and think a world without it would be worse.
In the time it takes to properly format and ask a question on Stack Overflow, an engineer can iterate through multiple bad LLM responses and eventually get to the right one.
The stats tell the uncomfortable truth. LLMs are a better overall experience than Stack Overflow, even after accounting for inaccurate answers from the LLM.
Don't forget, human answers on Stack Overflow were also often wrong or delayed by hours or days.
I think we're romanticizing the quality of the average human response on Stack Overflow.
Hey, can you show me the log files?
Sure here you go. Please help!
Hmm, I don’t really know what I’m looking for in these. Good luck!
Some people take that as a personal attack, but it can be more helpful than a detailed response to the wrong question.
Stack Overflow is explicitly not for "dialogue", recent experiments (which are generally not well received by the regulars on the meta site) notwithstanding. The purpose of the comments on questions is to help refine the question and ensure it meets standards, and in some cases serve other meta purposes like pointing at different-but-related questions to help future readers find what they're looking for. Comments are generally subject to deletion at any time and were originally designed to be visually minimal. They are not part of the core experience.
Of course, the new ownership is undoing all of that, because of engagement metrics and such.
Interesting question - the result is just words so surely a LLM can simulate an ego. Feed it the Linux kernel mailing list?
Isn’t back and forth exactly what the new MoE thinking models attempt to simulate?
And if they don’t have the experience that is just a question of tokens?
That won't be happening anymore, neither on SO or elsewhere. So all this hard won experience, from actually doing real work, will be inaccessible to the LLMs. For modern technologies and problems I suspect it will be a notably worse experience when using an LLM than working with older technologies.
It's already true for example, when using the Godot game engine instead of Unity. LLMs constantly confuse what you're trying to do with Unity problems, offer Unity based code solutions etc.
I think the name "Mixture of Experts" might be one of the most misleading labels in our industry. No, that is not at all what MoE models do.
Think of it rather like, instead of having one giant black box, we now have multiple smaller opaque boxes of various colors, and somehow (we don't really know how) we're able to tell if your question is "yellow" or "purple" and send that to the purple opaque box to get an answer.
The result is that we're able to use less resources to solve any given question (by activating smaller boxes instead of the original huge one). The problem is we don't know in advance which questions are of which color: it's not like one "expert" knows CSS and the other knows car engines.
It's just more floating point black magic, so "How do I center a div" and "what's the difference between a V6 and V12" are both "yellow" questions sent to the same box/expert, while "How do I vertically center a div" is a red question, and "what's the most powerful between a V6 and V12" is a green question which activates a completely different set of weights.
Perhaps the antidote involves a drop of the poison.
Let an LLM answer first, then let humans collaborate to improve the answer.
Bonus: if you can safeguard it, the improved answer can be used to train a proprietary model.
I'm more amused that ExpertsExchange.com figured out the core of the issue, 30 years ago, down to their site's name.
