I think it's a common shared understanding and a focal point, and there's huge value in preserving that. Having a different common understanding might be acceptable, and might even be an improvement, but only if people agree on it. Having no common understanding would mean something of great value was lost.
Right now, many different groups who may not all agree on all goals nonetheless gain value by sticking to Open Source, and not a dozen slight variations on additional restrictions. In the absence of that, we'd have the "non-commercial" group, the "educational use" group, the "don't compete with us" group, the "don't use for military applications" group, the "don't use for nuclear reactors" group, the "don't use to develop proprietary software" group, the "don't redistribute unmodified versions" group, the "don't distribute outdated versions" group, a dozen variations on "don't distribute if you disagree with our values" groups, the "don't distribute if you're a large company" group, the "don't distribute if you're a specific company" groups...
Every one of those is a license term I've seen advocated. Every one of those violates the OSD, so it thankfully stays obscure and unpopular, because enough people prefer using and contributing to software with less unusual licensing.