I have similar reservations about code formatters: maybe I just haven't worked with a code base with enough terrible formatting, but I'm sad when programmers loose the little voice they have. Linters: cool; style guidelines: fine. I'm cool with both, but the idea that we need to strip every character of junk DNA from a codebase seems excessive.
For code that is meant to be an expression of programmers, meant to be art, then yes code formatters should be an optional tool in the artist's quiver.
For code that is meant to be functional, one of the business goals is uniformity such that the programmers working on the code can be replaced like cogs, such that there is no individuality or voice. In that regard, yes, code-formatters are good and voice is bad.
Similarly, an artist painting art should be free. An "artist" painting the "BUS" lines on a road should not take liberties, they should make it have the exact proportions and color of all the other "BUS" markings.
You can easily see this in the choices of languages. Haskell and lisp were made to express thought and beauty, and so they allow abstractions and give formatting freedom by default.
Go was made to try and make Googlers as cog-like and replaceable as possible, to minimize programmer voice and crush creativity and soul wherever possible, so formatting is deeply embedded in the language tooling and you're discouraged from building any truly beautiful abstractions.
Chaos, sure, but beautiful chaos.
But in my case it was the other way around. I work in a Kowloon Walled City of code: dozens of intersecting communities with thousands of informally organized but largely content contributors. It looks like chaos, but it works ok.
Code formatting really did feel like a new neighbor declaring "you know what this place needs, better-marked bus lanes!" as though that would help them see the sky from the bottom of an ally or fix the underlying sanitation issues. As you might imagine, the efforts didn't get far and mostly annoyed people.
But as the GP said, it all depends on the culture. If you pick up and move to Singapore you'd damn well better keep your car washed and your code style black.
It was a confluence of a lot of bad design features and blunders and I can't blame the formatter for the mess it caused. So I understand your point but, I'd amend it a bit: version control is the reason many projects require a specific formatting style.
In projects without an explicit style, the number one formatting rule is don't reformat code you didn't touch.
One factor is "churn", that is, a code change that includes pure style changes in addition to other changes; it's distracting and noisy.
The other is consistency, if you're reading 10 files with 10 different code styles it's more difficult to read it.
But by all means, for your own projects, use your own code style.