Why not both? User testing won't help you with questions like "there's a bug in feature X, but the fix will potentially impact feature Y in unforeseen ways. Given we're shipping soon, is it a good idea to apply the fix now or should we apply a much simpler mitigation and postpose the full fix till next iteration so we can get more testing in or even implement a larger refactoring that would resolve this category of bugs by design?"

If feature X is used way less frequently than feature Y, the answer is obvious. If feature Y is used way less frequently than feature X, the answer is obvious. If its a wash, you move on to look at different metrics.

This isn't a hypothetical, I ran this exact scenario yesterday against the VS Code telemetry.

I applied the mitigation, the refactoring is coming next week. :)

For some products, yes, user shadowing, interviews and testing is enough.

But in the case of VS Code, I agree with the other commenters that you can benefit from both.

VS Code supports many languages, source control systems, workflows, external systems and extensions, and it's used by a very wide spectrum of users and developers. I can see why they believe telemetry is required to better understand how it's used.

They don't have to be mutually exclusive, do they? You can surely learn some things from large-scale telemetry collection that you can't learn from user studies (e.g., semi-representative "in the wild" performance metrics).

People act differently when they're being watched. Capturing the normal usage as they work with your product without any oversight can reveal a lot.