That's not true. The working memory is finite, you can't keep things in it indefinitely. You can imagine your working memory as a queue[1] - you put things into it through one end, and you take things out of the other end to put them into code. If you can't write fast enough, the queue will overflow and loose a portion of the information you fetched from long-term memory to solve a problem. If that happens, you will have to restart the solving of the problem and re-fetch the required information. This takes time, but more importantly, it disrupts your focus and has a chance of getting you out of the zone.

There's a speed of writing that, depending on the size and volatility of your short-term and working memory, is optimal for you: you don't need to work to improve the speed above that threshold. But if your writing speed is significantly below that threshold, it will slow you down and decrease your productivity. You probably naturally achieved the speed optimal for you long ago, and forgot how you felt before you got to that point.

[1] For the sake of argument only. Obviously, in reality it's more of a graph than linear queue, but it still has limited capacity.

There's a huge mental component to coding (I doubt it's 90% though). In any case, an IDE makes that easier by getting the boring physical stuff out of the way quicker.

And the speed at which you write code is quite meaningful. I can easily write 1000 lines of high-quality Rust code on a good day with just the LSP (rust-analyzer) for support. More recently, I started a project that hit some bugs in rust-analyzer -- working on it was a comparatively miserable experience.

Edit: Never mind, I see above that you're comparing a full-fledged IDE to just using an LSP. I consider LSP to be in the IDE category; I agree that LSP is just about where I want to be and that a full JetBrains-like IDE is overkill. What I don't like is not having any sort of code completion or rename support.