Politics has been a significant part of Twitter's success, driving relevance & audience size. Politics was fuel for growth and Twitter was influential partially because of it's openness. The moderation was there because being a relatively balanced & hate free spare without too much controversy was essential for the majority of advertisers.
To reverse those parameters and turn it into a walled garden of Musk-like right wing edge lords that increasingly promotes right wing edge lord content that significantly reduces its safety for advertisers and then complaining that it's purely down to other people making the advertisers flee requires some major cognitive dissonance or a very blinkered world view.
Musk's aim for X.com is to make it an everything app, which is obviously impossible if it does not appeal to almost everyone. As an owner he has every right to make it into a smaller, politically homogeneous message board but he shouldn't simultaneously complain about the very obvious & easily predicted effects of that.