I will not deny you have made valid criticisms of the matrix protocol, but Signal has some very broken design choices as well. One of these is much easier to fix by motivated technical end users than the other.

At least in matrix a user choice of client codebase and binary can control which server they trust, and if they wish to automatically encrypt to new unverified room participants or not depending on their threat model. Technical users have total control over their rooms and the UX will improve for less technical users.

Also it is possible for someone to run the server daemon in a remotely attestable system like a Nitro enclave allowing it to prove to users what administrative features are enabled on a given server. This is something I am building foundations for right now.

Unlike Signal, any organization is free to run their own matrix servers with server-pinned channels adjusted for different threat models while still having access to the wider network. You get to choose the server that can decide your room membership, including choosing a server that does not grant a central administrator any significant trust.

If forced to pick between two flawed protocols, I will choose an open network controlled by democracy that gives individual users total freedom to make it better over a dictatorship for the long run.