Well, this really depends on the protocol and architecture of the system.
If it's a system where the server is merely store-and-forward, where it forgets its knowledge of messages after the recipients have received them, then sure, your stance is reasonable. The client will decide on message ordering; it can either just display in the order received from the server, or use any timestamps stored in the messages to order them (including possibly reordering if messages arrive out of order). The client has no other source of truth it can draw from, and so different clients may order things differently. (Even in this case, though, for many systems like this I would expect the server to timestamp the messages, and for all clients to honor those timestamps, so in practice everyone should see the same ordering.)
But Matrix is not such a system: each homeserver is the system of record for what messages have been received, and what order they go in. In this case, I would expect all users to see the same ordering, assuming all clients are able to query and receive the full history from the server.