Destination IP will be the same for all sites on the server, SNI tells you exactly which site was asked for. Not meaning to be pedantic, sometimes the distinction isn't clear.
But in order to encrypt the SNI name, you'd first need to verify a certificate tied to a bare IP address. You'd also need to trust DNS completely. RTT would inflate significantly.
The CA system is a mess, but DNS is worse. Tying certs to bare IPs would create a deployment nightmare as well.
SNI is imperfect, but it is a big improvement over the previous status quo, which was single-IP per https host, which obviously did nothing to obscure the site hostname either.