Your approach would definitely protect you. In practice, many site owners don't do this, or they configure their web server with the whitelist instead of their firewall, denying direct access but exposing information about their domain.
For site owners who don't know about this, these are the IP addresses you can expect traffic from: https://www.cloudflare.com/ips/
I'd personally advice using IPv6 (with a high, random address rather than the common aa:bb:cc:dd::0) to make scanning for hosts a lot harder to accomplish, just in case your firewall fails for some weird reason.