The biggest problem I can see with default deny is that it makes if far harder to get uptake for new protocols once you get "we only allow ports 80 and 443 through the firewall".
Yes, I think blocking outgoing connections by port is not the most useful approach, especially for default deny. Blocking incoming makes more sense, and should be default deny with allow for specific ports on specific servers.
