In addition to custom hardware (like the crypto cards modoc mentions downthread) you also have to look at the potential liabilities you face. If you're involved in a HIPAA compliant environment, or one where you are storing large amounts of personal information, it may be a requirement that you have certain controls on physical access to the machinery and logical access to the data that are harder to guarantee with a VM Provider.

This isn't to say that a VM might not be perfectly acceptable for parts of your operation. But realistically, at this time if you look at the way most of the compliance documents for data security are written; you cannot be compliant in a suppliers shared virtualized environment.

There are some applications which benefit from custom setup hardware (cypto cards, extra NICs, etc...), you may want more network security than the single network or double homed boxes on simple public and private LANs (you may need isolated firewalled internal networks beyond what you can get from leased host providers). You also may want to drop in some of the various appliances out there for stats/analysis, spam/virus protection, google search boxes, etc...

GPGPU!