I personally find that if I don't use them, my socks, towels, etc all come out very stiff and dry.

Yes, I know it's not ideal for towels because the wax inhibits some absorption, but I couldn't stand it so I just went back to using dryer sheets.

Soon I'll be getting a good quality dryer with a steam sanitize mode, maybe that will help.