> If there's a Facebook binary on my phone that I've never logged into, is it doing anything?
Couldn't it? Apps can start without ever being launched by the user, or continue running in the background after they are "closed", and that means that they can collect data then send it home or to third parties. There's a ton of things an app can do without any permissions or indicating to the user that anything is happening. It's been used for things like listening for audio beacons and reporting them when overheard.
I wouldn't want to trust a company like facebook to not abuse every option available to them to collect data. There's also a problem with vulnerabilities. That "unused" facebook binary might contain a flaw that could be exploited. Getting rid of installed applications you don't need/want is a good way to reduce your attack surface.