That's trusting a third party.
> get a hardware wallet like Ledger
Again, trusting a third party, and one which had a data breach less than a year ago:
https://www.ledger.com/blog/security-incident-report
And even if you want to discount that, you still have the risk of physically losing the wallet or having it stolen.
> If, by hiring someone, you mean using a custodian like Kraken,
Yes, that's what I mean.
> then you're still solving a problem by not dealing with the traditional banking system.
And, like I said, replacing it with a different problem, of dealing with and trusting a crypto custodian.
> Is everyone supposed to have a personal contact from within a small bank?
In a perfect world, yes. In this world, anyone running a business should have at least one such contact.
> The closest compromise might be to work with credit unions instead of banks.
Yes, that's also a good solution, much better than crypto IMHO.