That's fine. I don't think any individual is foolish for preferring to keep these companies out of the process.
But it is just undeniable at this point that any authentication system other than raw passwords must come from any already ubiquitous ecosystem that doesn't require people to download, install, or buy anything new. Hoping that yubikeys take off is fantasy.