Your prospective employers do not expect every coffee, phone screen, or interview to result in a new employee. They understand that they will talk to people who are a bad fit, and that's part of the process.
I respect that you feel that way, and have felt that way myself, but it gets in the way of advantageous moves.