I'm not op, but I have had to do this a few times because of the same reasons. You pause, take a breath and kindly say "thank you for the opportunity, but at this time I don't think this is the right fit" and leave it at that. No need to embellish, or add extraneous detail or think you're being awkward because they will do the same thing if they don't want to go further in the process with you. It's just business, treat it as such.