Very true — it's not illogical to think that, or to use it as a reason to ignore the discussion. Responding to someone by pointing that reasoning out, however, is very seldom going to lead to productive discussion. It's usually best left unsaid.