I feel the need to emphasize. This was not a
silly comment. It was abuse. This wasn't misplaced careless language, overheard. This was a targeted and abusive insult to another person who was sitting across the table.
Brushing off the 'silly' comment and moving on is the easiest way to hurt the target more than the comment already has.
I agree that it's easier to say that "my opinion is different", but it's chicken-shit and offers a level of respect that signals unwarranted sympathy. "Your opinion is stupid" would be a profoundly more appropriate statement had it been be, because it much better describes my feelings.
Without making specific slippery-slope arguments, surely at some point we'd all find it ridiculous to say "Thanks, but my opinion is different" after witnessing something disgusting, right? So I guess the question is where that line is for us individually. "You got this pass because of your tits" is severely past that line for me. It's miles deep into "Fuck off" territory.