I do, to an extent, agree. But I also think it's impolite to have relationships where "we're going to talk about what I want to talk about, but when we go to talk about what you want to talk about I'm just going to pick up my laptop and ignore you or tell you I don't want to".
You could argue, in your description, the same about RMS - he might feel entitled to someone's time to talk about free software.