>I do not accept responsibility for feelings of offense others may feel in response to my speech.
If you know some people will be offended by a particular usage and you choose to use it anyway, then you are responsible for the resulting offence, whether you accept it or not.
Sometimes there are benefits which make the offence worthwhile, particularly when the offence is exceptionally uncommon. But using gender specific speech in generic/unknown contexts has no such benefits, and will cause offence fairly often. Singular they is easy and idiomatic, and I don't see any real excuse not to use it, other than laziness.
>It's not my job to manage their emotional state.
No, it's just a nice thing to do.