Compare an analogous concern to the one you raise: "what if caring about people of color comes at the cost of empathy for your fellow whites?"
[1] https://goo.gl/maps/iDZ2TZV4o12DiKtL8
[2] https://www.theatlantic.com/business/archive/2014/06/how-cit...
If you have a lot of pressures in your life, adding a diet change to that is not realistic. Pushing for meat alternatives in convenient places and making it easy for large groups of people to have small cognitive load (low cost of spoons) to go without meat for a meal is far more impactful than trying to convince everyone to go “full vegan or you are a monster”. If you have the time to consider your options and make a change to your habits that match your values, great, otherwise power to those trying to make those choices easier for everyone. Don’t let it weigh on you, take care of yourself and create space to make the changes you want to see in yourself.