Whether it you think it should or not depends on your personal preferences, but in practice the government does get a say in anything that it deems to be an undue way to reduce your taxes.
Barter would be much more common if it was a legal way of avoiding taxes.