And when two or more minority interest groups feel this way on each other's issues, it is a failure to enact the "will of the people" if you use strict issue-by-issue majoritarianism. After all, each individual pet issue fails on its own merits - they just aren't broadly popular enough.
Fundamentally, the issue is that strengths of preferences do not show up in a referendum on a topic. Furthermore, there's no way to credibly commit to a compromise that gets you something important in exchange for a relatively unimportant concession. I'm not saying these issues outweigh the benefits of direct democracy, they're just problems that can get solved by a representative system.