I used to wear your shoes in IE6/7 ages (no longer, I gave up during the "framework of the week" race and went all-backend), and it wasn't simple at all. Browser compatibility with all their rendering nuances, individual system oddities and all sort of fragile stuff.
And fortunately, no one bats an eye at a slightly broken site, but everyone hates even a slightly broken vehicle.