> Does a customer care if you shave off 2% of the final cost or do they care about having world-class customer support?

Times and times again, they'll tell you the latter and actually choose the former.

The same bargain as paying more to have no ads: people vocally push for no ads, some will ponny up the money, and the vast majority will make do with the ad supported model, while ad blocking or giving up on the service when they're fed up with it.

> Does a customer care more about how long payment processing takes or do they care that it takes their local mobile payment app?

I'm curious how you find the people to ask that ? If your current service doesn't provide support for the payment app, who would you ask if it was a deal breaker for them and refused to become your customer, not giving you anybof their information ?

We need a new "appeal to customer" logical fallacy. Talking to the customer is not a panacea for running a business, and the failed company graveyard is full of products that "delighted customers" but still couldn't cut it in the long wrong.

Part of running a business is having to explain to your boss how you spent millions of dollars, and building confidence that you're making sound decisions and not just shooting from the hip. Many times that will mean making decisions in the best interest of the company over the customer, and there's nothing wrong with that.

There's nothing perfect about A/B testing, and like any tool it can do both good and harm. But when I have to explain to my boss about how I'm spending their money, there's a limit to how much lip service I can play up about the customer journey before I have to put my money where my mouth is and demonstrate that I'm putting their cash to good use. A story that includes A/B testing along with qualitative customer research is better than a story that just includes one or the other.