If you’re free to be scammed out of your money, with no repercussions to others, sure it’s unreasonable to stop you. But with (American) credit cards, it’s the backing financial institution that bears the burden of fraud; merchants accepting fraudulent transactions are punished as well.
If you were talking about asking someone’s permission to surfs your own cash money, that would be outrageous.
A bank account or a credit card is a relationship where you rent someone else’s infra to make payments. Makes sense to work together to minimise friction for both parties.