I've seen your argument many, many times before, and it comes from people who have divorced their understanding of value from their understanding of work.
You don't get paid to work, you get paid to provide value, and I'm not saying that's right or how it should be, but that's how it currently is. If, through your labor, you can't provide value above your cost to a company, the work you're doing is not worth the cost. One way to get an idea of how much what you're doing will cost is to ask you and your peers what you think it'll cost, and one way of measuring that cost is a Story Point.
