The oft-repeated statement that "renting is throwing your money" is an implicit contrast to owning a home, where the mortgage payment "builds equity" in your asset that can be sold later.
"Throwing money away" means you don't get to own something that can be sold for money later on. That's why we "throw money away" on gasoline, but not "throw money away" in a savings account.
The second part of my argument is that throwing money away isn't necessarily a bad thing, because the alternative (such as paying to own something) can end up being more expensive and being a worse deal financially.