The market conditions are set by the number of people trying to buy the cars and how much they're willing to pay.
The dealership meets the market price, not sets it. If their fee was unreasonable people would buy other cars until the dealer dropped their prices to something reasonable.
People don’t want to buy other cars just because the dealers are colluding and raising the prices despite adding no value.
When they do decide to avoid a car just because of the leeching dealerships, it hurts the manufacturer too