They don't just make accelerators, they'll sell you the hardware too (unlike TPUs). They don't just sell you the hardware, the software ecosystem will work too (unlike AMD or Intel). That hardware won't just do a lot of computations, it'll also have a lot of off-chip memory bandwidth (vs Cerebras or others). Need to embed those capabilities in a device that can't fit a wafer cabinet or a server rack's worth of compute, Nvidia will sell you similar hardware that uses a similar stack, certified for your industry (e.g. automotive). Take any of that away and you're left with a significantly weaker offering.
Also they benefit from the priority of paying fabs a lot of money and placing a lot of orders.
If anything, Nvidia is less dominant than they should be because they've managed to ensure absolutely no one wants to buy from them when there are viable alternatives.