Inference will get much much cheaper. We're paying $30000 for a top of the line gpu right now, but that's only because everyone insists on buying nvidia, so nvidia has full incentive to charge the absolute maximum.
Long term that vendor lock in will go away and prices will go down to something reasonable.
Long ago CPUs were super expensive too, now they're so cheap we put them in toothbrushes
