I'd argue that the innovation with these models is making smaller. Just throwing compute resources to make a model with more parameters is easy and doesn't really expand our knowledge. IMO, larger and larger LLM's aren't that impressive. Being able to shrink that model down, retain its accuracy (to a degree) and be able to run it on smaller hardware is impressive and will more likely lead to AI/ML being intertwined within people's day-to-day