I see GPT-3 as an "advancement" rather than a "breakthrough". There were several differences to the previous model such as the transformer being used and the rest is more like data + compute power. Have a read from this post here about it. [0]
What the hype around this only shows is the market potential around GPT-3's capabilities and glosses over the issues around it because from a VC standpoint: "Who cares, I want that giant 10x exit."
Just like black-box models, GPT-3 is an opaque NLP deep learning model that takes an input to generate an output but doesn't know why and cannot explain itself or its own actions. It may be only text its generating but it is still unsuitable for very sensitive areas like law due to this despite several valid ideas in applying this into legal text. We need proper scrutiny towards GPT-3 from the tech industry and less of the hype manufactured by its merchants.
But that's all boring to everyone, look at what GPT-3 can do right now. Far more important than actual potential breakthroughs like explainability in deep learning, detection mechanisms or AGI these days /s
[0] https://pagestlabs.substack.com/p/gpt-3-and-a-typology-of-hy...
