undefined | Better HN

0 pointsbalder19911y ago0 comments

Or another question, do they still publish any research that’s relevant for the field nowadays?

0 comments

No. They publish PDFs that hype up their models, but they do not publish anything even resembling a high-level overview of model architecture

jacobgorm1y ago

Given that you can download and use the weights, the model architecture has to be includded as part of that. And I did read a paper from them recently describing their MoE architecture and how it differs from the original GShard.

awestroke1y ago

Excuse me? What weights can you download from OpenAI? gpt2 does not count

1 more reply

j / k navigate · click thread line to collapse

0 comments

awestroke1y ago

No. They publish PDFs that hype up their models, but they do not publish anything even resembling a high-level overview of model architecture

jacobgorm1y ago

awestroke1y ago

Excuse me? What weights can you download from OpenAI? gpt2 does not count

1 more reply

j / k navigate · click thread line to collapse