I confess I'm not sure what I'd do with this in the random grab bag of Deep Learning knowledge I have, but I think it's pretty fascinating. I might like to see a trained latent encoder that works well on a bunch of different neural networks; maybe that thing would be a good tool for interpreting / inspecting.
Or maybe some metaparameter that mucks with the sizes during training produces better results. Start large to get a baseline, then reduce size to increase coherence and learning speed, then scale up again once that is maxed out.
I.e., self-supervised training is done to produce semantically sensical results, and the RL-trained conditioning input steers to contextually useful results.
(Btw., if anyone has tips on how to not wreck the RL training's effort when updating the base model with the recently encountered semantically-valid training samples that can be used self-supervised, please tell. I'd hate to throw away the RL effort expended to aquire that much taking data for good self-supervised operation. It's already looking fairly expensive...)
If we can get that, then maybe we don't even need to train anymore; it'd be possible to start to generate NNs algorithmically.
(bit of trial and error from https://github.com/zedeus/nitter/wiki/Instances)
[0] https://www.lesswrong.com/tag/recursive-self-improvement
It's not "recursive self improvement", which is just a belief that magic is real and you can wish an AI into existence. In particular, this one needs too much training data, and you can't define "improvement" without knowing what to improve to.
Recursive self-improvement isn't "maybe magic is real", it's "maybe the magic we already know about stays magical as we cast our spells with more mana."
Is there a law of thermodynamics which prevents AI from writing code which would train a better AI? Never learned that one in school.
And FYI here's OpenAI plan to align superintelligence: "Our goal is to build a roughly human-level automated alignment researcher. We can then use vast amounts of compute to scale our efforts, and iteratively align superintelligence."
I guess people working there believe in magic.
> and you can wish an AI into existence.
Eh? People believe that self-improvement might happen when AI is around human-level.
You need to apply Wittgenstein here.
This appears to be true because you haven't defined "better". If you define it, it'll become obvious that this is either false or true, but if it is true it'll be obvious in a way that doesn't make it sound interesting anymore.
(For one thing our current "AI" don't come from "writing code", they just come from training bigger models on the same data. For another, making changes to code doesn't make it exponentially better, and instead breaks it if you're not careful.)
> I guess people working there believe in magic.
Yes, OpenAI was literally founded by a computer worshipping religious cult.
> People believe that self-improvement might happen when AI is around human-level.
Humans don't have a "recursive self-improvement" ability.
Also not obvious that an AI that was both "aligned" and "capable of recursive self-improvement" would choose to do it; if you're an AI and you're making a new improved AI, how do you know it's aligned? It sounds unsafe.
I've been thinking about this recently. Personally, I've yet to see any compelling evidence that an LLM, let alone any AI, can operate really well "out of distribution". It's capabilities (in my experience) seem to be spanned by the data it's trained on. Hence, this supposed property that it can "train itself", generating new knowledge in the process, is yet to be proven in my mind.
That raises the question for me: why do OpenAI staff believe what they believe?
If I'm being optimistic, I suppose they may have seen unreleased tech, motivating their beliefs that seemingly AGI is on the horizon.
If I'm being cynical, the promise of AGI probably draws in much more investment. Thus, anyone with a stake in OpenAI has an incentive to promote this narrative of imminent AGI, regardless of how realistic it is technically.
This is of course just based on what I've seen and read, I'd love to see evidence that counter my claims.
Saying "well that is not physically impermissible" doesn't make it real.
In any case nobody has ever shown that recursive self-improvement "takes off", and nor is that what we should expect a priori.
GANs are another example of self-improvement. It was famous for creating "deep fakes". It works by pitting a fake generator and a fake detector against each other, resulting in a cycle of improvement. It didn't get much further than that, in fact, it is all about attention and transformers now.
This is just a way of optimizing parameters, it will not invent new techniques. It can say "put 1000 neurons there, 2000 there, etc...", but it still has to pick from what designers tell it to pick from. It may adjust these parameters better than a human can, leading to more efficient systems, I expect some improvement to existing systems, but not a breaking change.
(Though I suppose this skips Neuralink / step 3 and jumps right to step 4.)
Furthermore, I posit that resnet especially in transformers allows the model into a more exploratory behavior that is really powerful, and is a necessary component of the power of transformers. Transformers is just such a great architecture the more i think about it. It's doing so many things so right. Although this is not really related to the topic.
Transformers are just networks that learn to program the weights of other networks [1]. In the successful cases the programmed network has been quite primitive -- merely a key-value store -- in order to ensure that you can backpropagate errors from the programmed network's outputs all the way to the programmer network's inputs.
The present work extends this idea to a different kind of programmed network: a convolutional image-processing network.
There are many more breakthroughs to be achieved along this line of research -- it is a rich vein to mine. I believe our best shot at getting neural networks to do discrete math and symbolic logic, and to write nontrivial computer programs, will result from this line of research.
