That's an awful lot of "mights".
Luckily, he won't be able to say "I told you so" when our universe runs out of memory.
The induction fallacy is what allows humans to get up in the morning and make decisions with incomplete data i guess.
For instance, perhaps they are a form of communication? We could be just a really complex story that one being is telling to another.
We already build very powerful intelligences all the time, in the form of children. But despite having a lot of raw computing power, they aren't very useful without knowledge. They need to learn a lot of things before they can live effectively, solve important problems, etc...
Artificial intelligence and intelligence are not fundamentally different things. If you want AI's that turn out better than children do, you'll need improved parenting and educational techniques, or an alternative. I haven't seen any commentary on how singularity people plan to deal with this. And if they could, why not just use that technique on human children and vastly improve the world, now?
The main difference between AI and children I've seen proposed is that the AIs can run on superior hardware. But it's not like people currently use all the hardware resources their brain makes available; maybe that will be a bottleneck in the future, but it isn't one we've hit yet.
Back to children, we're so good at teaching them that we manage to teach them many inexplicit ideas without ever having much understanding of those ideas.
