Humans are perfectly capable of assuming unproved axioms, changing their set of axioms, and accepting contradictory axioms. We do these things all the time, applying one axiom in one situation, and it's opposite in another. Just ask someone about their political beliefs for a while.
The fact is human intelligence is not an end in itself, it's a tool we use to achieve goals set by our evolutionary priorities, as encoded into our emotions and needs. These are the things that drive us, not logical axioms and proven truths. Even smart people have an emotional need to be correct, and many will resist having their beliefs challenged and changed tooth and nail. It takes constant effort and self discipline to maintain an open mind to new ideas and the rejection of existing assumptions, and certainly doesn't come naturally to us.
So this systematic theorising all seems somewhat beside the point. Don't get me wrong. It's interesting and useful philosophical work, no question, but it's not really applicable to actual human minds.
What these arguments claim is that if a digital computer could emulate a human mind, then the human mind is ultimately algorithmic (so there is an algorithm that can produce all the theorems that the human mind is capable of producing.) The Church-Turing thesis says that every algorithmically computable function is computable with a Turing machine. This is equivalent to the proposition that the collection of humanly knowable theorems can be recursively axiomatized in some formal theory T. This theory would then be consistent [2].
Given that this hypothetical system is consistent, it has at least one Gödel sentence: a true statement which cannot be proven from within the system. Lucas and Penrose assert, however, that humans could see its truth by "stepping outside of the system", as they do, for example, in proving the consistency of Peano arithmetic. Therefore, there would be at least one thing that a human mind could do, but which its supposedly equivalent digital computer could not.
So Lucas and Penrose are not assuming that humans are consistent; on the contrary, they are saying that Strong AI proponents themselves are implying that minds are reducible to a consistent formal theory. As Lucas and Penrose do not accept the Strong AI premise, they are not making this implication, and they do not have to reconcile it with the observed inconsistency of humans - that is, as it were, left as an exercise for Strong AI proponents to solve (this, however, is not the crux of their argument, which concerns deducing the truth of the implied system's Gödel sentence, as outlined above.)
I believe these arguments can be plausibly challenged, but not by anything so simple as observing that humans are inconsistent.
[1] https://iep.utm.edu/lp-argue/
[2] https://www.maa.org/press/maa-reviews/g-dels-disjunction
"Whether or not Gentzen's proof meets the requirements Hilbert envisioned is unclear: there is no generally accepted definition of exactly what is meant by a finitistic proof, and Hilbert himself never gave a precise definition.
The vast majority of contemporary mathematicians believe that Peano's axioms are consistent, relying either on intuition or the acceptance of a consistency proof such as Gentzen's proof. A small number of philosophers and mathematicians, some of whom also advocate ultrafinitism, reject Peano's axioms because accepting the axioms amounts to accepting the infinite collection of natural numbers."
- Me again - Frankly that doesn't seem to me to be a case of stepping out of anything to prove it. All they did was expand the set of axioms to include an additional set that allowed the construction of a separate proof, but that still leaves you in the same position you started in because you can't prove the new expanded system in terms of it's aggregate axioms either. In fact it seems we don't even have a clear definition of what a finitistic proof even is. I see no reason why an algorithmic system couldn't engage in such games.
Which to me makes more sense at least than the first option, which seems to argue that human minds cannot be mechanized because humans are capable of proving mathematical truths and machines have mathematical truths they cannot prove.
[EDITED to add:] Contra simonh, though, one can refute the argument without going so far as to say that human mathematicians are definitely inconsistent. (Maybe a sufficiently careful human mathematician is consistent.) All that's required is that we not be able to prove that we are consistent, and I think it is extremely clear that we can't.
In other words, the Lucas(/Penrose) argument was refuted before it was ever published.
(Penrose's version isn't really any improvement on Lucas's.)
Note: The video is 2 hours long and highly technical. I haven't watched anything like all of it. The speaker is _not_ endorsing Lucas's or Penrose's conclusion that Goedel's theorem shows that minds cannot be mechanized; he makes observations similar to Putnam's, simonh's, and mine, but clearly makes them with more subtlety and intricacy :-).
We don't have absolute knowledge in Physics, or science generally in the strict sense that this video and philosophers of logic discuss. We only have useful theories that describe the world very closely to reality across most situations.
We also very importantly get to conclusively rule out an awful lot of ideas and theories. That's very important work. It is all about levels of confidence though. If you think about it, you probably already knew all this, you just hadn't considered the consequences yet. We're all in that boat about a lot of things. I'm afraid in the grand scheme of things, despite all out truly incredible accomplishments, it's also true that we're stuck with fairly limited mental faculties.
As to why? Well, we need to feed and clothe our children somehow. That means figuring out how to get stuff done. Working out explanations of the world that verifiably work, more or less, has turned out pretty well for us.
I think I'm just too dumb to understand this discussion. I think for me we could split the discussion in two potentially separate questions:
- Can a machine emulate a human mind so well that it would be indistinguishable from a "real" human to an external observer (that's the Turing test, effectively)
- Can a machine emulate human consciousness
And maybe a third bonus question:
- Is there a meaningful difference between these two propositions from a scientific perspective? I.e. can we make falsifiable claims that would let us suss out philosophical zombies?
At this point I'm absolutely convinced that the answer to the first question is affirmative. We're not there yet and there's quite a long way to go until we do, but I really don't see why there would be a fundamental mathematical hurdle on the way there. Maybe it exists, but I have yet to find a really compelling argument of where that hurdle would lie concretely. Give me an example of a thought that we couldn't teach a machine.
Regarding the 2nd question (and the 3rd) it just boils down to "what's consciousness exactly? Is it even knowable?" and I don't think anybody has an answer to that. My personal intuition is that it's unknowable.
You can't even know if anyone other than you is conscious. For all you know, the rest of us might all be robots being good at fooling you.
I think that if we achieve AGI, at some point there will be a division between people: Those who acknowledge them as living, feeling entities, and those who don't.
Just like even nowadays there are still people who feel about others (based on race, skin color, etc), that they're not really people.
This claim comes up quite often in these sorts of discussions, but it is very hard to maintain this level of skepticism, about the external world, consistently (to start with, do you believe any of your thoughts are about an external world?) To take this extremely skeptical position only about other peoples' minds would be quite tendentious.
Given that we can't define consciousness, even that might be implying more knowledge than someone can possess.
I'll take a stab at what seems to me to be an insurmountable problem for AI: vision. And not just vision, but "seeing". Pretend I am standing next to a table, and the AI successfully identifies me, and identifies a table next to me (which, in itself, is a very difficult, if not impossible, thing for AI to do right now). Now, let's say I sit on the table. Now, we ask the AI, "Am I sitting on a table, or sitting on a chair?"
Further to the point (and a less "human" scenario): Musk has been attempting for years to make a lights-out facility for building cars. As of yet, there are still many things that a robot cannot do, or a human can do faster and more reliably, in spite of throwing billions of dollars and millions of work-hours at the problem. Another example: shoe manufacturing robots. Another example: brick laying robots. Another example: pipe welding robots. None of those can even come close the a human's ability to adapt on the fly to small variations, or to learn new behaviors. AI sees the world in a very low-resolution way, and humans are generally unaware of how much "constructing of the world" our brains do compared to the input data it receives from our senses. Replicating this is going to take more than a GPT3, for example.
I feel the missing ingredient is to be able to combine this fuzzy or intuitive understanding with some kind of rule engine.
It's like in the book "Thinking Fast and Slow". It feels like GPT and the best image recognition algorithms we have today are good at the fast kind of thinking but doesn't have the slow and logical part.
Is that right? I thought computer vision was getting pretty good at object detection in the past few years.
Wikipedia has a short summary of some of the objections[3], and I think Minsky's is particularly to the point: as humans can believe false ideas to be true, human mathematical understanding need not be consistent and consciousness may easily have a deterministic basis.
[1] http://cogprints.org/356/1/lucas.html
[2] https://www.maa.org/press/maa-reviews/g-dels-disjunction
[3] https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument...
If you program a mind specifically to say "I'm concious.", it does not prove that it is.
However, if conciousness depends on something we cannot simulate and that conciousness is the causal reason we think (or say) we are concious, I would not expect that an exact simulation of a brain would be able to do the things we attribute to conciousness.
So if you emulate a mind to the best accuracy available, and it suddenly says "I'm concious.", it is at least a very strong indication that it really is, at least in the same sense that humans are.
So, for example, if by "consciousness" we mean "intentionality" or at least that it entails intentionality, then you have to explain a) how anything within a computer can be about anything in the world, and b) how abstract concepts can exist in a computer. And you have to think very carefully about this because the internet is full of flippant and unexamined responses that confuse the interpreter with the interpreted.
Let's say I take a picture of a bike wheel and store it on a computer. What makes this about the bike wheel? I know that the picture is about the wheel, but the picture does not contain that information. No "metadata" can accomplish that either because metadata is of the same nature as the image. Metadata isn't even intrinsically about the image in question. It's there, maybe adjacent to it, but its aboutness is, like the image's aboutness, entirely in my mind. A program can be written to simulate aboutness, but again, the aboutness is simulated in the sense that it is not a real feature of either the metadata or the program. It is an interpretation the programmer or user brings to the computer. So again, no aboutness is to be found in the program either.
Then there's the question of abstract concepts. A wheel is a concrete thing, but to know that it is circular is no longer a concrete matter. Circularity is nowhere to be found in the world as such. All you ever have are particular circular things. The concept of circularity requires abstraction from particulars. So why can't a computer do that? The reason is the same as the reason why circularity does not exist in the world on its own and that is that matter is always particular. Matter is not abstract. But computers are material things. So how could a computer abstract circularity from an image of a wheel? Sure, you can have an algorithm that is written to match circles in images, but this is not the same as having a concept of circularity on its own. For that you need an intellect.
The problem of qualia is also related. A materialist account of physical reality entails the denial of qualities like color because all that a materialist accepts is a world of geometric extension in space. Qualities are therefore taken to be features of consciousness. But if consciousness is a physical phenomenon, then how can that be? Here again computers provide no answer. Sure, I can represent the color "red" using #FF0000, but this is not the color red. I can construct a monitor that, when fed a signal that corresponds to #FF0000 activates some physical elements to produce red light, but this is just a conventional alignment between that representation and the construction of the monitor. There is nothing intrinsically red about #FF0000. And besides, the monitor isn't really producing red light according to the materialist. It is producing light that is in a state that, when observed by a conscious being, appears red (which, again, runs into the problem of how even that is possible given that consciousness is physical).
So the whole idea that computers can be understood as truly autonomous things apart from human beings is bankrupt. They are better understood like all other technology: an "extension" of human beings. While most animals have all they need in themselves to achieve their ends, human beings can construct technological artefacts to perform actions that they could not otherwise. A turtle has a shell to protect himself, but he is also limited to that shell. Human beings are on the other hand lack any such protection, we're soft and easier to hurt, but we have a mind that allows us to make protection when we need it and in an (in principle at least) uncountable many ways depending on the needs facing us. We make coats for winter weather, astronaut suits for the low-pressure environment of space, a beekeeper's helmet when extracting honey from hives, and so on. Computers are just another expression of the same principle.
We do have kinematic simulations of this kind, but what we don't have is a way to map a visual image of a bike to generate an internal kinematic model, or associate them together in a useful way, and then contextualise that within operational and intensional models, then a way to relate runs of the simulation to the actual physical environment in an intensional way. I don't think there's anything impossible about any of that, but we certainly don't have this capability at the moment. There's a long, long way to go.
