I bet a lot of these experiments would already solvable by putting the LLM in a simple loop with some helper prompts that make it restructure and validate its answers, form theories and get to explore multiple lines of thought.
If an LLM would be able to do that in a single prompt, without a loop (so the LLM always answers in a predictable amount of time), then it would mean its entire reasoning structure is repeated horizontally through the layers of its architecture. That would be both limiting (i.e. limit the depth of the reasoning to the width of the network) and very expensive to train.
But if a human is allowed time and internal reasoning iterations, so should the LLM when determining if it has deep insight. Right now we're simply observing input -> output of LLMs, the equivalent of snap answers from a human. But nothing says it couldn't instead be an input -> extensive internal dialogue, maybe even between multiple expert models for seconds, minutes or hours, that are not at all visible to the prompter -> final insightful answer. Maybe future LLMs will say, "let me get back to you on that".
From a computer science point of view: a single prompt/response cycle from a LLM is equivalent to a pure function; the answer is a function of the prompt and the model weights and is fundamentally reducible to solving a big math equation (in which each model parameter is a term.)
It seems almost self evident that "reasoning" worthy of the name would involve some sort of iterative/recursive search process, invoking the model and storing/reflecting/improving on answers methodically.
There's been a lot of movement in this direction with tree-of-thought/chain-of-thought/graph-of-thought prompting, and I would bet that if/when we get AGI, it's a result of getting the right recursive prompting pattern + retrieval patterns + ensemble models figured out, not just making ever-more-powerful transformer models (thought that would certainly play a role too.)
The LLM isn't the whole brain. Just the area responsible for language and cultural memory.
spoilers warning:
Is that basically the plot to westworld?
The great part is with clear enough directions it also knows how to evaluate whether its done or not.
No, it's the equivalent of putting a gun to someone's head and asking them "what are my intentions?" Which is readily available to any being with a theory of mind.
Because, obviously, training data probably includes a decent amount of motivation breakdowns as a function of coercion.
It doesn’t know why, but it knows what to say.
Put gun to persons ahead.
Ask them to do a division.
Then screaming at them "HOW DID YOU DO THAT, TELL ME NOW, OR YOU'RE TOAST".
Even most humans would splutter and not be able to answer.
Are they faking it? Will they murder me? Are they just trying to scare me?
I don't understand the point you're trying to make.
Or am I misunderstanding something about that technique?
Of course, yes, we do know one thing that's missing in LLMs, which is "loop and helpers" like you describe. Which I'm sure many people are currently hacking at - one way being for the LLM to talk to itself.
But as for "a theory of mind", if enough writings served as input, then LLMs do have plenty of that.
Another question is whether LLMs are raised to behave like humans (which might be where they most NEED some theory of mind). Of course not. The ones we know most about are only question answerers. The theory of mind they might have (that is not negated by the lack of loop and internal deliberation) may be overwhelmed by the pre- and post-processing: "no sex, no murder plots, talk to the human like they are 5, bla bla bla". And yet you can ask things like "Tell it like you are speaking to 5 year olds who want to have a fun time". Some theory of mind makes it through.
While this is a simple example, the concept extrapolates to more complex issues as we develop cognitively and emotionally. Not only is theory of mind about recognizing that your thoughts and feelings are separate and distinct, but also being able to project your understanding into others to predict how they might be thinking and feeling in their own separate circumstance. This is the fundamental basis of empathy, or being able to predict and understand how others around you may be feeling given their unique current circumstances even if they are different from your own.
LLMs have not demonstrated any of the above understanding. Again, I'm sure they could generate a definition for it, and on any given prompt they could probably even generate some generic-ish text that could pose as empathy, but so can a horoscope. But there is not the deep, continuous comprehension of the user as a separate thinking and feeling agent that is fundamental to the idea of theory of mind.
These responses are logically the same as "No You".
https://www.sciencedirect.com/science/article/abs/pii/S00100...
I get frustrated often when people argue "well, it isn't really intelligent" and then give examples that are clearly dependent on our brain's chemical state and our bodies' existence in-the-physical-world.
I get the feeling that when/if we are all enslaved by a super-intelligent AI that we do not understand its motives, we will still argue that it is not intelligent because it doesn't get hungry and it can't prove to us that it has Qualia.
This paper argues that gpts are bad at understanding human risk/reward functions, which seems like a much more explicit way to talk about this, and also casts it in a way that could help reframe the debate about how human evolution and our physical beings might be significantly responsible for the structure of our rational minds.
It doesn't appear until the early 20th century, in the shadow of compulsory education and the challenges it presented, first as a technical label for attempts to sort students -- and later soldiers -- into the tracks in which they're most likely to succeed, and then being haphazardly asserted (but not scientifically evidenced) as some general measure of mental aptitude.
At that point it shifts from something qualitative (which mental tasks might someone be good at) to something quantitative (how much more might one personal excel at all mental tasks than another), and the burgeoning field of modern American psychology goes "Aha! A quantitative measure! Here's our meal ticket to being recognized as a science instead of those quacks from Vienna", with far too much at stake to either question the many assumptions at play or the inconsistent history of usage.
Momentum takes hold and the public takes the word into its everyday vernacular, even while it's still not a clear and sound concept in its technical domain. [Most of this is history is more academically covered in Danziger's 1987 "Naming the Mind" which is excellent, and critical foundational reading to contextualize recent hot discussions in AI]
The way you're using it when you worry about "super-intelligence" is in the sense of intelligence being some universal, unbounded, quantitative independent variable along the lines of "the more intelligent something is, the more cunningly it can pursue some rationalized goal" -- some master strategist.
That's fine, and you're not alone in that, but there's not really any sound scientific groundwork to establish that there exists some quality of the world that scales like that. You're fear, and what you try to distinguish conceptually from what the paper addresses, is an inductive leap made from highly unstable ground. It's in the same invented, purely abstract idea-space of "omnipotence" or "omniscience" where one takes a practical idea like "power to influence" or "ability to know fact" and inductively draws a line from these practical senses towards some abstract infinite/incomprehensible version of that thing. But that inductive leap a Platonic logician's parlor trick and ends up raising all kinds of abstract paradoxes, as well countless physical impracticalities about how such things could exist.
So a lot of people (academic and lay) just aren't with you in taking that framing of intelligence very seriously. For many, an "super-intelligent" software whose "motives" we don't understand is just a program that produces incorrect outputs and ought to be debugged or retired, and the more interesting questions around machine "intelligence" are practical ones like "what tasks are these programs well-suited for". Here, the authors point out that the current batch of programs are not good at tasks that benefit from a theory of mind.
Knowing the answer to that kind of question reaches back to the earliest and least disputable sense of the word, where we saw that some new students and soldiers excelled at certain tasks and struggled with others, and wanted to understand how best to educated/assign them. And likewise, as we look at these tools, the pressing question for engineers and businesses is "what are they good for and what are they not good for" rather than the fantastical "what if we make a broken program and it wants to kill everyone and we don't notice and forget to shut it off"
It wouldn't have to want to kill everyone. As long as it doesn't want to not kill everyone, the side effects of it getting what it wants could be catastrophic.
> and we don't notice
How well do we understand what's going on inside ChatGPT? How well will we understand the next?
> and forget to shut it off
Earlier I would have argued that sufficiently advanced AI could prevent itself from being shut off via Things You Didn't Expect, and would instrumentally want to preserve its existence. But these days, people are giving ChatGPT not just internet access but even actively handing it control over various processes. At this rate, the first superhuman AI will face not an impermeable box but a million conveniently labeled levers!
I'm not sure what you mean here, since the word dates back to the late 14th century with roughly the same meaning as now. Perhaps you're thinking of "intelligence quotient"?
I appreciate the highlighting of the term intelligence being ill-defined. Moreover, it's certainly true that "AI safety analysts" takes intelligence as a sort magic wand term and this seems to drive their arguments.
All that said, since both computers and human brains are material artifacts, it doesn't seem impossible to create a device that combines their properties. It seems plausible that such a thing could have a variety of dangers.
For many, an "super-intelligent" software whose "motives" we don't understand is just a program that produces incorrect outputs and ought to be debugged or retired, and the more interesting questions around machine "intelligence" are practical ones like "what tasks are these programs well-suited for".
We saw early Bing Chat behave, not in ways we couldn't understand but like a deranged and vengeful human. Certainly, it was merely simulating human behavior but if today's methods produce artifacts that unselectively amplify human behaviors, it's not hard to imagine problems appearing.
We can hope that there's a fundamental difference between programs that simulate human language and programs able to plan and carry out long term goals (and carrying out long term goals is something people do so there's no good reason some kind of program couldn't do that).
I think you're right that particular weirdness of the "doomers" makes some other portion of the population dismiss concerns. But that isn't an argument that the doom isn't possible - it should be an argument to clarify how we talk of computation and human capacities (see, I don't to say "intelligence" unless I want to).
>Here, the authors point out that the current batch of programs are not good at tasks that benefit from a theory of mind.
Not good at tasks that benefit from a theory of mind extracted from visual data.
I'm just saying that I don't think there's any point on that line where we will be comfortable admitting that the machine is "intelligent" or "conscious" or "AGI," or whatever, and that I appreciate attempts to quantify (or at least qualify) what we MEAN when we say that, rather than just goalpost-moving.
We (mostly) don't want unaligned A(G|S)I. The outcomes of that could be extenstential.
A big part of the problem is that "is" has a wide variety of inconsistent meanings, and that this fact is sub-perceptual, and that it is culturally very inappropriate to comment on aspects of our culture like this, preventing knowledge of the problem from spreading.
/u/Swatcoder makes essentially the same point but in much more detail, though regarding less important words.
It's not just you. It hit me almost a year ago, when I realized my then 3.5yo daughter has a noticeable context window of about 30 seconds - whenever she went on her random rant/story, anything she didn't repeat within 30 seconds would permanently fall out of the story and never be mentioned again.
It also made me realize why small kids talk so repetitively - what they don't repeat they soon forget, and what they feel like repeating remains, so over the course of couple minutes, their story kind of knots itself in a loop, being mostly made of the thoughts they feel compelled to carry forward.
I would claim that most people use intuition/assumptions rather than internal chain-of-thought, when communicating, meaning they will present that simplified concept without second thought, leading to the same behavior as the toddler. It's actually trivial to find someone that doesn't use assumptions, because they take a moment to respond, using an internal chain-of-thought type consideration to give a careful answer. I would even claim that a fast response is seen as more valuable than a slow one, with a moment of silence for a response being an indication of incompetence. I know I've seen it, where some expert takes a moment to consider/compress, and people get frustrated/second guess them.
The problem is consciousness is a vocabulary word that establishes a hard boundary where such a boundary doesn't exit. The language makes you think either something is conscious or it is not when the reality is that these two concepts are actually extreme endpoints on a gradient.
The vocabulary makes the concept seem binary and makes it seem more profound then it actually is.
Thus we have no problem identifying things at the extreme. A rock is not conscious. That's obvious. A human IS conscious, that's also obvious. But only because these two objects are defined at the extremes of this gradient.
For something fuzzy like chatGPT, we get confused. We think the problem is profound, but in actuality it's just poorly defined vocabulary. The word consciousness, again, assumes the world is binary that something is either/or, but, again, the reality is a gradient.
When we have debates about whether something is "conscious" or not we are just arguing about where the line of demarcation is drawn along the gradient. Does it need a body to be conscious? Does it need to be able to do math? Where you draw this line is just a definition of vocabulary. So arguments about whether LLMs are conscious are arguments about vocabulary.
We as humans are biased and we blindly allow the vocabulary to mold our thinking. Is chatGPT conscious? It's a loaded question based on a world view manipulated by the vocabulary. It doesn't even matter. That boundary is fuzzy, and any vocab attempting to describe this gradient is just arbitrary.
But hear me out. chatGPT and DALL-E is NOT hype. Why? Because along that gradient it's leaps and bounds further than anything we had just even a decade ago. It's the closest we ever been to the extreme endpoint. Whichever side you are on in the great debate both sides can very much agree with this logic.
But it's not obvious at all. It may possess consciousness in a way we can't relate to or communicate with.
This is the whole problem with consciousness and has been discussed by philosophers for centuries. We each appear to be conscious but can't be certain anything else is or isn't.
Lots of software engineers have spent their lives reading sci-fi that features AGI, and they're excited by/lost in that fantasy.
It's interesting to see that in people who often view themselves as hyper-rational.
Because you have to solve 10,000 different problems. And a huge number of those problems are going to have significant overlap, but sharing lessons between them is going to be difficult unless you have a generalized algorithm.
Hence AGI is the trillion dollar question.
Also there is the general hubris in all this to only look at the new and shiny, I remember when there was that pizza robot (some multi-dimension axis hand thing) that cost whatever in building and research, when the costco pizza "robot" is pretty darn good, but doesn't sell as "futuristic/cool" because its a spigot on a servo.
The real impacts will come when they are properly integrated into the current computational fabric, which everyone is racing to do as we write this.
Hinton is one of these individuals and with no definition of what intelligence is it is an understandable of dogmatic position.
This whole problem of not being able to define what intelligence is pretty much allows us all to pick and choose.
In my mind BPP is the complexity class solvable by ANNs and it is a safe and educated guess that most likely BPP=P.
BPP being one of the largest practical complexity classes makes work in this area valuable.
But due to many reasons that I won't enumerate again AGI simply isn't possible and requires a dogmatic position to believe in for people who have even a basic understanding of how they work and the limits from the work of Gödel etc...
But many of the top scientists in history have been believers of numerology etc...
Associating math with LLMs is a useful too to avoid wasted effort by those who don't believe AGI is close, but it won't convince those who are true believers.
LLM's are very useful for searching very large dimensional spaces and for those problems that are ergotic with the Markov property they can find real answers.
But for most of what is popular in the press will almost certainly be a dead end for generalized use of the systems are not extremely error tolerant.
Unfortunately it may take another AI winter to break the hype train but I hope not.
IMHO it will have a huge impact but overconfident claims will cause real pain and misapplication for the foreseeable future.
You might say, well thats not AGI, AGI must also do such and such. Well, we can get arbitrarily close to that definition as well via RLHF.
Another objection might be: well, if thats the definition of AGI, that seems really underwhelming compared to the hype train. This says nothing about autonomy, sentience, free will -- exactly. Those concepts can or should be orthogonal to doing productive work.l, IMHO.
So, there it is. We can now make a reward model for folding socks, and use gradient descent with RL to do the motion planning.
Maybe thats AGI and maybe its not, but I'd really love it if we had a golden period between now and total enshittification that involved laundry folding robots.
Now it is here and it's like "No, what we really meant is it has to be the next Einstein".
People are forgetting how stupid people are.
GPT is already better than average human.
Most people can't do what we claim GPT must be capable of to qualify as AGI.
The only logical conclusion is that many people are also not conscious and don't qualify as being able to reason.
Let's say I'm deep in a coding problem. A co-worker comes by and says "How did your team do in the game yesterday?". I say, "Um, uh... sorry, my head's not there right now." It takes us time to swap between mental "spaces".
So, if I have an AGI (defined as having a trained model for almost everything, even if that turns out to be a large number of different models), if it has to load the right model before it can reason on that topic, then that's pretty human-like. (As long as it can figure out which model to load...)
The one thing missing is that (at least some) humans can figure out linkages between different mental "spaces", to turn them into a more coherent holistic mental space, even if they don't have (all of) each space at front-of-mind at any moment. I'm not sure if this flavor of an AGI could do that - could see the connections between different models.
An AGI should be able to solve any creative problem a human could, with drive and knowing purpose and coherent vision. The LLMs are still narrowly focused and require human supervision.
We might well get there with chained AIs automatically training new reward models for each new problem, or by some other paradigm, but I don't feel like we're past the threshold yet.
So it seems like maybe a better title would be "LLMs don't have as advanced a theory of mind as a human does... for now..."
LLM names an specific product, aimed at solving an specific problem.
If you've found some, please let everyone know.
If you've found some, please let everyone know.
I disagree with the topic sentence.
The goal should not be to "build machines that think like people", but to build machines that think, period. The way humans think is unlikely to be the optimal way to go about thinking anyways.
Instead of talking about thinking, we should be talking about function. Less philosophy and more reality. Can the system reason itself through various representative challenges as well as or better than human? If yes, it doesn't much matter how it does it. In fact, it's probably for the best if we can create AI that thinks completely different than humans, has no consciousness or self awareness, but still can do what humans can do and more.
Game AIs are functionally much better than humans but no one believes they can think, right?
Oh, but if you are arguing for AI from a specialized tool standpoint and not a general intelligence standpoint, if you are talking about "weak" AI rather than "strong" AI, then I'm right there with you. :-)
Many scientists outside the AI field have long shared an interest in the objective of how to "think like people" using software. Far fewer care if the AI is inexplicable (or if it can't be dissected into constituent components, thereby enabling us to explore the mind's constraints and dependencies among its cognitive processes).
The paperclip optimizer is a great parable here. If you build your intelligence to build as many paperclips as cheaply as possible don't be surprised when said intelligence disassembles you and the rest of the universe to do so.
So yea, HOW starts mattering a whole lot when you want to ensure it understands that it shouldn't do some particular things.
To be clear, I think this is in fact a correct assessment of the architecture of intelligence. You can suspend thought and still function throughout your day in all ways. Discursive thought is entirely unnecessary, but it is often helpful for planning.
My observation of LLMs in such a construction of intelligence is they are entirely the thinking mind - verbal, articulate, but unmoored. There is no, for lack of a better word, “soul,” or that internal awareness that underpins that discursive thinking mind. And because that underlying awareness is non articulate and not directly observable by our thinking and feeling mind, we really don’t understand it or have a science about it. To that end, it’s really hard to pin specifically what is missing in LLMs because we don’t really understand ourselves beyond our observable thinking and emotive minds.
I look at what we are doing with LLMs and adjacent technologies and I wonder if this is sufficient, and building an AGI is perhaps not nearly as useful as we might think, if what we mean is build an awareness. Power tools of the thinking mind are amazingly powerful. Agency and awareness - to what end?
And once we do build an awareness, can we continue to consider it a tool?
While you're adding a bunch of eastern philosophy to it, we need to take a step back from 'human' intelligence and go to animal and plant intelligence to get a better idea of the massive variation in what covers thought. In animal/insects we can see that thinking is not some binary function of on or off. It is an immense range of different electrical and chemical processes that involve everything from the brain and the nerves along with chemical signaling from cells. In things like plants and molds 'thinking' doesn't even involve nerves, it's a chemical process.
A good example of this at the human level is a reflex. Your hand didn't go back to your brain to ask for instructions on how to get away from the fire. That's encoded in the meat and nerves of your arm by systems that are much older than higher intelligence. All the systems for breath, drink, eat, procreate were in place long before high level intelligence existed. Intelligence just happens to be a new floor stacked hastily on top of these legacy systems that happened to be beneficial enough it didn't go extinct.
Awareness is another one of those very deep rabbit hole questions. There are 'intelligent' animals without self awareness, but with awareness of the world around them. And they obviously have agency. Of course this is where the AI existentialists come in and say wrapping up agency, awareness, and superintelligence may not work out for humans as well as we expect.
Is this actually true? I thought it just involved a different part of the brain. Is there actually no brain involvement? Sure it does not need your awareness or decision making, but no brain? I find that hard to believe.
If that's how it works, then the "soul" is more like an emergent phenomenon created by the interplay between the various layers of conscious thought and the base layer of nothingness when it's all turned off. That architecture wouldn't necessarily be so difficult to replicate in AI systems.
This is simple to experience for yourself since it'd mean we stop being aware when listening so intently thoughts stop. Obviously we don't cease to be aware at such times.
You've also misunderstood what is meant by nothingness ("no thingness").
It's not. They don't realize it, they're merely referring to stopping your internal monologue. There are dozens of other mental processes going on in any given waking moment. Even actual top shelf cognition is going on, it just occurs in a "language of thought".
But to my understanding the idea of nothingness being some objective in Buddhism isn’t the case - but it’s often described as such because that state of pure awareness without encumbering thought and attachment in many ways to an unpracticed person feels like nothingness. After all, the awareness is silent, even if it is where all thought and feeling spring from.
Finally, awareness isn’t that moment you snap back to thought. You’re always aware. We just tend to be primarily aware of our thoughts and emotions. We walk around in a haze of the past and future and fiction as the world ticks by around us, and we tend to live in what isn’t rather than what is. You don’t disappear in the sense that you cease to be as an individual mind, you are always yourself - that’s a tautology. What you lose is the sense of some identity that’s separate from what you ARE in this very moment. You aren’t a programmer, you aren’t a Democrat, you aren’t a XYZ. You are what you are, and what that is changes constantly, so can’t be singularly defined or held onto as some consistent thing over time with labels and structure. You just simply are.
Meditation techniques that focus on breath or the body are an attempt to make you do the breathing/sensing consciously. If you film yourself and later look at what you did, you'll notice you aren't breathing well when you're breathing consciously, so you're probably depriving yourself of oxygen, lowering blood concentration in certain brain regions and you hope it will be the brain region associated with conceptualizing, language etc.
You can do the same with sleep. You can try to consciously fall asleep, and just like breathing, you will have a hard time because there's a reason why falling asleep is not conscious (or in other words it does not go through the regions of the brain that conceptualize). You can experience the balance center shutting down (feels like falling or turning) and you can go even deeper and feel the fear of the "ego" dying (temporarily). What remains is definitely much different than waking or dreaming state. But it is still not that "awareness/nothingness".
If you assume that "the eyes are the window to the soul", you notice some interesting properties.
1. It is far more observable from the outside (eyes open/lidded/closed, emotion read in eyes)
2. It affects behavior in a diffuse way
3. It pays attention but does not dictate
My pet theory about human consciousness is that is that consciousness is simply recursive theory of mind. Theory of mind [1] is our ability to simulate and reason about the mental states of others. It's how we predict what people are thinking and how they will react to our actions, which is critical for choosing how to act in a social environment.
But when you're thinking about what's in someone's head, one of the things might be them thinking about you. So now you're imagining your own mind from the perspective of another mind. I believe that's entirely what our sense of consciousness is. It's our social reasoning applied to ourselves.
If my pet theory is correct, it implies that the level of consciousness of any species would directly correlate to how social the species is. Solitary animals with little need for theory of mind would have no self awareness in the way that we experience it. They'd live in a zen-like perpetual auto-pilot where they do but couldn't explain why they do what they do... because they will never explain it to anyone anyway.
And people say LLM output is nonsense.
I'm blind with glass eyeballs. Does this mean my soul is easier to access than yours? Or is it harder because there's something specific about the eyeball that makes it the window?
Decision making requires imagination or the ability to envision alternative future states that may result from various choices.
Imagination is the start of abstract thinking. Consciousness results from the individual thinking abstractly about itself and how it interacts with the world.
I believe this is more or less the definition of human mental illness. I have to say that while I know it's really not possible, I wish people would stop pulling on these threads. I got into this line of work because I thought video games were cool, not because I wanted to philosophize about theories of mind and what intelligence is. I really don't like thinking about whether I'm just some sort of automaton made out of meat rather than metal and silicon.
Maybe that's their goal.
But for many users of AI, the goal is to have easy and affordable access to a machine that, for some input (perhaps in a tightly constrained domain), gives us the output that we would expect from a high-functioning human being.
When I use ChatGPT as a coding helper, I really don't care about its "theory of mind." And its insights are already as deep (actually more deep) as I get from most humans I ask for help. Real humans, not Don Knuth, who is unavailable to help me.
IIRC one of the prompting techniques developed this year was to ask the model who are some world class experts in a field and then have it write as if it was that group collaborating on the topic.
This was my thought as well. But then I figured if I can't get someone to give me thoughtful feedback, I might have bigger problems to solve.
[1] normally I find HN discussions about what if chatGPT is human or "humans are just autocompletes" to be highschool-level scifi and cringe respectively
And by "us" I mean "those of us who choose to use ChatGPT," and not that I was forcing you to use ChatGPT.
It's true, I don't morally object to asking ChatGPT to "do my labour." It raises no red flag for me. (Okay, there's the IP red-flag about how ChatGPT was trained, but I don't think that's what you mean.)
Maybe and maybe not. Being sentient or even sapient doesn’t mean it has human emotions, feelings or motivations. You have to be very careful when ascribing naturally evolved emotions and motivations to something that is not a naturally evolved intellect.
LLM's are basically a validation of Searle's Chinese room. What they've proven is that you can build functioning systems that perform intelligent tasks purely at the level of syntax. But there is no (or very little) understanding of semantics. If I ask a person on how to end the world, whether I ask in French or English or base64 or perform a 50 word incantation beforehand likely does not matter. (unless of course the human is also just parroting an answer)
The Chinese room argument is bad in that it hides an assumption of mind/body dualism. If you believe that humans have "souls" and other things do not, then you have a qualitative difference between a human or a machine. On the other hand, if you are a materialist then you are faced with the problem that humans don't have much understanding of semantics either. We're all chemical processes and it's hard for those to get much into semantics.
But then, the difference between LLMs and humans becomes quantitative, sort of, and since I cannot say that LLMs and humans are qualitatively different, the only argument I can find is that in my experience, LLMs have never responded in a way that leads me to believe that they are anything other than a statistical model of language. Humans, on the other hand, are not a statistical model of language.
It's because our bandwith and monkey brains are so slow that we're forced to operate at a level of semantics. We can't just make inferences from almost infinite amounts of data the same way we can't play chess like Stockfish or do math like a calculator. The dualism is precisely in the opposite view, that computation is somehow "substrate independent". Searle argues we can have AI that has understanding the way we do, just that it's going to look more like an organic brain as a result.
The important insight from LLMs is that they're not like us at all but that doesn't make them less effective or intelligent. We do have plenty of understanding, we need to because we rely on a particular kind of reasoning, but artificial systems don't need to converge on that.
Human insight is really easy to break, confidence men wouldn't really be a thing if it were hard to break. Simply putting a statement like "I love you" in front of a statement commonly overrides our intellect. Or offering a chocolate bar in trade of our passwords. If you want a human to tell you how to end the world, you'd just convince them to be your friend first.
That's not how LLMs work though, and I'm increasingly convinced that "syntax" and "semantics" are turning into annoyingly useless ideas people forget are descriptive, in the same way grammar books and dictionaries are descriptive.
My model of LLMs is that, in training, they're positioning inputs in an absurdly high-dimensional[0] latent space. That space is big enough to encode anything you'd consider "syntax" and "semantics" on some subset of dimensions. As a result, the model sidesteps the issue of "source of meaning" - there is no meaning but that formed through associations (proximity). This is pretty much how we do it, too - when you think of "chair", there is no token for platonic ideal of a chair in your mind. The word "chair" has meanings and connotations defined via other words, which themselves are defined via other words, ad infinitum, with the only grounding being associations to sensory inputs.
--
[0] - On the order of 100 000 dimensions for GPT-4, perhaps more now for GPT-4V / GPT-4-Turbo.
Not even slightly. See Cantor's diagonal argument.
See also Plato's cave for syntax vs semantics.
As for the rest of it, the LLM is basically "raw compute". You need a self-referential loop and long-term memories for it to even have the notion of self. But looking at it at that level and discounting it as "incapable of thinking" is missing the point - it's the larger system of which LLM is one part, albeit a key one (and which we're still trying to figure out how to build) that might actually be conscious etc.
Basically inventing a board game and play against ChatGPT and see what happened. It was not able to do a single move, even having provided all the possible start moves in the prompt as part of the rules.
Not that I had a lot of hope about it, but it was definitely way worst than I expected.
If someone wants to take a look at it:
https://joseprupi.github.io/misc/2023/06/08/chat_gpt_board_g...
Here's my attempt at similar conversation — it seems GPT-4 is able to visualise the board and at least do a valid first move.
https://chat.openai.com/share/98427e21-678c-4290-aa8f-da8e93...
The model was whatever was up that that time, so probably was 3.5 if you say so.
Also in this game if I don't move the queen I force a draw, right?
I don't know, take your own conclusions, I tried what I tried with the results I got. And the reason I created a Monte Carlo Engine to play the game was specifically because of this, I expected ChatGPT to be able to make moves but actually not being good with the game. You can try yourself, the code is available.
> Also in this game if I don't move the queen I force a draw, right?
I don't know as there is no time but I assume it is mandatory to move when, what happens in a chess game with no time if someone does not want to move? Same applies here.
I've bought 'new' board games for kids.
Then, I have been un-able to play because the instructions were pretty bad.
Humans also need to 'learn'. Need a few play-throughs.
No human is going out and 'in a vacuum' with no experience, buying Risk and from scratch, read instructions and play perfect game winning strategy.
> If it is not memorizing, how do you think is doing it? (me)
> by trying to learning the general rules that to explain the dataset and minimize its loss. That’s what machine learning is about, it’s not called machine memorizing.
Consider a typical LLM token vector used to train and interact with an LLM.
Now imagine that other aspects of being human (sensory input, emotional input, physical body sensation, gut feelings, etc.) could be added as metadata to the the token stream, along with some kind of attention function that amplified or diminished the importance of those at any given time period -- all still represented as a stream of tokens.
If an LLM could be trained on input that was enriched by all of the above kind of data, then quite likely the output would feel much more human than the responses we get from LLMs.
Humans are moody, we get headaches, we feel drawn to or repulsed by others, we brood and ruminate at times, we find ourselves wanting to impress some people, some topics make us feel alive while others make us feel bored.
Human intelligence is always colored by the human experience of obtaining it. Obviously we don't obtain it by getting trained on terabytes of data all at once disconnected from bodily experience.
Seemingly we could simulate a "body" and provide that as real time token metadata for an LLM to incorporate, and we might get more moodiness, nostalgia, ambition, etc.
Asking for a theory of mind is in fact committing the Cartesian error of making a mind/body distinction. What is missing with LLMs is a theory of mindbody... similarity to spacetime is not accidental as humans often fail to unify concepts at first.
LLMs are simply time series predictors that can handle massive numbers of parameters in a way that allows them to generate corresponding sequences of tokens that (when mapped back into words) we judge as humanlike or intelligence-like, but those are simply patterns of logic that come from word order, which is closely related in human languages to semantics.
It's silly to think that we humans are not abstractly representable as a probabilistic time series prediction of information. What isn't?
Then the next research step could be to study those properties so as to reconstruct/reproduce a theory of mind-body AI, without needing any embodiment process at all to obtain it. Is that, in principle, possible? It is unclear me.
... a hardware interface that generates a token stream from a living human's body would seem to enable this at some level.
Not sure how it would work at scale. Maybe something much simpler like phones with built-in VOC sensors that can detect nuances of the user's perspiration, combined with real time emotion sensing via gait, voice, along with metadata that is already available would be sufficient to produce such a token stream... who knows.
The eval is a weird, noisy visual task (picture of astronaut with “care packages”). Their results are hopelessly narrow.
A better eval is to use actual scientifically tested psychology test on text (the native and strongest domain for LLMs), for example the sort of scenarios used to gauge when children develop theory of mind (“Alice puts her keys on the table then leaves the room. Bob moves the keys to the drawer. Alice returns. Where does she think the keys are?”) which GPT-4 can handle easily; it is very clear from this that GPT has a theory of mind.
A negative result doesn’t disprove capabilities; it could easily show your eval is garbage. Showing a robust positive capability is a more robust result.
Aren't you confusing having a theory of mind with being able to output the right answer to a test? Isn't your proposed evaluation especially problematic because an "actual scientifically tested psychology test" is likely in the training data along with a lot of discussion and analysis of that test and the correct and incorrect answers that can be given?
Of course, another approach you can take if you suspect contamination is to ask follow-up questions that are less likely to be in any training data; if the LLM is a mere stochastic parrot without a ToM it will not be able to give well-formed answers to follow-up questions.
Perhaps if I'd said "established psychology research methodology" the intent would have been clearer?
As far as I'm concerned, the only person I can be certain is concious is me.
It doesn't have to be a "scientifically tested psychology test"
Construct your own story with multiple characters of varying knowledge and beliefs and see how it does.
Nothing I’ve seen shows evidence of any sort of abstract concepts in there.
The conscious "why" comes after the decision. In that sense it's exactly the kind of bullshit machine that LLMs are.
The brain is trying to 'predict' the next sensory input, and that prediction is our awareness. What we would call our 'conscious self'.
It makes point of calling it a 'controlled hallucinations', in that what we experience as our self. "Hallucination" being the experience we have as our brain 'predicting/controlling' for the sensory input. So All inputs come together in a 'hallucination', but it is averaged 'Bayesian', with the actions we are taking at same time. So Action + Prediction = Self.
It is funny that using the word 'hallucinate' in AI has become so common and it is also used in Humans. And so few people seem to make connection that they are actually very similar, and far from being an argument against AI consciousness, is argument for how similar they are.
I perceive a moving of the goalposts as machine intelligence improves. Once we'd have been happy with smarter than an especially stupid person, now I think we're aiming at smarter than the smartest person.
Goal posts only exist in games.
These systems are engineering products to be leveraged in enginenering processes. We want to understand what they're good at and what they're bad at, and what potential they show for further refinement. There are no goal posts or "happy with" criteria in that context, and when we find ourselves adjusting the language we use to describe them because of how we see them work, we're trying to refine our ability to express their capabilities and suitabilities.
Intelligence, in particular, is a very poor and ambiguous word to be stuck using in technical contexts and so we're likely to just gradually shed it over time to reduce confusion as we hone in on better ways to talk about these systems. We've repeatedly done the same for earlier advances in the field, and for the same reason.
We get a better and better idea of what this hazy term "intelligence" means as we DIY tinker with making our own new ones.
Once we'd have been happy with smarter than an especially stupid person, now I think we're aiming at smarter than the smartest person.
We're going to get there sooner than we think. When we get there, we will have new things to regret in ways we'd never thought of before.
I'll take that. My own expectation is I'll have a few minutes-to-months to say "I told you so".
If you regard "an especially stupid person" as someone with significant cognitive or communication limits, then Parry and Eliza's Doctor are pretty fair simulations of paranoid schizophrenia (as it was understood at the time) and Rogerian therapy. Likewise, chess and go AIs are pretty damn smart, except they can't do anything else.
The point is that, if you accept limits on what the machine needs to do, then "intelligence" as defined by behavior you can recognize becomes trivially and meaninglessly easy.
(It's sort of like evaluating a person's competence: a minority person has to be more competent than their cohort because non-minority people get the benefit of the doubt.)
"A chief goal of levers (cranes, etc.) engineering would be to build devices that lift like people"
There is gain in implementing desirable qualities. Just that.
Odd restriction. Why not investigate text-based ones?
Or is “vision-based” a technical term that encompasses models that were trained on text?
Funnily enough, this statement also applies to people that are scared of AI.
Maybe a bit off topic but does anyone else have that friend who sends them fear mongering AI videos with captions like "shocking AI" that are blatantly unimpressive or completely fake?
What is the best way to subdue this kind of fear in a friend, sending them written articles from high level researchers like Brooks does not work.
Edit: this is may turning to a search for truth and definitions of reality question. When the last person alive is no longer able to tell whether they're speaking to an AI, does it actually matter whether it's true generalized intelligence or just an emergent approximation?
Similarly, if you want to approximate a human, an LLM may be the best we can do right now, but it's hardly a good approximation.
