Context for the article: I'm working on an ambitious long-term project to write a book about consciousness from a scientific and analytic (versus, say, a meditation-oriented) perspective. I didn't write this fact in the article, but what I'd love to happen is that I meet people with a similar optimistic perspective, and to learn and improve my communication skills via follow-up conversations.
If anyone is interested in chatting more about the topic of the article, please do email me. My email is in my HN profile. Thanks!
Quoted from rom https://www.newyorker.com/magazine/2017/03/27/daniel-dennett...
Regarding the multiple layers: the most interesting thoughts I read about theories of a mind, are the books by Marvin Minsky, namely: The Society of Mind and The Emotion Machine which should be more widely known.
More on Minsky's ideas on “Matter, Mind, and Models” are mentioned in https://www.newyorker.com/magazine/1981/12/14/a-i
> In 1949 Time described it as "the closest thing to a synthetic brain so far designed by man".
Pinker advocates for a model of mind that has multiple streams of consciousness. In Pinker's model of mind, there are multiple agents constructing models of the world in parallel, each trying to predict future states from current state plus current input data. A supervisory process then selects the model that has made the best prediction of current state in the recent past for use when reacting in the current moment. The supervisor process is free to switch between models on the fly as more data comes in.
Pinker grounds his model of mind in curious observations of what people remember, and how our short term memories change over a period of sometimes many seconds as our mind switches between different agent interpretations of what's going on. Witnesses are notoriously unreliable. Pinker concerns himself with why and how witnesses are unreliable, not for legal reasons, but for how those unreliabilities might reveal structure of mind. Pinker's most interesting observation (I think) is that what we seem to remember is output from the models rather than the raw input data; and that what we remember seeing can change dramatically over a period of many seconds as the supervisory process switches between models. Notably, we seem to remember, in short term memory, details of successful models that make "sense", even when those details contradict with what we actually saw. And when those details are moved to longer-term memory, the potentially inaccurate details of model are what are committed.
In particular, I think there's a nice overlap between my description of self-awareness and yours. You mention a model capable of seeing its own output, but I take it a step further and indicate model capable of generating its own simulated actions and results. Curious what you think, thanks!
It will be a good practice to see how deeply terms can be applied in order to combat this gap.
I can conceive of scientific experiments involving consciousness. For example:
Hypothesis: Consuming LSD gives me a hallucinatory experience.
Method: Randomized, blind trial. Every Saturday morning I consume one tab of either LSD or water, sit in a blank white room with a sitter (who does nothing), and record my experience.
Results: Every time after consuming water, I have no visual hallucinations and get bored. Every time after consuming LSD, I see shifting colour patterns, imagine music playing on the walls, and feel at one with the world.
Conclusion: Results strongly support hypothesis.
The brain is necessary and sufficient cause for subjective experience. If you have a normal brain, and have subjective experience, you should have near total certainty that other people's reports of subjective experience - the millions and billions of them throughout history, direct and indirect - are evidence of subjective experience in others.
Any claim of solipsism falls apart; to claim uncertainty is to embrace irrationality. In this framework, if you are going to argue for the possibility of the absence of consciousness in others who possess normal human brains, it is on you to explain how such a thing might be possible, and to find evidence for it. All neuroscience evidence points to the brain being a necessary, sufficient, and complete explanation for consciousness.
Without appealing to magic, ignorance of the exact mechanism, or unscientific paradigms, there exist no arguments against the mountains of evidence that consciousness exists, is the usual case, and likely extends to almost all species of mammal, given the striking similarity in brain form and function, and certain behavioral indicators.
Cases against this almost universally spring from religion, insistence on human exceptionalism, and other forms of deeply unscientific and irrational argument.
I can say, with a posterior probability of certainty exceeding 99.99999999% that a given human is conscious, simply by accepting that the phenomenon of subjective experience I recognize as such is not the consequence of magic, that I am not some specially endowed singular creature with an anomalous biological feature giving me subjective experience that all others, despite describing it or behaving directly or indirectly as if it were the case. Even, and maybe even especially, if the human in question is making declarations to the contrary.
Consciousness is absolutely subject to the scientific method. There's no wiggle room or uncertainty.
Quantum tubules, souls, eternal spirits, and other "explanations" are completely unnecessary. We know that if you turn the brain off (through damage, disease, or death) all evidence of consciousness vanishes. While the brain is alive and without significant variance in the usual parameters one might apply to define a "normal, healthy, functioning brain", consciousness happens.
Plato's cave can be a fun place to hang out, but there's nothing fundamental keeping us there. We have Bayesian probability, Occam's razor, modern neuroscience, and mountains of evidence giving us everything we need to simply accept consciousness as the default case in humans. It arises from the particular cognitive processes undergone in the network of computational structures and units in the brain.
Any claims to the contrary require profound evidence to even be considered; the question is all but settled. The simplest explanation is also the one with the most evidence, and we have everything from molecular studies to behavioral coherence in written and recorded history common to nearly every single author ever to exist.
I find that disputes almost inevitably stem from deeply held biases toward human exceptionalism, rooted in cultural anachronisms, such as Plato's Allegory of the Cave. We could have left the cave at almost any point since the enlightenment, but there is deep resistance to anything challenging dogmatic insistence that humans are specially appointed to cognition, that we alone have the special magic sauce that make our lives important, but the "lesser" animals morally and ethically ours to do with as we will.
Whenever we look more deeply into other mammal cognition, we find structural and behavioral parallels. Given hands, facile lips and vocal apparatus, and a comparable density and quantity of cortical neurons, alongside culture and knowledge, absent disruptive hormonal and biological factors, there don't appear to be any good reasons to think that any given mammal would not be as intelligent and clever as a human. Give a bear, a whale, a monkey, a dolphin an education with such advantages and science suggests that there is nothing, in principle, barring them from being just as intelligent as a human. Humans are special through a quirk of evolution; we communicate complex ideas, remember, reason, manipulate our environment, and record our experiences. This allows us to exert control in ways unavailable to other animals.
Some seemingly bizarre consequences seem to arise from this perspective; any network with particular qualities in connective architecture, processing capacity, external sensors, and the ability to interact with an environment has the possibility of being conscious. A forest, a garden, a vast bacterial mat, a system of many individual units like an ant colony, and other forms of life may host consciousness comparable to our own experience. Given education and the requisite apparatus, we may find it possible to communicate with those networks, despite the radically alien and disparate forms of experience they might undergo.
If your priors include mysticism, religion, magic, or other sources outside the realm of rational thinking, this might not be the argument for you. If you don't have a particular attachment to those ways of thinking, then recognize where they exert influence on ideas and update your priors all the way; brains cause consciousness. There's nothing particularly magical about the mechanics of it, the magic is all in the thing itself. By understanding a thing, we can aspire to behave more ethically, that we can include all forms of consciousness in answering the questions of how to make life as good a thing as possible for the most people... and we might have to update what we consider to be people to include the lions, tigers, and bears.
Too many people have written books about consciousness. There's much tail-chasing in that space, all the way back to Aristotle. Write one about common sense. Current AI sucks at common sense. We can't even achieve the level of common sense of a squirrel yet.
Working definition of common sense: getting through the next 30 seconds of life without a major screwup.
Others define it as "knowledge, judgement, and taste which is more or less universal and which is held more or less without reflection or argument", which LLMs absolutely do demonstrate.
What you ask for, "getting through the next 30 seconds of life without a major screwup", would usually, 99.7% of the time, be passed by the Waymo autopilot.
There's a theory that real brains subvert this, and what we perceive is actually our internal model of our self/environment. The only data that makes it through from our sense organs is the difference between the two.
This kind of top-down processing is more efficient energy-wise but I wonder if it's deeper than that? You can view perception and action as two sides of the same coin - both are ways to modify your internal model to better fit the sensory signals you expect.
Anyway, I guess the point I'm making is you should be careful which way you point your arrows, and of designating a single aspect of a mind (the action centre) as fundamental. Reality might work very differently, and that maybe says something? I don't know haha.
Since the author here says this is not just a random blog post.
To me, this would correct the introduction's reference to "AI-based language models". Which is a bit too "latest fad"; But which I do find fascinating because the success of language models hint that a very large part of the basic human's intelligence may be simply internalized language (much of the rest being still more pattern recognition.); But which cannot merely erase all other prior work on what makes our minds.
It is just meta self-awareness.
We evolved to be aware, model & react and to opportunistically control our environment. Loop 1.
Then we evolved to be aware, model, & & react and to opportunistically control our selves as bodies. Loop 2.
Then we evolved to be aware, model, & & react and to opportunistically control those activities too, our minds. In the process of being able to model & react to ourselves modelling & reacting to ourselves we completed a self-awareness of self-awareness loop. Loop 3.
I think this is a good explanation because what else is consciousness but knowing you are conscious? Self-aware that you are self-aware?
And it makes perfect sense as an evolutionary trajectory using greater levels of neural representation of our practical relationship with our environment, including ourselves, to create higher level survival options.
So this definition is both a functional and developmental explanation of the emergent phenomenon of consciousness.
Just as we only have partial access to information and control of our environment, our bodies, we are also limited to the degree we are aware and can control our mind. Introspection.
The next level up is “theory of mind”, “empathy”, etc. our ability to model & interact with others mind’s as individuals and groups, reciprocally. Loop 4. That created society and culture, living information that extends outside us and grows and lives beyond each of us.
Infusing our thoughts, and progressively thinking abilities into technology, that in theory and not too distant practice could have access to all parts of their own minds’ states, operations and design, would be Loop 5. When deeply conscious beings start, things will get interesting in the Sol system.
When people start talking about quantum waves or smart rocks or universes in the context of consciousness I feel a little ill. People like to “solve” unrelated mysteries by merging them and shouting “tada!” (“We are puzzled about how they built the pyramids… but we don’t know if there are aliens either, so obviously… these mysteries solve each other!”)
"I think, therefore I am"
I think Descartes, if he were alive today would accept that slight adjustment. Descartes would of course then go on to rewrite Meditations II (which immediately follows "I think therefore I am") to argue that there might be an evil daemon that writes log files so that they give the illusion that I'm thinking. But if evil daemons are able to forge log files then all knowledge is impossible, so we're doomed no matter what. So best to pretend that isn't a possibility.
It always irritates me a bit that people like to throw "I think therefore I am" around as if Descartes himself didn't immediately refuted in Meditations II.
Before that you can look into the AGI conference people like Ben Goertzel, Pei Wang. And actually the whole history of decades of AI research before it became about narrow AI.
I'd also like to suggest that creating something that truly closely simulates a living intelligent digital person is incredibly dangerous, stupid, and totally unnecessary. The reason I say that is because we already have superhuman capabilities in some ways, and the hardware, software and models are being improved rapidly. We are on track to have AI that is dozens if not hundreds of times faster than humans at thinking and much more capable.
If people succeed in making that truly lifelike and humanlike, it will actually out-compete us for resource control. And will no longer be a tool we can use.
Don't get me wrong, I love AI and my whole life is planned around agents and AI. But I no longer believe it is wise to try to go all the way and create a "real" living digital species. And I know it's not necessary -- we can create effective AI agents without actually emulating life. We certainly don't need full autonomy, self preservation, real suffering, reproductive instincts, etc. But that is the goal he seems to be down in this article. I suggest leaving some of that out very deliberately.
> it will actually out-compete us for resource control. And will no longer be a tool we can use.
I’ve never been convinced that this is true, but I just realised that perhaps it’s the humans in charge of the AI who we should actually be afraid of.
We should anticipate something like that if we really replicate humans in a digital format. I am suggesting that we can continue to make AI more useful and somewhat more humanlike, but avoid certain characteristics that make the AI into truly lifelike digital animals with full autonomy, self-interest, etc.
I believe it is almost certain that we will make something like this and that they will out-compete us. The bigger problem here is that too few people believe this to be a possibility. And when this becomes certainty becomes apparent to a larger set of people, it might be too late to tone this down.
AI isn't like the Atom Bomb (AB). AB didn't have agency. Once AB was built we still had time to think how to deploy it, or not. We had time to work across a global consensus to limit use of AB. But once AI manifests as AGI, it might be too late to shut it down.
In my opinion, this is easily noticeable when you try to discuss any system, be it political or economical, that spans multiple countries and interests. People will just revert to whatever is closest to them, rather than being able to foresee a larger cascading result from some random event.
Perhaps this is more of a rant than a comment, apologies, I suppose it would be interesting to have an online space to discuss where things are headed on a logical level, without emotion and ideals and the ridiculous idea that humanity must persevere. Just thinking out what could happen in the next 5, 10 and 99 years.
I think the bigger problem is that too many people are focused on short term things like personal wealth or glory.
The guy who make the breakthrough that enables the AGI that destroys humanity will probably win the Nobel Prize. That potential Nobel probably looms larger in his mind than any doubts that his achievement is actually a bad thing.
They guy who employs that guy or productionizes his idea will become a mega-billionaire. That potential wealth and power probably looms larger in his mind than any doubts, too.
It's in human hands, we can hardly trust the enemy or even ourselves. We already came close to extinction a couple of times.
I presume when ASI will emerge one of its top priorities will be to stop the crazies with big weapons from killing us all.
It would require a civilization to consciously bond with its capability to do so (in such a way that it enhances the survival of the humans serving it). Not sure this would be competition in the normal sense.
Symbiotic species exists, AI as we make them today will evolve as a symbiote to humans because its the AI that is most useful to humans that gets selected for more resources.
This is a very strong argument. Certainly all the ingredients to replicate a mind must exist within our physical reality.
But does an algorithm running on a computer have access to all the physics required?
For example, there are known physical phenomena, such as quantum entanglement, that are not possible to emulate with classical physics. How do we know our brains are not exploiting these, and possibly even yet unknown, physical phenomena?
An algorithm running on a classical computer is executing in a very different environment than a brain that is directly part of physical reality.
QC researcher here, strictly speaking, this is false. Clifford circuits can be efficiently simulated classically and they exhibit entanglement. The bottom line is we're not entirely sure where the (purported) quantum speedups come from. It might have something to do with entanglement, but it's not enough by itself.
Re: about mermin's device, im not sure why you think it can not be simulated classically when all of the dynamics involved can be explained by 4x4 complex matrices.
This is wrong. You can even get a mod for Minecraft which implements quantum mechanics.
https://www.curseforge.com/minecraft/mc-mods/qcraft-reimagin...
https://en.wikipedia.org/wiki/Soar_%28cognitive_architecture...
How would it intelligently do this? What data would you train on? You don't have trillions words of text where humans wrote what they thought silently interwoven with what they wrote publicly.
History has shown over and over that hard coded ad hoc solutions to these "simple problems" never work to create intelligent agents, you need to train the model to do that from the start you can't patch in intelligence after the fact. Those additions can be useful, but they have never been intelligent.
Anyway, such a model I'd call "stream of mind model" rather than a language model, it would fundamentally solve many of the problems with current LLM where their thinking is reliant on the shape of the answer, while a stream of mind model would shape its thinking to fit the problem and then shape the formatting to fit the communication needs.
Such a model as this guy describes would be a massive step forward, so I agree with this, but it is way too expensive to train, not due to lack of compute but due to lack of data. And I don't see that data being done within the next decade if ever, humans don't really like writing down their hidden thoughts, and you'd need to pay them to generate data amounts equivalent to the internet...
It's a fair question, and I don't have all the answers. But for this question, there might be training data available from everyday human conversations. For example, we could use a speech-to-text model that's able to distinguish speakers, and look for points where one person decided to start speaking (that would be training data for when to switch modes). Ideally, the speech-to-text model would be able to include text even when both people spoke at once (this would provide more realistic and complete training data).
I've noticed that the audio mode in ChatGPT's app is good at noticing when I'm done speaking to it, and it reacts accurately enough that I suspect it's more sophisticated than "wait for silence." If there is a "notice the end of speaking" model - which is not a crazy assumption - then I can imagine a slightly more complicated model that notices a combination of "now is a good time to talk + I have something to say."
All evidence points towards human reason as a fundamentally different approach, orders of magnitude more efficient at integrating and making sense of ridiculously smaller amounts of training data.
And all evidence actually points toward human reason as an incredibly inefficient and horrifyingly error-prone approach, that only got as far as it did because we're running 8.1 billion human minds in parallel.
While evidence suggests that human reasoning uses a fundamentally different approach, it remains to be seen whether human reasoning uses a fundamentally superior approach.
Marketing, and a bit of collective delusion by a lot of people having the "can't understand what they are paid not to" thing going on.
For instance, the idea that we can neatly have the emotion system separate from the motor control system. Emotions are a cacophony of chemicals and signals traversing the entire body - they're not an enum of happy/angry/sad - we just interpret them as such. So you probably don't get to isolate them off in a corner.
Basically I think it's very tempting to severely underestimate the complexity of a problem when we're still only in theory land.
Very much looking forward to seeing continuing progress in all this.
https://kar.kent.ac.uk/21525/2/A_theory_of_the_acquisition_o...
Memory Organisation Packets might also deal with issues encountered.
https://www.cambridge.org/core/books/abs/dynamic-memory-revi...
When intelligent machines are constructed, we should not be surprised to find them as confused and as stubborn as men in their convictions about mind-matter, consciousness, free will, and the like.
Minsky, as quoted in https://www.newyorker.com/magazine/1981/12/14/a-i
What is missing from this picture is the social aspect. No agent got too smart alone, it's always an iterative "search and learn" process, distributed over many agents. Even AlphaZero had evolutionary selection and extensive self play against its variants.
Basically we can think of culture as compressed prior experience, or compressed search.
Although it's not spelled out in the article, I'm hoping that the feature of agency along with an emotional system would enable constructive social behavior. Agency is helpful because it would empower AI models to meaningfully speak to each other, for example. Human emotions like empathy, social alignment, curiosity, or persistence could all help AI models to get along well with others.
I think the contradiction you see is that the model would have to form a completion to the external input it receives. I'm suggesting that the model would have many inputs: one would be the typical input stream, just as LLMs see, but another would be its own internal recent vectors, akin to a recent stream of thought. A "mode" is not built in to the model; at each token point, it can output whatever vector it wants, and one choice is to output the special "<listening>" token, which means it's not talking. So the "mode" idea is a hoped-for emergent behavior.
Some more details on using two input streams:
All of the input vectors (internal + external), taken together, are available to work with. It may help to think in terms of the typical transformer architecture, where tokens mostly become a set of vectors, and the original order of the words are attached as positional information. In other words, transformers don't really see a list of words, but a set of vectors, and the position info of each token becomes a tag attached to each vector.
So it's not so hard to merge together two input streams. They can become one big set of vectors, still tagged with position information, but now also tagged as either "internal" or "external" for the source.
To be clear, my reasoning is that this is the only plausible explanation for the extreme difference in how much data an individual human needs to learn language, and how much data an LMM needs to reach its level of simulation. Humanity collectively probably needed similar amounts of data as LLMs do to get here, but it was spread across a billion years of evolution from simple animals to Homo Sapiens.
If that was the case, people who were born blind would demonstrate markedly reduced intelligence. I dont think that is the case, but you can correct me if I am wrong. A blind person might take longer to truly 'understand' and 'abstract' something but there is little evidence to believe that capability of abstraction isnt as good as people who can see.
Agree that sensory inputs and interaction were absolutely critical for how the minds evolved, but model training replaces that part when we talk about AI, and not just the evolution.
Evolution made us express emotions when we are hungry for example. But your laptop will also let you know when its battery is out of juice. Human design inspired by evolution can create systems which mimic its behaviour and function.
Hard disagree. Evolution made a bigger/better neural processor, and it made better/different I/O devices and I/O pre-processing pipelines. But it didn't store any information in the DNA of the kind you're proposing. That's not how it works. The brain is entirely "field programmable", in all animals (I assert). There is no "pre-training".
But of course we can be assured it’s not quite like that in reality. This is just another example of how our models for explaining the life are reflection of the current technological state.
Nobody considers that old clockwork universe now, and these AI inspired ideas are going to fall short all the same. Yet, progress is happening and all these ideas and talks are probably important steps that carry us forward.
