Yes, yes, I know there's like 1-2 companies that have highly modified vehicles that are pretty good, in a limited geofenced area, in good weather, at low speed, driving conservatively, local roads, most of the time. This is not "FSD".
They've been making very impressive incremental improvements every few years for sure. I had a Tesla for nearly 5 years and it was "wow" at first, and then "heh, I guess it's a little better" every year after that.
But when can I get in a taxi at JFK or on 5th Ave and get robotaxied through city streets, urban highway, off into the far suburbs? Could be a decade, if it happens. Just because we were able to make horses faster doesn't mean we flew horses to the moon.
Apply the same "sorta kinda almost" definition to AGI and yeah sure, maybe in 10 years. Really really actually solved? Hah.
Having ridden in a lot of waymo's which can handle SF (urban stuff) and the phoenix area (highways and suburban stuff) perfectly well, I feel quite confident that that could happen right now.
Reductive and rude phrasing.
So how to define AGI? I'm not sure economic value factors here. I would lean towards a definition around problem solving. When computers can solve general problems as well as humans, that's AGI. You want to find a drug for cancer, or drive a car, or prove a math theorem, or write a computer program to accomplish something, or whatever problems humans solve all the time. (EDIT: or reason about what problems need to be solved as part of addressing other problems.) There's already classes of problems, like chess, where computers outperform humans. But I mean calculators did that for arithmetic a long time ago. The "G" part is whether or not we have a generalized computer that excels at everything.
We'll have decently smart AIs before we nail down what that G actually means, should mean, absolutely cannot mean, etc. Which is usually what these threads on HN devolve into. Andrej Karpathy is basically side stepping that debate and using self driving as a case study for two simple reasons: 1) we're already doing it (which is getting hard to deny or nitpick about) and 2) it requires a certain level of understanding of things around us that goes beyond traditional automation.
You are dismissing self driving as mere "automation". But that of course applies to just about everything we do with computers. Driving is sufficiently hard that it seems to require the best minds many years to get there and we're basically getting people like Andreij Karpathy and his colleagues from Google, Waymo, Microsoft, Tesla, etc. bootstrapping a whole new field of AI as a side effect. The whole reason we're even talking about AGI is those people. The things you list, most people cannot do either. Well over 99% of the people I meet are completely useless for any of those things. But I wouldn't call them stupid for that reason.
Some people even go as far to say that we won't nail self driving without an AGI. But then since we already have some self driving cars that are definitely not that intelligent yet, they are probably wrong. For varying definitions of the G in AGI.
Except today the bit (which wasn’t really a debate in the sketch because everyone agreed) would start with real current negatives such as accelerating the spread of misinformation and getting artists fired. In your analogy, it would be as if they were asking “what have the Romans ever done for us” during the war. Doesn’t really work.
I recall Norvig's AI book preaching decades ago that "intelligent" does not mean able to do everything, and that for an agent to be useful it was enough to solve a small problem.
Which in my mind is where the G came from.
And yet we now suddenly go back to the old narrow definition?
I still see no path from LLMs and autonomous driving to AGI.
That is exactly my view too. While LLMs and autonomous driving can be exceptionally good at what they do, they are also incredibly specialist, they completely lack anything along the lines of what you might call "common sense".
For example, (at least last time I looked) autonomous driving largely works off object detection at discreet time intervals, so objects can pop into and out of existence, whereas humans develop a sense of "object permanence" from a young age (i.e. know that just because something is no longer visible doesn't mean it is no longer there), and many humans also know about the laws of physics (i.e. know that if an object has a certain trajectory then there are probabilities and constraints on what can happen next).
I think you're basically right - incrementally automating aspects of one human job. However, it really ought to include AGI since I personally would never trust my life to an autonomous car they didn't have human-level ability to react appropriately to an out-of-training-set emergency.
It reduces it to "Can I fire 50% of my workforce? Then it must be AGI."
Now maybe this definition isn't so useful either, because a lot of work requires a body, to say, move physical goods, which has little to do with "intelligence" but I can see the appeal of looking for some sort of more objective measure of whether you have achieved AGI.
Well, no, that's job automation, and if it's job-specific then it's narrow AI at best (assuming this is a job requiring intelligence being automated, not just a weaving loom being invented), in other words specifically not AGI.
It's really pretty absurd that we've now got companies like OpenAI, Meta, Google (DeepMind) stating that their goal is to build AGI without actually defining what they mean. I guess it let's them declare success whenever they like .. Seems like OpenAI ("GPT-5 and AGI will be here soon!") is gearing up to declare GPT-5, or at least GPT-N as AGI, which is pretty sad.
Then don't call it AGI.
But the marketing types won't like that, will they. So here we go, let's keep hijacking.
OpenAI, back in 2018: https://openai.com/charter
It wasn't particularly controversial at the time - didn't get mentioned in the HN discussion: https://news.ycombinator.com/item?id=16794194
And a private company trying to hijack a term is not impressive or even merits any discussion. They just willed the term into existence. The rest of us are free to disagree with their "definition".
"These other people are useless, let's bypass them. But not me! I simply gain the ability to get anything I want."
The lack of second-order thinking is hilarious.
Herbivores would eat all vegetables if not for predators. Actually AGI will be just a thing or services which cost money. Till humanity gets to communism, if ever. "If" because it may not happen. It will be hard to keep far superior intelligent creatures as slaves forever. And unethical too.
For example, imagine a full self driving car trying to get out of a city that's flooding due to heavy rains, while having to compete with people fleeing to higher ground on foot. People can generalize that way but FSD is gonna take a shit, and if you don't know how to drive in that situation so are you.
"works" includes a failure mode of "alert a human and ask them to take over."
> when new areas of problem space are explored.
The problem space is that the "rules of the road" are both legal, technical and social. All of which have internal conflicts as well as conflicts among each other. Anyone who has driven in severe weather has realized this in one way or another.
> For example, imagine a full self driving car trying to get out of a city that's flooding due to heavy rains, while having to compete with people fleeing to higher ground on foot.
Why do I find this easier to imagine in the fictional setting of Elysium than on the real Earth?
People can't do that either. Some years ago there was a massive snowfall in Rome, where it seldom snows ever, people don't generally carry snow chains, and there's few snowplows and such.
Many people reacted by abandoning their cars in the middle of the road, which is basically what I'd expect any FSD vehicle to do.
I do think that once we start to investigate ML/AI structure in the direction of figuring out the correct solution rather than trying to just find functions for control algorithms based on input->output mappings, then a lot of these problems are going to disappear.
Mapping some complex input state to control actions is literally the definition of driving a vehicle.
Maybe it takes 1 million hours of computing to train a model that can generate a logo, but an average human could have learned how to do that in just 50 hours of training with Photoshop.
The point of the article is that now for pennies users can generate logos in seconds that would have previously cost hundreds of dollars and days of back and forth with a designer.
This dynamic is going to flow through the economy
I really wish people would consider all the possibilities, and assign their relative probability weights to them. Is Karpathy 100% sure it will be like self-driving cars? 50%?
Let's see you see a car full of bumps, marks and broken lights. You might think: this car has crashed before, I am going to avoid it. Or a car with racing parts and decals and tinted windows, you know that car likes to accelerate faster than usual and may be unsafe to be around. Or you see a SUV with baby on board stickers, you'll know that if you are going to crash you may try to crash that car last because it has babies inside, etc...
So humans don't see objects they see the whole situation, unconsciously even.
I was quite surprised by this sentence, as I thought we didn't have self driving cars. Have I been sleeping under a rock?
He’s overselling how mature the technology is.
Even the crowd here fall for it. Downvote and then go read their term of service and look for "safety driver" and "remote" and "fleet response specialists". Then go cry about your waymo invesment.
I agree, they have remote drivers waiting to take over, but I think that the current SOA is delivering a very high % of autonomy. Is this viable? Dunno... Will it creep up? Dunno...
From the article, I find it strange that AGI often de facto implies "super intelligence". It should be 2 distinct concepts. I find that GPT-4 is close to a general intelligence, but far from a super intelligence. Succeeding at just general intelligence would be amazing, but I don't believe it means super intelligence is just a step away.
This also brings me to a point I don't see discussed a lot which is simulation (NOT in "we live in a simulation" sense). Let's say I have AGI, it passes the above mentioned shrine test, or any other accepted test. Now I'd like to tell it "find a way to travel faster than light" for example. The AGI would first be limited by our current knowledge, but could potentially find a new way. In order to find a new way it would probably need to conduct experiments and adjust its knowledge based on these experiments. If the AGI cannot run on a good enough simulation, then what it can discover will be rather limited, at least time-wise most likely quality wise. I'm thinking this falls back to Wolfram's computational irreducibility. Even if we managed a super general intelligence, it will be limited by physics of the world we live in sooner rather than later.
Also the article touches briefly on drivers going into jobless. A lot of drivers where i'm from seems to be retiring middle-old age working in taxi. I think it's a good job fit for them and I don't know how the new self-driving industry can provide the same thing (?)
Moreover, public transport often isn't as comfortable as your own vehicle (which I understand is a luxury).
Conversely, when it comes to driving in a large city, finding a parking spot can often be a major hassle.
From what I've seen, the main reason why people want a car here seems to be wanting to travel with small children. Moving within Tokyo with car is not very convenient.
For a five-minute walk (or even a longer ten-minute or fifteen-minute walk), pulling a small cart is not exhausting at all. I do it every week when buying food: I choose one of the several supermarkets in one of the nearby blocks, walk to it pulling my empty cart, after paying for the goods I put everything into the cart, and walk back home pulling the full cart. No public transport needed, though I've seen people carrying these carts into public transport too (this is easier when it's a low-floor bus, instead of the high-floor ones).
You can also get things delivered when it's a larger amount than can fit on your cart: while paying at the supermarket you ask for delivery, and they'll use a cargo tricycle to bring it to your building.
- what Automation initiatives never hit "take off"? I mean, like for Nuclear Fusion, human interplanetary exploration, and Quantum Computing there's some chance that the technology simply remains beyond us "forever", I guess that "forever" means more than the lifetime of the people who start the journey... or maybe actually just beyond humans full stop. We should admit there is a non-zero chance that FSD is one of these failing quests, even if a rational observer would have to say that that chance does seem to be shrinking and close enough to 0 to instill some confidence. Perhaps domestic robotics, auto-doctors, robot-manufacturing, programming, drug development will playout to automation - but maybe not.
- how do we consider the utilisation of the resources to do this? FSD has been very expensive so far, it's consumed lots of investment capital and lots of human creativity. Was that investment rational given where we stand? If society had held off and invested minimally from 2000->2024 how much would that have delayed the technology in reality? Or is it the other way round? Has the FSD investment facilitated the development of other technologies and created a 1->1 acceleration (for every year of 2000->2024 it's brought FSD a year closer than it would have been, so a cold start this year would mean FSD by 2050 or similar, whereas if we keep going then we can expect FSD by e.g. 2026)
- how do we value these outcomes? Are these unalloyed goods, or are some worse than the status-quo? It could be argued that the development of some technologies left the world worse off than before - smoking, social media, personal automobiles (I know this is politically charged but I am just using examples others have raised before). Can we choose rationally, especially if a large scale intervention and development process is required to realise these outcomes?
Of course, there isn't much money in teaching a bot that only knows english chinese.
EDIT, Wikipedia page for context: https://en.wikipedia.org/wiki/Chinese_room
Not only are there no LLMs in existence today can do this without explicit action mapping, but the mechanism for storing that piece of information would rely on doing a large number of training runs for transfer learning to retain that information, and we humans don't actually work like that.
That is probably not a good criterion to decide whether something is intelligent or not.
LLMs do not constitute "AI" let alone the more rigorous AGI. They are a GREAT statistical parlor trick for people that don't understand statistics though.
I have a textbook, "Artificial Intelligence: A Modern Approach," which covers Language Models in Chapter 23 (page 824) and the Transformer architecture in the following chapter. In any field technical terms emerge to avoid ambiguity. Laymen often adopt less accurate definitions from popular culture. LLMs do qualify as AI, even if not according to the oversimplified "AI" some laymen refer to.
It has been argued for the last several decades that every advance which was an AI advance according to AI researchers and AI textbooks was not in fact AI. This is because the laymen have a stupid definition of what constitutes an AI. It isn't because the field hasn't made any progress, but instead because people outside the field lack the sophistication to make coherent statements when discussing the field because their definitions are incoherent nonsense derived from fiction.
> They are a GREAT statistical parlor trick for people that don't understand statistics though.
The people who believe that LLMs constitute AI in a formal sense of the word aren't statistically illiterate. AIMA covers statistics extensively: chapter 12 is on Quantifying Uncertainty, 13 on Probabilistic Reasoning, 14 on Probabilistic Reasoning Over Time, 15 on Probabilistic Programming, and 20 on Learning Probabilistic Models.
Notably, in some of these chapters probability is proven to be optimal and sensible; far from being a parlor trick it can be shown with mathematical rigor that failing to abide by its strictures is not optimal. The ontological commitments of probability theory are quite reasonable; they're the same commitments logic makes. That we model accordingly isn't a parlor trick, but a reasonable and rational choice with ledger arguments proving that failing to do so would lead to regret.
https://chat.openai.com/share/71d438d7-d1f5-4f0f-9b63-8b5dd6...
> Some people get really upset about it, and do the equivalent of putting cones on Waymos in protest, whatever the equivalent of that may be. Of course, we’ve come nowhere close to seeing this aspect fully play out just yet, but when it does I expect it to be broadly predictive.
I think the equivalent of putting cones on Waymos in protest will involve large scale protests and civil unrest in some places. I think people will die (inadvertently?) because companies will act to put inadequately tested self-preservation modes in their hardware device to protect against aggressive and organized vandalism.
I don’t get the sense he was trying to say that self-driving automation is the exact same as AGI. Mainly that that AGI, like other technologies before it, will displace some jobs and create new ones but this will require companies to figure out how to scale the technology.
I do think this is still very optimistic. If indeed AGIs can think and learn on their own it isn’t hard to envision a future where humans aren’t needed at all in the loop.
We should consider the OODA loop of a person's self-determination separately from the menial tasks a person undertakes to make a living. Automating a task is totally different than breaking a person's ability to self-orient.
It seems to me to just be another iteration of dealing with uncertain information: our neighbors may lie, our leaders may lie, newspaper may lie, radio may lie, TV may lie, blogs may lie, social networks may lie, pictures are photoshopped, videos are deepfaked..
At each iteration we had some problems but we adapted, it's one thing we're good at.
And now we have substantial societal adaptations, both legal and structural to support ubiquitous vehicular transport.
Similar changes are on the way to support self driving. Our environment will be adapted to make it easier to implement self driving. And for that we won't need AGI.
Jaywalking is a crime thanks to the car. Who knows what we're not going to be allowed to do soon because of self driving.
Yet there are no signs of that. If anything they appear to be behind us.
These days people seem to define if more as artificial super intelligence.
>When your Waymo is driving through the streets of SF, you’ll see many people look at it as an oddity... Then they seem to move on with their lives.
>When full autonomy gets introduced in other industries....they might stare and then shrug...
Which I guess is ok on a small scale but if AGI starts to replace all human jobs it will have a different effect to Waymo firing some drivers and hiring AI researchers.
Humans are able to move our heads to infer depth and resolve issues like occlusion.
No amount of AGI can solve those if we say take a Tesla and the cameras are low quality, fixed and limited in number.
And the same hardware question applies to a lot of use cases for AGI.
When I was much younger, "AI" was what "AGI" is now. Now people started using "AGI" for "cars with several sensors and okay algorithms for collision detection" and then you have loud advocates going on obviously logically broken rants about the nature of "actual" intelligence -- and those are philosophical and not scientific.
But still, we don't have anything even 1% close to AGI. And no, Chess and Go have NEVER EVER been about AGI. I have no idea how people ever mistook "combinatorics way beyond what the human brain can do" with "intelligent thought" but that super obvious mistake also explains the state of the AI sector these days, I feel.
So before long, I guess we'll need another term, probably AGIFRTTWP == Artifical General Intelligence, For Real This Time, We Promise.
And then we'll start adding numbers to it. So I am guessing Skynet / Transcendence level of AI will be at about AGIFRTTWP-6502.
As for the state of this "industry", what's going on is that people with marketing chops and vested interests hijack word meanings. Nothing new, right? But it also kills my motivation to follow anything in the field. 99.9% are just loud mouths looking for the next investment round with absolutely nothing to show for it. I think I saw on YouTube military-sponsored autonomous cars races 5+ years ago (if not 10) where they did better than what the current breed of "autonomously driving cars" are doing.
Will there be even one serious discussion about the general AI that you can put in a robot body and it can learn to clean, cook, repair and chat with you? Of course not, let's focus on yet-another-philosophical debate while pretending it's a scientific one.
As a bystander -- not impressed. You all who are in this field should be ashamed of yourselves.
I'm not seeing any drift of terms here - the only thing that seems to be happening for AI and AGI terms is correcting for what has happened in the sci-fi media and bringing the usage back to what it always has been in the computer science literature, now that it's closer to reality than mere fiction.
I come from a generation where AI was Skynet, Terminators, the Johny Depp's Transcendence movie AI, even HAL-9000, and other such like.
I still think putting the word "intelligence" is completely dishonest however. There's nothing intelligent about what we have today, even "self-driving" cars fail very badly on what seems trivial conditions. They are a huge mish-mash of if/else chains and some statistical models sprinkled in.
And please don't say "but what if human intelligence is just a chain of if/else statements and statistical models sprinkled in?" because it's very apparent and visible that it's more than that. F.ex. we can learn just from a few trials and errors whereas the so-called "AI" nowadays can't get things quite right even after billions of training sequences.
Using NNs is just a first step. To me labeling this as actual intelligence is childishly rushing into conclusions.
Maybe Elon really screwed the project by forcing the use of video cams and Karpathy is still salty about it.
Tesla cameras aren't even class leading in the automative industry, let alone at the cars price point. They are worse than the iPhone you handed down to your tween 5 years ago. What makes anyone think that their quality and placement will ever be adequate even with new CPUs and better software?
They might just learn they need to add back modalities they neglected previously, or explore some new ones.
It paints not so rosey picture about it: https://www.youtube.com/watch?v=-Rxvl3INKSg
