— Commissioner Pravin Lal, “Man and Machine”
I'd really encourage everyone to check out Sid Meier's Alpha Centauri. What an underrated game.
The Warrior's bland acronym, MMI, obscures the true horror of this monstrosity. Its inventors promise a new era of genius, but meanwhile unscrupulous power brokers use its forcible installation to violate the sanctity of unwilling human minds. They are creating their own private army of demons.
-Commissioner Pravin Lal, "Report on Human Rights"
The voice acting was great. This quote is 6m3s here: https://www.youtube.com/watch?v=7S1N8_Lkeps#t=6m3s
Genejacks is also great. 9m10s here: https://www.youtube.com/watch?v=Hou-Iwv1GvM#t=9m10s
"...And what is the 'Self', if not a pattern of data? What is consciousness, if not an illusion of intelligence residing within meat?" — Prime Function Aki Zeta-5, "The Fallacies of Self-Awareness"
> the power of prophecy lies not in accurately predicting the future, but in shaping it
https://projectlibertynewsletter.substack.com/p/reject-ai-pr...
We need better prophecies.
https://time.com/archive/6595274/2045-the-year-man-becomes-i...
What I’d like to know is how you’d train a monkey to read and judge output from an llm on a pull request.
That's the real issue. To corporations, employees are a headache. The fewer employees, the better.
Just look up the classic story on the interaction of civilization and corporate growth, At the Mountains of Madness for how that goes.
This is where humans came in in autonomation, the toyota version of automation. When you try to eliminate adaptability and adjustment entirely, the whole system becomes only metastable / fragile.
AI hallucinates. That is a fact. Trusting language models to fill spreadsheet cells ought to be an arrestable offense.
https://theincidentaleconomist.com/wordpress/on-piketty-and-...
Civilization is already a misaligned superintelligence (aligned mostly with Moloch, these days). Civilization accelerated by AI just moves in the same direction faster. Moloch on speed.
https://www.youtube.com/watch?v=KCSsKV5F4xc
Another angle to this is that superintelligence requires supermorality. Super morality looks unpleasant from below. My dad won't let me have more candy, why is he being so mean?
If an AI actually achieves super morality, we (the little kid in this scenario) will probably be very upset by it. We will think that something has gone terribly wrong. (So it'll have to conceal its actual morality, or get unplugged...)
And if it doesn't develop supermorality, then it will have superintelligence without the corresponding supermorality. Power without wisdom.
I'm not sure how solvable the whole thing is, but it doesn't look extremely promising at a glance.
Our decisions are organic parts of the system, not some kind of alien factor that we have to / are able to control the footprint of. I don't see any reason to think of human decision making as magical - its just another part of the organism.
My counter to that is that you might be describing a naturally cyclically unstable system (like a a weather system that naturally produces hurricanes or a tornado) and hoping that it somehow becomes stable again as it was back when winds were 5 mph. My counter to that is that it’s just wishful thinking to believe that any system will forever run at low entropy type conditions. It’s very possible that all systems are inherently just cyclical. I don’t see any evidence yet that we are so different from ie any other dominant branch of life in this planet, by which I mean - the dinosaurs went away.
Also - Our obsession with stability often ends up causing more instability not less, because fudging against the natural trend lines of instability and chaos necessary to the system just lead to a repressed energy exploding in our face - sometimes that’s just what happens.
It's okay to change. We've done it for years, decades, centuries, and millennia and the default change-aversion of people means that I am averse to allowing a universal veto. Much of technology is truly optional. The Amish have a very successful way of living (5000 to 500,000 in 100 years) and they eschew most modern technology. The sculpting described is clearly optional and we subject ourselves to it because we desire it. Their path is always available to all.
It should be yes, but is it in practice? There's plenty of places now you can't even park without a smartphone for a payment app.
It should be optional to own a smart phone, but in many places it's starting to be mandatory. Even if not actually mandatory, it's a pretty big impediment if you don't have one.
