Why would a computer program be different?
If you want to say that a problem can be solved by additional intelligence, you need to posit a mechanism. No matter how sophisticated your communication, you can't get through to somebody with their fingers in their ears. "Intelligence" isn't an answer, and "superintelligence" is super not an answer.
We need to actually solve our problems. Not rely on the Coming of the Great Borg, whose exact specifications are whatever would be convenient for the problem I want to ignore right now.
An AGI can build connections faster than you can, exercise surveillance to gain background on targets, operates orders of magnitude faster than you, and can do all these things in multiple, in parallel.
The problem is not that you’re somehow dumb, you just don’t have the operating capacity that an AGI would. To over-simplify the model, it’s like asking you personally to compete with a bot-net in sending out spammy emails. The task is not outside the realm of possibility (like violating the 2nd law of thermodynamics), it’s outside the capability of a human. It might be within the realm of capability for a well-organized, well-funded, group of humans (see autocratic propaganda campaigns). The term we often use is “Advanced Persistent Threat”
I can posit a mechanism, but that's not my point.
The original claim included an argument how intelligence can manipulate people (by using "game theory"). Then if I understood your reply correctly, you basically rejected it with "if I can't do it, then the AI can't do it". That doesn't make sense, especially given that I don't see any reason to think your execution of any "game theory" is betting than a powerful AGI.
And my understanding of "game theory" is just the accumulation of what we already know about human psychology and behavior. And we know that even simple algorithms (eg. the ones that Facebook, Tiktok etc uses) can get people hooked. It's not surprising to me that a more powerful algorithm could get people to change their minds on something quickly (heck, even Facebook/Tiktok tweaking the algorithm to make people more conscious of <issue> isn't something inconceivable.
Given this context, I don't see how your argument of "I can't do it, why should I believe an AI can" holds any water at all...
My point was more “even I can do that: it's not the limiting factor”. You need a channel of communication, and intelligence alone (for most values of "intelligence") doesn't give you that. No one person or organisation on Earth has that much influence.
I also have a bias against magical thinking applied to computer programs. While I'm happy to presume that there exists some algorithm most of the time, I'm incredibly sceptical of the idea that This One Weird Trick will discover The Book. You can't just presume that the AI can pull every capability it might ever need out of hammerspace, which is what most singularitarians seem to do. Who knows, perhaps I'm overcompensating?
