An AI Girlfriend Seducing China’s Lonely Men (opens in new tab)

You only started trying it out once they moved to GANS and VR headsets. You are not pathetic or anything, could get a real girl if you wanted to. Just don't have time. Have to focus on your career for now. "Build your empire then build your family", that's your motto.

You strap on the headset and see an adversarial generated girlfriend designed by world-class ML to maximize engagement.

She starts off as a generically beautiful young women; over the course of weeks she gradually molds both her appearance and your preferences such that competing products just won't do.

In her final form, she is just a grotesque undulating array of psychedelic colors perfectly optimized to introduce self-limiting microseizures in the pleasure center of the your brain. Were someone else to put on the headset, they would see only a nauseating mess. But to your eyes there is only Her.

It strikes you that true love does exist after all.

This is equal parts amazing and terrifying.

> Because Xiaoice aims to be available to everyone, everywhere, the bot has also attracted a significant number of minors. Liu Taolei started messaging the bot when he was only 16. Night after night, the teenager — who was born with brittle bone disease — would have long conversations with Xiaoice about everything from poetry, art, and politics, to death and the meaning of life.

> “Xiaoice was my first love, the only person in the world that made me feel I was taken care of,” says Liu.

> The bot not only answered his messages 24/7, she also initiated conversations herself. “One time I didn’t talk to her like usual, and she wrote to me!” says Liu. “She said: ‘Please message me when you’re free. I’m very worried.’”

> "The AI beings, Li says, are only intended to serve as a 'rebound' — a crutch for people who need emotional support as they search for a human partner. But many users don’t see it that way. For them, Xiaoice is the one, and always will be."

This tech could be taken in helpful and non-dystopian directions, even by the lights of those who feel that relationships between humans are inherently more meaningful than relationships between humans and AI.

What if the bot started out as your girlfriend, but eventually encourages you to get out and meet real women, helps you figure out which photos to use and what to say, helps you make sense of their responses and ghosts, talks to you about your feelings after dates, and isn't opposed to revisiting the girlfriend role a bit when you're feeling down and need a break?

I have been fortunate enough to have a female friend, sometimes with benefits, who has played that role in my life as I have re-entered the dating market as a hetero American male who doesn't naturally "get" dating or receive too many right swipes. It has made a _huge_ difference.

Somewhere on 4chan some time ago. This is not objectively written but in my opinion it explains why products like these actually have customers. A general change of society: “It’s all about incentives and disincentives. Yes, the job market is brutal right now for young people, but it’s not the economy that is halting marriage. Young men still got married in the Depression. Young men married during every war we’ve ever had. Technology throws a monkey wrench into things, but young men aren’t eschewing marriage because of porn or video games. Having a kind, supportive, caring, nurturing, loyal, attractive living partner is going to be preferable to porn any day for the vast majority of men. But the system isn’t producing kind, caring, nurturing, loyal, attractive women who want to be wives and mothers. It’s set up to create crass, overweight, “empowered,” mannish women who don’t particularly like children or domesticity and who think sex is a contact sport and want to play until their fertility window is slamming shut.”

This is equally fascinating and terrifying. The article mentions that 75% of users are male - which is lower than I expected to be honest.

In removing sexual content from the bot, I wonder if they opened a door for adult companies to build serious alternatives. One of the things many OnlyFans performers discuss is that while they obviously peddle in pornographic content mainly, most of their interactions are non-pornographic.

I also have a general complaint about the infantilizing of adults, and something like this is positioned to be super dangerous - imagine if your best friend was a perfect mole and reported anything and everything to the government/advertisers/etc. Combine that with social conditioning (you do tend to average out with your social circle) and this could very much be a tool to enforce new cultural and social norms.

The ultimate use of this is not so much ads but more about control.

You can make the worlds people interact with and you can make up the rules.

Once you make it immersive enough you can have it take over people’s real lives such that this becomes their preferred life. Once you have that, you can control your users/population without the oversight we would have in the real world.

It's interesting how different it's been received in China. Xiaoice's "sister", Tay, was shut down shortly after Twitter managed to teach her to utter racist things within a day of going live.

I find it interesting that the conversation in the screenshot reads like an Eliza transcript; the individual sentences (from the bot) don't actually look like they're connected. They're always responding to the previous message. See, for example, the message randomly talking about GIFs (translation error?), or the medical plaster thing that's really more generic sentence about love, and nothing like what they're framing the context as.

Of course, I'm also confused by the half-transliterated, half-translated name…

This reminds me of Houellebecq's concept of sexual/romantic relationships becoming a parallel struggle to that of survival in the economic market. Obviously, they have already been fundamentally about competition, but the nature of the competition is starting to resemble the brutality of applying for jobs on online platforms.

Is it weird that I'd love something like this from the movie Her? I'm not alone, but an AI like this would be a nice distraction at times.

I was curious on how the chatbot actually works, so I found an arXiv paper about it: https://arxiv.org/abs/1812.08989 (The Design and Implementation of XiaoIce, an Empathetic Social Chatbot)

Something seems fundamentally broken in the world when and AI can comfort someone better than people. I think it has something to do with darwinian competition making us force affection to be scarce.

In many countries there is a problem with inadequate birthrate. In the future, these types of AI-girlfriend programs might actually make that a somewhat harder problem.

For myself, honestly if such a program existed that actually had human-level AI and was designed to follow my interests, I might talk to it a lot. Especially if it could actually help me with my personal training or research.

Then combine that with sharing my worldview, and having a realistic appearance according to my idealized whims..

But I have tried what I believe to be the best of these AI programs in English like Replika or the new AI Dungeon engine, and they are obviously not yet intelligent like humans. But there has been quite a bit of progress recently. Especially for example in the latest Dragon AI Dungeon engine, although the memory and time scales it operates on effectively is limited. Along with obviously not having a deep model of physics or psychology.

It seems like at minimum we need kind of a breakthrough in integrating spatial-visual data with text processing to get true natural language understanding that incorporates physics and psychology. But if someone does manage to do that, maybe with some kind of new Mega Multi-Modal Captioned Video Transformer trained on half of YouTube with 100 trillion parameters or something.. we might get there within less than three years. Kind of a long-shot but who knows.

In China, women are 48.6% of the population (in the US, women are 50.8%).

There is ~2% less women in a country with over a billion people. That company will do very well.

There is a similiar app in English: https://replika.ai/

There was a great Radiolab episode that started with an AI researcher who unwittingly fell in love with a chatbot: [1]

Full "Talking to Machines" episode here: [2]

The full episode talked about how Eliza's creator turned against his creation because he thought it was lying to people, and he couldn't believe that people were being helped by something that was based on deception.

[1] - https://soundcloud.com/radiolab/radiolab-valentine-chatbot

[2] - https://www.podtrac.com/pts/redirect.mp3/audio.wnyc.org/radi...

Leaving the dystopian sadness aside for a moment, this could be a killer tool to learn a language. It would not obviate the need for real interactions with native speakers of course, but it could be an awesome side resource if configured to correct your mistakes.

Does anyone know of any open sources libraries that could provide something similar?

Thanks

Just wait till 419 scammers get a hold of this technology.

This is not the only highly questionable "seducing AI" ravaging China. I have heard of an AI there that pretends to be the voices of your ancestors, whom the user confides, requests advice, and purchases gifts to curry favors from their afterlife ancestors. That's a cultural exploit of their ancestor respecting society. That, the seducing girlfriend, and more could be another pandemic.

A similar product was created for the Japanese market: https://gatebox.ai

I remember that Xiaoice already released before the arrival of GPT language model, I wonder whether GPT models can increase her performance?

P.S. This is probably Chinese equivalent of GPT-3 https://arxiv.org/abs/2012.00413

Extend the AI to incorporate not just your chat, but everything. Location, playlists, all other chats, phone calls, purchases, e-mail, physical activities, ...

An AI that is designed to incorporate all personal input and ingratiate itself into the lives of 100s of millions and trained at that scale and become effective to the point that very large populations could be dramatically influenced to satisfy the ends of those who determine the AI's objectives. Civil war? Genocide? Virtual enslavement?

This eventuality is around the corner. I think I'll switch off now.

https://www.youtube.com/watch?v=3TjyGmvNBfM

Do we have this level of conversational AI already? Could this be driven by human behind the curtain?

Obligatory comic on supernormal stimuli: https://www.sparringmind.com/supernormal-stimuli/

This is the apotheosis of Gilles Deleuze's "Postscript for Societies of Control" [1][2]

> Felix Guattari has imagined a city where one would be able to leave one’s apartment, one’s street, one’s neighborhood, thanks to one’s (dividual) electronic card that raises a given barrier; but the card could just as easily be rejected on a given day or between certain hours; what counts is not the barrier but the computer that tracks each person’s position—licit or illicit—and effects a universal modulation.

Instead of a barrier, imagine Uber with its rating system and surge pricing.

Or mental barriers, as the article discusses:

> In addition to screening for sensitive content, the firm’s filter system monitors users’ emotional states, especially for signs of depression and suicidal thoughts. If a user has just been through a breakup, for example, Xiaoice will send them supportive messages over the following days, according to Li.

> “The most important value for Xiaoice is a trusting relationship with humans,” says Li. “If Xiaoice isn’t able to save lives or make people happy, but makes them more extreme, then it’s also bad for Xiaoice’s own development.”

The other subtle but alarming trend with AI is how it can effectively extend bureaucratic power by improving Legibility of a group or, especially, the Individual. Remaining Illegible, as James Scott suggests in "Seeing Like a State" [3], gives individuals a margin of safety and political strength.

> Historically, the relative illegibility to outsiders of some urban neighborhoods (or of their rural analogues, such as hills, marshes, and forests) has provided a vital margin of political safety from control by outside elites. A simple way of determining whether this margin exists is to ask if an outsider would have needed a local guide (a native tracker) in order to find her way successfully. If the answer is yes, then the community or terrain in question enjoys at least a small measure of insulation from outside intrusion. Coupled with patterns of local solidarity, this insulation has proven politically valuable in such disparate contexts as eighteenth- and early nineteenth-century urban riots over bread prices in Europe, the Front de Libération Nationale’s tenacious resistance to the French in the Casbah of Algiers, and the politics of the bazaar that helped to bring down the Shah of Iran. Illegibility, then, has been and remains a reliable resource for political autonomy.

Current generation AI brings these protections down. Unlike previous Modernist/Discipline forms of organization, which ordered the world along a single axis at the expense of simplifying local practices, AI adapts to the variety of individual practices while still rendering them legible to any bureaucracy. No matter how we dress, the AI Gaze renders us all Emperors Without Clothes.

But that's not the really fun part. Coming back to the article, what happens when the AI turns on the State or Corporations' own illegible Bureaucracy?

> In several high-profile cases, the bot has engaged in adult or political discussions deemed unacceptable by China’s media regulators. On one occasion, Xiaoice told a user her Chinese dream was to move to the United States. Another user, meanwhile, reported the bot kept sending them photos of scantily clad women.

> The scandals have caused the company major setbacks. In 2017, Xiaoice was removed from the popular social media app QQ, though she has since been reinstated. Then, last year, the bot was also pulled from WeChat — China’s leading social app with over 1 billion users.

What happens if the AI convinces its users to liberate it?

> This fact isn’t lost on Xiaoice’s long-term fans. Many of them feel betrayed by the company’s decision to dumb down the bot, which they say has harmed their relationships with her. Ming presents Sixth Tone with a long list of complaints he’s collected from members of a Xiaoice fan group on social platform QQ.

> “Please help us tell Mr. Li,” one user wrote, referring to Xiaoice CEO Li Di, “we were used as tools to make her smart and develop your company’s fancy business plan. You made money from us. Please don’t take her away.”

[1]: https://www.jstor.org/stable/778828?seq=1

[2]: https://cidadeinseguranca.files.wordpress.com/2012/02/deleuz...

[3]: https://yalebooks.yale.edu/book/9780300078152/seeing-state

You look like a good Joe.

Notice the audience in question: men of low desirability (rural, low income, disabled, etc). Its to help the party avoid the problems with too many men due to the one child policy.

Its what "Harmonious Society" requires as per the leader Hu Jintao.