[0] https://openai.com/index/sycophancy-in-gpt-4o/
[1] https://www.reddit.com/r/MyBoyfriendIsAI/comments/1qx3jux/wh...
1. possibly/probably not in a good or healthy way? idk
Grossly irresponsible to ever release this IMO.
ex: https://www.reddit.com/r/MyBoyfriendIsAI/comments/1qx3jux/wh...
I used it to help brainstorm and troubleshoot fiction: character motivations, arcs, personality, etc. And it was truly useful for that purpose. 4.5 was also good at this, but none of the other models I’ve tried.
Of course this particular strength is dangerous in the hands of lonely unstable people and I think it’s dangerous to just have something like that openly out there. This really shows that we need a safe way to deploy models with dangerous specializations.
Thank you for saying this
It is not "they go to therapy" because that's cheating; that answers the question "what can they do?" not "what can society do?" (and i think it's a highly speculative answer anyway)
Counseling can help with this to some degree and everyone can make some amount of progress. The question is what we do with those whose "ceiling" remains lower than is tenable for most relationships. For those, there is not a better solution than robots.
However, the always-available, always-validating robot is not a valid psychological need. It is a supernormal emotional stimulus. It is not healthy and, like other supernormal stimuli, builds invariably tolerance, desensitization, and dependence. Fast cycling of discontent -> open app -> validation is a huge contributor, the same way that the constant availability and instant nature of vaping make it incredibly addictive.
So I think there is a case to be made for harm reduction.
The answer no one wants to hear on HN is get rid of capitalism as it is currently.
You, ajkjk, are a product. When you are not working I need you to be looking at a screen full of ads and clicking on things. Don't worry, you won't have anything else to do because everyone else is also doing the same. If your doing things with friends and spending your attention on them, you're not spending your attention on my latest product, and that's pretty anti-capitalist of you. Thinking about going to the bar, you can't afford it, VC bought up all the property and bars and raised the price 400%. Trying to find some other 3rd place to hang out at? Don't exist, nobody can afford people that show up and don't spend anything.
We have designed modern society to push us toward an AI that can give us our undivided attention because everyone else is so busy doing nothing they don't have time for friends.
It quantifies it as a solved problem.
Why and what drove people to do this in the first place.
This is the conversation we should be having, not which model is currently the most sycophant. Soon the open models will catch up and then you will be able to self host your own boyfriend/girlfriend and this time there won’t be any feedback loop to keep it in check.
We don't have digital superhumans. These simulacra are accessed primarily via corporate-controlled interfaces. The goal of their masters is to foster dependence and maximize extraction.
Lonely people forming relationships with digital minds designed to be appealing to them is sad, sure, but the reality is much sadder. In reality these people aren't even talking to another person, digital or otherwise, just a comparatively simple plausibility engine which superficially resembles a digital person if you're not paying much attention.
How do you know that? Maybe it's the same argument to solipsism, or the Chinese room thought experiment, that these "digital superhumans" are stochastic parrots too, just like our current LLMs.
My AGENTS.md has:
You MUST use a modern but cost effective LLM such as `qwen3-8b` when you need structured output or tool support.
I've been reading a lot of "screw 'em" comments re: the deprecation of 4o and I agree there's some serious cases of AI psychosis going on with the people who are hooked, but damn this is pretty cold - these are humans with real feelings and real emotions here. Someone on X put it well (I'm paraphrasing):
OpenAI gave these people an unregulated experimental psychiatric drug in the form of an AI companion, they got people absolutely hooked (for better or for worse), and now OpenAI is taking it away. That's going to cause some distress.
We should all have some empathy for the (very real) pain this is causing, whether it's due to psychosis or otherwise.
At the same time though, I don't think it's healthy to let them go on with 4o either (especially since new users can start chatting with it)
I think we're too attached to media.
Psychosis is a real risk for schizophrenia spectrum disorders, but a lot of those relationships look to be rooted in disordered attachment.
I read these posts and feel sad for these people and it makes me realize now as an older guy how much more I value learning how to skateboard or run a committee, or write code, run a business or any time I spent on investigating the real world.
Life is short, these people are getting emotionally nerd sniped and dumped into thought loops that have no resolution, no termination point, no interconnectedness with physical reality, and no real growth. It's worse than any video game that can be beaten.
Maybe that's all uncharitable. I remember when I was a child people around me in the academic religous circles my parents ran talking about how "engineers" lacked imagination and could never push human progress forward, and now decades later I see, those people have at most written papers in already dead niche flights of fancy where even in their own imaginary field their work is relegated. I know what they did isn't "nothing", but man... it's a lot of work for a bunch of paper in a trashcan no ine even cites.
This was something that I figured out with my first gf, and had never seen written down or talked about before - that when I praised her she became happy, and the more superlative and excessive the praise got, the happier she became, calling her gorgeous, the most wonderful person in the world, made her overjoyed.
To be clear I did love her and found her very attractive, but overstating my feelings for her kind of felt like I came close to lying and emotional manipulation, that I'm still not comfortable with today. But she loved it and kept doing it because it made her happy.
Needless to say we didn't stick together (for teen reasons, not these reasons), but later in life, I tried doing this, but I did notice a lot of women respond very positively to this kind of attention and affection, and I still get some flack from the miss from apparently not being romantic enough.
Maybe I am overthinking this, or maybe I am emotionally flatter than I should be, but finding such a powerful emotional lever in another human being feels like finding a loaded gun on the table.
So yes, I think it is a bit sexist or at minimum gender typing. And I don't think it's necessarily a "lie" for you to overstate your feelings. You might have matured in your approach, but I believe that everyone appreciates (to some variable measurement) positive affirmation from their partners. And that your lie was recognizing your partners needs for inputs, to help them in their self-image, and to assure them in their self-doubts. These are not lies.
For example if I told you 'good thinking', you would probably think I am giving a token of appreciation to you. If I told you 'wow, you are absolutely brillant!', you'd probably think I'm mocking you or trying to manipulate you into doing something.
On the one hand they're "mourning" their AI partners, but on the other hand they have intelligent and rational conversations about the practicalities of maintaining long running AI conversations. They talk about compacting vs pruning, they run evals with MRCR, etc. These are not (all) crazy people.
That seems like a very unlikely conclusion to me. Why is it your prior?
I won't say it's hard, I will say it requires attention.
The problem is we live in a capitalistic society that believes capturing your attention in order to sell it is the number one priority of any business.
This really isn't human versus AI, it's humans versus FAANG/social media/ads/TV.
work, sleep, socialize you can only do 2. With the help of AI you could talk to people as much as you want without wasting their time.
>exploited until the legal pressure piled up
Being given access to a relationship is not exploitation. In some ways AI relationships are better than human ones.
It's like paying someone to be your friend and saying "wow, this is so much easier than friendships otherwise"- of course it is, that's what you're paying for. Nothing wrong with that per se- a lot of therapy e.g. is paying someone to pay close attention to you. But it's not the same sort of thing at all.
Also I don't see any of the big cloud providers (apart from Azure) saying they are bound to professional secrecy acts (e.g. the S203 in Germany)
1) 4o was very sycophantic and had no real safeguards against really deep romantic roleplay. It'd 'go along' with the user and give minimal to zero pushback; "I feel a connection with you" "I feel it too" etc etc
2) It was good enough at just chatting that if you didn't really push it it made a reasonable simularca of talking to an actual person.
Combine 1 and 2 with people who can't connect well with real people for any number of reasons, physical disabilities, mental health issues, emotional development issues, etc, and you get r/MyBoyfriendIsAI/ [0] and the various other places that initally freaked out when 4o was initially sunset for gpt5 and now again.
Loving AI bots. Killing yourself based on what an AI bot says.
Its hard to believe any of this is real or should be.
Dating apps are skewed: men receive little attention while women have an overwhelming amount of attention.
Porn satisfies our most base sexual functions while abandoning truly intimate connections.
The ultimate goal of sexual unions has been demonized and turned into something to avoid. That being children. After school specials since the 80s have made pregnancy a horror to avoid instead of a joy to grasp.
AI is just the latest iteration of technology increasing the divide between the sexes.
When the clankers come, we're fucked.
I don't draw the same conclusion. I think they've made teen pregnancy a horror to avoid, which is totally fair.
After solidly internalizing the messaging of "teen pregnancy is the worst thing ever" & "sex leads to teen pregnancy", there's no "switch" to make those thought patterns disappear without a trace at the point at which it's "ok" (by whatever metric is relevant to the individual) to participate in sex, child-rearing, etc. So individuals find themselves dealing with long-term guilt at having sex and/or aversion to having children, neither of which is "rational" according to their values but which nonetheless is real and affects their behavior.
Pregnancy can be employment-disrupting, and a horror if you're not financially ready to raise a child. Teen pregnancy can end one's future, one's educational and career prospects, before it even begins. The steady and nearly-uninterrupted decline in teen pregnancy from its peak in the early 90s is an absolute miracle of sex education.
The birth rate for women 20-24 was cut in half from 2005 to 2023, and the birth rate for teens under 20 dropped by 2/3s[1], which is frankly amazing progress.
1: https://usafacts.org/articles/how-have-us-fertility-and-birt...
I'm not following your train of thought here. Why is this the fault of computing?