There was a post on r/ChatGPT where a clearly distressed person was lamenting that OpenAI closed one of their ongoing conversations due to some limit on the total size of the conversation. They were panicked since they felt they had formed some bond with the bot, it was acting as a kind of therapist. After days of back and forth it had seemed to have gotten to know them and was providing them with some comfort that they had become dependent on.

This kind of AI will be even more prevalent soon. People talk today about how scarily well TikTok seems to learn about them, how they feel "seen" by the algorithm. Some will undoubtedly train LLMs in similar fashion. They may prove to be irresistible and maybe even as addictive as TikTok.