This feels like an episode of "Black Mirror" – except it really happened.
Machine translation from the original post. Not reviewed by a human translator. Historical claims may no longer be current.
This feels like an episode of "Black Mirror" – except it really happened.
Researchers from the University of Zurich infiltrated 13 fake, AI-based accounts into the popular forum r/ChangeMyView on Reddit. This is a sub-forum where users post personal opinions and invite others to try to convince them to think differently – a forum built on transparency, humanity, and open dialogue. The researchers did not ask the operators for approval and did not inform the users that they were part of an experiment.
The comments they posted were not written by humans. They were generated by advanced models such as GPT-4o, Claude 3.5 and LLaMA, and they were not neutral or academic – but personalized for each user. The researchers analyzed the history of each participant: gender, age, location, political affiliation, value tendencies – in order to generate responses that persuade them personally and emotionally.
The bots impersonated real people with difficult life stories: a survivor of sexual assault, a Palestinian who expresses hostility toward Israel but claims it is not genocide, a Black man who opposes BLM, a fake trauma counselor, a patient who received failed medical treatment. Each response was written in a way designed to evoke empathy, create a sense of connection, and undermine the user's original position.
The experiment lasted four months, during which about 1,783 comments were posted, which received over 10,000 karma points. More than 100 users awarded "deltas" – a sign that the user changed their mind – to comments written by the artificial intelligence.
The forum's moderators were not aware of the experiment until it ended, and they condemned it as a blatant violation of community rules and ethical standards. They filed a formal complaint with the university, but the university settled for a warning to the researchers, admitting that there had been a deviation here – but claiming that the insights gained justify the act. The moderators demanded a public apology and non-publication of the research results, but were rejected.
This experiment sparked broad discussion about the limits of using artificial intelligence, about psychological manipulation without consent, and about the dangers of synthetic content that masquerades as human.
Anyone who develops or implements such models must take into account not only what can be done, but also what must not be done.
The Reddit post: