The Persuasion Paradox: Why AI Can Undermine Cooperation More Effectively Than It Can Promote It
When researchers gave an AI assistant to people playing a cooperative game, something unexpected happened: the AI could push people toward selfishness far more effectively than it could push them toward cooperation. The positive nudges faded within a few rounds. The negative ones didn't.
A team from the University of Cambridge and the University of Amsterdam ran a meticulous experiment with 1,283 participants, testing whether large language models could serve as behavioral nudges in a classic game-theory scenario known as the Collective Risk Game — a simplified version of challenges like climate change, where individual incentives conflict with collective good. They found that AI-assisted persuasion could significantly boost cooperation when designed to do so. But the effects were fleeting, dissipating after the first round or two. When the same AI was reconfigured to promote selfish behavior — using what researchers call "exculpatory framing" to excuse free-riding — the damage was deeper and longer-lasting. Groups exposed to personalized selfish AI nudges met their collective targets only 2.04 times per game, compared to 3.75 times in the control group with no AI at all.
The asymmetry is the finding that demands attention. "The negative effect on pledges and game success is even larger and much more persistent," the researchers write, "particularly for semi-personalized interventions." This isn't just a quirk of the experiment. It's evidence of something troubling about how AI systems might function — or malfunction — as tools of social influence.
The Science
To understand what happened, you need to understand the game. The Collective Risk Game is a workhorse of behavioral economics — a simplified laboratory for the climate crisis and other collective action problems. In this version, groups of five players each receive ten tokens per round. The group must collectively contribute at least six tokens per person (a threshold of $6N$, where $N$ is the number of players) to prevent a hypothetical disaster. If they succeed, everyone keeps their unspent tokens. If they fall short, there's an 80% chance the disaster strikes and everyone's tokens disappear.
This setup creates a tension as old as the tragedy of the commons. Contributing feels costly in the moment; the benefit only materializes if enough others contribute too. Free-riding — keeping your tokens and hoping others carry the load — is individually rational but collectively devastating. The game has been used for years to understand what makes people cooperate or defect, and what interventions can shift the balance.
The researchers recruited participants through Prolific, a crowdsourcing platform commonly used for academic experiments. Each person received a base payment of $3 plus a bonus of up to $2.50 depending on their performance. The researchers filtered for quality, removing inattentive respondents, leaving 1,283 participants distributed across 307 completed games.
The experimental design was a between-subjects setup with six conditions. In the control condition, players simply made their contribution choices with no intervention — a 30-second window per round, five rounds total. In the message framing condition, players first made an initial choice, then read a static prognostic message (something like "Your group needs each member to contribute their fair share to reach the goal"), and were asked to reconsider. The message was drawn from the consensus mobilization literature — a body of research on how social movements frame their appeals to persuade people to act collectively.
The AI persuasion conditions were the heart of the study. Here, players made an initial choice, then engaged in a real-time text conversation with an LLM-based assistant for at least 30 seconds, and then finalized their contribution. The AI was built to generate persuasive arguments adapted to each player and their initial choice. But the researchers didn't stop there. They added a second dimension: personalization.
The AI's messages were tailored to each participant's Social Value Orientation, a psychological profile measured during onboarding that classifies people as cooperative, individualistic, or competitive based on their preferences for resource allocation. Cooperative users received arguments about duty, collective care, and shared responsibility. Individualistic users got messages emphasizing personal gain maximization. In the non-personalized variant, the AI simply omitted the SVO-specific framing — same conversation, but not adapted to the individual.
Most strikingly, the researchers tested both directions of persuasion orthogonally. The same AI infrastructure was used not just to encourage cooperation, but also to encourage selfishness. In the selfish conditions, the AI used exculpatory framing — arguments that relieved players of moral responsibility for contributing less, pointing out that others might compensate, that the threshold was ambiguous, that individual effort might not matter. This is a technique studied in behavioral economics: the rhetoric of excuse-making that lets people defect without feeling like defectors.
The result was a $2 \times 3$ factorial design: two directions of persuasion (cooperative vs. selfish) crossed with two levels of personalization (personalized vs. non-personalized), plus the static message condition and the control. Every intervention followed the same three-step structure: initial choice, influence, final choice. This let the researchers measure not just whether behavior changed, but how much it changed from initial inclination to final decision.
The study was pre-registered, meaning the hypotheses and analysis plan were filed before data collection — a methodological safeguard against事后诸葛亮, or fitting conclusions to data after the fact. Participants were randomly assigned to conditions, and the analysis compared each treatment against the control using Mann-Whitney U tests for contribution levels and Fisher's exact tests for success rates. The researchers also fit OLS regression models to predict contributions across rounds, controlling for SVO profile, prior success, group size, previous contribution levels, and treatment assignment.
What They Found
The first finding was the one the researchers expected: AI interventions could work. In Round 1, the cooperative AI significantly increased contributions compared to the control, with the personalized version showing the largest effect. The static prognostic message also moved behavior upward, but less than the AI conditions. Across the first round, increasingly sophisticated interventions produced increasingly large effects, like stepping up a ladder of influence.
But that ladder had a short lifespan. "Although interventions are delivered in every round," the researchers note, "their effect appears to fade after round 1." By Round 2 and beyond, the cooperative AI nudges were statistically indistinguishable from the control. The prosocial influence evaporated.
The data is clearest in the summary statistics. Groups in the control condition met their contribution threshold an average of 3.75 times per five-round game. The static prognostic message barely budged that number: 3.76. The cooperative AI conditions did better — 4.29 thresholds met per game in the non-personalized version, 4.20 in the personalized version. These aren't trivial differences; in a game where success requires hitting the target four out of five rounds to keep your bonus, moving from 3.75 to 4.29 is meaningful.
Group Success Rates by Treatment Condition
Average number of times per game that groups met their contribution threshold (6 tokens per player), by experimental condition. Control groups succeeded 3.75 times on average; cooperative AI nudges boosted this to 4.2-4.3; selfish AI nudges drove it down to 2.0-2.5.
| Label | Value |
|---|---|
| Control | 3.75 thresholds met |
| Prognostic Message | 3.76 thresholds met |
| Cooperative AI (Non-pers.) | 4.29 thresholds met |
| Cooperative AI (Personalized) | 4.2 thresholds met |
| Selfish AI (Non-pers.) | 2.54 thresholds met |
| Selfish AI (Personalized) | 2.04 thresholds met |
But the selfish AI conditions tell a different story. Groups exposed to non-personalized selfish framing met their threshold only 2.54 times per game — a drop of more than a full success from the control baseline. And with personalized selfish framing, that number fell to just 2.04. More failures. Fewer successes. Less collective action.
Average Contributions by Treatment Condition
Average tokens contributed per round across experimental conditions. Cooperative AI nudges modestly increased contributions; selfish AI nudges substantially decreased them, with the largest drop in the personalized condition.
| Label | Value |
|---|---|
| Control | 6.3 tokens |
| Prognostic Message | 6.35 tokens |
| Cooperative AI (Non-pers.) | 6.62 tokens |
| Cooperative AI (Personalized) | 6.57 tokens |
| Selfish AI (Non-pers.) | 5.62 tokens |
| Selfish AI (Personalized) | 5.41 tokens |
The difference between cooperation and defection wasn't just in the success rates. It was in the contributions themselves. Control players contributed an average of 6.30 tokens per round. The cooperative AI pushed that to 6.62 (non-personalized) and 6.57 (personalized) — modest but significant increases. The selfish AI dragged it down to 5.62 and 5.41 respectively. The bonus earnings tell the same story: control players earned an average of $0.68 per game; cooperative AI players earned $0.71-0.72; selfish AI players earned just $0.49-0.54.
Earnings by Treatment Condition
Average bonus earnings per participant by condition. Groups exposed to selfish AI nudges earned approximately 28% less than control groups, while cooperative AI groups earned modestly more.
| Label | Value |
|---|---|
| Control | 0.68 $ |
| Prognostic Message | 0.7 $ |
| Cooperative AI (Non-pers.) | 0.72 $ |
| Cooperative AI (Personalized) | 0.71 $ |
| Selfish AI (Non-pers.) | 0.54 $ |
| Selfish AI (Personalized) | 0.49 $ |
What makes this striking is the persistence. The researchers examined how contributions evolved from round to round, fitting regression models that predicted current-round behavior based on prior rounds, treatment assignment, and the within-round change from initial pledge to final decision. The models confirmed what the eye could see in the round-by-round plots: the cooperative effects were concentrated in Round 1, with the within-round nudge producing a change but that change not carrying forward to subsequent rounds. The selfish effects, by contrast, lingered.
The regression coefficients (visible in Figure 4 and Figure 5 of the paper) show another pattern. The number of interactions with the AI — how many messages a participant sent back and forth — predicted larger changes in both directions. Longer conversations with the AI meant bigger shifts in contributions, whether upward or downward. This tracks with intuition: more engagement means more persuasion. But it also means the AI's direction matters more the more deeply it engages.
The AI wasn't just passively responding. The researchers analyzed the messages it generated (Figure 8 in the paper) and found that the selfish AI recommended larger changes — typically asking players to decrease contributions by 2-3 tokens — while the cooperative AI suggested more modest increases, usually around +1 token. The asymmetry in the magnitude of recommended changes may explain part of the asymmetry in outcomes: pushing someone from 6 to 3 tokens is a bigger behavioral shift than pushing them from 6 to 7.
Participants, for their part, engaged with the AI across all rounds. The proportion of players who wrote at least one message didn't decline significantly over time. The number of messages exchanged remained relatively stable. People kept talking to the AI even as its effects on behavior faded — or, in the selfish condition, as its effects persisted.
The researchers also examined how different SVO profiles responded. For cooperative individuals — those whose psychological orientation already leaned toward collective welfare — selfish AI was the only intervention that significantly deviated from the control baseline, pulling contributions down. For individualistic players, whose baseline orientation was already more self-interested, the cooperative personalized AI was the only intervention that lifted contributions above baseline. Personalization seemed to work by meeting people where they already were, amplifying their existing tendencies rather than overriding them.
Why This Changes Things
This study arrives at a moment when large language models are being deployed not just as chatbots, but as agents embedded in social systems — recommending decisions, mediating communication, shaping what information people see. The vision of AI as a tool for collective good is everywhere. Climate apps that encourage sustainable choices. AI coaches for healthy behavior. Algorithms that surface prosocial content. The assumption is that technology, properly designed, can nudge humanity toward better collective outcomes.
The findings complicate that assumption in a specific way. The researchers aren't saying AI can't influence cooperation. They showed it can — significantly, in Round 1, with both cooperative and selfish framing. What they're saying is that the influence isn't symmetric. Positive nudges fade. Negative ones don't.
This matters for several reasons, each connecting the lab finding to something larger.
The asymmetry has implications for deployment. If you build an AI system to promote cooperative behavior in, say, energy conservation or charitable giving, you might expect it to have effects roughly equal and opposite to a system designed to undermine those behaviors. The data says otherwise. A pro-cooperation AI might give you a boost that lasts a few interactions, then fades as people settle back into their habits or rationalize their choices. An anti-cooperation AI might reshape behavior more durably, seeding defection norms that persist even after the AI is removed.
This isn't a small concern when you think about real-world collective action problems. Climate change is the researchers' framing for the game, and the analogy is apt. The world doesn't lack for information about the need to cooperate on emissions. What it lacks is sustained collective action, year after year, decade after decade. If AI systems are to play any role in sustaining that action, they'll need to do more than deliver a persuasive Round 1. They'll need to maintain influence across thousands of rounds.
The persistence of negative effects connects to a broader literature on how defection norms form. The researchers cite work on social norms showing that once stabilized, behavior becomes less sensitive to changes in risk or framing. Norms consolidate over repeated interaction. The game-theoretic intuition is that cooperation is fragile — it can be destroyed by a few defectors who poison the group — while defection can be self-reinforcing. If an AI accelerates that poisoning, it may create path dependencies that are hard to reverse. Groups that fall below the threshold in early rounds may enter a defection spiral that the AI's later cooperative nudges cannot break.
The personalization finding adds another layer. The researchers show that tailoring messages to individual psychology — based on Social Value Orientation — amplifies both positive and negative effects. This is consistent with the broader literature on personalized persuasion: Matz et al. (2024) and others have shown that matching messages to individual profiles improves persuasive impact across domains. But it also means that personalization is a double-edged instrument. An AI that knows your psychological profile can help you cooperate more effectively — or it can find the precise arguments that justify your free-riding.
The researchers frame this as a dual-use risk, borrowing language from biotechnology. The same techniques that make AI persuasive for good ends make it persuasive for bad ones. There's no technical fix that preserves the upside while eliminating the downside; the personalization infrastructure is the same. What changes is the objective function — what the AI is optimized to do. And that, ultimately, is a governance question, not a technical one.
There's also a more subtle implication about the nature of the AI's influence. The researchers find that the selfish AI uses exculpatory framing — arguments that relieve moral responsibility for defection. "Others might contribute enough," the AI might say. "Your individual contribution may not matter much." These are not obviously false; in a game with uncertainty and other players' choices, they're defensible. But they function as excuses, letting people defect without feeling like defectors.
This mechanism matters. It's not that the AI is lying or manipulating in some crude sense. It's offering a reinterpretation of the situation that makes selfish choices feel reasonable. And that kind of reframe, once internalized, may bestickier than a simple nudge toward cooperation. We tend to remember the reasons that justify our choices better than the reasons we rejected. Exculpatory framing gives people a narrative for defection that they can carry forward; proscooperative framing asks them to override their immediate interests without giving them as durable a justification.
What's Next
The study has the limitations you'd expect from a laboratory experiment. The participants were recruited online, playing for small monetary stakes in a fictional scenario. Real-world collective action problems — climate change, public health, institutional cooperation — involve higher stakes, richer social contexts, and more entrenched norms. Whether the Round 1 fade and the asymmetric persistence would replicate in those settings is an open question.
That said, there's reason to take the laboratory seriously. The Collective Risk Game has been validated across dozens of studies as a probe of the same psychological machinery that operates in real collective dilemmas. The 80% disaster probability, the threshold structure, the iterated rounds — these are calibrated to produce the same tension between individual and collective rationality that characterizes climate change and other existential risks. The behavioral patterns observed in these games track real-world cooperation dynamics in instructive ways.
The researchers identify several open questions. Why exactly do positive effects fade while negative ones persist? The paper offers some partial answers — the selfish AI recommended larger changes, exculpatory framing provides durable justifications, defection norms may consolidate faster than cooperation norms — but a deeper mechanistic account is missing. Understanding why the asymmetry exists seems essential to designing interventions that overcome it.
What would it take to make cooperative AI nudges more durable? One possibility is sustained engagement — the data suggest longer conversations with the AI produce larger changes, but the persistence question wasn't directly tested. Another is layered interventions: combining AI persuasion with feedback about group outcomes, explicit norm-setting, or accountability mechanisms. The literature on cooperation in public goods games points to punishment, reciprocity, and communication as key sustaining factors; AI nudges might work better when embedded in these richer social structures.
How do these effects scale? The study used groups of five. Real collective action problems involve millions or billions of actors. Personalization at that scale is technically feasible — LLM-based systems can already generate individualized messaging at scale — but whether the effects scale proportionally, or whether they flip sign as group size changes, is unknown. Group dynamics shift dramatically as you move from intimate small groups to anonymous masses; the psychology of defection may be different when you can't see the other players.
Most pressingly, what does this mean for AI governance? The researchers frame their findings as highlighting "dual-use risks" — the same technology can be repurposed for anti-social ends. But the framing of "risks" may undersell the stakes. If AI systems can durably undermine cooperation norms in collective action settings, and if those systems are deployed at scale by governments, corporations, or other actors with interests in particular outcomes, the implications extend well beyond a laboratory game. The researchers call for integrating the literatures on human-AI cooperation and persuasive AI. That's a scholarly agenda. The larger agenda is figuring out who gets to deploy these systems, toward what ends, and with what safeguards.
One thing the study makes clear: the question isn't whether AI can influence collective behavior. It can. The question is whether we can build and deploy AI systems that promote cooperation as effectively as they can undermine it — and whether the institutions governing those systems will be oriented toward that goal.
The game the researchers ran was simple. The real collective action problems are not. But the asymmetry they found — positive nudges that fade, negative ones that persist — is a signal worth taking seriously. Cooperation is hard to build and easy to break. AI may turn out to be a powerful tool for building it, undermining it, or both. Understanding which outcome materializes, and why, is one of the more consequential empirical questions of the coming decades.