Picture yourself merging into a single lane during rush hour. The polite move is to take turns, letting one car in at a time. Now imagine the car beside you has no driver, just an empty seat and a computer at the wheel. Do you still wait your turn, or do you nose ahead, figuring the machine will yield?
That everyday moment sits at the heart of a study published in the Journal of Economic Psychology. As algorithms increasingly negotiate contracts, drive cars, and stand in for human owners, researchers wanted to know whether people treat these artificial agents the same way they treat other people, and if not, why the behavior changes.
The question behind the experiment
Iván Barreda-Tarrazona, Marina Pavan, and their colleagues at Jaume I University in Spain built their investigation around a classic setup called the Prisoner’s Dilemma. In this game, two players each choose to either cooperate or defect. If both cooperate, they each earn a solid payoff. If both defect, they each earn less. But if one defects while the other cooperates, the defector walks away with the biggest prize while the cooperator gets nothing. The tension is built in: the choice that helps you most in the short term hurts the pair overall.
Earlier work had found that people cooperate less with machines than with other humans in a one-time version of this game. But the authors noticed a gap. In those studies, the machine had no human stakeholder behind it, so defecting on the algorithm cost no real person anything. In real life, an artificial agent usually represents someone. The self-driving car carries a passenger. Walmart’s negotiation bot, which the authors cite as an example, bargains with suppliers on behalf of the company.
So the team designed their experiment around a central question: do people cooperate with an artificial agent acting on behalf of a real human the same way they would with that human directly? And if there is a difference, what drives it?
How the study worked
The researchers ran the experiment in a physical laboratory in Castellón, Spain, in April 2022, before ChatGPT’s public release, so participants had little exposure to modern AI tools. In total, 346 people completed the study, recruited from various degree programs. Students who had studied game theory or the Prisoner’s Dilemma in economics courses were excluded.
Participants were split into two conditions. In the “Humans” treatment, 178 people played the games against other people in the room. In the “AA” treatment, participants were told they would play against an artificial agent. Half of these were active decision-makers; the other half were passive “receivers” who sat back while an artificial agent played on their behalf and collected whatever payoff it earned. In other words, defecting on the machine really did cost a real person money.
The artificial agent itself was built to imitate people. Programmed in Python, it calculated its cooperation odds using the actual decisions made by the 178 human participants, drawing on up to two previous rounds of play. Participants were told the agent made probabilistic choices based on how humans had behaved in comparable situations. This design let the researchers hold one thing constant: the uncertainty of facing an unpredictable partner. The only thing that changed between treatments was whether the partner was labeled human or machine.
Everyone played four sets of games. The first was a series of ten one-shot games, each against a fresh partner. The next three were repeated games, each lasting ten rounds against the same partner. Before every round, participants reported what they expected their partner to do. After every decision, they noted whether reason or a specific emotion drove their choice.
What the data showed
In the one-shot games, the identity of the partner made no measurable difference. People cooperated at similar rates whether they faced a human or a machine, and cooperation started low and dropped toward zero for both. This contrasts with the earlier research the authors were building on.
The repeated games told a different story. When people played the same partner across ten rounds, they cooperated less with the artificial agent than with humans, and the gap grew after the first repeated game. In the second and third repeated tasks, the likelihood of cooperation was roughly 15 percentage points lower with a machine. Among human partners, cooperation tended to build and hold high until the final rounds, when it collapsed, a pattern the researchers call an “end-of-the-world” effect. Against the artificial agent, cooperation started lower and drifted down.
The researchers also spotted a learning gap. People playing other humans grew more cooperative from the first repeated game to the last, suggesting they learned that early cooperation paid off. Those playing the machine showed no such improvement across games.
Not distrust, but opportunity
The natural guess is that people simply trusted the machine less. The beliefs data pointed the other way. When facing a new partner, participants generally expected the same level of cooperation from a machine as from a human. In several cases, they actually expected the artificial agent to cooperate more than a human would.
This led the authors to a behavior they label “exploiting the partner”: choosing to defect while expecting the partner to cooperate. In effect, taking advantage of someone you think is playing nice. Across all the games, people were 16 to 21 percentage points more likely to exploit the artificial agent than a human partner. The pattern held even though the machine’s winnings went to a real person sitting in the same room.
“Our results suggest that people do not trust artificial agents less but are more inclined to exploit them when they anticipate cooperation,” the authors write.
The self-reported motives added another layer. Participants facing the machine cited “reason” as their main driver more often than those facing humans (88.8 percent versus 83 percent), and reported emotions like empathy less often. Greed was the most common negative motive in the machine condition.
Possible explanations and limits
The researchers offer several interpretations, presented as possibilities rather than settled conclusions. Because the machine made probabilistic choices, participants may have underestimated how likely it was to punish a defection. They may also have felt entitled to a larger share, since the passive receiver had done no work. And they may have viewed each new instance of the agent as less capable than a human of learning from past encounters, which would explain why cooperation never climbed across games.
A few caveats are worth keeping in mind. The study used university students in a single lab, and it measured emotions through self-reports rather than physiological tools. The design always paired the machine with a human beneficiary, so the authors note they could not fully separate how people would treat a machine acting purely on its own. They also point out that the experiment predated the wave of familiarity with AI that followed ChatGPT, and that growing comfort with these tools could shift behavior in either direction.
For organizations handing negotiation, scheduling, or bargaining over to algorithms, the findings offer a note of caution. Even when people know a machine represents a real person and behaves just like a human would, they appear more willing to take advantage of it when they sense an opening. The label on the counterpart, it seems, can change how fair a deal people are willing to strike.
