Punching Snakes
Don't Do It
The Results of Riot Games’s Priming Experiments
Last week I published a piece on Ars Technica about the player behavior experiments being conducted at Riot Games. Long story short, Riot decided that just banning players for bad behavior wasn’t good enough, so they have tried to include the entire League of Legends community in policing and improving toxic behavior, defined as basically all the horrible things we half-jokingly associate with Call of Duty online play and open Xbox Live chat rooms in general. Riot Games has an entire player behavior team led by a trio of PhDs, and their efforts are demonstrably successful.
All of that was interesting to me in its own right when I heard it at the GDC panel where their lead player behavior guru, Jeffrey Lin, shared some results of the experiments at Riot, but what made me jump out of my seat after the panel session ended, walk up to the podium to introduce myself, and ask Lin for an interview, was the final experiment he talked about which involved the psychological principle of priming.
Priming is giving someone a stimulus to affect their reaction to the following stimulus. I talk to you about old people for an hour and then you leave the room walking slower than you otherwise might have, probably because you’re thinking about old people, and old people move slowly.
Riot Games was kind enough to let myself and Ars Technica use some slides from their GDC presentation. That material is now available in the GDC Vault for anyone who attended the conference this year to view, but that’s not most of you, and when I encounter a GDC panel with excellent visual references I like to share them, like I did with last year’s piece over on G4 about level design in Gears of War 3, which I saw at a GDC 2012 panel.
I want to share with you some of the slides we couldn’t use for the feature, because I think these charts nail home just how cool the priming experiment is.
To summarize again from the Ars piece, Riot Games used game tips as their priming text, displayed them in either red, blue, or white fonts, and placed them in loading screens, in-game, or both.
A priming message about not harassing teammates displayed in red on the loading screen…
…had better effects on decreasing toxic behavior than when the same message was displayed in white.
A priming message about cooperating with other players was effective at decreasing toxic behavior when presented in blue…
…but ineffective when it was presented in red.
There was one quote from the interview with Jeffrey Lin and T. Carl Kwoh, a producer from Riot Games,  that we couldn’t use for the feature and which I think bears sharing. Lin said:
“For a long time now psychologists have done studies on priming and the color effects there. And they’ve noticed that in the West, there are certain associations with red already.
In some cases, for example, there’s a story of college students going to take an exam but the font of the college exam was in red, and they performed worse than another course that had different colored fonts for the exam. There’s this weird association with red and failure and other studies have shown that red has links to error avoidance behaviors. So why not test red?
But red is also interesting if you consider cross-cultural differences. For example, red is a very lucky color in the East. Or [in some] places in Europe, red is not the grading color for exams and is not the color linked to failure. So what we’re interested in seeing now is if we do red what are the effects across all these different countries.”
That gives a layman like me an idea as to the factors that could lead to variable results in this priming experiment, but this last slide I want to share with you is both fascinating and hilarious. We mentioned these results in the Ars piece, but again, seeing the chart really nails the point home.
Asking players who will be the most sportsmanlike during the match in a red font produced dramatic results in precisely the wrong direction.
Now consider that there were 217 different combinations of specific tip message, color, and location(s) in this experiment and you understand why they needed PhDs on the player behavior team to figure out what all this data means.
The idea that Riot Games could have such measurable and clear effects on player behavior is both interesting and scary. I have no doubt Riot is using these priming messages solely for the benefit of the entire League of Legends community, but imagine a video game company that decides to try this sort of thing to drive in-app purchases, for example?
Hell, maybe someone is already doing that, but I don’t want to research the question and find out. It’s like when Frank in They Live didn’t want to put the glasses on.
The problem with psychology is that the entire “science” crumbles when you apply it to someone with a spine, unyielding will power, pride, and ambition. And people like that are far more common than modern media and egalitarian propaganda would have you believe.
These experiments of manipulating players like cattle may work to a certain degree indeed, and they’re certainly of interest to the company, which has its interests rooted in money, but the whole idea is so cynical at its core that I find it difficult to grasp sincere interest in it on an individual level.
What kind of stereotypical cackling villain do you have to be to find the idea of manipulation on a mass scale appealing?
Although I’m sure there’s a piece of psychology to “explain” my view on the matter as well.
“What kind of stereotypical cackling villain do you have to be to find the idea of manipulation on a mass scale appealing?”
Manipulation on a mass scale? You mean like advertisement? Which is basically trying to manipulate people into buying / using your product?
The fact is that (mass) manipulation is used a lot and you come across it most days without even realizing it. For example, when you go into a supermarket, the cheapest brands might be on the hard to reach places on the bottom, so less people will buy them (compared to the more expensive brands). When you enter the shop, the fruit and vegetables might be in the beginning of the shop, to make it look more tempting to people outside. When you’re ready to pay, there might be a place to buy some chewing gum, candy bars or whatever near the exit. Which you didn’t intend to buy when you came to the shop, but they’re still trying to get you to regardless. Often, supermarkets tend to rearrange products, so you can’t just walk your standard route buying the things you always buy. Instead, you’re forced to see other things as well, tempting you into buying them as well.
And so on.
It doesn’t require a super villain to be interested in such things. It simply takes a company that wants to make money.
I agree with Shadow Master. Reading about the effects that the font color has on someone’s behavior made me immediately think of how weak minded the person was. Maybe these things will be able to influence the behavior to meek, nerdy individuals like the community of League Of Legends, but I know myself and my friends just laugh at these messages. They are complete jokes.
Psychologists have no business working at player enforcement because they have no idea what they are doing. Ask any psychopath what they think of a prison psychologist and you will hear tales about just how easily fooled and incompetent they are.
All the player behavior team is is an annoyance to the community. The fact of the matter is they, as well as their beloved Tribunal, are failures. They can point to all these statistics about how toxic behavior is being decreased by all these things they are implementing but it isn’t reflective of reality. It’s like when New York City reported record lows in crimes, which was true, but then the only reason for it was that they stopped counting certain things as crimes. The reality of the matter is there is more toxic behavior than ever on League of Legends and I think most gamers who play it will tell you this. I played two games with a friend of mine last night and we ran into 10 trolls. That’s 10 trolls out of 16 players we met. Mr. Lin has tweeted that only 1% of the community is Toxic, so seeing 63% of the players we meet being trolls should never happen.