I worked in freemium games for 4 years, first at Zynga, then at Red Hot Labs. In my time at Zynga, I made a ton of friends and got to know a lot of people, many of them hardcore gamers just like me. We had weekly board game nights, we had an internal starcraft tournament, and we would often chat about the newest PC and console games during our lunch breaks. Even the people who weren’t hardcore gamers were still good people, and I never actually met anyone that fit the greedy fatcat stereotype.
Meanwhile, we were making games that were almost universally reviled in online communities of hardcore gamers, and we caused some nasty addiction related problems for some of the people that played our games.
The mental conflict between my professional life and the views of just about everything I saw on Reddit and when playing games bothered me for a long time. My thoughts about the matter didn't really crystallize until long after I’d left Zynga, but I think I now understand why the games we were making wound up so bad - as well as what the developers of future games can do to try to avoid going down the same path.
Some Background
The first thing that gamers need to realize is that freemium social and mobile games are frequently delivered as a “service”. This really means they’re more like a TV series than a movie, where the initial game sets the tone of the entire series, but the development team delivers new gameplay and features every week. This means that the game is constantly evolving and changing as new features are added.
Zynga’s primary method of judging the success of these added features (and thus deciding what to add in subsequent weeks) was to look at how player statistics change when they interact with the feature, and to run A/B tests where some players would see slightly different features than others.
At first, there were three key metrics that Zynga judged features on:
1) Retention: how long do people play the game before they get annoyed / tired / bored and leave
2) Virality: how good are players at bringing in other people to play the game. Measured by how many FB posts the player would make, and how many people would click on those posts.
3) Monetization: how much does the player spend.
In the beginning...
When I first joined Zynga the Fall of 2009, I was put on the Farmville team as a programmer. At the time, new game features were things like “support arches the player can walk through”, “make it so that the mouse picking on buildings is pixel perfect”, or “lets add a new animal”.
A feature that moved the three key metrics upwards was a successful feature - and at first they were fairly innocent. The ability to send gifts to other players was a huge virality hit. Significantly improving game performance drove up retention and monetization. Getting rid of an annoying bug in the tutorial segment significantly increased first day retention. From the perspective of designing a normal game improvement, then judging that improvement, all of these metrics were great at distinguishing a high value improvement from a change the players didn't even notice.
Of course, that means that the people who designed features that moved those metrics a lot gained significant say in future feature designs (and bonuses / raises).
Over time, features became more and more optimized to drive those key metrics - and disregarded all other considerations. Fun, ethics, long term player retention (which was very difficult to measure), and everything else fell by the wayside. People still talked about it, of course, but we had improving those key metrics down to a science. Trying to argue “but this way will be less annoying to the players” when someone can look at the data and say “well the metrics are 10% better with this” just didn't work.
We weren't completely oblivious to this, even as it was happening. We made a number of efforts to start tracking how much the players were actually enjoying the game, including a big company push to track and optimize “Net Promoter Score” (NPS). Unfortunately, we still had to keep the money flowing in to support the company, which meant the weekly features to prop up the 3 key metrics still had to keep going out. I don’t know if Zynga ever managed to actually increase NPS or if they eventually gave up on them, but I do know even targeting metrics like NPS isn't a sure track to a good game - eventually someone will figure out how to easily raise NPS at the cost of other parts of the gaming experience, and it will be downhill from there.
Post Zynga
A lot of developers have left Zynga to other companies, and I’ve seen a lot of other gaming companies that have been heavily influenced by Zynga’s metrics first strategy. From a business and investor viewpoint, that strategy has been tremendously profitable, and so there’s huge pressure to adopt it. From a hardcore gamer viewpoint, that strategy is a nightmare that’s ruining the mobile gaming market, and bleeding out into other markets.
In most Zynga spinoffs I've heard about, the three key metrics are what are tracked - not any of the harder but potentially better metrics that Zynga tried to adopt later.
The feature sets we used at Zynga are also spread throughout the entire industry now, often poorly implemented to be even worse experiences than they were originally. The big hubbub about the new Dragon Keeper game seemed to be largely caused by the energy mechanic and the plant/harvest mechanics being forced into the game, with prices set abusively high.
So what can people do?
I really think its time for developers to start standing up and pointing out the problems with purely metrics driven development. I’d love to hear other people’s thoughts on problems and solutions, cause I don’t think what I’ve got so far will be enough to convince the investors and entrepreneurs that really need to be convinced for change to happen.
The problems I wish I could go back 3 years and point out would include
1) Features need to be designed for fun, then optimized for metrics. Designing for metrics leads to gaming the system in order to drive up those metrics, and leads to sacrificing everything else.
2) The common feature set is over optimized to the key metrics - while they may move these metrics positively, they likely move most other metrics negatively. Many of the features didn't consider player experience at all, and adding them to a game can significantly detract from the appeal of the game.
3) Optimizing for these metrics can be abusive and amoral. Sure you can get a player to pay $1000 for your game and spend 4 hours a day playing it, but should you? Whether by accident or design, many of these features exploit psychological tricks that higher risk players are vulnerable to. This leads to addiction and sacrificing real life needs to pay for the game.
4) Long term retention analysis is too difficult to be practical, and short term retention doesn't really notice when you add straw to the camel’s back.
5) Support the games that aren't using this strategy. Flappy Bird was a random success, which I doubt can be replicated easily. However, it showed quite clearly that the standard freemium features aren't needed for a game to become popular. Pixel Dungeon is a great game that doesn't bother with any of the freemium mechanics, while Jet Pack Joyride does a great job of maintaining the fun of the game despite having several of the standard feature set.
6) If you work at a place that’s drifting down this path, speak up. Talk to the analysts and designers about what their features are actually doing to the enjoyability of the game, and point out to them what can happen if people get too addicted. I succeeded in preventing a real money gambling game from gaining social addiction mechanics just by pointing out how dangerous it was (although that eventually made Red Hot Labs stop making games at all, which meant I was out of a job…)
Going Forward
I’m now getting away from the freemium market altogether to focus on making a fun game (which you can learn more about here). I fully expect that part of the mobile games market will still be miserable and abusive for years to come, since it’s seen as the easy route to stupidly large amounts of money. It will probably never go away completely, but I’m hopeful that the modern 3-key-metrics strategy will fall away from the mainstream relatively quickly - as a combination of players getting sick of it, and developers either abandoning or refining the strategy to focus on fun and quality.
