MMO Tidbits

Perspectives on MMO Design and Production

  • Categories

  • Archives

  • Meta

  • Copyright Notice

    Contents Copyright 2009-2010 Arnold Hendrick

A Customer Development Strategy for Building Online Games

Posted by Arnold Hendrick on August 10, 2009

The MMO Waterfall to Failure

The traditional process is a sequential “waterfall:” concept > prototype > pre-production > production > beta > live.  In MMO development the full cycle typically takes three to four years. Gameplay assumptions and decisions made during concept and prototype aren’t “field tested” until years later in beta. Even the most gifted designer does not have an infallible crystal ball. Even if they did, limitations and compromises creep in during development. Occasionally everything aligns correctly to allow a great success like World of Warcraft. Far more commonly well-funded teams, led by industry veterans, end in resignations and layoffs. In just the last few years the roster of defeat is long: Vanguard, Tabula Rasa, Age of Conan, Warhammer Online. Each of these long, well-funded efforts failed to meet its financial goals. In each case purges swept aside everyone from famous leaders to the obedient rank-and-file.

I suggest the problem is not in the people but the process. MMO developers may be using better methods for executing their work, such as Scrum, but the overall strategy of game development remains firmly rooted in the 1990s. Below is a diagram of that traditional approach:

 1 Traditional Waterfall MMO Development

The  2-3 year time lag between concept and first customer exposure (beta) is the fatal flaw. Prior to beta the only player feedback comes from within the development team, perhaps some publisher representatives and possibly “friends and family.” Are these people the product’s target audience?

Even worse than the time lag, 80% to 90% of the game’s development budget is consumed before beta test data reaches the development team. The “waterfall” strategy insures that game development becomes a gigantic bet placed years in advance.

A half year ago Eric Ries penned a very insightful article titled “Achieving Failure.” He described how an all-star team achieved near-flawless execution of a product intended to surpass major competitors. The team spent years of time and millions of dollars executing an extremely well-thought-out design. Unfortunately, the designers lacked perfect crystal balls. In addition the marketplace slowly changed during those years. Only at launch did the true scope of their failure manifest. The company madly scrambled to reorganize and exploit what little success it found. In the process the original plan was quietly forgotten and most of the staff laid off. That story parallels the many “waterfalls to failure” in the MMO industry.


Customer Development & Games

Finding a Better Way: This “waterfall to failure” pattern inspired Silicon Valley thinkers to ask “Why wait years to find out if something new will succeed?” From this was born the idea of  “Customer Development Engineering.” This philosophy meshed nicely with the evolving philosophy of continuous development (see “Continuous deployment and continuous learning).

Customer Development as a philosophy is simple. Put a minimally viable product into the customer’s hands as soon as possible.. Monitor customer reaction to determine what succeeds and fails. Adjust and expand the product based on that information. Obviously, an initial vision is needed for creating a “Minimum Viable Product.” Nevertheless, the emphasis is on learning from customers as soon as possible.

Today the Customer Development strategy has born fruit in one corner of the games industry. Companies like Zynga and Playdom use it when building social networking games. Of course that genre is ideal with its rapid development cycles and small teams.


Building an MMO Using Customer Development

How can Customer Development be applied to MMOs? Isn’t it impossible to apply small, quick, lean development to gigantic MMO projects? Actually it is possible, but the traditional multi-stage development process must be restructured into an iterative process with constant customer interaction. Here is a vision of the Customer Development MMO project:

2 Customer Development for MMO Games

Concentrating on the Core Game: The Customer Development game project gives a rudimentary working game to players quickly. Obviously a team must acquire a platform, engine, and/or tools. They also need to conceptualize system design and create some concept art. However, this is directed at one immediate goal: building an initial level of core gameplay. This “initial level” has just enough characters, enemies, terrain, and advancement to provide the minimal core game experience.

For example, a Customer Development style team making Warhammer Online would build an “initial level” for the first 8 levels of play. They would concentrate on one race, three basic character types (melee, ranged and healing), framework quests (title and objective, minimal prose) and the enemy “mobs” to go with them. A team making Age of Conan would build the newbie zone Tortage (levels 1-19) with a similar mix of character types, framework quests and enemies.

Early Live: The very next step brings the core gameplay to players. This is “Early Live.” The entire game infrastructure to support a few hundred to a few thousand players must function. Rudimentary customer support, community management, and financial transactions must exist. The game will actually charge money according to its business model (subscription, microtransaction, etc.).

The goal is real users paying and playing for the core game immediately. Nothing measures player behavior better than their spending. Free games don’t pay salaries or please stockholders. The amount charged in “Early Live” need not be “full freight.” A subscription game might have a “founders club introductory offer” of just $3/month instead of $15/month. A microtransaction store might regularly offer a 75% discount to “Alpha VIPs.” The existence of actual user spending, so later development work can measure comparative gains and losses. The absolute amount earned is secondary. Do not expect Early Live income to support the development team.

The length of time needed to reach Early Live varies. A startup studio with a new staff busy selecting an engine and tools could take a year. A veteran MMO team at a studio with solid tools and an operational infrastructure might need six months. What nobody needs is spending a half year on a 500-page design document with a 500-illustration “creative bible” of concept art and backstory. Instead the development team should focus on actually making the core game.


Development During Early Live

Adjusting the Initial Level: When gamers start playing the initial level, the development team evaluates what works and what falls flat. If the game under development is a cookie-cutter fantasy MMORPG, discovering that nobody wants to spend money on the initial level might be expected, but still must be addressed. Obviously something more is needed. The team might try making the game really easy to learn (like WoW), or have really challenging PvE team play (as Vanguard once promised), or be really violent and sexy (like the topless babes and beheadings in Conan).

Within a month or two a revised version goes to Early Live customers with a spiffy new feature. Player metrics are examined and dollar volumes observed. The customer support team supplements this with forum post summaries and in-game player observations. Metrics might show that players frequently purchase and use the sexy new outfits despite no stat benefits. The metrics might also show that the beheadings rarely occur. The team can investigate to see if game mechanics make beheadings too hard to achieve, or players are simply min-maxing their combat moves without regard for the “gore level” in the graphical results. Conversely, the team might discover that nobody bothers getting the sexy outfits but customers are clearly working hard to behead their opponents. Success depends on finding and fostering each game’s specific audience.

Building the End-Game: Once the initial level is deemed successful, the next step is to “bracket” the game by constructing an end-game level. What will players do once they reach the highest level, get all the skills and/or get the best weapons? How well can the game hold these players? This problem must be solved. If it is not, players will leave in droves once they reach the game’s level maximum. One solution might be stretching out leveling so that nobody will achieve max level, as in EVE Online. Another solution might be guild vs guild PvP territorial play (as in Lineage, Lineage II and the recently deceased Shadowbane). A third might be RvR play (as pioneered by Dark Age of Camelot).

Testing this endgame in Early Live is critical. Does it maintain player interest for an acceptable period? If not, the development team must experiment with changes or alternatives. Endgames are notoriously tricky to build in MMOs.

Building the “middle levels” must occur after the initial level and endgame are validated through customer testing. How can anyone create a player growth path until the start and end points are known? Furthermore, once you know the endpoints and have preliminary measurements of audience type, customer engagement and churn, the business types can use “return on investment” and “time to market” calculations to determine how much time and money is available for building those “middle levels.” Armed with this information, the development team can move forward with confidence.

Systems Lockdown: Early in the development of the additional “middle” levels a “systems lockdown” must occur. This is the point where key gameplay decisions, progression curves and data ranges become fixed. All future work must conform to the lockdown standard, or else invoke a formal “change control” process to judge the cost-benefit of a change. Lockdown facilitates speedy content creation by giving designers, artists and programmers a framework for future work.

Systems lockdown also allows the development group to address “technical debt.” After lockdown the team can reasonably estimate the benefits of refactoring code, documenting design data and algorithms, writing up pipeline procedures, settling on coding practices, etc.

Curiously, systems lockdown is not specifically identified in the traditional “waterfall” game development process. While it should occur by the end of pre-production, failure to lockdown is a common source of failure (see “An Intervention”).

Platform Improvements: Throughout development weakness emerge in the engine, tools, pipelines, build speed and processes. Instead of having a separate “tools team” trying to either catch up with development or guess in advance what development needs, let the development teams judge when a sidetrack into tools improvement pays off. For example, a level-building team learns from experience that current tools mean four weeks per region layout. The programmers on the team estimate that two weeks work improving the tool will save everyone one week of time per region. Therefore, if three or more regions are still needed, the sidetrack into tool improvement will save time.


The Benefits of Early Live

Early Live will be controversial. Publishers and development teams are used to making their own decisions about a game. Designers and producers have strong personal opinions about what customers really want, passionately held beliefs concerning what is “best.”  These beliefs are still valuable – they fuel the initial concept for any game. The mistake our industry makes is insufficient testing and refinement with actual gaming customers.

Train As You Fight: A generation ago the US Army adopted the slogan “Train As You Fight.” The goal was simple: all training should be applicable to real combat, and imitate combat conditions as much as possible. Soldiers should use the same equipment they would have in real battle, receive orders the same way, move as they would in combat, over terrain and in climates similar to real-world battlefields. Soldiers train for Iraq in the desert, for Afghanistan in the mountains. In recent wars the US Army learned once again that sending troops into combat without these benefits resulted in less successful missions and higher casualties.

An MMO’s Goal is Live: The goal of building an MMO is to run it “live,” just like the ultimate purpose of having an army is to fight a war. Early Live insures that all necessary functionality is present. It highlights what is still missing, and quickly demonstrates what is overkill.

Making a Functional Product: An MMO needs solid code that doesn’t regularly crash client or server software. It needs a comm infrastructure that routes data quickly and efficiently between players and servers. It needs a load-balancing system that prevents too many players from piling onto one server and a design that discourages players from clumping up in the same spot. It needs a regular deployment cycle with a rollback option for fatal bugs. It needs an offsite backup system to minimize data loss if a flood/hurricane/tornado/earthquake destroys the data center. It needs a metrics system that tracks what players are really doing in the game. It needs a customer service to deal with customer problems and community management to mediate between developers and gamers. The list goes on and on. It is just as easy to over-plan and over-support these requirements as to forget them or under-provide for them. The only way to achieve truly cost-efficient operation is through experience.

Find & Fix Early: An Early Live philosophy places high demands on code reliability. Subtle memory leaks and nasty bugs become painfully evident when a thousand players suffer client lockups and crashes. If the comm infrastructure cannot scale the team finds out immediately. If designers accidentally create a “nexus point” of excessive player concentrations, stalls and slowdowns reveal it. If artists overload the graphic engine everyone sees it.

In the past it was easy for members of a development team to ignore a problem. They could assume that “someone else” would fix it later… during beta, after launch, whenever. The Early Live philosophy prevents that. Now the development team is constantly balancing the need to fix problems with the need to push forward development. They constantly decides what’s important enough to fix now.

Integrating Development with CS & CM: Early Live brings customer service (CS) and community management (CM) into the process of development much sooner. Properly handling customer problems and building a positive customer community is essential to an MMO. A development team needs to work with the CS and CM groups to guide and prioritize future development, just as CS and CM rely on the development team for their product and tools.

Financial Snapshots: Early Live provides financial data on actual operational costs, from bandwidth usage to operational staffing. It takes time to find the delicate balance between maximum earning power and maximum customer support. More than once A game development team “assumed” impossibly large amounts of customer support. Early Live reveals these assumptions and helps prevent “fatal financial flaws.”

Marketing Benefits: Early Live is a marketing department’s dream. Instead of guessing about potential audiences they can examine information about hundreds to thousands of players who are paying customers! Initial growth and churn rates can be extrapolated to better predict potential income levels. Early Live should prevent marketers from estimating a million subscribers with 5% monthly churn before launch, only to discover three months after launch that over half the purchasers dropped their subscriptions!


Yes But…

Questions arise about how applicable the Customer Development process is in certain situations. Here are some questions (Q) and answers (A).

Sidetracked by Live   Q:  What if a horde of players flood into Early Live, forcing the development team to concentrate on polishing the first level or two? Won’t that prevent forward progress? What if everything falls down in a heap because we don’t have enough hardware, the software isn’t robust, or the support staff is too small?

A: First, if operational resources are limited, you can meter players coming into Early Live via keys; as players churn out you release additional keys. Second, if your software and staff can’t even handle one server’s worth of players, you need to work on the operational side of your business. The longer you postpone making “massive” functional, the riskier the whole project becomes.

The Minimum Viable Product  Q:  Customer Development talks about a “minimum viable product” (MVP). Our MVP is a fantasy MMORPG with at least 20 races, 30 character types, 50 regions, 500 monsters, and must support PvE, PvP, guild vs guild and RvR. If we don’t have all this we’ll fail.

A: The Roman Empire wasn’t conquered in a day. It began small and grew, piece by piece. A successful MMO must do the same. Within that vast shopping list of features you need to identify the core gameplay that will attract and keep customers playing your game. You are being distracted by grandiose visions with lots of “chrome.” Chrome in games isn’t necessarily good. It can create complicated interfaces and confusing options that become obstacles to enjoyment. Blizzard’s greatest successes were simpler and easier than other games of their day. Diablo was easier to play than any previous fantasy RPG. WoW was easier to get into than any previous MMORPG. It is important to “find the fun” early and mitigate risks by constantly evaluating the ability of gamers to learn, play and enjoy your product.

Fear of Competition  Q:  If we reveal our product in Early Live, our competitors will steal the idea and benefit from our work. Even if they don’t, we’ll have competition far sooner than if we kept everything secret.

A: Customer Development brings the product to customers within 6-12 months. Even if your competitors follow in your footsteps with a process just as efficient, they’ll be a half year to a year behind. More importantly, you gain invaluable knowledge about who the customers are and how to sell to them. Use that advantage to build customer loyalty. Your would-be competitors are just imitating your success. You know the reasons for that success. Exploit your knowledge to insure competitors are left with crumbs from the table. Best of all, if your competitors aren’t using Customer Development they’ll spend years developing something that’s outdated the day it ships.

Licenses & Product Secrecy  Q:  Our game is linked to a movie (or TV or book) license that prohibits us from revealing characters or content in advance. We can’t go live early.

A: Yes you can, with stand-ins for the confidential material. Replace specific licensed names with generic equivalents. For example, replace “Enterprise Class Starship” with “Galactic Exploration Cruiser”, or rename “Mordor” as “The Dark Wastes.” Certain object may need alternate textures and/or concealing geometry during Early Live. You can still test all the play mechanics of the final game. Your development team is forced to pay attention to gameplay, which greatly improves the chance that your licensed title will be one of the rare few that’s actually fun to play.

Customer Exhaustion  Q:  Enticing players to come back to a game is much harder than acquiring new ones. Early Live will “poison the well” by giving a weak, incomplete game to our best customers.

A: If the potential customer pool is so small that one server’s worth of players represents your key audience, it might be wise to pick a different audience! However, if that audience is wealthy enough to support your product, invest heavily community management. Make them fell like well-informed, heavily-involved partners in the development process. When you reach “launch” you can promote the game as Version 2.0. Alternately you could make cosmetic changes and attach a new title for a “fresh” PR campaign.


Scrum and Customer Development

At its core Scrum is a day-to-day, week-to-week development methodology. Customer Development is a long-term strategic philosophy. Nevertheless, there are many parallels. Customer Development is “Scrum writ large.”

Scrum sprints (iterations) concentrate on delivering a shippable increment of functionality – customer development concentrates on delivering increments of a playable game.

Scrum requires review and process examination after each sprint – customer development uses customer metrics and buying patterns to guide future work. 

Scrum expects plans will be adjusted from sprint to sprint – customer development expects that both designs and gameplay will change as the team learns about player preferences throughout Early Live.

Given a choice, I would use Scrum exclusively in a Customer Development project. Scrum principles such as backlogs feeding multiple teams working in 2-4 week sprints is a perfect way to monitor and control game creation before and during Early Live. It allows you to balance the needs of live operation with the needs of new development through frequent reprioritization. Scrum “releases” match up perfectly with periodic updates to Early Live players. A philosophy of Customer Development ideally matches the Scrum process.

3 Responses to “A Customer Development Strategy for Building Online Games”

  1. You make some excellent points. The huge amounts of time and money invested in developing games without any reliable way to estimate likely success or failure is perhaps the biggest business risk in gaming.

    Some time ago the elaborate preproduction model Hollywood uses for films was adopted for games. Films don’t typically have to create the technology needed to complete them. If the technology can’t be made to work or where the infrastructure to support it is impractical to put in place, all that preproduction is pretty much wasted. A side issue is the difficulty in predicting how long it will take to develop needed technology. This is one area where Hollywood is not a good model for games.

    Iterative development is a much better way to run projects. Scrum, which delivers demonstrably working components at the end of each sprint, is a more practical way to go. I am speaking here of actual Scrum and not the “Scrum-like” bastardizations which skip the requirement of delivering something that works. It also, as you note, allows for quick responses to changing market conditions.

    In some instances it would be better to solve the technical challenges before making large investments in art, design, stories, and other nontechnical assets.

    This next gets away from Project Management, so it is not something for which I can claim any particular expertise.

    The problem is with the financial model. I don’t see players paying to play a game that is in development and I don’t see players who have been allowed to play for free being very happy when they are expected to start paying.

    Following up on one of your suggestions, it should be possible to release the beginning portion of the game in a reduced time frame. Players could pay a small amount for that. New players could always start with the limited game play at the low starting price. As more is added to the game, players could access that for an additional fee that would be kept competitive with other games. Players could be given free access to new areas for a limited time, or as a reward to loyalty, etc. As stated, this is not my area.

    I think you are on the right track as this is a challenge the games industry cannot afford to leave unmet.

  2. […] A Customer Development Strategy for Building Online Games […]

  3. skyydragonn said

    Interestingly enough if you take a good hard look at Blizzards World of Warcraft you can draw more parallels to the Customer development strategy than you can to the waterfall strategey outlined first. When it launched it was a fairly basic fantasy MMORPG with a solid license and fanbase. over the next 5 years the core conecepts haven’t changed much but many many additional features/systems have been added.

Sorry, the comment form is closed at this time.