Earlier this year, GI.Biz wrote about the era of the perennial game and reported that the lifespan and popularity trajectory of a game is longer than ever. Utilising data from NewZoo’s PC & Console Gaming Report, they surmised that ‘the average years elapsed since launch of the top ten most-played games in the world [judged by monthly active user figures] was more than seven years’. Titles including Apex Legends (released in 2019), Minecraft (released in 2011), GTA 5 (released in 2013) and Call of Duty: Modern Warfare II (released in 2022) all dominated the top 10. So what is it about these titles that continues to draw players month after month, year after year? It’s that these games are able to understand what their players want, where they can add new dynamics or challenges, and continually improve and fix issues. They do this by bringing in Live Ops teams and Data Experts to work behind the scenes to crunch numbers, predict trends and analyse player behaviour long after the game’s initial launch window.

But how does this data work impact the lifespan of a video game and its success, what impact do gamers have on their favourite games, what data are game developers collecting and – more importantly – what are they doing with it?

Two people who know data like the back of their hands are Laine Baker and Sean Quinn. Together they have over 40 years of data, product and project management experience in games, technology and finance sectors, and currently head up the Live Ops departments at Sumo Sheffield and Sumo Leamington, respectively. Here, they break down some of the ways game developers can acquire and utilise player data, best practices for implementing live ops into a development team and the many ways players have huge, long-lasting impact on the success of a game without ever realising.

WHAT IS DATA IN GAMES?

Data [in relation to live service or GaaS games] refers to collected information about how players get exposed to a game, how they play and interact with it and – sometimes – how they spend within a game.

In 2024, you’ll be hard-pressed to find a game – live service or box release – that doesn’t utilise data in some way, with engines like Unreal and Unity both offering built-in analytics tools, and a huge variety of off-the-shelf tools and technology available. With more access comes more understanding, and developers, publishers and marketers are able to gain a deeper insight into the playstyles, habits, needs and challenges of their players to continue to improve and evolve their games.

But it hasn’t always been this way: data in games has gone through a journey; from a scattered start to arrive at the common practice we see today. Prior to the mid-2000s, Data in games wasn’t something widely explored by developers, with teams taking the anecdotal observations of the team and immediate players around them to try and understand what was or wasn’t working. This approach still provided useful feedback, but often the experiences of these groups didn’t give an accurate overview of how millions of players were interacting with the game, or even what those numbers looked like. Luckily, the growing popularity of MMORPGs, free-to-play games and mobile games was right around the corner, and the use of data was accelerated alongside it – with experts began to be brought in (often from other, more established industries like tech or finance) to begin diving into the data details of a game.

Fast forward roughly 20 years, and Data Experts have evolved to become Live Ops teams – who specialise in the ongoing management and enhancement of a live service game post-launch. Over two decades, they’ve worked to utilise tools and technology, hone their craft and grow their presence, and are now able to more efficiently and effectively work with their players to create better, more sustainable and enduring games.

[Data taken from Stampede: Racing Royale during its time in the Xbox Insider Program. Data teams are able to see and understand how often certain ‘events’ are happening within the game – from how many times people land at the final screen, to how often they acquire and use a power up.]

GETTING STARTED | TELEMETRY EVENTS AND METRICS

When we say data, most people think of personal data such as demographics (age, gender, territory etc.) … but players can rest easy in the knowledge that Live Ops teams aren’t usually interested in their personal lives or identifiers. Instead, data teams are looking to collect behaviour analytics to understand what, how, why and when players are engaging with games: they’re called telemetry events.

There are several types of telemetry events that a game might collect – including:

Acquisition: How are players first finding the game?

Technical Performance: How is the game running for its players?

Interface Usage: How are players interacting with the UI – where are they navigating to, when and how long is it taking them?

Setting Usage: What settings are players using?

In-Game Play or Behavioural Data: How are players playing?

Monetisation Data: Are players purchasing items, or how are they interacting with a storefront?

Player Sentiment (both in and out of game): Using both in-game data and external (social media, community spaces etc), how are players feeling about the game?

These telemetry events can inform a huge range of metrics that Data and Development teams can use to measure success, areas that need improvement or any common trends. Examples of these metrics include:

Retention: After installing the game, how long do players interact with it? How often do they come back between sessions and how long are they?

Heat Maps: Where are players getting stuck on a level? Are there particular enemies or obstacles that are causing them significant issues? How often are they finding and using power-ups? What line of sight do they have when using weapons or fighting?

Peak Concurrent / Concurrent Users (PCU / CCU): How many players are playing at any moment? What’s the maximum number of players that were active at one time and what caused this peak?

Using these metrics, data teams are able to measure success, suggest changes, improvements, fixes or even out-of-game additions to create a smoother experience for players. This can then be relayed to Developers for them to gain greater understanding of how players play the game, the content or gameplay tweaks needed, how players react to customisation of the experience and more.

[Data taken from Stampede: Racing Royale during its time in the Xbox Insider Program. The devil’s in the dashboard. The above shows how the Sumo Leamington team receive and read information about Stampede: Racing Royale. Having data which is easy to visually see means identifying trends is simpler and more efficient.]

TECH, TOOLS AND GROUND RULES | THE DATA PROCESS

The key starting point for any data pro looking to take their game to the next level is to figure out what you want to know. A sharp, concise and focussed pipeline of enquiry is a great way to produce high-quality data that can make a real impact on your game, and – though it might be tempting – going too broad and collecting data on every mouse click or UI interaction can significantly impact not just running costs, but the overall performance of the game.

As you delve into more complex game design and behavioural questions, you can begin to define a game-specific layout that will help you understand and answer them. For kart-racing chaos game, Stampede: Racing Royale, the data team at Sumo Leamington use a combination of services and technology to capture, aggregate and analyse the data – the data pulled here can then be digested by an expert team of Data Engineers and Data Scientists. When playing a game – using Stampede: Racing Royale as an example – there’s a process that happens to record your activity:

A player picks up the game: All players in Stampede: Racing Royale play on real-time servers which collate the actions of a player in relation to where they are in the game. This data is then collected per player and sent to a database.

The player engages with the game: Each in-game event triggered by a player – like logging in, choosing a kart, using a power-up etc – is logged on a column and tied together by an anonymised player ID, and a timestamp to tell the data team not only at what time these occurred, but how long each took.

Player activity is logged and grouped: This data is then formatted, cleansed and stored, and is then grouped by whatever definition the data team require – for Stampede: Racing Royale, data is grouped per minute to give the best indication of how players are interacting and engaging with the game.

Player data is analysed: Data is loaded into a pipeline tool and database which orchestrates common query patterns. In practice, this data may be used to create a picture of the amount of time between power-up acquired events and be able to work out how often players are picking them up.

Common themes and connections are found: Technology which creates logical connections then enables the data team to access the information in a clean, ordered and logical way – miles away from the million+ rows of data that the process began with.

Numbers become visuals: Visuals representations of data are then created for the team to easily see real-time data analytics. The team can then see an overall view of player activity ranging from the amount of time they play, the Power-Ups they run into and use, the karts and customisation selected at the start and how long races last.

The human touch: Whilst most of the data is processed using technology, the human touch involved in data collection for games is crucial. Having a solid understanding of not only data, but of player behaviour and general games industry trends is the best way to make the most of the information you collect. A knowledgeable Live Ops team is an extremely valuable addition to any development team – and their curiosity and thirst for knowledge is what keeps games evolving, fresh and players coming back!

[Data taken from Stampede: Racing Royale during its time in the Xbox Insider Program. Playing all over the world! Having access to data about where your players are accessing the game can help to build and support its future.]

DAY-ONE DATA | IMPLEMENTING LIVE OPS INTO YOUR DEVELOPMENT TEAM

It’s easy to assume that until a game is in the hands of players, that a data team has no function – but that couldn’t be further from the truth. Incorporating a data team early in development is a great way to set your game up for success throughout its lifespan, and having a clear data team structure working within a development team grants clearer communication between developers and data, and can help to build a culture of curiosity around design. That foundation from the beginning means data teams can understand how an update, new feature or fresh content will impact the game.

Having a data team working on a project, even while it’s still in development, means that when a game does get into players’ hands, the data team are prepared and ready to get to work. For Stampede: Racing Royale, the game has had Data Experts working on it since its early inception in 2021 and they’ve been able to identify the key metrics they want to capture and analyse, build the correct tools, prepare the tech for the incoming data, and develop the best ways to analyse and present their findings to the development team. These data pros will also work hand-in-hand with community management teams who are at work creating a thriving and engaged player-base for the game who are ready to pick the game up at launch, provide feedback and give insight into what they’d like to see next. This, combined with the analysis of player behaviour, builds trust within the game’s community, and can ensure long-standing success.

And the proof couldn’t be clearer: Stampede: Racing Royale has undergone three Steam Playtests and over six weeks on the Xbox Insider Programme between its announcement and its launch in the Xbox Game Preview. Heaps of changes have been made during that time, thanks to player feedback in community tools like Discord, in-game activity captured by the data team and player behaviour observed throughout.

[Not everything seen in an initial playtest will make it through to the final version of a game. Maybe a power up isn’t being used correctly or a kart is proving unpopular, the data team will the able to identify common trends and make recommendations for how to supercharge a projectile or glow up a vehicle.]

DATA DRIVEN GAME DESIGN

Before launch, it’s vital that developers have created a game that is fundamentally fun to play, has an engaging and enjoyable core-loop and that it has the potential to keep players returning. All game content will be focus tested ahead of launch, either in testing phases like alpha or beta, with specific groups of testers, or in a soft-launch phase to make sure it has those key features. These testing phases are extremely valuable to data teams, as it allows them to test how features of the game are being used, which customisation is the most (or least) popular, bugs or major glitches within the game or any challenges that players are experiencing.

The data gathered during these testing phases is then used to make changes to features within the game that aren’t working as intended. Here, Sumo Leamington Live Ops Director Sean shares a look at how these testing phases can be used by data teams:

“If, during a testing phase, my team and I are tasked with investigating the usage of power-ups throughout Stampede: Racing Royale, we would first start with a series of questions:

What Power-Ups are the most effective? | Which Power-Ups are used most frequently? | How quickly are players using a Power-Up when they get it?

“From here, we’ll set out a series of telemetry events that will help us answer these questions:

Hit rate of every Power-Up | Frequency of every Power-Up | Elapsed time between receiving and firing a Power-Up.

“Then, the data we receive will go through the collection process and we’ll be able to answer our initial questions and identify the top and bottom performers when it comes to power-ups:

The Sticky Bomb is the least-used power up – players are holding onto it longer than any other power-up in game and it has the lowest hit rate.

“This presents us with a whole new question: why? We can then dive into the group of players showing this information and work out their traits:

Are they new players? | Is their skill-level lower? | Have we made any changes in-game that might be confusing players?

“When we think we’ve worked out the why, we can take these finding to both the Game Designers and the community team to collectively work out a solution. We could:

Create a tutorial so new players understand how the power-up works! | Tweak the VFX in-game so players can see how it’s used and what its impact is! | Do some Community Engagement with a focus on the Sticky Bomb and its purpose, uses and best-practices.

This process allows Data, Community and Design teams to work in harmony and create a more player-friendly experience… without players ever having to ask! In a game that is high-speed, high-stakes and high-chaos, you need every element to be working together to create the desired experience… otherwise you might end up with Sticky-Bomb haters losing interest and logging off.

THE FUTURE OF LIVE OPS

Looking ahead, Sumo Sheffield Live Ops Director Laine predicts that a combination of Big Data and Machine Learning will lead to improved and more precise customisation for players in the future. He said: “There’s been a noticeable move towards Big Data – where Data Experts take large amounts of both structured and unstructured data from various sources, and then process, compute and use it in various applications – in recent years.

“This large-scale digesting of data, combined with Machine Learning that is able to tailor the player experience, like building player profiles automatically based on behaviour and serving them customised content – like varied daily quests, items to suit their preferences in the store, or tailoring notifications and hints to their playstyles and patterns – could see Data Scientists being able to deeper their understanding of what players want to see.”

Thanks to the winning combination of data experts, great community managers and reactive devs, games are able to continue to thrive and grow their audience. The brightest and best games are the ones which utilise hybrid insights (data + community) to keep the game fresh, deliver content that their players are asking for, are agile when it comes to implementing fixes and are always on the pulse of what players want to keep them engaged and onside.

These hybrid-insights really show their value in games like Devolver Digital’s Cult of the Lamb, which is able to quickly respond and show flexibility when it comes to updates, patches and new content thanks to its hyper-engaged community. Last year, the community team for the game told players it would introduce an, erm, spicy new update if they got the channel to 100k followers in a designated amount of time – a smart way to rally the community with a clear call to action, and then deliver on content for the game which was likely already in development.

Despite recent discussions regarding a perceived ‘overload’ of live-service games entering the market in recent years, there is light at the end of the tunnel. Titles like Helldivers 2 have been praised for ‘setting the new standard for live service games’ for their commitment to growing thriving communities, keeping players engaged with regular, high-quality content and, of course, making the most of the Data captured by their expert teams to keep watch, to track and ultimately improve the player experience.

Stampede: Racing Royale is now on Xbox Game Preview and is available to play for free! You can get more information and download the game on the Xbox website.

To discover how Sumo can support your live service title, please visit the Services page.