Fornite is one of the biggest games in the world at the moment. It drew in 125 million players in less than a year, making it a resounding success. This multiplatform game had battle royale-type gaming fans salivating for its release on Android until it finally happened just recently. And with successful games come successful streaming events, as well as tournaments.
All these activities generate massive amounts of data which Epic Games has access to. Fortnite processes 92 million events a minute, and around 54 billion events a day. At its peak, it generates 40 GB of data per minute. All that ingested data from game clients, servers and services brings it up to the tune of 2 petabytes a month. That’s how massive the data is for Epic Games.
And when you have the most popular game in the world, you need a system that matches that scale, which is why they’ve elected to go with Amazon Web Services (AWS) to build their architecture on. They’re all-in on AWS, a well as its other machine learning tools such as Amazon SageMaker.
What makes Epic Games’ usage interesting is the millions of players’ concurrent and telemetry data that they can use for analytics, KPI tracking and product improvements. The game has grown more than 100% in recent months, which pushes their capacity to the limits. Fortnite outages are common and the company has used the data gathered from those incidents to refine its architecture.
There was an in-game event recently which Epic invited all its players to participate in at the same time. A game traffic record was set because of that, with 37 terabytes per second delivered across its platform.
The data is used in everything from everything from finding out average revenue per unit (ARPU) to game analysis and improvements. The analytics architecture has many moving parts and micro services, but generally speaking it’s a real-time pipeline where everything is stored in AWS S3.
That makes it a data lake for Epic, with datasets at the petabyte level. It’s then integrated and sent to Spark to scores to telemetry data to Tableau and SQL.
AWS is also good for Epic Games because architecture is critical. Growth isn’t organic in many cases, with the potential for a million customers to show up one day in one big burst. AWS’ architecture team makes sure everything is tested completely from day one and they have the solutions architects involved immediately, as well. MapReduce and open source tools for big data are also critical for Fortnite’s needs.
The big data requirements for Fortnite may be massive, but any outsourced data workers would also know what’s needed. Data-driven systems are part of tomorrow, so enterprises have to be prepared for it today.