Trouble on the horizon? Blizzard's Diablo IV beta stats seem like a return to a troubling form

With Diablo IV's beta week-ends (closed/pre-order and open/free) in the rear-view mirror, it was clearly time for Blizzard to use their "not a marketing beta" exercise into a marketing exercise. And, in the best Blizzard tradition, they've done this with an infographic:

Most commentary, including the article linked above, fell into the obvious trap of any context-free statistic, and simply gushed about the size of the numbers. 61 million hours played! Blizzard's largest beta ever! Wow!

Blizzard have pulled this trick several times with Diablo III's metrics, from their first anniversary infographic, when they claimed a daily average of 2.1 million players since launch... without mentioning that 10 million people were trying to play the game in the first month, which definitely skewed that average upwards. That was followed by their Q3 2013 non-GAAP investor call, in which they answered a question about sales of D3's just-launched console version with a number "across all SKUs and platforms since the game's launch," naturally rolling in the 14.5M units of sales they'd already reported, six months previously.

And then there's the Q2 2014 non-GAAP investor call, in which they redefined "Reaper of Souls" to include both the base game on PC and the recently-launched Reaper of Souls expansion, giving them a pretext for "counting" those same 14.5M units of sales as having happened in yet another quarter. They got away with it, too; their share price went up. Truly, this is the stuff of legend.

Activision Blizzard are not alone in their love of context-poor infographics, of course; essentially every major publicly-traded corporation plays this same game, while a compliant business news media politely decline to call them out for it. As the wise man once said, there are lies, damned lies, and statistics, and spotting the shenanigans is an essential survival skill, especially when dealing with any large corporation.

In the world of corporate metrics, the missing context is absolutely key to having any idea what's actually going on. So let's add some context, shall we?

Largest. Beta. Ever. 

What was Blizzard's second-largest beta? Do you know? Have Blizzard ever said? How many players took part in their second-largest beta? Hell, how many players took part in Diablo IV's beta?

None of the released numbers is, directly, a player count. So, let's do some napkin math, and estimate the player count from the numbers given.

Assume that roughly twice as many players paid for beta access than took part in the free week-end. Each paid player takes part in 2 week-ends, with each free player taking part in just one, which averages to 1.666666... week-ends per player. Call it 1.67 (napkin math)

Assume that the average player will not be able to no-life this thing; they'll have errands to run, house-cleaning to do, kids to wrangle, and so on, which will limit their play time to, say 10 hours each week-end. Limiting average hours played will inflate the players count, but this is just napkin math; it's the ballpark, not the ballgame.

Now, divide your total hours played (61.5M) but the number of weekends per player (1.67), and then divide again by the number of hours per weekend (10), to estimate the number of players:

61.5M hours played ÷ 1.67 weekends per player ÷ 10 hours per weekend ≃ 3.65M players

Now, let's sanity check that number, by comparing with a second stat which Blizzard themselves have provided: the number of beta wolf pup backpacks earned: 2.6M.

Comparing this to our player count estimate, this is roughly 71% of players; since some of those players would have made the mistake of starting with Druid, for example, and likely not have had very much fun, I'd say that this looks like a  reasonable ratio. I'm going to say that a total beta player count of 3.65M is reasonable, as an estimate.

That certainly seems respectable, especially if 2.43M of them had pre-ordered the game. It's respectable enough that I wonder why Blizzard didn't just cite the player count themselves. Also, respectably as it may be, it does feel like a stretch to be lavishing a 3.5M or so player count, or 2.4M pre-order count, with superlatives. D3's Reaper of Souls expansion sold 2.3M copies in its launch week, after all; if Diablo IV has only managed 2.4M pre-orders to this point, then it suggests that Blizzard have failed to grow their D3 player base at all in the decade since D3's launch.

Pandering to Diablo II fans has not bought them back to the franchise.

I can't stress enough just how disastrous that launch was. Diablo III launched broken, buggy, unfinished, and ultimately turned out to be so badly designed that Josh Mosqueira ended up leading a complete revision of essentially every gameplay element: every system which dealt with the way players dealt, healed, or mitigated damage was reworked, along with all of the items, and the system which generated items. In an ARPG, that's basically everything, and it still wasn't enough to actually fix all of the game's issues.

Essentially everything that Diablo II fans might have wanted from Diablo III was absent at launch... and so D2 fans mostly left after that launch. They did not come back for Reaper of Souls; they did not come back to buy Diablo II: Resurrected; and they have not come back to Diablo IV, either, if we're to judge by the numbers that we have.

And why would they? Blizzard never publicly acknowledged the extent of the D3 launch disaster; they never apologized; and they steadfastly refused to do the one thing with D3 that D2 fans kept asking for, i.e. to add an offline solo play option to the PC game, even after releasing four different console ports of the same game which had the feature.

Also, Diablo IV is more of an MMO than a classic ARPG, something which Blizzard's lost Diablo fans didn't ask for, and may not be willing to pay for. Blizzard is trying to lure these players away from Path of Exile and Last Epoch, remember, both of which offer a classic ARPG experience (with multiplayer being optional, rather than being unavoidable); Last Epoch is even playable offline.

And don't forget Lost Ark, which offers exactly the same sort of MMO-like experience as Diablo IV, but which is much more full-featured than Blizzard's offering. Honestly, who is this game even for?

Diablo Immortal did not expand the appeal of the franchise.

Porting D3 to mobile was clearly intended to do two things:

1. Make a pile of money for Activision Blizzard, and

2. Expand the Diablo franchise to appeal to people who hadn't previously been Diablo fans.

It clearly accomplished its #1 goal; whether they money made reached (or surpassed) ActiBlizz's expectations is not known, but Diablo Immortal "whales" definitely dumped a lot of money onto the top line of Blizzard's balance sheet.

There's currently no evidence that it accomplished goal #2. Diablo Immortal racked up 10M+ downloads on Google Play alone, and it's reasonable to assume that iOS downloads were about the same. But Diablo IV's only managing something like 2.4M - 2.6M pre-order/beta-tester customers. If 20M people played and loved Diablo Immortal, where are they all?

And, if Blizzard hasn't managed to lure Diablo II fans back the franchise, and if they haven't managed to expand beyond the remaining Diablo III fan base... how well can Diablo IV actually sell?

Is Diablo IV doomed?

I don't have a crystal ball. I don't know what the future will hold. There are a bunch of excited gaming bloggers who are breathlessly predicting that Diablo IV will be a smash hit, and even a game of the year contender.

Of course, those are some of the same people who gave Diablo III's disastrous launch state an 88 metascore, which users did not agree with:

Personally, when I look at the coverage of the game by people who've played the game, I'm getting a much more luke-warm vibe. The people who like the game do seem to like it a lot, but the people who don't like it really don't like it, and there are very few people in the middle. That's a pretty mixed reception, which is not what I'd expect of a future GOTY-caliber smash hit.

Expecting the 3M or so people who bought the Reaper of Souls expansion to also buy the next main-line game in the series seems reasonable. Expecting the game to pull in another 12M customers from somewhere else feels like a stretch. I'm feeling like Diablo IV might well struggle to hit Diablo III's first-year sales pace, and I don't think it has legs enough to be the best-selling installment in the franchise.

Especially not with ExileCon 2 happening only weeks after that launch.

And this time, there will be no "across all SKUs and platforms" aggregate sales reporting tricks. The game is releasing day and date on all platforms at once; its sales numbers will be its sales numbers. There won't be any "hours played" cuteness, either; trying to obfuscate poor sales numbers by citing a bunch of made-up metrics might have worked in the past, but there are too many eyes on Diablo IV, and too much interest in whether it succeeds, for Blizzard to get away with those tricks a second time. The next metrics we get from Blizzard need to include a sales figure.

So, yes, Diablo IV might be mildly doomed. It won't be an outright flop, but it won't be a hit, either, instead living in a liminal space between those two states of being... which might be the worst possible outcome for Blizzard, and for their Diablo franchise. A clear result, positive or negative, would at least offer Blizzard clear guidance as to whether they should change course, or stay the course. A middling result will give them no such guidance, and likely result in Diablo continuing on its current apparent glide path to irrelevance in a genre that Diablo invented.