April 05, 2018

Numbers may not lie, but statistics without context can be incredibly misleading

Consider, for a moment, the state of VR as described by Windows Report:
The Steam Hardware Survey numbers for March 2018 are out and things are not looking good for Microsoft as its Windows Mixed Reality (WMR) fortunes seem to be dwindling.
As you will recall, the WMR headsets have been facing a slow growth in market share, despite the company implementing deep price cuts on the headsets, which initially promised easy setup and support for lower-spec PCs.
Steam’s latest figures place Facebook’s Oculus Rift at the top, with its market share at 50.62 percent, up from 47.31 percent in February this year. This compared to WMR’s share which fell from 5.36 percent to stand at 5.32 percent, while HTC’s Vive also dropped to 44.06 percent from 45.38 percent.
Those statistics, as presented, make it sound like WMR headsets are struggling while Oculus Rift sales are soaring. This is reinforced with this awful graph:

Seriously, this is awful. I could understand sorting in descending order by percentage share, or in alphabetical order, but WTF kind of order is this supposed to be? W, H, O, O, O order? Leader 3rd order?

Would you like to know what the actual numbers are, as presented by the Steam Survey itself?


So, Rift's utter dominance of Steam's VR users actually amounts to 0.20% of Steam's total user base, a gain of 0.06% over the previous month's result; given Steam's installed user base of 125 million or so, that translates into 250K headsets sold, total. HTC Vive, meanwhile, also grew, by 0.05% to 0.18% of Steam's total installed users, or 225K -- not as good, especially since the Vive was leading at one point, but hardly the calamitous drop implied by Windows Report's post.

Windows Mixed Reality, meanwhile, also gained, and though they gained less than either the Rift or the Vive, they're still up 0.01% over the previous month, putting them at 0.01% total for their first appearance on the Steam Survey.

These are not great sales numbers, obviously, and I'm on record as being very bearish on VR's prospects, generally, but it's not like WMR is being clobbered by Rift and Vive; none of them selling well. So, the question really isn't, will SteamVR support save WMR? The question is, why are none of the SteamVR-supported headsets selling at all?

Remember, SteamVR is essentially a new platform, which means that millions of units need to be sold for it to be viable. A a point of comparison, the WiiU platform moved 13.2 million units and still failed; VR probably needs to start doing Switch-like sales numbers to turn its fortunes around. 500K in sales among all of the platform's top sales performers won't be enough to make SteamVR itself viable; and if SteamVR isn't viable as a platform, then none of its compatible hardware will survive, either. WMR won't be the only VR headsets to be added to the drawer.

Windows Report don't see this, because they're not looking; they're still trying to call the horse race, as if it were still possible to anyone to win a race which, currently, they're all losing:
It is clear that Microsoft’s WMR headsets are struggling to stay afloat, which isn’t what they envisioned for the product that ideally had greater advantage over its rivals, what with the close relationships with PC OEMs, and the built-in support feature on Windows 10.
[...]
Microsoft may have to do a lot more to raise buyer interests in the technology if they’re going to continue having any impact. But numbers don’t lie, Oculus is winning at attracting more gamers to its side.
Do you think Microsoft should bow out?
Yes, MWR is struggling to stay afloat, but only because VR as a whole is struggling to stay afloat; nobody is making money on VR, and if the various competing VR hardware developers can't work together to grow the overall VR market substantially, then VR as a whole will fail.

Is also misses the mark on yet another important detail, here. Microsoft does not make VR headsets. Their hardware partners (Acer, Asus, Dell, HP, Lenovo, and Samsung), are the ones spending money to make headsets that aren't moving. Microsoft can't "bow out" because they don't have any skin in the game; they can only stop supporting the WMR format, which will sour their relationships with the biggest makers of not only VR headsets, but also Windows laptops.

That's something which Microsoft might eventually do, having admitted that their Windows 10 strategy is a failure, and that the Windows platform is no longer seen internally as being integral to the firm's future, but they're not going to do it now. Everyone is still pinning their hopes, and significant fortunes, on VR being a thing; Microsoft can't be the first to burst that bubble without looking weak, something their shareholders would not react well to.

And all of that is obvious, really... if you start by looking at what the numbers actually say, instead of reframing them to support a different narrative that's more in line with your expectations. People who lie with statistics aren't always doing it maliciously; sometimes, they're just lying to themselves.