September 20, 2017

Lying with statistics, the Microsoft way

Microsoft Touts Windows 10 Quality and Reliability

Here's a classic trick from the statistical liars' arsenal: the misleading graph. I spotted this particularly egregious example on Thurrott.com, and it really has it all: improper scaling, truncation, omitted data, missing labels, you name it.

The source? Microsoft, naturally.

I'll let Paul Thurrott himself describe the context from whence this springs:
On the eve of the Windows 10 Fall Creators Update release, Microsoft is touting the quality and reliability of the previous version, the Creators Update.
“The Windows 10 Creators Update is the best version of Windows 10 ever,” Microsoft director John Cable writes. “What makes Windows 10 Creators Update the best version of Windows 10 ever? Quality. Our dedicated focus on customer obsession – listening and responding to user and partner feedback – are key to the quality improvements in Windows 10.”
[...]
Frankly, this whole thing is rather curious, both for its timing and because it leaves out any data from before the Anniversary Update. Rather than belabour the point, I’ll just note that this is what Microsoft provided. You can come to your own conclusions, but I don’t quite understand why they are revealing this now.
Thurrott is being a lot more generous here than I will be; this graph does not reveal, and is not intended to reveal, anything at all. It is, in fact, almost entirely information-free, and utterly meaningless; we're expected to take it on faith that it means what Microsoft says it means, but have no means of verifying that, and no reason to trust them.

Notice how the y-axis is unlabelled; there's a "+500 million" number slapped down on top of those vertical bars, but we have no way of knowing which of those bars is meant to be 500 million users high (if any). There are thirteen vertical bars, but only four months on the x-axis, and no way of knowing which bar belongs to which month (again, if any). There's an orange line zig-zagging its way down the graph (presumably over time?) but no way of knowing how many Customer Support Contacts it represents at any point of that slope, let along whether its vertical scale is the same as the bars'. And so on.

This is bullshit. Thurrott is bends over backwards to be diplomatic, but I don't mind belabouring the point; bullshit like this is why people don't trust Microsoft anymore.

UPDATE:

MakeUseOf also covered this, and had this take on Microsoft's "statistics."
The timing of this release is a little confusing. But we suspect it’s an attempt, ahead of the release of the Windows 10 Fall Creators Update, to convince Windows 10 users (especially corporate customers) that things are getting better.
Possible... but I'm still betting on this being aimed at Windows 7/Server 2008 customers, who they're still trying to start on the path to Windows 10 migration.