Showing posts with label #howtoliewithstatistics. Show all posts
Showing posts with label #howtoliewithstatistics. Show all posts

September 26, 2017

With the Fall Creators Update less than three weeks away, over a quarter of Windows 10 customers don't have Spring's CU yet.

I'd already blogged about Microsoft's claims about the Creators Update's reliability, but Wayne Williams at betanews has a pretty good take, too.
The Windows 10 Fall Creators Update is set to begin its rollout in a matter of weeks, yet over a quarter (27.5 percent) of Windows 10 users still haven’t received the Creators Update.
According to the latest figures from AdDuplex, while the Creators Update found its way on to another 7.5 percent of computers this month to give it a 72.5 percent share, it’s still well short of the 91.2 percent that the Anniversary Update reached before the Creators Update was released. AdDuplex warns that as a result of slow rollouts like this, fragmentation will only increase in the future.
[...]
A week ago, Microsoft proudly announced that the Creators Update is much more reliable than the Anniversary Update, with a "39 percent total reduction in operating system and driver stability issues" and the number of support calls diminishing "significantly." But as I pointed out at the time, a large portion of that improvement can be attributed to the fact that many Windows 10 users simply don't have it.
And this is the point where I remind you that the Fall Creators Update, a.k.a. version 1709, is launching a full month late. With over a full month of extra time in which they were rolling out the Creators Update, they're still nowhere near where they should be, or need to be. 

Can someone remind me again why Windows-As-A-Service was supposed to be such a great way to receive Microsoft Windows from the Gods of OS? Because I don't see it.

September 20, 2017

Lying with statistics, the Microsoft way

Microsoft Touts Windows 10 Quality and Reliability

Here's a classic trick from the statistical liars' arsenal: the misleading graph. I spotted this particularly egregious example on Thurrott.com, and it really has it all: improper scaling, truncation, omitted data, missing labels, you name it.

The source? Microsoft, naturally.

I'll let Paul Thurrott himself describe the context from whence this springs:
On the eve of the Windows 10 Fall Creators Update release, Microsoft is touting the quality and reliability of the previous version, the Creators Update.
“The Windows 10 Creators Update is the best version of Windows 10 ever,” Microsoft director John Cable writes. “What makes Windows 10 Creators Update the best version of Windows 10 ever? Quality. Our dedicated focus on customer obsession – listening and responding to user and partner feedback – are key to the quality improvements in Windows 10.”
[...]
Frankly, this whole thing is rather curious, both for its timing and because it leaves out any data from before the Anniversary Update. Rather than belabour the point, I’ll just note that this is what Microsoft provided. You can come to your own conclusions, but I don’t quite understand why they are revealing this now.
Thurrott is being a lot more generous here than I will be; this graph does not reveal, and is not intended to reveal, anything at all. It is, in fact, almost entirely information-free, and utterly meaningless; we're expected to take it on faith that it means what Microsoft says it means, but have no means of verifying that, and no reason to trust them.

Notice how the y-axis is unlabelled; there's a "+500 million" number slapped down on top of those vertical bars, but we have no way of knowing which of those bars is meant to be 500 million users high (if any). There are thirteen vertical bars, but only four months on the x-axis, and no way of knowing which bar belongs to which month (again, if any). There's an orange line zig-zagging its way down the graph (presumably over time?) but no way of knowing how many Customer Support Contacts it represents at any point of that slope, let along whether its vertical scale is the same as the bars'. And so on.

This is bullshit. Thurrott is bends over backwards to be diplomatic, but I don't mind belabouring the point; bullshit like this is why people don't trust Microsoft anymore.

UPDATE:

MakeUseOf also covered this, and had this take on Microsoft's "statistics."
The timing of this release is a little confusing. But we suspect it’s an attempt, ahead of the release of the Windows 10 Fall Creators Update, to convince Windows 10 users (especially corporate customers) that things are getting better.
Possible... but I'm still betting on this being aimed at Windows 7/Server 2008 customers, who they're still trying to start on the path to Windows 10 migration.

August 08, 2017

Selection bias in action: Windows 10's data collection policies really haven't been as positively received as they'd like you to think.

From Wikipedia:
Selection bias is the bias introduced by the selection of individuals, groups or data for analysis in such a way that proper randomization is not achieved, thereby ensuring that the sample obtained is not representative of the population intended to be analyzed.[1] It is sometimes referred to as the selection effect. The phrase "selection bias" most often refers to the distortion of a statistical analysis, resulting from the method of collecting samples. If the selection bias is not taken into account, then some conclusions of the study may not be accurate.
Keep this definition in mind when reading ZDNet's latest reportage on Microsoft's Windows 10 privacy officer's latest press release (and, OMG, did that ever become a bear of a sentence):
After being pummeled by critics and regulators for Windows 10's overzealous personalization efforts, Microsoft says it's received "positive" feedback about privacy-enhancing changes it introduced in the Windows 10 Creators Update.
Those changes, which Microsoft rolled out in Windows 10 in recent months, include an online privacy dashboard and finer controls for location, speech recognition, diagnostics, tips and recommendations, and relevant ads.
It made those changes under the watch of European data-protection authorities, amid the French National Data Protection Commission's (CNIL) year-long probe over Windows 10's "excessive data collection" and tracking browser data without user consent.
CNIL in June lifted its formal notice on Microsoft, noting it had halved the volume of telemetry data collected under the Basic Diagnostic setting, and now provided "clear and precise information" about web tracking for personalizing ads.
With that episode in the rearview, Microsoft is now highlighting signs that customers do trust it with their data and that its responsiveness to customer feedback -- rather than just legal threats from watchdogs -- is driving Windows privacy improvements.
"Feedback we've received about the Creators Update has been positive. This is great news to us because what we hear from you directly impacts the improvements we make," says Marisa Rogers, Microsoft privacy officer for the Windows Devices Group.
Despite the basic Diagnostics setting collecting far less data than before, Rogers points out that 71 percent of customers select the Full option, which sends browser data, app and feature usage, and inking and typing data to Microsoft.
The setting is on Full by default, but can be toggled to Basic.
So, where's the selection bias here? you're probably asking by now.

Let's start with the fact that a significant number of the Windows users most likely to have objected to Microsoft's Windows 10 privacy regime are still using Windows 7. They never switched to Windows 10, precisely because of Microsoft's broken data collection and privacy policies, and that has not changed, even after the changes which Microsoft made while under threat of regulatory action by the Article 29 Working Party.

We'll continue with the fact that "Full" data collection is still the default, and that leaving it on may not represent approval so much as apathy on the part of those who have done so. The 71% of customers who are still set to "Full" data collection didn't select it; they just didn't care enough to change the setting. How many times have you clicked through an annoying pop-up screen to get to whatever you'd turned on your PC to do, intending to go back and look at it later... only to forget to go back later? Yeah, me, too.

And, about that positive feedback? All I can say is, "Duh."

January 28, 2017

Microsoft's fake stats show Windows 10 market share growing steadily

It's just about the end of the month, which means we're only days away from seeing new OS market share stats from the likes of StatCounter and NetMarketShare. And, true to their form of the past year, Microsoft are trying to preempt the actual stats with their own version of these numbers, apparently hoping to keep the hype train rolling, regardless of what reality has to say about Windows 10's actual adoption rate.

MS's stats have always been rather... generous, shall we say? to Windows 10, but their latest pronouncement has at least one tech blogger just straight up calling them bullshit.

From Wayne Williams at betanews:
While NetMarketShare’s OS usage share figures show the new operating system doing fine, but lagging some distance behind Windows 7 (as you'd expect), Microsoft’s figures paint an entirely different picture.
Five months ago, the software giant showed Windows 10 hitting 50 percent in the US, and two months ago, it had the new OS overtaking Windows 7 globally. Today’s update though stretches the believability just a little too far.
[...]
So why do I say the numbers are fake?
Well, the growth of Windows 10 and the decline of Windows 7 is just way too neat to be real. According to the Windows Trends figures, Microsoft grew by exactly one percentage point a month, every month from July to November (42 percent, 43 percent, 44 percent, 45 percent, 46 percent). At the same time, Windows 7 declined by exactly the same amount.
And if that doesn’t sound unlikely enough, Windows 8.1’s share went up one percentage point in August (to 13 percent) and hasn’t changed since. Windows 8’s share (2 percent) hasn’t wavered either. And that's ridiculous. Windows 8.x is an OS in decline.
It’s possible the new numbers are just placeholders, and Microsoft has dropped them in while awaiting the real figures (although why it would do that I have no idea). But if that's not the case, then I call BS. Microsoft’s Windows share numbers have been raising eyebrows for a while, but this latest lot smacks of just being made up.
You tell 'em, Wayne. It's great to see that some folks in the professional tech media are starting to tire of Microsoft's PR BS. Suddenly, I can't wait to see the reaction to the actual numbers, when they come out on Wednesday.

November 18, 2016

Reminder: Microsoft are still dicks, and Windows 10 is still annoying

From TrustedReviews:
Windows 10 has effectively become that annoying vegan friend who judges you when you order a burger and tells you the harrowing story about how it came to your plate.
As part of a new series of Windows 10 tips, Chrome and Firefox browser users are being informed ‘you know what, actually, Edge is far safer for you.’
As spotted by illCodeYouABrain on Reddit, Users are seeing a pop-up on the Edge icon saying “Microsoft Edge is safer than Firefox. It blocks 21% more socially engineered malware.”
It then offers a link to do “learn more,” which in this instance is Microsoft’s “you should really educate yourself about the issues.”
...
If you don’t want to hear Microsoft’s ‘annoying vegan friend’ act, the firm told VentureBeat that tips can be easily switched off in system settings.
...
Or you could listen to Microsoft, which has gleaned that 21% safer figure from a report it commissioned NSS Labs to create.
I'm genuinely puzzled by this one. Not because I expected Microsoft to learn anything from the GWX backlash that has 48% of all PC users sticking with Windows 7, but simply because I can't figure out who the target audience is, here.

People who are tech-savvy enough to still be using other browsers aren't going to fall for this; all Redmond is accomplishing is to irritate them. Meanwhile, everybody who lacks the technical savvy to know better is probably already using Edge, just as they were probably using IE before being switched to Windows 10. Who is this for?

Also, didn't Microsoft already try this same tactic, with zero results? I'm pretty sure they did, before trying outright bribery to try (and fail) to lure users to their new browser. What makes them think they'll get better results this time?

Also, commissioning a dubious report that you then cite in propaganda? That's lying with statistics, again, and they aren't even trying to be subtle about it.

Also, how exactly is Edge supposed to be protecting users from social engineering? The whole point of social engineering is that it relies on bad habits on the part of users to bypass the systems that would otherwise protect them.

The single biggest problem with this approach, though, is that it does nothing to address the issues with Edge that sent people back to Chrome, Firefox, et al. Edge didn't support extensions out of the box, and doesn't have any extensions available to download now that Redmond finally have added this critical feature of modern browsers, because there are no users of Edge. Software developers don't develop for users that don't exist.

Which is why users of Chrome outnumber users of Edge by 10 to 1, and why Edge only grew its browser share by 0.1% last month. So, yeah, I understand why they're desperate; they're losing, and they're out of ideas. Desperation isn't attractive, though; I don't they're going to catch any more fish this time around, using this same rancid bait.

September 27, 2016

How to lie with statistics, Microsoft edition: Update

From Venture Beat:
Microsoft today announced that Windows 10 is now installed on over 400 million devices. It took the latest and greatest operating system from the company about 14 months to hit the new milestone.
Windows 10 was installed on over 75 million PCs in its first four weeks. It passed110 million devices after 10 weeks, 200 million in under six months, 270 millionafter eight months, 300 million after nine months, and then 350 million after 11 months. Microsoft was aiming for 1 billion devices running Windows 10 “in two to three years,” but recently the company backpedaled on that goal.
Here’s the progress in a chart, with Windows 8 thrown in for context (Microsoft stopped reporting Windows 8 milestones after 200 million):
As you can see, Windows 7 and Windows 8 milestones were similar for the first six months or so, though it quickly became clear that Windows 8 was not selling as quickly as its predecessor did. Windows 10 has been ahead of both from the start, and while the slope of its growth has become less steep, it’s still very strong. Windows 10 got to 400 million devices about 7 months sooner than Windows 7.
Oy, vey. Do we have to do this every fucking month?