Microsoft engineer: ‘Definitely problems’ with test process after crippling Windows patch
By Gregg Keizer, Computerworld | August 22nd, 2014
Employee says latest ‘Blue Screen of Death’ screw-up affects only 1 in 10,000 PCs.
A week after Microsoft pulled a Patch Tuesday update that crippled an unknown number of Windows 7 PCs, the company has yet to provide a working fix for either the original vulnerability or the resulting problem for people affected by the broken update.
Nor has Microsoft, which still retains a reputation for more transparency around security events than its rivals, including ultra-secretive Apple, issued any public statements outside the narrow confines of the MS14-045 “bulletin” that accompanied the three-patch update.
But someone claiming to be a Microsoft employee stepped up to fill some of the information void.
“We’re working as hard as we can to fix this and release that fix as quickly as possible, so stay tuned for the re-release announcement soon,” said Kurt Phillips on Wednesday. Phillips added that although he was not an official company spokesman, he was an “engineer on a very busy graphics team trying to fix our problem.”
Computerworld was unable to confirm Phillips’ identity; the only “Kurt Philips” at Microsoft listed on LinkedIn.com was a high-level manager on an Exchange team. Microsoft declined to either confirm or deny Phillips’ identity, and also declined to comment on a timetable for a re-release of the MS14-045 patch.
“We are aware of some issues related to the recent updates and we are working on a fix,” a Microsoft spokesman said via email late Thursday. Phillips gave more detail than the company has offered publicly, and also acknowledged the screw-up. “The reason we pulled this patch was that IF you ran into the problem specified, it’s a horrible user experience,” Phillips wrote. “We made a fairly invasive change in font handling as part of a security patch and thought we had it tested properly, but there are definitely problems in our test coverage and design process that we need to address. We definitely have lessons to learn from this and we will.”
As of early Friday, Microsoft had not re-released the flawed part of MS14-045, one of nine updates it shipped on Aug. 12; it then told users to uninstall the patch on Aug. 15. The company later removed the buggy patch from the Windows Update service.
In the absence of information from Microsoft, it was inevitable that customers filled the vacuum. In a long and still-growing discussion thread on the Microsoft support site, the overall impression has been sharply negative. Among the most benign messages were those wondering why Microsoft has not said more or simply asking when a fix would be available.
“Any idea when we will have a fix? What should we do in the mean time?” asked someone identified as heshie on the thread, which has grown to more than 500 messages and has been viewed over 97,000 times, both large numbers for Microsoft’s support discussion forum.
On the same thread, however, Philips downplayed the scope of the problem that crippled some PCs with the notorious “Blue Screen of Death.”
“One thing to keep in perspective here — the actual numbers we get through telemetry (clearly not exhaustive, but definitely representative) are that the failures are only happening in ~0.01% of the overall population,” Phillips said. “So, about 1 in 10,000 machines are crashing.”
While the percentage cited by Phillips was indeed very low, it translated into about 85,000 Windows 7 PCs — the machines that seem to be most affected — or if all the estimated 1.52 billion personal computers running Windows worldwide was the population he had in mind, approximately 152,000 systems.
In either case, that’s a lot of angry customers, many of whom spent hours either recovering or trying to recover their PCs.
“Just wanted to clear up some of the hyperbole — Microsoft isn’t crumbling, all of our testers weren’t fired, etc. 99.99% success is pretty good in most jobs in this world, but clearly we need to strive for higher,” Phillips continued.
Phillips was referring to speculation on the support thread and elsewhere by end users and IT administrators alike, who have all tried to explain what they see as a decline in the quality of Microsoft’s software updates. Some of that speculation has revolved around the July job cuts \ Microsoft made in the U.S., where according to many accounts a large number of software test engineers were let go.
Phillips also gave a mixed message about whether Windows users should follow Microsoft’s advice and uninstall the pertinent patches. “If you installed [the update] and haven’t seen a Stop 0x50 [error], there’s no guarantee you won’t see one before we fix it, but look at the odds,” Phillips wrote. “I’m not uninstalling. You need to make your own decision on that, of course.”
Additional information on how customers should deal with the buggy updates can be found on Microsoft’s support site.