I feel like reading Jakob Nielsen articles is a mandatory assignment in the UX field. So along with everyone else in the community last week I read his latest article, “Windows 8 — Disappointing Usability for Both Novice and Power Users.”
What if Nielsen had tested Windows 8 with a different audience? Like teens. What would the headline have read?
“The new design is obviously optimized for touchscreen use (where big targets are helpful), but Microsoft is also imposing this style on its traditional PC users because all of Windows 8 is permeated by the tablet sensibility.
How well does this work for real users performing real tasks? To find out, we invited 12 experienced PC users to test Windows 8 on both regular computers and Microsoft’s new Surface RT tablets.”
Are 12 experienced PC users the primary audience Microsoft is designing Windows 8 for?
(It’s worth mentioning that Nielsen never describes testing novice users in the article though his headline suggests otherwise.)
When we teach usability testing we emphasize the importance of testing with the right audience in addition to the right number of people. Nielsen even points this out:
“If, for example, you have a site that will be used by both children and parents, then the two groups of users will have sufficiently different behavior that it becomes necessary to test with people from both groups. The same would be true for a system aimed at connecting purchasing agents with sales staff.”
Given the amount of press the article has received in mainstream media, it’s worth thinking about whether the 12 experienced PC users were the right users to test with.
In 2008 Dan Harrelson interviewed Jensen Harris about designing the Microsoft Office Ribbon interface that first appeared in Office 2007. He described the struggle they faced with user testing:
“With a user base of 400 million Office users, just slightly changing the demographics of a study can vastly influence the results. Do you want beginners, intermediates, or advanced? Advanced with Office or with computers in general? Which version of Office do they currently run? What region of the world are they in? What kind of work do they do? Do you measure the first day they get the new version—or after three weeks—or after three months? Or a year?” – Jansen Harris
Why would usability testing be any less difficult for the Windows 8 OS? Yet, here’s Nielsen saying it’s a disappointing experience for both novice and power PC users. And remember, the article says nothing about testing novice users…
I hated the ribbon release. I could appreciate it from a design perspective, but the transition to using it was excruciating. I failed basic tasks all day, every day. If you took 12 users like me, an experienced PC user and an advanced Office user, and conducted a usability test on Microsoft Office 2007, I bet the resulting headline would have included the word disappointing.
What if Nielsen’s Windows 8 study had been conducted with 12 experienced smartphone teen users?
Maybe the headline would have read, “Windows 8 – Cramazing Usability for Novice and Power Teen Users!”
You see if I were Microsoft I might be thinking about the audience of the future and what their environment might be like. I’d gamble on things like:
- Touch interfaces becoming the norm everywhere (even in the office)
- The way teens interact with technology today will be the foundation for tomorrow’s knowledge workers
With these things in mind, you might design differently. You might take risks and break rules. That’s what I appreciate about the Windows 8 release, even if Microsoft got some of it wrong.
By the time today’s teens enter the workforce Microsoft will probably have worked out the kinks and heck, they may even be ahead of the game.
Would Nielsen’s negative finding about low information density and overly live tiles that “backfire” be the same if he tested with teens? With Pew Internet reporting that one in four teens between the ages of 12-17 reporting they own a smart phone and teens sending 60 text messages a day in 2011, it seems this generation is comfortable with less information and frequent updates. Perhaps the reported “negatives” might not be so negative.
And what about that great line in Nielsen’s article?
“On a regular PC, Windows 8 is Mr. Hyde: a monster that terrorizes poor office workers and strangles their productivity.”
If a group of teens had to work from a tablet or smartphone instead of a regular PC, would their productivity be strangled today? MIT Technology Review just published an article talking about how mobile computers (smart phones) appear to be reaching mainstream use as fast as, if not faster, than the television. If this is the environment U.S. teens are growing up in, isn’t it possible they could be more productive on these devices now than those of us who are used to a “regular PC”?
I recognize Microsoft has to sell a product designed for today’s users with today’s technology. Nielsen presents findings that will negatively impact every audience and they shouldn’t be dismissed. But I’m gambling that Microsoft released the radical changes seen in Windows 8 at just the right time. They will work out the kinks and be ready for the next generation of knowledge workers. Even Nielsen acknowledges that Microsoft has “a history of correcting their mistakes.”
November 28, 2012
One day after I pressed the publish button on this post I learned my friends at Adaptive Path released videos of the talks from UX Week, which I was not able to attend. If I had been able to attend, I know Jensen Harris’ talk The Story of Windows 8 would have influenced the development of this article further. I’m 13 minutes in and loving it.