Jakob Nielsen Got It (Mostly) Wrong

6
1928

Why Jakob Nielsen got it (mostly) wrong

Summary: Do not assume, as Jakob Nielsen’s Alphabetical Sorting article implies, that A-Z listings are usually not the best way to present a list to users. Instead, carefully evaluate your typical users and their tasks, based on the following listing selection parameters, and make a more informed decision about whether to use A-Z listings, or something else.

It’s not every day I disagree with Jakob Nielsen, arguably a living legend, a guru, an Einstein of usability, but today is that day. I’m a big fan of Jakob Nielsen and his Alertbox articles, but this time I think he left out critical information in an article that could lead you astray. So I am here to humbly set the record straight, and fill in that missing information.

Nielsen photo Nialkennedy via Flickr Creative Commons license
Jakob Nielsen got his A-Z listings article (mostly) wrong

Jakob Nielsen’s October 4, 2010 Alertbox titled ‘Alphabetical Sorting Must (Mostly) Die’ is in my opinion (mostly) wrong.

Actually, it’s not that it’s so much ‘wrong,’ more that it completely left out crucial information on a rather complex and critical choice designers and information architects face every day. That choice is:

‘What type of ordered listing should be used so a user can pick from a set of items?’

Here’s Nielsen’s summary from his Alertbox article:

‘Ordinal sequences, logical structuring, time lines, or prioritization by importance or frequency are usually better than A–Z listings for presenting options to users.’

Note the word ‘usually.’ I take issue with the word ‘usually.’ Now if Nielsen had picked any other words, such as; ‘frequently, somewhat, potentially, sometimes, typically,’ or other variations, I would have been fine (mostly).

But the word ‘usually’ infers that other choices are better more often than A-Z listings. That my friends is (in my humble opinion) just plain information architecture crazy talk.

Determining which type of listing to use should not be made until more information is know about the typical users and the task or tasks that are to be performed. For some instances, A-Z listings won’t be the best solution, but for many others, they will.

Alphabetical or ‘A-Z’ listings are your friends:

Alphabetical listings, aka A-Z listings, are incredibly powerful and useful sorting systems and can be an information architect’s best friend. On a side note, the actual term to use when defining ordered listings of items (such as in A-Z listings) is ‘collation,’ as defined by WikiPedia.

What is missing in my opinion from Nielsen’s article is identification of the parameters that should be evaluated prior to making the decision about what type of listing (A-Z or otherwise) to use.

Parameters for evaluating a type of listing to use:

Click here to view latest job postings


Domain Expertise of Persona

By Domain Expertise I am referring to the general knowledge the Persona (typical user) has in relation to the subject matter of the website or application. For example, a consumer shopping for new tires for a car may have very limited domain expertise regarding the subject, but a tire-store owner may have very great domain expertise about the subject.

A-Z sorting may be helpful in situations where Personas with lack of expertise in the domain can be provided all the possible choices, allowing them to scan the listings of items. Indeed, this is a very common and useful way to present groupings of names or Brands, as TireRack.com demonstrates with their ‘Tires by Brand’ A-Z listing as shown below.

Although A-Z listings of brands of tires may be better for the limited domain expertise user, an alternative listing technique, such as Frequency of picked items, might be better for the more advanced domain expertise user, our tire-store owner for example (assuming such frequency of use data is known).

TireRack.com uses an A-Z sorting of Brands

As shown above, the A-Z listing of Brands on TireRack.com can be helpful for low domain expertise users.

It is important to consider the domain expertise the Persona has prior to making a choice of a selection tool because familiarity (or lack thereof) of subject matter may impact the ease of use of various sorting systems. Enabling low domain expertise users to ‘reorient’ themselves by using an A-Z listing might be helpful when users are trying to scan the total set of choices, in an attempt to pick from one or two choices from the entire set.

Specific Label Expertise of Persona

Going into a bit more detail than with Domain Expertise, by Label Expertise I refer to the knowledge the typical user has in regards to the actual words (labels) used in a listing of items. For users with limited exposure and knowledge about the meaning of specific labels, an A-Z listing can be a helpful way of enabling them to familiarize themselves with the listing terms. Likewise, users with advanced knowledge regarding the topics may not find it necessary to have an A-Z listing, and may find it easier and faster to use frequency of use or other types of sorting schemes.

Side note: A best practice is to use card sorting (including completed and blank cards) with participants who closely match the Personas when determining the words to use in a listing of items.

As an example, Fluitek uses an A-Z listing of industrial filtration parts on the home page. This enables users who may have limited understanding of the definitions of some terms to be able to quickly scan the entire list of choices in an orderly fashion. This enables them to easily navigate the choices and thus zero-in on likely candidates.

Fluitek uses an A-Z listing of items for industrial filtration parts

As demonstrated above, Fluitek.com uses an A-Z listing of items for industrial filtration parts.

However, as seen in the next example from Purwater.com, a non-alphabetical listing of products is used down the left hand side of their product page. Whether to aid recognition of terms or to help users navigate to the specific product, or both, a visual categorization to the right is also used on the page.

The visual categorization uses thumbnail photos of each product type under its corresponding label. This visual aid can help provide additional clues to help users find the correct item, if they are unfamiliar with definitions of labels. However it is a limited use tool as it is problematic if more than a handful of thumbnails are presented for users.

Pur uses a non A-Z listing on the left, and thumbnails on the right, for product identification

As shown above, the Purwater.com products page uses a non A-Z listing on the left, and thumbnails on the right, to assist with product identification

Listing Tool Frequency of Use

Another parameter that can be considered when choosing a listing type is the amount of listing tool usage a typical user experiences, or will experience. Frequently utilized listing tools may not require alphabetical ordered listings, and indeed other forms of ordering, such as listing item frequency of usage or even keyboard entry may be a better choice (picking a U.S. State as an example, keyboard entry of the two letter abbreviation of the State can be significantly faster and less error-prone than choosing from a drop down list).

For selection tools that are used rarely, A-Z listings can provide a more effective and efficient way to present items in the list.

As an example, the Texas Department of Motor Vehicles uses an A-Z drop down menu tool of ‘How Do I…’ choices to assist users in finding the right information for various actions. Most likely, a typical user rarely needs to visit their site and use this selection tool. Thus this tool’s use of an A-Z drop down list helps organize the total set of choices for those low tool frequency users.

Texas Department of Motor Vehicles uses an A-Z drop down menu tool

As shown above, the Texas Department of Motor Vehicles uses an A-Z drop down menu tool for their ‘How Do I…’ selection listing.

Listing Item Frequency of Use

Akin to the tool frequency of use, the listing item frequency of use is the frequency of which users pick each item in a listing of items. With a listing of items by frequency of use typically the most picked (popular) items are placed at the top of the list, with less popular items below. This is a handy way to present listings, especially when the popularity data directly links to the actions of the typical users.

However, where design teams get in trouble with popularity based listings is when they make assumptions about popularity, without the actual user contextual data to back it up. Design teams who simply utilize web log data to extrapolate which items are used ‘most frequently’ often don’t realize their data may be suspect. For example, some users will just ‘pick’ items that are near the top of the list because they either do not know which item to pick, or don’t care.

As a case in point, while at Blue Cross I examined the most commonly picked ‘no-sale’ reason codes from the sales call-center’s non-sales reports. My goal was to use the no-sale reason code data to try to better predict which advertising sales audiences might provide better sources of leads, thus better sales.

The Agents had to choose a non-sale reason code from a list of over 40 items. They did this by using a drop down tool to select from this long list of codes. The listing order was by frequency of use, thus the most picked reasons were at the top.

Photo of a call center by vlima.com via flickr creative commons license
Call center workers are under high time pressure to accomplish tasks, designers must consider this when selecting listing types

According to the Blue Cross non-sales reason code data, only the top 2 or 3 reasons were picked. It was very rare to find another code other than the top 2 or 3. Those top 2 reasons made up about 90% of all reason code responses. Why was there such a large discrepancy between the number of codes and the very few that were picked?

Instead of just taking the data at face value and using it to predict target audience potential sales activity, I decided to go and observe and interview the agents. I wanted to learn the ‘why’ of their selection behavior.

In interviewing the agents, and conducting contextual observations, I learned Agents were extremely pressed for time. Thus they just picked one of the first reason codes in the list, instead of scrolling through the entire list. Furthermore, agents were not rewarded for the quality of the reason code response provided, just for providing a response. So picking any reason code was good enough, from their perspective.

In essence, the agents were cherry-picking the top-most responses because it was the fastest way to complete the task and take another call.

Knowing this, I worked with the marketing team to try to dramatically cut down the list of non-sales codes, by determining which reasons were really needed, versus just a nice to have.

The new list cut down the set of reasons from over 40 to just 6. These 6 reason codes would still provide the marketing department with valuable data, but would easily fit on a drop down menu with no scrolling. Thus, agents could pick from one of the 6 reasons with little to no slow-down. This proved to be a highly effective way to better capture non-sales data that ultimately was used to determine which advertising audiences were more likely to be better prospects.

The screen shot below demonstrates an example of a reason code list that is short enough to fit in a drop down box (this one does not use A-Z listings).

Photo of reason code list

The moral of this story is designers must fully understand the ‘why’ of user behavior when determining which type of listing type to use. For frequency of use listings, guessing the frequency, or not fully exploring user reasons for frequency data, can potentially lead to a bad user experience.

In many cases, starting with an A-Z listing is the best way to proceed to gather frequency of use data prior to determining the order of a frequency of use list.

Number of Items in the List:

Really long lists are going to need organization, either by clustering in like groups, or more often by A-Z listings. Related to this is the very long listing most designers deal with when asking users to pick a State or Country. Especially for Country lists, placing the list in A-Z order is a best practice, but with the typical user’s Country listed first, if known (and again down in its normal place in the list, just in case they miss it).

As can be seen in the example below, Google’s profile editing page includes a country selection drop down menu in A-Z order.

Google Country listing is in A-Z order

However, it is often faster and less error-prone to enable users to key in their address data, especially when choosing for example U.S. States in an address entry order flow.  The point is, designers and information architects must carefully consider how many items will comprise an A-Z listing prior to determining whether to use it for a specific task.

Conclusion: Why Jakob Nielsen was (mostly) wrong

So now that I’ve provided a more complete list of parameters that can help you choose a type of listing, whether A-Z or otherwise, I hope you’ll see why Jakob Nielsen’s Alertbox was (mostly) wrong. Do not assume, as his article implies, that A-Z listings are usually not the best way to present a list to users.

Instead, carefully evaluate your typical users, based on the selection parameters, and make a more informed decision about whether to use A-Z listings, or something else.

Parameters for evaluating a type of listing to use:

  • Domain (overall subject matter) expertise of persona
  • Specific label expertise of persona
  • Listing tool frequency of usage
  • Listing item frequency of usage
  • Number of list items

6 COMMENTS

  1. Great post! It’s nice to see someone point out when a “guru” doesn’t necessarily tell the whole story.

    It’s also important to note that the type of items in the list can impact the proper listing paradigm. Take the TXDMV example in the post. Had they chosen to list full-length questions, rather than fragments with distinct “action” words, an A-Z listing wouldn’t make as much sense. Paying close attention to the type of information in your list and choosing a listing method from there can greatly impact usability of the list.

    Thanks again for a great post!

    Chris

  2. Thanks for going into more depth on this interesting topic! I think you made a good case that there’s a lot to consider when ordering these lists, and that simple rules (whether biased toward or against A-Z order) are all subject to the usual “it depends” maxim.

    I think that the type of task you’re performing is one of the most important considerations. Your tire example is, for me, impossible to evaluate without knowing why the personas you describe are choosing tire brands from a list.

    For example, if the consumer’s goal is to choose a popular and presumably decent quality tire brand from the list, a “most frequently purchased” list would be much more helpful. Popularity doesn’t guarantee quality, but it conveys some information about the tires, where an A-Z conveys none at all. And if the tire-store owner’s goal is to look up information about the consumer’s current, obscure brand of tires, an A-Z listing might indeed be more helpful. To my mind the personas are much less important to that example than the user goals and the type of tasks supported by that interface.

    I’m confused, too, by your call center example. An A-Z order would definitely have been worse in that case because at least with the frequently of use there’s a chance that some agents occasionally have time to pick the correct codes, so even though the ones at the top are artificially popular, there might be a grain of truth in there somewhere. Your solution, to narrow the choices, seems quite correct to me, but I don’t think it rebuts Nielsen’s point; it just indicates that list ordering isn’t a panacea.

    Even in your DMV example: is the A-Z ordering really helpful? Are people who are worried about buying a stolen vehicle looking for the word “avoid,” or are they scanning for “stolen”? Is “find” the right keyword, or “dealer license”? Again, these may be deeper design issues; maybe a “How do I…” dropdown isn’t a rich enough widget to do the job it’s been assigned.

    In conclusion, I think Nielsen was wisely provocative in saying “A-Z is usually wrong” and you are wisely provocative in saying “Nielsen is mostly wrong.” The truth — “it depends, and you have to think about each case carefully” — would never have attracted reader attention. 🙂

  3. Chris – good point about the ‘action’ type words used on the Texas motor vehicles drop down menu. I would be curious to know if those action words were the best to help users find the right information, or not.

    Drew – You’ve brought up some good points, but I will disagree with your summary that Nielsen is wise in saying ‘A-Z is usually wrong.’ The point I was driving at was until you have more information with which to make a choice, you cannot say ‘usually wrong.’ In fact, although I’ve not done so it would be interesting to take a random sample of web drop down menus on some websites and determine the number of times A-Z listings are used appropriately vs. inappropriately.

    My guess? Appropriate use will out-rank inappropriate use by a huge factor. Thus the term ‘usually wrong’ provided by Nielsen would be, well, mostly wrong.

    One thing we both agree on however, and that is each case must be reviewed, using some or all of the parameters I’ve provided, before choosing which type of listing to use.

  4. Your position is compatible with Neilsen’s. He says that generally, A-Z is usually the wrong choice: in most cases, lists should be collated in some other order than A-Z. This sounds perfectly plausible. You say that in any specific case, we should review all relevant information before deciding. This sounds like good advice. But you haven’t said anything about the general case, and so your advice doesn’t conflict with Neilsen’s.

    A couple of your examples seem tangential to your argument. For instance, the Texas Department of Motor Vehicles question listing is awful. If you want information on dealer training, you don’t look under D for Dealer, or T for Training. You look under F for Find. That’s clearly bogus — it seems they couldn’t be bothered thinking through any sort of useful categorisation. It’s a classic example of developers making things easy for themselves and hard for the users. A-Z is not the right choice here.

    The Blue Cross story is an interesting and informative example of good UX principles at work, but it has nothing to do with A-Z ordering.

    In short, you have shown very nicely that an A-Z listing is sometimes the right choice. But even if it’s sometimes right, still it might usually be wrong.

  5. Craig, I should have been clearer in my last, somewhat tongue-in-cheek paragraph: I was praising both of you for taking strong positions designed to produce debate (the word “wrong” is always a conversation-starter). That’s why I used the word “wise” rather than “accurate.” 🙂

Comments are closed.