In Part I of “Design Patterns for Mobile Faceted Search,” I looked at the challenges and opportunities of mobile faceted search. To address the well-known challenge of limited screen real estate on mobile devices, I covered the Four Corners, Modal Overlay, Watermark, and Full-Page Refinement Options design patterns, which maximize the real estate available for search results on a mobile device. This month’s column covers strategies for making people more aware of the filtering options that are available to them, as well as methods of improving transitions between the various states a user encounters in a search user interface.

Teaser Mobile Design Pattern

Teasers are nothing new on mobile phones. Most mobile user interfaces implement this pattern in some manner or another. Consider, for example, the Yelp iPhone application, shown in Figure 1.

Figure 1—Teaser design pattern in Yelp’s mobile app

Yelp displays five complete search results, plus a teaser at the bottom of the page that shows a partial view of the next result set. The Teaser design pattern is very useful in showing that there are more search results below the fold. As I mentioned in a previous column, “Choosing the Right Search Results Page Layout: Make the Most of Your Width,” my usability testing has shown that Amazon is an absolute master of this pattern. Regardless of a screen’s resolution, they always display partial product descriptions and pictures just above the fold. Showing part of what is at the fold is a very effective means of inviting visitors to scroll down and see more content.

Recently, when Microsoft came out with Windows 7 Mobile, they extended the Teaser pattern well beyond just showing the availability of search results below the fold. In Figure 2, an excellent video from Luke Wroblewski’s blog entry “Windows Phone: User Interface Teases & Transitions” effectively demonstrates this concept. As you can see, Windows 7 Mobile shows there is more content available on the left, right, above, and below what’s currently visible on the screen.

Figure 2—Teasers and transitions in Windows 7 Mobile (Compiled by Luke Wroblewski)

In its implementation of the Teaser pattern, Windows 7 Mobile uses lots of very obvious cues, including screen titles that are cut off in midword, partially visible screen widgets, image fragments, and abundant use of animated transitions.

A teaser is a specific application of the more general design principle: fixing imperfection, in which a design immediately engages people by having them fix something that is intentionally not quite right with a user interface or object. In the process of fixing an imperfection, people learn the user interface’s interaction model. Studies have shown that this process of fixing imperfection, or seeking symmetry, is very natural to humans and quite immersive, even to the point of alleviating anxiety and pain in burn trauma.

The problem some may see with the Windows 7 Mobile design is that people cannot win at this particular game of fixing imperfection. There is absolutely no way to fix the user interface to simultaneously show all of a title, widget, or image that exceeds the size of the screen. It’s kind of like a blanket that is intentionally made too small: if you cover your legs, your chest is exposed; pull the blanket up to your chin and your legs get cold. As UX designers, we often seek a kind of minimalistic, authentic beauty and symmetry in our designs, so some designers might find this kind of user interface profoundly disturbing. However, one cannot argue against the effectiveness of this design approach.

The wireframe shown in Figure 3 takes advantage of the Teaser design pattern to show a partial view of some faceted search filters on the right and, thus, expand the available virtual screen real estate.

Figure 3—Teaser design pattern facilitates the discovery of faceted search filters

The search results are on the left in this wireframe; filters, on the right. Following a basic convention of mobile user interfaces, buttons on the right let people drill down deeper into the application’s information architecture (IA) space, while tapping buttons on the left moves them closer to the top of the IA or to the home page. In this example of what Microsoft calls panoramic applications, the mobile device acts as a sort of viewfinder that displays only a small part of a much larger virtual space.

The Teaser design pattern very effectively facilitates discovery through its use of partially exposed screen elements—in my example, faceted search results filters. This pattern also enables people to make rapid transitions from looking at search results to narrowing down the search results, so it is highly suitable for applications in which it is advantageous for people to discover a set of filters quickly and use them often.

Basic/Advanced Parallel Architecture Pattern

Another good design pattern for efficient discovery and use of filters is the Basic/Advanced Parallel Architecture design pattern, depicted in Figure 4. This design pattern is especially effective in applications that offer a fixed set of filters—for instance, booking air travel.

Figure 4—Basic/Advanced Parallel Architecture design pattern

The idea behind the Basic/Advanced Parallel Architecture design pattern is that there are two modes of interacting with filters: basic and advanced. The basic mode lets customers easily engage with your application and dramatically shortens their learning curve. However, people who want an advanced mode of filtering can easily obtain this functionality—in the example shown in Figure 4, by returning to the filter page and tapping the Advanced Search tab.

The parallel architecture part of this pattern comes in when a customer needs to adjust the filter values. For customers who started their query from the Basic Search tab, tapping Back returns them to that tab. For customers who started from the Advanced Search tab, tapping Back returns them to that tab, where they can adjust the expanded set of filters. While the relationships between these three contexts—Basic Search, Advanced Search, and Search Results—are a bit difficult to explain, they offer a very natural interaction model that most people have found very intuitive to use during testing.

The Basic/Advanced Parallel Architecture design pattern offers the added advantage of a flatter architecture. Instead of an advanced mode that requires people to go deeper into an application’s IA, the advanced flow sits in parallel with the basic flow, so both are equally accessible with a single tap.

It is important to note that, in both cases, the search results page has a persistent status bar across the top of the screen, displaying an entire search query, plus any applied filters. Recall that a persistent status bar was one of the features I described in Part I of “Design Patterns for Mobile Faceted Search.”

One application that successfully implements the Basic/Advanced Parallel Architecture design pattern is the Thirsty Pocket iPhone app, shown in Figure 5.

Figure 5—Basic/Advanced Parallel Architecture design pattern in Thirsty Pocket

Notice that, instead of labeling this feature Advanced, Thirsty Pocket uses the label More Search Options. In Thirsty Pocket, the purpose of the advanced search page is to provide options for searching outside the device’s current GPS location, as well as narrowing a search by price. Most people would not use the location filter, and the price filter might, in fact, severely limit the effectiveness of search by too frequently narrowing a search to zero search results. The price filter is a great example of the “ejector seat lever” Alan Cooper so eloquently describes in his book About Face 3. Therefore, this price filter is best hidden on a separate page. In the Thirsty Pocket app, the label More Search Options describes the purpose of the advanced search page perfectly and has the added bonus of being less intimidating.

In another variation on the Basic/Advanced Parallel Architecture design pattern, instead of Basic Search and Advanced Search tabs, Thirsty Pocket presents two buttons on its home page: a prominent Search Near Me button for basic search and a de-emphasized More Search Options button for advanced search options. Tapping More Search Options lets you set filters, then tap Advanced Search to see the filtered search results. Nevertheless, the Thirsty Pocket user interface still preserves the essential features of this pattern, because tapping Modify Search on the search results page takes you back to the appropriate search page—basic or advanced—depending on how you initiated a search.

In Conclusion

A lot of people have asked me about these design patterns and the best ways to use them. Thank you all for your questions. Unfortunately, the answer would go well beyond the scope of even a two-part article. The best recommendation I can make is that you learn as many design patterns as possible; understand why these patterns have emerged, what customer and business goals they support, and what limitations they have. Then, once you understand the unique goals of your own customers and business leaders, you can either choose the right design pattern or combination of patterns or develop your own design patterns that fit the task at hand. Our understanding of user interfaces is still evolving—and for mobile design, still in its infancy. Mobile design patterns and paradigms are just beginning to emerge, but they represent the beginning of something magnificent.

References

Bucolo, Sam et al. “The Design of a Tangible Interaction Device to Alleviate Anxiety and Pain in Pediatric Burns Patients.” CHI ’06 Extended Abstracts on Human Factors in Computing Systems, Montreal, Quebec, Canada, April 22–27, 2006.

Cooper, Alan, Robert Reimann, and David Cronin. About Face 3: The Essentials of Interaction Design. 3rd ed. Indianapolis, IN: John Wiley & Sons, Inc., 2007.

Nudelman, Greg. “Design Caffeine for Search and Browse UI.” Information Architecture Summit 2010, Phoenix, Arizona, April 9–11, 2010. Retrieved April 30, 2010.

—— “Design Patterns for Mobile Faceted Search.” UXmatters, March 8, 2010. Retrieved April 30, 2010.

—— “Experience Design for a Viral Mobile Community.” Net Squared Conference, San Jose, CA, May 2009. Retrieved April 30, 2010.

—— “Choosing the Right Search Results Page Layout: Make the Most of Your Width.” UXmatters, March 9, 2009. Retrieved April 30, 2010.

Wroblewski, Luke. “Windows Phone: User Interface Teases & Transitions.” LukeW Ideation + Design, February 17, 2010. Retrieved April 30, 2010.

In my previous Search Matters column, “Mobile Finding: Turning Limitations into Opportunity,” I discussed how mobile search user experiences differ from those on the Web. In this and my next column, I’ll look specifically at the challenges and opportunities of mobile faceted search. This column covers design patterns for maximizing the real estate available for search results, while the next will cover strategies for making people aware of filtering options.

Faceted search is extremely helpful for certain kinds of finding—particularly for ecommerce apps. Unfortunately, the designers of mobile applications do not have established user interface paradigms they can follow or abundant screen real estate for presenting facets and filters in a separate area on the left or at the top of a screen. To implement faceted search on mobile devices, we need to get creative rather than following established Web design patterns. Join me in exploring the Four Corners, Modal Overlay, Watermark, and Refinement Options design patterns for mobile devices. Following these patterns can move us one step closer to making faceted search a usable reality on mobile devices.

But first, let’s take a look at the challenges of designing mobile faceted search, which include navigational elements that use up precious screen real estate, limited search-refinement options, and the general lack of an iterative refinement flow.

Mobile Faceted Search Challenges

Recall that one of the mobile applications I discussed in my last column was Amazon’s iPhone app. Figure 1 shows its search results screen.

Figure 1—Faceted search in Amazon’s iPhone app

On the Web, Amazon’s faceted search is nothing short of gold standard, so comparing their Web faceted-search experience to that of their iPhone app is instructive. Looking at the Amazon iPhone app in Figure 1, the first thing to note is the amount of screen real estate Amazon has devoted to things other than search results. Their fairly standard iPhone page layout has a navigation bar at the top and a tab bar at the bottom, which together take up about 74 pixels. In all, Amazon has permanently devoted almost 22% of their precious screen real estate to navigation and other features. As a consequence, they can display only three search results at a time. And Amazon is actually fairly frugal with its screen real estate—at least in comparison to other mobile ecommerce applications.

Another interesting issue is that the user’s keyword query—in this case, green lantern—does not appear in its entirety in the application’s navigation bar—both because the user interface displays the query in a really large font and because the query text has to compete with a large search refinement button that is also on the navigation bar. With the screen‘s limited real estate, the query gets cut off, allowing only green… to appear there.

Last, but not least, the only search refinement Amazon presents to customers is By Category—a button that takes a customer to a different category-selection screen. While category is definitely one of the most useful and frequently applied facets, offering category selection alone hardly compares to the rich array of search-refinement options that is available on Amazon’s Web site, as shown in Figure 2. Conspicuously absent are various sorting options, as well as filtering by price.

Figure 2—Search on Amazon’s Web site presents many refinement options

However, in these screenshots, you cannot see one of the most important and striking differences between search on the Web and on mobile devices, which you can understand only in terms of the app’s search-refinement flow. Figure 3 illustrates the differences between Amazon’s Web and mobile search-refinement flows.

Figure 3—The differences between the Web and mobile search-refinement flows

On the Web, Amazon’s search supports progressive, faceted search refinement and exploration as one of its primary goals for the search process. As Peter Morville said in his brilliant book, Search Patterns, “Faceted navigation … helps us learn. Search becomes an iterative, interactive experience where what we find changes what we seek.” For example, on the Web, a customer who starts with the keyword query Nike can narrow down the search results by category (Shoes), additional keywords (Nike Air), product type (Cross Training), and size (12).

In contrast, Amazon’s iPhone app does not preserve search refinement as part of its finding flow. The customer who starts with the keyword query Nike, then refines the results By Category (Shoes) is unable to refine the results further. If a customer attempts to refine a query by adding more keywords (Nike Air), the app removes the customer’s prior category selection (Shoes), doing a completely new search for the query Nike Air and searching all categories. This behavior has the unfortunate effect of interrupting the flow of step-wise exploration and discovery that is central to the faceted search experience. Obviously, there is a real need for better patterns for the step-wise narrowing, expanding, and refining of faceted search results on mobile devices.

Mobile faceted search needs to balance customers’ needs to both refine results and maintain their search-refinement flows against the limited screen real estate and fat-finger issues. We must especially guard against introducing false simple-mindedness in the name of simplicity. Simplicity is a great goal, but as John Maeda said in his masterful book, The Laws of Simplicity, “Some things can never be made simple.” Using the words of Albert Einstein, we have to strive to make the most of the tools available to us and make user interfaces “as simple as possible, but not simpler.” The four design patterns that follow make the most of the available screen real estate, while providing intuitive and useful search-results refinement capabilities and improving the mobile faceted-search experience.

Four Corners and Modal Overlay Patterns

As I described in my column “Search Results Satori: Balancing Pogosticking and Page Relevance,” maximizing the number of results on a search results screen is key to improving information scent and overall finding efficiency. One of the most successful design patterns for maximizing the use of screen real estate is the Four Corners pattern, shown in Figure 4.

Figure 4—Four Corners and Modal Overlay patterns

The Four Corners pattern devotes almost the entire mobile screen to search results. Users can display particular functions by tapping the semitransparent corners, which provide access to filters, the home page, and two additional menus. This pattern includes a thin, persistent status bar across the top of the screen, which displays an entire search query, plus any applied filters, rather than just the few characters Amazon Mobile displays in a large font. Displaying an entire query is highly beneficial, given users’ mobile context of use and the frequent interruptions inherent in the mobile finding experience.

Although the design pattern is called Four Corners and triangles are the most effective shape for the corners, they don’t necessarily have to be triangles. Other alternatives include semitransparent buttons or floating icons. However, for transparent buttons, it’s best to keep icons simple and avoid text labels.

Another pattern that works really well with Four Corners is the Modal Overlay pattern, shown on the right in Figure 4. A modal overlay can display various search-refinement options such as filtering and sorting. As I wrote in “The Mystery of Filtering by Sorting,” it is not necessary to place filtering and sorting in two different areas of a screen. Most people have, at best, just a vague idea of the differences between the two features. This is particularly true for the mobile experience, where screen real estate is at a premium and minimizing the number of taps necessary to achieving a goal is key.

The Four Corners and Modal Overlay patterns work particularly well in combination. In Figure 4, each user interaction on the modal overlay refines the search results interactively. This dynamic interaction model works great, because users can always see where they are and what filters they have applied, and they have easy access to further refinement options, all without ever leaving the context of the search results screen.

Some touch phones—like the iPhone—do not provide an API (Application Programming Interface) for modal overlays, but instead deliver content one screen at a time, with transitions between screens. On such devices, you’ll need to fake modal overlays. You can achieve this effect by rendering the same search results on a new screen—this time with a slightly darkened background—with a modal overlay over the results. When displaying this new screen, an application should minimize any transition to help to preserve the illusion of maintaining the search results context.

Watermark Pattern with the Full-Page Refinement Options Pattern

The latest generation of smartphones—including the Apple iPhone, Palm Pre, and Google Android—provides designers with a whole bevy of novel interaction models. Some of the ways in which people must interact with new smartphones may not be obvious to them, because they may not match their existing mental models or the affordances of devices with which they’re familiar. Nevertheless, once people discover these interactions, they seem surprisingly intuitive and fun, and people do not easily forget them.

In mobile computer games, it is common practice to teach players by allowing them to manipulate their mobile devices through various gestures, including tilting, turning, and shaking them—or even using a device’s accelerometer to mimic the way one would use a magic wand: “Incendio!”

Figure 5—Fun ways of employing the iPhone’s accelerometer in the Harry Potter game

In ecommerce applications, customers’ object in interacting with a mobile device is not typically entertainment, but the efficient acquisition of goods, services, or information—so necessity typically shortens their learning curve. If we want people to be able to perform a nonstandard interaction to use a device’s essential features, the interaction must be fairly obvious—and it must not require them to read instructions they would perceive as obnoxious or distracting. This is where careful use of the Watermark design pattern can be especially powerful.

A watermark is a transparent outline that appears either over or behind the primary content on a screen. In some ways, a watermark is similar to a splash screen, but it is considerably less distracting because watermarks let users see the content on the screen at the same time. Figure 6 shows two variations of the Watermark pattern, plus a full-screen example of a Refinement Options pattern.

Figure 6—Two variations of the Watermark pattern and a Refinement Options pattern

The two examples of the Watermark pattern, shown in Figure 6, demonstrate a circular swiping gesture and a shaking gesture on a multitouch device. In both cases, once a user launches an application, each watermark appears just once or twice, gently dissolving to let the user view the screen’s main content. Watermarks can also use animation—for example, to demonstrate the motion of the circular swipe by displaying one arrow at a time on the screen. However, animation greatly increases a watermark’s visibility, so you should use it with care. Circular swiping emulates the motion of digging into a pile of puzzle pieces to find the right one, while shaking recalls the action of shaking up small, loose objects in a bag or a box.

In each case, the idea behind the watermark is to educate or remind people about the gesture or interaction that is necessary to view a screen containing additional search-refinement features. Once people decide to dig in further or shake up their search results, the application displays a full screen of refinement options, as shown at the bottom of Figure 6. This screen provides an example of the Refinement Options pattern. Again, the thin, persistent status bar across the top of the screen shows the query. Together with the watermark, this helps maximize the real estate the screen can devote to search results, while making exploration intuitive.

With a full screen of refinement options, a user can view four or five of the most popular values for each of the facets and select a facet value with a single tap. This is almost like having the entire array of refinement options on a typical faceted search results Web page. Having the search box integrated into the refinement options page also re-enforces the idea of applying search query terms and facets together as a set, just as they are on a Web page. Together, these patterns let us duplicate the intuitive and efficient step-wise refinement and expansion flow our customers expect from a faceted search results page on an ecommerce site. Using a scrollable screen of facets rather than a modal overlay like that shown in Figure 4 allows us to devote the full screen width to the search results, sprucing them up with icons for popular refinement options and letting customers refine the results with fewer taps.

Does this mean presenting a full page of refinement options is better than a modal overlay? Not at all. A modal overlay displays its refinement options within the context of the search results, while displaying a separate results-refinement screen, along with its associated transition, yanks customers out of the search-results context and presents what some people might think is an overwhelming variety of refinement options.

When discussing design patterns, avoid getting caught up in judging one pattern to be better than another. Each pattern is useful in specific contexts and applications and for particular purposes and types of customers. Design patterns are like a language we can use to communicate with our customers. It helps to have a vocabulary of patterns that is as complete and varied as possible, as well as to use each pattern appropriately to communicate your design message with clarity and precision.

Thinking of porting your Web finding experience to iPhone, Android, or Windows Mobile? Just forget about the fact that these devices are basically full-featured computers with tiny screens. Having gone through this design exercise a few times, I have realized that designing a great mobile finding experience requires a way of thinking that is quite different from our typical approach to designing search for Web or desktop applications. To put it simply, designing a mobile finding experience requires thinking in terms of turning limitations into opportunities. In this column, I’ll discuss some of the limitations of mobile platforms, as well as the opportunities they afford, and share a few design ideas that might come in handy for your own projects.

Understanding Mobile Platforms

One of the challenges of mobile application design is understanding both the capabilities and limitations of each platform. Let’s use the iPhone finding experience as an example. On the plus side, the iPhone has a high-resolution screen, Multi-Touch controls, accelerometer, persistent data storage, cool video transitions, push content delivery, GPS, and a device ID. The benefits of these features have been pretty much beaten to death in advertisements, so I will not discuss them here. On the other hand, the problem constraints and limitations of mobile devices are much more interesting. I have found few sources that discuss these in detail, so in this column, I’ll attempt to describe the most important challenges of designing for the new generation of smartphones—at least as they pertain to finding:

• the difficulty of typing
• the small amount of screen real estate
• awkward touch controls
• the so-called fat-finger problem

The Difficulty of Typing

Searching requires users to type a search string, and typing on a mobile phone is difficult—partly because of the size of the device. In a July 2009 blog post on Alertbox, Jakob Nielsen called the mobile experience “miserable” and reported, “Text entry is particularly slow and littered with typos, even on devices with dedicated mini-keyboards.”

For many users, touch devices like the iPhone exacerbate this problem. Their screens display a virtual keyboard and other buttons. One issue with touch devices is poor visibility, because a user’s fingers must, by necessity, cover buttons on the screen when a user is pushing them. Thus, when pushing a button on a virtual keyboard, a user’s hand obscures his view of the keyboard.

The lack of tactile response on touch devices presents a particular problem when a user is multitasking—whether riding in taxi, using public transportation, or walking down a city street. On any mobile device, it’s very difficult to type when a person is crammed into a bus or subway car and being jostled around. A user’s desire to avoid typing becomes a necessity when using a mobile device while driving.

Another challenge users encounter when searching on smartphones is that they’re likely to lose anything they’ve laboriously typed into a search box if a device receives an incoming phone call, because mobile applications cannot block phone functions. Nor should an application, however useful, be able to prevent a user from receiving incoming calls.

The Small Amount of Screen Real Estate

Mobile screens are, by necessity, small, because a mobile device has to fit into a person’s pocket or purse. The small size of mobile screens limits the number of controls and the amount of content that can appear on them. In that Alertbox post I mentioned earlier, Jakob Nielsen reported, “Unsurprisingly, the bigger the screen, the better the user experience.” According to Nielsen, users’ success rates with touch phones like the iPhone are about double their success rates with feature phones.

Awkward Touch Controls

One of the consequences of mobile devices’ having smaller screens and controls that users must manipulate through touch interfaces is that some controls no longer look like their Web and desktop counterparts. For example, rather than the usual drop-down list or set of option buttons, the selection control on an iPhone is instead a spinning control called a picker, shown in Figure 1.

Figure 1—Three pickers in the Urban Spoon iPhone app

A picker that a user can quickly manipulate with a finger eats up precious real estate, so it’s not possible to show users much outside the picker. Large pickers also place limitations on how users can filter search results on mobile devices. Similar size restrictions exist for all of an application’s touch controls.

The Fat-Finger Problem

The high-resolution screens on better mobile devices—like the iPhone, Android, and Palm Pre—can accommodate fairly high information density. Unfortunately, at the same time, touchscreens limit the on-screen density of controls that users can accurately manipulate with a finger. Thus, placing multiple controls close together on a touchscreen mobile device presents difficulties. This challenge is known as the fat-finger problem.

Typically, an iPhone touchscreen can comfortably support a maximum of only three to five clickable buttons or tabs across a screen. More than this leads to frequent frustrations due to users’ inadvertently pressing the wrong control. While five controls across a screen is the absolute limit for relatively frustration-free mobile computing for people with relatively small fingers, I highly recommend a limit of four or fewer controls. Of course, if one control is bigger than the others, you’ll have to reduce the overall number of controls accordingly. For example, Figure 2 shows the Yelp mobile application, which has only three controls across the navigation bar at top of the screen, but four controls across tab bar at the bottom.

Figure 2—Yelp iPhone app

Because Yelp’s search box takes up a large part of the navigation bar, the designers had to reduce the overall number of controls to three to ensure users’ wouldn’t press the wrong controls with their fingers too often.

Understanding User Experience within a Mobile Context of Use

The challenge in designing mobile applications is the need to accommodate the design constraints and usability challenges mobile devices impose, while focusing on users’ goals within a mobile context of use. Simply duplicating the functionality of a Web application—while trying to work around the mobile design challenges I’ve described—always results in a subpar mobile application. It’s not enough to think: How can I duplicate our Web application’s user experience within the limitations of the mobile platform? Instead, it’s better to start from scratch, focusing on What experience would work best for mobile users? Putting users’ goals first allows a design team to concentrate on the new opportunities a mobile application presents rather than seeing the challenges of mobile simply as barriers to implementing a Web application’s existing functionality.

Next, I’ll present some ideas about how to approach the design of finding experiences for mobile devices in a way that lends itself to taking full advantage of their capabilities within a mobile context of use. These ideas by no means represent an exhaustive catalog of all possibilities. I merely want to provide a few examples that may inspire further exploration as part of your own finding projects.

Preloading Pertinent Search Results

As I discussed earlier, typing on mobile devices is difficult. Plus, a phone call, a text message, or an opportunity to take a picture is likely to interrupt a user’s finding experience at least once. Saving a user’s previous searches is an obvious and simple way of re-engaging users in a finding task and provides useful context when an application first opens.

Unlike Web applications, native mobile applications are persistent, so it’s easy to cache their search results. Cached results load quickly, users’ re-engagement is immediate, and there is little to compete for a user’s attention—that is, at least until another phone call comes in. Some mobile device APIs even enable native applications to detect whether a phone call interrupted a user’s previous session or the user exited an application normally and determine how much time has elapsed since a user last opened the application. These capabilities present interesting possibilities for fine-tuning an application’s welcome-back screen to re-immerse users in their previous tasks or offer pertinent new content and present new possibilities for interaction.

Providing Local Results

On mobile devices that support GPS, Wi-Fi, or other location-tracking mechanisms, you can determine the current location of another person who is using a mobile device, allowing applications to offer location-aware services. Mobile applications with search capabilities can serve highly relevant, fresh results that perfectly match a user’s current mobile context. For example, Loopt is just one of a whole cadre of social networking mobile applications that allow users to track the locations of their friends who are currently nearby and exchange messages with them, get coupons from local merchants, and discover neighborhood happenings, as Figure 3 shows.

Figure 3—Local search results in the Loopt iPhone app

Offering a Value-Added Interpretation of the Real World

Using mobile devices for sense-making in the real world offers one of the most intriguing possibilities for mobile applications. Luke Wroblewski described some interesting possibilities, including augmented reality applications, in his Smashing Magazine article “Enhancing User Interaction with First Person User Interface.”

When it comes to finding, however, the Amazon Mobile iPhone app offers the Amazon Remembers feature, which is a forerunner of many exciting things to come. Amazon Mobile lets customers take pictures of any real world items and add them to their lists of things to remember. Once a user uploads a photo, Amazon figures out what the item is and, if there is a corresponding item available for sale on Amazon, displays that item and sends the customer an email alert, encouraging her to purchase the item. Customers often get a response in a matter of minutes, but the search can take up to 24 hours. This application lays a solid foundation for the idea of a mobile Internet of objects that I described in one of my previous columns, “Brave New World of Visual Browsing.”

Providing Various Sorting Options

As I previously mentioned in my column “The Mystery of Filtering by Sorting,” sorting options are an excellent way of opening up an ecommerce site’s inventory for browsing. One of the best ways of doing this is to provide two or more buttons on a search results screen that allow multiple ways of dissecting an inventory, without ever failing to serve some results. By using sorting options together with geolocation, customers can even avoid having to type in any query at all. As Figure 4 shows, the ThirstyPocket iPhone app lets customers simply press the Search Nearest or Search Newest button to see a sample of local results without having to type anything in the search box.

Figure 4—ThirstyPocket iPhone app

Of course, a customer can always type keywords in a search box to narrow down the results. As I explained in my presentation at the Net Squared Conference in May 2009, using this design pattern lets customers engage with an inventory of items or content immediately, then invest the effort of typing in keywords once they have caught the scent of something that interests them or to refine the search results further. Of course, given the fat-finger problem and a mobile device’s limited screen real estate, we can’t provide more than three to five sort options on the screen at one time. However, as the ThirstyPocket example shows, even a couple of sort options is often enough for customers to begin exploring.

Considering Custom Controls

One consequence of mobile devices’ having a smaller screen is not having the space to create a navigation bar of filters, facets, or categories on the left, providing a key to the properties by which a user can effectively narrow down a result set. Consider, for example, the all-important category filter. If an application were to use the standard iPhone picker, shown in Figure 1, it would obscure most of the search results on the screen. Various applications deal with this challenge in different ways. Some mobile applications have created custom category-filter controls—like those in Amazon Mobile, shown in Figure 5.

Figure 5—Custom category-filter controls in Amazon Mobile

Clicking By Category takes a customer to a different, full-sized screen on which he can select a category. The ThirstyPocket application, shown in Figure 4, uses the same principle for its custom category selector, with an added twist: the visual design of the category selector is reminiscent of the familiar drop-down list, taking full advantage of customers’ existing mental model and helping them understand what behavior to expect.

Changing Search Paradigms

Because of the unique mix of constraints and opportunities that mobile application design presents, this design space is rich with possibilities for changing the existing paradigms for search and finding. Consider speech recognition, for example. While, on the desktop, speech recognition does not yet enjoy widespread popularity and use, mobile represents an entirely different context—where speech recognition can offer an ideal solution. Not interpreting a spoken word correctly on a mobile device might not be quite as big a deal as it is on the desktop, because the accuracy of speech recognition may actually approach, if not exceed, that of typing on a mobile phone’s awkward mini-keyboard. Combine speech recognition with the use of an accelerometer and magnetometer, allowing gestural input, and you have the Google Mobile search application for the iPhone, shown in Figures 6 and 7.

Figure 6—Google Mobile iPhone app

Figure 7—Google Mobile Voice Search on the iPhone

The iPhone application Google Mobile recognizes the gesture of a person’s swinging the phone up to his ear to know when to record a search command. When the user speaks, the search engine accepts and interprets his voice commands, then serves up search results. This user interface implements what is literally a game-changing design paradigm, because its designers have taken the time to truly consider the mobile context of use and map natural interactions like speech and gestures to mobile device functions. As Peter Morville said in his book Search Patterns, “We simply raise our phones to our ears and speak our search, relying on Google Mobile to derive what we want from who we are, where we stand, and what we say…. Like placing your hands under a tap to turn on the water, this is the type of smart design that ‘dissolves in behavior’.”

When it comes to designing for a mobile context, we are just starting to scratch the surface. As Tom Chi of OK/Cancel famously quipped in his interview with Luke Wroblewski, “A well defined-and exciting problem (and its associated constraints) is the catalyst that makes design go.” When we stop thinking about the limitations of mobile platforms and, instead, truly focus on the user goals we are working to support, we might just find those limitations turning into opportunities for redefining how people find, remember, and discover things in their world.

References

Morville, Peter. Search Patterns. Sebastopol, CA: O’Reilly, 2010.

Nielsen, Jakob. “Mobile Usability.” Alertbox, July 20, 2009. Retrieved February 27, 2010.

Wroblewski, Luke. “Enhancing User Interaction With First Person User Interface.” Smashing Magazine, September 21, 2009. Retrieved February 27, 2010.

Wroblewski, Luke. “Defining the Problem: Q&A with Tom Chi.” Functioning Form, April 27, 2006. Retrieved February 27, 2010.

Faceted search has been around for a long time and has become the de facto standard for search on most ecommerce sites. However, filters with numeric values remain among the most confusing, because many sites have not able to design usable numeric filters that people can use in an intuitive manner. Recently, powerful user interface controls called sliders have become all the rage for specifying numeric attributes in finding user interfaces. Unfortunately, in their rush to implement this latest, greatest feature, many companies have not designed easy-to-use sliders. Rather than solving usability problems, poorly designed sliders create even more issues around numeric filter usability. In my experience, the following three usability issues surface most often with numeric filters:

• representing discrete values for aspects as sets of ranges
• inadvertently emphasizing overly constrained filter states
• being parsimonious with inventory information

In this column, I’ll examine each of these issues and present the best practices that solve these problems. 

Representing Discrete Values for Aspects as Sets of Ranges

One of the most common pitfalls with numeric aspect filters is their representing discrete values as sets of ranges. What makes a filter value discrete? Certain types of products typically have widely recognized, discrete values—for example, a digital camera might have a 10MP resolution and a lens with 3X zoom—which can often consist of either whole numbers or fractions. These discrete values tend to be consistent across competing brands and models and rarely change from one vendor to the next.

Many sites offering faceted search do not present discrete numeric values for aspect filters in the most useful fashion. Designers are often tempted to present such values to customers as sets of ranges. Presenting discrete filter values as a set of ranges typically results in a user interface similar to that in Figure 1, showing a camera-resolution filter on a major ecommerce site.

Figure 1—Camera-resolution filter with values presented as a set of ranges

My usability studies have shown that many people have trouble finding 10MP within a 9.9MP-11.9MP range and applying the filter correctly. The problem is a mental model mismatch. People don’t think of 10MP as falling within some range. When customers look for discrete values in aspect filters, they are most often looking for a specific value—for example, 10MP—that forms a critical part of the information scent. Most people typically think of a camera resolution as a specific value—not as a numeric value within some range.

When users try to look for the value 10MP within a range, usability problems arise, because the resolutions 9.9MP and 11.9MP simply do not exist as discrete values. While, numerically, it is true that 10MP lies within a 9.9MP-11.9MP range, this range has very poor information scent and requires additional thinking and interpretation on the part of users, which does not make for intuitive and efficient finding. Having to think hard just to find something causes problems for users.

In my user research, I have often observed that people who are in a rush, distracted, or simply not that great at working with fractions in math often select the wrong range of discrete values, wonder why they are not getting the results they expected, get frustrated, and quickly leave a site to navigate to a more usable one.

Instead of providing ranges, it is much more effective to present lists of discrete values, showing all of the possible values for an aspect filter. If doing this results in a filter with too many options, simply rounding off the value to around 10MP works very well, as shown in Figure 2. (Note that the word around is usually implied. Although it sometimes shows up in a short form such as ~10MP or 10+MP. However, this extra precision is usually unnecessary, as I will explain in a future column.)

Figure 2—Recommended redesign of camera-resolution filter

When a customer clicks a link representing a discrete value, one option is to expand a list of subvalues—for example, clicking an around 10MP link would show all of the available subvalues: 10.0MP, 10.2MP, 10.3MP, 10.4MP, as shown in Figure 2A. Should you bother designing and implementing this? Usually not. Unless your customers are professionals who might be looking for a particular value, they typically would not drill down past the around 10MP link. Is there really that much difference between digital-camera resolutions of 10.2MP and 10.3MP to a typical consumer? So, unless your company’s inventory is extremely large or you have a very specialized clientele, it is often much more useful to present an approximate value like 10MP.

Should you support single or multiple selection for displaying discrete filter values? The answer is always: It depends. Up until a few years ago, guidelines generally recommended the simplicity of single-selection links for discrete values. Today, however, I highly recommend that my clients implement multiple selection for discrete filter values—provided the finding user interface also supports multiple selection. Mark Burrell of Endeca echoed this sentiment in the 2010 UIE Webinar, “Leveraging Search & Discovery Patterns for Great Online Experiences,” in which he emphasized that most ecommerce customers no longer find multiple selection hard to use and prefer the flexibility it offers.

Numeric Sliders for Filters

Numeric sliders have recently become the rage for faceted search. Sliders can add a touch of interactive fun to what many business people refer to as a boring finding interface. Note that boring user interfaces usually work extremely well, because they are intuitive and easy to use, and UX powerhouses like Amazon and Netflix do quite well without any sliders whatsoever.

That said, sliders can be helpful in some applications, because they give people tremendous filtering power when they are implemented correctly. With great power, however, comes great responsibility: Sliders deserve special consideration from designers—precisely because they make it so easy for your customers to screw up and overconstrain their queries, leading to search results that are either incorrect or of poor quality. This, in turn, results in frustrated customers who leave your Web site quickly without buying anything. The two issues I see most often with sliders are:

• inadvertently emphasizing overconstrained filter states
• being parsimonious with inventory information

Let’s take a closer look at each of these issues and what we can do about them.

Issue: Inadvertently Emphasizing Overconstrained Filter States

To examine the issue of overconstrained filter states, let’s take a look at an example—the slider rating control on TripAdvisor, shown in Figure 3. Ratings seem deceptively simple, yet raise a host of usability issues when they are implemented incorrectly.

Figure 3—Slider rating control on TripAdvisor

TripAdvisor has implemented ratings using a double slider control, which presents a perfect example of inadvertently emphasizing overconstrained filter states. This double slider control allows a wide variety of ranges—some of which are much more useful than others when it comes to finding particular items or content of interest. For example, rating ranges spanning only a single star—such as 0-1 stars, 1-2 stars, or 2-3 stars—are simply not useful, because short ranges do not match the mental model of the people using the system.

Most of the time, people click a rating filter to get all of the merchandise above a certain rating—that is, as a way to find higher-quality items in their search results. However, a double slider control overemphasizes the ability to constrain the range from both sides, making it very easy to get this wrong by overconstraining a query.

A much more useful way of approaching this design problem is to use a single slider or a set of links like those shown in Figure 4, which shows ratings as they are currently implemented on Amazon.

Figure 4—Amazon’s more useful implementation of ratings as links

We could further improve the rating control shown in Figure 4. Since the goal of people using this control is to get better-quality items, it is not actually that useful to show only a single star—the lowest possible rating. Instead, it might be more intuitive to replace the single star with the word Any and make that the default filter state. The filter shown in Figure 4 also has the important added advantage of providing vital inventory information. Item counts following each set of stars help people using the ratings filter to clearly understand the consequences of their actions and builds appropriate expectations. Understanding what to expect from their actions lets people be more efficient and effective, leading to higher customer satisfaction and better usability. Next, we’ll explore how we can use sliders to show inventory information.

Issue: Being Parsimonious with Inventory Information

Do all dual sliders emphasize overconstrained filter states? Not necessarily. For some filters dual sliders are entirely appropriate—for example, for price ranges. However, the problem that I described earlier of overconstraining queries and, as a result, providing inadequate information never really goes away. As Figure 5 shows, the dual slider control for the price range filter on TripAdvisor provides no inventory information, so customers would have no idea what the effect of manipulating the sliders might be. Any action customers take might be a hit or a miss—but it is never clear in advance which it will be, because the system simply does not provide the necessary inventory information.

Figure 5—Dual-slider filter for price range on TripAdvisor

Compare the dual slider for price range on TripAdvisor to the price control on Staples.com, pictured in Figure 6.

Figure 6—Price control on Staples.com with multiple check boxes

As I mentioned in one of my previous columns, “The Mystery of Filtering by Sorting,” overconstraining search results by price is one of the most common human errors we see in usability testing for search user interfaces. Which control would you expect to result in more overconstrained queries: a dual slider or a set of check boxes? The control that makes overconstraining results easier is the dual slider, because it gives no clue as to what to expect from a particular action. On the other hand, the dual slider provides the bling many business people crave as a means of differentiating their user interface from the competition.

In this age of highly interactive Ajax interfaces, clicking links or typing in numbers to specify a price range seems so old-fashioned. Is there a control that would provide the interactivity and fun of the slider, yet offer the inventory information so necessary to helping customers make informed decisions?

One approach would be to use a dual slider for setting a price range, combining it with a sparkline that graphically represents the available inventory information for every price in the range. Figure 7 depicts my suggested redesign of the TripAdvisor price filter, which combines a dual slider with a sparkline, showing the available inventory for each price in the range.

Figure 7—Suggested redesign of the price range filter on TripAdvisor

In his book Beautiful Evidence, Edward Tufte describes sparklines as “data-intense, design-simple, word-sized graphics.” Although I have no idea who first thought of combining a slider with a sparkline, I have been recommending this solution to my clients for over four years, and I can claim to have arrived at this idea independently. Unfortunately, so far, most of my fellow designers and the business people with whom I have worked have remained cold to this idea, stating that a slider with a sparkline would be “above and beyond what an average user could understand.” However, my own experience as a user researcher backs the opposite conclusion. Every usability test participant who has seen the user interface in Figure 7 has confidently stated that he or she understood that the sparkline represented the number of items available, eloquently proving once again that “clarity and simplicity are completely opposite of simple-mindedness,” as Tufte said in his book Envisioning Information.

Recently, during the UIE Webinar I mentioned earlier, I was supremely gratified to hear Mark Burrell of Endeca recommend dual sliders with histograms as one of the best ways of showing price ranges. Burrell’s experience with dual sliders was similar to mine: he said that most people have no trouble understanding that histogram bars—a step-wise variant of a sparkline—represent item inventory for each part of a slider’s range and that histograms clearly help people to avoid overconstraining their queries.

In Conclusion

In our age of rapid development, new ideas and new controls hit the Web almost every day. Even as companies are struggling to overcome the old challenges of numeric faceted filters, designers are innovating interesting new controls with which they can solve those challenges. One such control is the slider, which gives unparalleled power to customers. However, successfully offering this power requires careful UX design in order to avoid potential pitfalls. Again, the following three issues with numeric filters surface most often:

• representing discrete values for aspects as sets of ranges
• emphasizing overconstrained filter states
• being parsimonious with inventory information

Designers’ uninformed use of sliders for filters often exacerbates these issues. In this column, I have presented best practices as well as some novel approaches to help designers overcome these challenges. But there is simply no substitute for empathy and solid qualitative user research. Understanding why customers do certain things is extremely important in designing effective user interfaces. No matter what metrics we have, we cannot improve our user interfaces unless we understand what our customers’ goals are and why people fail to reach them. With every new filtering innovation, it becomes ever more important to stay focused on our customers, with patience, empathy, and understanding.

References

Morville, Peter, and Mark Burrell. “Leveraging Search & Discovery Patterns for Great Online Experiences.” UIE Virtual Seminar, 2010. Retrieved February 7, 2010.

Tufte, Edward. Beautiful Evidence. Cheshire, CT: Graphics Press, 2006.

—— Envisioning Information. 4th ed. Cheshire, CT: Graphics Press, 1990.

In my last installment of Search Matters, “Cameras, Music, and Mattresses: Designing Query Disambiguation Solutions for the Real World,” I presented several design strategies for query disambiguation.

Unfortunately, most sites do not make sufficient use of this pattern and some that do use it design and implement it incorrectly.

Show Me More

The idea behind the More Like This pattern is very simple: within each group of items representing a particular category from a catalog or accompanying each item in search results, provide a prominent link or button with a label that is some variation of More Like This ». Of course, the devil, as they say, is in the details. Figure 1 shows one of the more successful implementations of this pattern, the Amazon.com home page.

Figure 1—Amazon’s successful implementation of More Like This

On close examination, this design yields a few important lessons:

• Make group organization simple and obvious. Each group title should be prominent and simple, not inviting deep thought or much examination. The words and the color treatment of group titles should flow in a way that lets customers easily skim groups. Their titles exist to clearly separate various groups and direct the eye to the items they contain.
• Focus on helping customers make decisions. Although the items in each group are themselves fairly relevant, the subtle focus of the whole page is not on finding exactly the right item on the current page, but instead on displaying some relevant entry points to an unfathomable cornucopia of relevant choices on Amazon.com that each group exemplifies.
• Format groups differently from search results. The items in each group have a horizontal layout, in a gallery format, so it’s easy to differentiate them from the search results, which typically appear in a vertical list.

The most important thing to keep in mind is that all of these design features support a single key goal: to help customers find what they want by providing the information scent and motivation that makes their navigational decisions easy and intuitive. Let’s examine what happens when sites ignore these lessons by looking at some other implementations of the More Like This design pattern.

Make Group Organization Simple and Obvious

Any cognitive friction around the organization and format of different groups makes navigational decisions more difficult. Instead of making decisions primarily on the basis of the information scent embedded in each group’s content, customers must try to understand why certain groups have a different format—a confusing and usually futile endeavor. Figure 2 shows one example of such an over-designed More Like This page, displaying search results for the query Thai Restaurant San Francisco on Yelp.com.

Figure 2—Yelp’s over-designed More Like This page

The aim of this page is not to present all possible choices, but to help customers make a quick choice of the method that would best let them explore a bevy of Thai-food restaurant recommendations. Unfortunately, the purpose of the page is far from obvious. Observe that the page has no fewer than seven different groups, each with its own branding and formatting, in an overly complex, nonstandard layout. Together, all of these differences create a cacophony of results, making it very hard to compare the different groups without studying them in detail. Too much fancy formatting and too varied a layout hinders customers’ ability to make navigational decisions quickly, which should be the page’s primary aim.

Contrast this page with the Amazon home page in Figure 1, in which each group has the same simple format, drawing the eye to the content. On Amazon, with the aim of making customers’ navigational decisions quick and easy, groups’ simple, intuitive organization lets customers easily scan them and decide whether to explore a particular group further.

Last, but not least, the first section, Best of Yelp, does not even look like the other groups. It is the only one that contains breadcrumbs. As a rule of thumb, on More Like This pages, breadcrumbs should appear above all of the More Like This groups, not within any of them.

Focus on Helping Customers Make Decisions

To help customers make navigational decisions within More Like This groups, is more content better? The answer is: it depends. Amazon’s More Like This pattern includes one subtle, but very powerful feature: if a customer increases the page width, the number of items in each horizontal group in this liquid layout increases, so higher resolution screens and wider windows show more items in each group. I’ve already discussed the benefits of liquid screen layouts in my column “Choosing the Right Search Results Page Layout: Make the Most of Your Width,” so I will not cover that topic in detail here. Be aware that this kind of horizontal expansion is no simple trick, so don’t expect it to work out of the box, as it were. Though, if you can afford the labor that developing the right CSS stylesheet involves, this is one feature well worth implementing.

On Amazon, more content is better. Partly, this is because every additional item a wider screen shows might be the one that catches the interest of a prospective buyer. But most important, showing a greater number of items presumably creates better information scent for each group, helping to motivate customers to explore a group in more detail.

What if there were a way to show even more content in each horizontal group on a page? One way to do that would be with a carousel widget that scrolls from side to side like that shown in Figure 3, on a category page on Netflix.com.

Figure 3—Netflix’s More Like This carousel pattern

In many ways, the Netflix implementation of a More Like This carousel is very similar to the Amazon implementation of More Like This. The page organization is simple and obvious, and the groups look different from the search results. There is, however, one important difference: clicking an arrow at either side of each group scrolls the contents of the group to the right or left, displaying an additional four items. Does the addition of this carousel widget help or hinder the primary goal of the page?

Well, the jury is still out on that. However, I can tell you that the carousel takes a customer’s focus away from making a decision about which group to explore and instead invites the customer to linger on the page a bit longer, while exploring each of the groups. This is not necessarily a bad thing. But keep in mind that, even though a carousel can show customers twice as many items in a group, it still shows them only 8 to 10 items in total. This is in contrast to the hundreds, if not thousands of items a customer could see in a complete list of search results by clicking the See all > link for a group. Providing larger secondary targets—the left- and right-pointing arrows—also makes people think they should click them. If customers do click the arrows, there is a pretty good chance they might become mesmerized by the fancy scrolling interaction and miss the fact that this page is supposed to help them choose a subcategory to explore. A customer might look at the eight items the widget presents, find none of them relevant, and leave the site, thinking those eight items are all there were to see in a particular subcategory.

A carousel might seem like a fun, beneficial feature, but remember that the main purpose of the More Like This design pattern is to direct customers to explore the entire cornucopia of items under each category—that is, to select a category to explore—not to find exactly the item they want among the 8 to 10 items each carousel displays. To drive people to explore, the overall design of each group must be fairly Spartan, so customers can make their decisions quickly and move on to exploring the main body of the search results. If fancy group formatting or Ajax carousels make customers disregard the more important More Like This buttons, such a page fails to meet its primary objective.

If you are still thinking about using carousels for your More Like This groups, consider that Netflix has one of the best recommendation engines in the world and can usually select very relevant items to include among the 8 to 10 options their carousels display. Amazon, which also has an exceptional recommendation engine, tried incorporating carousels for all of their groups, but now uses the carousel feature more sparingly, presumably because the results underperformed the Spartan group design, which is optimized for quick scanning.

Although a typical More Like This page does not warrant the use of carousels, in some circumstances they can be appropriate. One place where a carousel widget might come in handy is in the Two-Dimensional More Like This design pattern I’ll discuss later. But first, let’s take a quick look at what happens when groups look just like the actual search results.

Format Groups Differently from Search Results

So far, we’ve covered More Like This pages that, for the most part, look pretty different from search results pages. What happens if you apply the More Like This pattern to subgroups containing vertical lists of results?

The query disambiguation page on TripAdvisor.com, shown in Figure 4, does a particularly poor job of helping customers choose either a means of exploring or a category of items to explore.

Figure 4—Search results for the query San Francisco on TripAdvisor

This is clearly a page that does not inspire or invite further exploration. There is much that really does not work on this page, including the bevy of no less than ten generic, monochrome icons; intermixed, ungrouped results that seemingly show up in random order; the small number and poor quality of results; and the absence of any pictures. My list could go on and on. However, as bad as all of these problems are, customers might actually overlook them, if the page did not also commit the cardinal sin of any grouping presentation: making the groups on the page look like actual search results.

In all of my user research, I have found that nothing is more confusing to participants or generates more sheer exasperation in customers than dealing with a page that looks and feels like a search results page, but is actually some kind of fancy, special-purpose page containing groups, whose aim is to take customers to actual search results. TripAdvisor’s query disambiguation page for San Francisco, shown in Figure 4, is exactly such a page. Take care to avoid this approach in your designs.

Two-Dimensional More Like This Pattern

How could we improve TripAdvisor’s query disambiguation page for San Francisco? In my user research, I’ve discovered that precisely because it is possible to show only a very small number of items for each topic—typically 4–5—customers’ do not expect to discover specific items of interest on this kind of page, but instead to see a representative set of items in each specific subcategory.

What if, instead of showing a fairly random sample of 4–5 specific hotels, attractions, or restaurants, we showed 4–5 generic items that represent a more granular subdivision, or aspect, of each category? In other words, instead of getting item details for 4–5 specific hotels, customers could drill down to one of the most popular hotel types—for example, Family, Business, Bed and Breakfast, or Boutique. Clicking one of the 4–5 generic hotels in the Hotels group would take customers to a search results page with results filtered by both the category Hotels and a type of popular hotel, making it possible for them to rapidly perform a visual, two-dimensional drill down into their area of interest. This is the idea behind my Two-Dimensional More Like This pattern. Figure 5 provides an example, showing my suggested redesign for the TripAdvisor query disambiguation page.

Figure 5—Suggested redesign for the query disambiguation page

This Two-Dimensional More Like This pattern is particularly appropriate for cases where the content includes thumbnail images and both the category and aspect are clearly defined—as they are in the case of hotel types, for example. In many cases, brand is another popular search facet that provides an easily recognizable secondary aspect you can apply across a group.

Interestingly, I have found that there is little need for secondary aspects to be consistent across categories. Let me give you an example. In my last column, “Cameras, Music, and Mattresses: Designing Query Disambiguation Solutions for the Real World,” I showed how HomeDepot search results for the query drill fell into three categories: Drills, Drill Bits, and Hammer Drills. If we were to apply the Two-Dimensional More Like This pattern to that example, it would be highly appropriate to use Brand—for example, Black & Decker, Dewalt, and Makita—as a secondary segmentation for the Drills category. On the other hand, it would be best to segment the Drill Bits category by the type of drill bit—for example, Sets, Drill Bits, Philips Screwdrivers, and Flat Screwdrivers.

In my user research, I have found there is little problem with applying different secondary aspects to different More Like This groups on a page, so long as they make some intuitive sense. I have theorized that this is because the customers using a search results page usually quickly narrow the results down to a specific category, by following its information scent. So, they need to figure out the organizational pattern behind a single subcategory in which they are interested rather than to compare different subcategories to one another.

As I mentioned earlier, the Two-Dimensional More Like This pattern is an excellent use case for using a carousel widget. As Figure 5 shows, if there are more aspects—like brands or hotel types—than can fit in the 4–5 thumbnails that can appear on a page at once, you can safely hide the less popular aspects on the carousel. In this case, the expectation would be for customers to first narrow down their search to a specific category, then invest more attention to selecting the appropriate secondary aspect. If customers cannot find the secondary aspect they want by scrolling from side to side with the carousel, they can still click the group’s See All » link to view all of the search results for an entire category.

The Two-Dimensional More Like This pattern works particularly well for resolving ambiguous queries and creating category pages on which customers can clearly express search refinements by clicking a single label or image. However, it would be less appropriate for information categories that are somewhat arbitrary and less clearly defined. In the Netflix example shown in Figure 3, is there really that much difference between the categories Anime Action and Anime Sci-Fi? Applying the Two-Dimensional More Like This pattern to the recommendations on the Amazon home page would also be problematic. What single image could you choose to represent the categories New for You or Recommended for You and their subcategory aspects?

One difficulty in designing a Two-Dimensional More Like This page is selecting which specific items to show as representatives of each aspect. I recommend that you select the first thumbnail in the set of search results that would appear if a customer clicked a subcategory plus an aspect. Selecting the first thumbnail in a list of search results as the representative aspect performs particularly well for sorting a search result set by Best Selling or Best Match, because the first item usually provides an excellent representation of the constrained results set. Sometimes customers click a specific item not to select a subcategory and aspect, but simply because they liked the item in its thumbnail for some reason. Placing the item that represents both a subcategory and an aspect first in a set of search results satisfies both the display criteria and the I-want-this-specific-item mental model perfectly.

What if a specific criterion like price is important in a customer’s decision making when selecting an item on a Two-Dimensional More Like This page? In this case, as Figure 5 shows, it is perfectly appropriate to use a sorted list format—for example, Priced from $XX.XX, which shows the lowest-priced item in a given set of search results first—even if the set of search results customers see once they click a link is sorted by Bestselling or Most Popular.

Our language is limited and imperfect. Typically, people type search queries quickly and with little forethought, so queries are definitely less than perfect. When a customer constructs a query that may have more than one meaning, a good search user interface provides tools to help the customer define the query in less ambiguous terms, so the search results more closely match the person’s intention. This process is known as disambiguation, and best practices for effectively supporting the disambiguation of search queries are the subject of this column.

Recently, I came across a new search engine—which shall remain nameless—that promised a combined search and browse approach to finding products. I was curious, so I put this new search application through its paces by typing the query Canon. In addition to results for cameras, the search engine displayed results including the company’s profile for investors, Pachebel’s Canon—a form of music—and, to my great surprise, a Canon mattress and a Canon ottoman, which the products section featured prominently.

Unfortunately, these search results represented a fairly typical situation that occurs when a search application does not correctly understand the meaning of a query. Especially frustrating was the fact that the user interface did not provide any tools to help people to refine their queries and, thus, improve the quality of their search results. The only way people could improve their search results was by typing more keywords into the search box, which takes both thought and work—two things any busy, distracted Internet user can do without.

In her recent column on UXmatters, “First, Do No Harm,” Pabini Gabriel-Petit quoted Jef Raskin: “A computer shall not waste your time or require you to do more work than is strictly necessary.”

What can we do to remedy this situation? In this column, I’ll discuss three simple strategies designers of search applications often use to help people resolve ambiguous queries:

1. Show related searches.
2. Select a default category automatically.
3. Prominently display a category selector.

Let’s look at the advantages and limitations of each approach.

Showing Related Searches

Related Searches is a fairly standard module on mainstream ecommerce sites and search engines. Figure 1 shows an example on Amazon.com.

Figure 1—Amazon’s Related Searches module in the search results for Canon

Most search applications track how people modify their original queries, then apply an algorithm to this tracking data to display the most common modifications to queries in a Related Searches module. As Figure 2 shows, people who typed Canon augmented their queries by typing Canon camera, Canon lens, or other more specific queries.

In her recent book, Search User Interfaces, Marti Hearst discusses the related searches algorithm in detail and quotes several impressive studies that document the effectiveness of this approach. Indeed, in my own user research, most people have found the alternatives a Related Searches module presents very relevant and clicked them to find what they wanted—mainly because of their very good information scent. Related searches often seem to present the keyword combinations people had unconsciously desired, but could not quite formulate themselves without more thought.

Essentially, a Related Searches module eliminates the effort and thought creating a good query requires, making a search user interface more intuitive and enjoyable, without using a great deal of precious screen real estate. The back-end algorithm for this module does require a fair bit of infrastructure to implement the query tracking. However, some APIs like Yahoo! BOSS are now available to make the job easier.

In my studies, I discovered people had a positive response to most suggestions revolving around modifications of their original queries, because of their very strong perception that the recommended queries represented the combined wisdom of crowds—as with tagging and folksonomies. However, many people I observed were highly suspicious of any orthogonal queries that recommended competing brands or products.

For example, in Figure 1, a related query that recommends Nikon in response to the query Canon would quickly cause people to become mistrustful and question the value of all suggestions in the Related Searches module. For this reason, I recommend that search user interfaces concentrate on showing only query modifications in their Related Searches modules. Show competing products and brands in a different module instead—for example, in a sidebar. You can easily limit search results to those that contain the original keywords with a simple grep command. Done properly, a Related Searches module is a very useful device for helping people disambiguate their queries and improving the quality of their search results—thus, creating a better finding experience.

Selecting a Default Category Automatically

Displaying only search results belonging to a specific category or topic by default is another very common strategy for disambiguation. An ecommerce system can determine what category or topic to display by default, using supply data, demand data, or some combination of the two. Supply data refers to the number of distinct product or content entries currently available in a site’s inventory that match a certain keyword query. In contrast, demand data refers to the number of people who selected products from a specific category after typing the same keyword query during a given period of time.

To automatically determine the default category using supply data, an ecommerce system might fairly easily measure the number of items currently in the inventory that match a specific keyword query. For example, if 70% of the items that match the query Canon belong to the Digital Gadgets category, the search algorithm might automatically display results in the Digital Gadgets category whenever a customer submits that query—omitting results in Music and other categories. A more sophisticated way of doing the same thing would be to use demand data to determine how many customers in the last week proceeded to view or buy items in the Digital Gadgets category after searching for Canon versus products in all other categories; then, if the number exceeds a certain threshold, to display the Digital Gadgets category of results by default.

What are some advantages of selecting a category of results by default? Automatically interpreting the query Canon as a request for Digital Gadgets, then showing a Canon brand product catalog lets Best Buy create a very compelling visual browsing experience, complete with custom subcategories and aspects, as shown in Figure 2.

Figure 2—Best Buy search results for Canon

I’ll cover best practices for product catalogs in more detail in future columns, but for now, notice how extremely committed this category selection is in comparison to the Amazon search results for the same query, shown in Figure 1. Not only is the Canon brand selected, there is no trace of any results matching Canon for other types of products on the site. This provides a great user experience for someone looking for electronic gadgets, but it could be very confusing to someone looking for something in the category Music, for example.

A final word of caution: Automatically selecting a default category or topic is a very committed action, so communicating how to undo this action can be a challenge. Evidence shows that half-hearted measures for indicating the presence of other results on a Web site do not work particularly well.

Here is an example. When doing some user research for one of the top Web retailers, I tested a memorable user interface that automatically selected the category Shoes when a user typed the query Nike. However, unlike the Best Buy user interface shown in Figure 2, the user interface I was testing also provided a prominent link to let users undo the category selection—Not looking for Nike Shoes? The usability test task involved finding Nike bags. The study’s significant finding was that the vast majority of participants did not discover or click the link Not looking for Nike Shoes?

I theorized that this might have occurred, because the link did not contain any information scent for the category Bags for which participants were searching. Additionally, when people quickly scanned the page—as most people tend to do—mentally processing this negative statement was fairly difficult. Dynamically displayed links for other Nike categories that started with strong keyword scent words might have been much more successful—for example, Other Nike products: Bags, Shirts, Pants, Jackets, More….

The moral of this story? If you do commit to using a default topic or category as part of your disambiguation strategy, make sure it is for a good reason, based on metrics that help your company meet its business goals. Decrease your company’s risk by providing a clear way out of the default category, using links that start with prominent keywords and provide strong information scent for other popular categories or topics matching a keyword.

Prominently Displaying a Category Selector

In addition to a Related Searches module, most ecommerce sites also provide a category selector widget. Notice that, in Figure 1, Amazon presented its available categories in a navigation bar on the left. While this strategy is pretty standard and widely accepted, some sites go still further, emphasizing their categories as a strategy for disambiguation. For example, compare the Amazon search results shown in Figure 1 to The Home Depot search results for drill, in which the categories appear prominently above the search results, as shown in Figure 3.

Figure 3—Home Depot search result categories

There are many good reasons for emphasizing categories as part of your disambiguation strategy. Prominently displaying categories above search results clearly signals to customers that their queries may have more than one meaning on a site. Providing categories also lets customers correctly customize search results by selecting a category, then aspects or other finding tools matching a specific category. For example, the category Drills might feature the aspects Power Tool and Hand Tool, while the category Drill Bits might have the aspects Size and Hardness. Showing categories above the search results—as opposed to in a navigation bar on the left—also allows the display of longer category names without wrapping, providing improved information scent without compromising readability.

However, as we can see in Figure 3, showing categories above the search results can also lead to some pitfalls. If you look carefully at Figure 3, you’ll notice that The Home Depot search results actually feature two separate category selectors—one in the navigation bar on the left, the other above the search results—which could be pretty confusing. The two sets of categories are not the same, leading customers to wonder where they should click and why. The links in the navigation bar are sorted alphabetically—not in their order of popularity—which is suboptimal. It’s not clear how the categories above the search results were selected or sorted or why they appear in such a prominent location. As a result, the categories above the search results feel a bit like a Band-Aid—letting customers jump directly to popular areas of the site.

Providing additional information scent for the main categories on The Home Depot site would serve that function better. Customers might expect each of the subcategories to be a separate link—so, for example, clicking Power Tools would display a level-two subcategory—but this is not the case. Instead, each link constitutes an independent selection, forcing customers to navigate three or four levels deep into the site hierarchy and make committed category selections before they’ve had a chance to see the wider inventory. Since each link exposes just a subset of a very detailed hierarchy, these links force customers to learn the site’s category structure to be able to make informed decisions.

Last, but not least, hierarchical categories can introduce a huge readability problem if many categories start with the same keywords. For example, seven of The Home Depot’s categories start with the keywords Tools & Hardware > Power Tool…, making it hard to distinguish the Tools and Accessories categories from one another and make the right choice.

This page displays far too many categories—fourteen, in all—above the search results. Plus, despite the generous amount of screen real estate the page allots to categories, the category names are too long, so they wrap. Together, these two factors push the actual search results way down on the page, placing all products below the fold at many screen resolutions. My research has indicated that many customers might be confused by this. Instead of scrolling, they might feel the site is forcing them to click a category before they can see any actual search results.

A better choice would be to commit to showing the first-level categories just once, above the search results, while avoiding wrapping, and augmenting them with additional keywords—as space allows—to help customers make the right choices. As Daniel Tunkelang notes in his book Faceted Search, showing four to seven values for each aspect seems like the sweet spot. His finding is well supported by my own research. Sticking to Daniel’s recommendation, I would reduce the number categories on The Home Depot site at least by half. The resulting user interface might look something like that shown in Figure 4.

Figure 4—Redesigned Home Depot results, with an expanding category widget

Figure 4 demonstrates an Expanding Category Widget design pattern that is very similar to a search user interface I recently redesigned for an enterprise client. The original design had very long category names, forcing categories in the navigation bar on the left to wrap three or four times.

Placing categories above the search results solved the problem, effectively handling long category names and providing a great way of disambiguating complex queries. However, be aware that even this improved user interface distracts customers from shopping. Prominently displaying categories practically forces your customers to deal with the complexities of your user interface and the category hierarchy on your site rather than actually shopping.

For a less ambiguous query, a good middle ground might be to automatically collapse the wide category widget to its standard size in the navigation bar, allowing users to drag its handle if they want to expand the category hierarchy again, as shown at the bottom of Figure 4. Judicious use of a simple animation, showing an expand/collapse transition might be very helpful in improving the attractiveness and usability of this widget. The Expanding Category Widget is a useful design pattern that might be worth exploring when determining your category-driven disambiguation strategy.

Conclusion

It is a fact of life that ambiguous queries are fairly common. In a post-Google world, most people expect a very high degree of relevance for search results after typing just a single keyword. They are often surprised and dissatisfied when your search user interface does not deliver. In this column, I’ve covered a wide range of tools and design patterns that can help reduce the thought and effort necessary to convey the desired meaning of a customer’s ambiguous query.

Numerous variations on disambiguation strategies exist, from adding a fairly simple Related Searches module to completely redesigning your search results page, following the Expanding Category Widget pattern. However, this column has not provided an exhaustive list of design patterns for query disambiguation. It is my sincere hope that this installment of Search Matters will inspire you to seek more information and explore new ideas for query disambiguation. It is only through trying out new ideas and careful experimentation that you can find the optimal approach for your Web site and your customers. Your customers depend on you to help them find what they are looking for. Do not fail them.

In my next column, I’ll cover another powerful, but underused design pattern, More Like This, which—among its many uses—can also be very helpful in query disambiguation.

In this installment of Search Matters, we’ll continue our discussion of ads in search results. If you missed it, read Part 1, which covered these best practices:

• Integrate your ads’ appearance with the rest of your site.
• Make sure customers can easily distinguish ads from content.
• Keep ads relevant and appropriate.
• Understand how your customers interact with ads.

In Part 2, we’ll discuss the following best practices:

• Understand what makes a good ad.
• Limit cannibalization.
• Provide ads for internal merchandise instead of third-party advertising.
• Pay special attention to ads on pages that appear if there are no search results.

Let’s dig in!

Understand What Makes a Good Ad

Based on the findings of eyetracking studies, we’ve seen that users spontaneously pay attention to things that are

• concrete
• actionable
• not like marketing

These findings apply to both typical search results and third-party ads. Thus, a good ad is concrete and to the point rather than full of generic, marketing jargon. For example, it might present a picture of the actual item being advertised or indicate clearly that there’s free shipping rather than saying something like We have thousands of cheap items in stock! As shown in Figure 1, the ads on Cars.com are actually pretty good.

Figure 1—Good ads with appropriate content on Cars.com

This ad on Cars.com follows an important principle: It is highly relevant to the search results on the page, calling customers’ attention to the specific item in the search results. Even though the ad at the top of the page is a banner ad, it doesn’t have the drawbacks of typical banner ads—such as presenting irrelevant content or being positioned too far away from the main content. The price quote, showing good value, is prominent and provides a good call to attention. The presentation of the ad is very straightforward, with concrete information and little marketing language or visual noise.

Is it enough that an ad’s content be appropriate? While content is important, it is just one aspect of the marketing message. Another important principle of ad appropriateness is keeping an ad’s style appropriate to the topic. Contrast the style of the Cars.com ad to the ad on the Yahoo! Movies page, pictured in Figure 2. Even without seeing the flashy animation, it is clear that the ad is quite vibrant.

Figure 2—An appropriate ad style on Yahoo! Movies has bling

An ad with this animated style is highly appropriate if we expect customers to be getting ready for an action-adventure movie experience complete with special effects. In this case, tasteful, yet somewhat loud movie ads, providing links to previews, are not only appropriate, but expected. This example shows how carefully chosen ads can form a large part of a Web page’s useful content, making it unnecessary for Yahoo! to work so hard at keeping the page exciting. The ad in Figure 2 calls out a specific item within a set of search results that appears on the page.

When it comes to hosting ads, it definitely pays to get creative. In our research, we learned that the way an ad is presented matters a great deal in how customers perceive both the ad and the site hosting it. Creative, interesting presentations get extra points, especially if they are unobtrusive. For example, the peel-the-corner ad on EddieBauer.com, shown in Figure 3, is a great example of an ad that draws attention and invites participation, without obnoxiously dominating an experience.

Figure 3—Getting creative with an interactive, peel-the-corner ad

Anecdotal evidence suggests that most people do not mind interacting with this kind advertising and consider peel-the-corner style ads acceptable or even positive on a wide range of sites—if the ads contain appropriate content.

What kind of customer action should an ad invite? In Figure 1, the ad on Cars.com asked customers to purchase or consume a specific item on the same Web site—for example, purchase a Lexus. The ad in Figure 2 invited customers to participate in an offline interaction they will perform after viewing the ad—see a movie after reading a movie review and referring to a theater schedule on Yahoo! Movies. These are both examples of passive ads—that is, they do not invite customers to click, so much as capture eyeballs, with the purpose of instilling a subliminal desire to select a specific item among many that a site offers. This mirrors a classic bricks-and-mortar advertising strategy, in which certain companies pay stores to feature their brands on banners and place their products in prominent locations.

However, the Web is a very dynamic medium, leading to all sorts of novel advertising models. One advertising model involves carrying your competitors’ ads, advertising similar products for sale on a different Web site or in another store. On the one hand, customers’ clicking these ads generates marketing revenue. On the other hand, this marketing revenue literally eats into the sales revenue from products and services your Web site provides. Thus, competitors’ ads cause what is known as cannibalization.

Limit Cannibalization

Generally, carrying competitors’ ads is a dangerous and losing proposition, because cannibalization involves many hidden costs that are often hard to quantify. Most companies spend a lot of money on advertising, to bring customers to their own Web sites. If customers click a competitor’s advertisement, not only is there an opportunity cost—because those customers do not buy your own service or product—there is also a hidden cost—you’ve wasted your marketing dollars.

In addition to cannibalization’s threatening your bottom line, the experience of your customers’ clicking your competitors’ ads is likely to be disconcerting to them, leaving them less satisfied with your site as a result. Most competitors’ ads just dump customers on the home page of their site or, at best, on a search results page, leaving them to figure out how to navigate to the one item they want—for instance, to compare a competitor’s price for an item they were looking at on your site. Thus, clicking an ad almost always involves extra work for your customers. For this reason, if your customers are getting the results they want on your site, most do not click a competitor’s ad—unless

• the ad makes a very attractive offer or
• the competitor’s brand is stronger

Some sites understand their customers’ reluctance to do extra work and consciously decide to exploit this fact by hosting competitors’ ads—boosting their revenue without much cannibalization of their own sales. The key to doing this successfully is making rational judgments regarding which ads to host, based on your overall site revenue strategy and hard site traffic statistics. Some successful strategies for carrying competitors’ ads include the following:

• making competitors’ ads look like advertising—Customers deliberately ignore such ads, and they get very few clicks. This strategy works great if you get paid for ad impressions, not by the number of clicks.
• carrying only ads for weaker brands—This limits the number of customers who click away from your site.
• not allowing ads to display competitors’ prices—Displaying the actual prices for specific competing goods or services encourages customers to click ads.

The competitor ad on Yahoo! Cars, shown in Figure 4, provides a good example for all three of these strategies.

Figure 4—Competitor advertising on Yahoo! Cars

Another interesting strategy involves completely embracing the experience of offering the best marketplace price from whatever source and being fully committed to always carrying ads from various competitors. You can see one example of this strategy on Buy.com, which we discussed in Part 1 of this column. Buy.com shows prices from eBay and Dell, as well as the price for the product from its internal search engine. This strategy works well in any of the following cases:

• Your site has the best price.
• Differences in price are not significant.
• Competitors’ prices do not give a complete picture—for example, items for sale are used or have high shipping costs, but their ads do not communicate these details.

Provide Ads for Internal Merchandise Instead of Third-Party Advertising

Amazon.com uses an interesting variation on this strategy, carrying the same items from both Amazon’s own store and their online marketplace, as pictured in Figure 5.

Figure 5—Ad for the Amazon marketplace

This is a win-win situation for both Amazon and their customers. Amazon collects rich fees from sellers in their marketplace, so they make money regardless of which items customers purchase. Having a marketplace also benefits consumers by providing multiple, highly relevant shopping options without presenting anything that looks or feels like an ad. This strategy is very tricky, because Amazon is essentially competing with their own sellers. However, despite the challenges, this strategy has been quite successful—in large part because Amazon has been very proactive, constantly adjusting their price points and the inventory of items they carry to keep their marketplace thriving and provide customers with the comfort of knowing they are getting very good prices from a brand they trust.

Amazon’s strategy reflects a more general principle: Whenever possible, providing and upselling merchandise to customers is almost always better than hosting third-party ads. There are many good reasons to host your own merchandising rather than third-party ads, including the following:

• control—No matter how carefully you screen third-party ads, undesirable content sometimes slips through, harming your brand and sending your hard-earned customers to your competition.
• visual design—Despite your best efforts at ad customization, third-party ads look like, well, ads. On the other hand, you can fully integrate merchandising into your site, make the best use of your available space.
• specificity—Third-party ads are often not very specific to a site’s own products—nor would you want them to be, because that would drive your customers to the competition. On the other hand, you can make merchandising as specific as necessary, because in this case, all ads ultimately drive traffic back to your own site, where transactions convert directly to your bottom line.

Merchandising simply offers a better user experience. People like specific, targeted content that does not look like an ad, but helpfully provides ideas and choices.

Pay Special Attention to Ads If There Are No Search Results

When there are no results, search results pages are an especially dangerous place to host advertising. All of the negatives of displaying competitors’ ads that we discussed earlier become further exacerbated on a no search results page. Such a page is generally devoid of your own content, making third-party ads their primary calls to action.

As I described in my earlier column “Starting from Zero: Winning Strategies for No Search Results Pages,” no search results pages indicate to customers they’ve over-constrained their query and need to reformulate or broaden their search. Displaying third-party ads on a no search results page invites customers to abandon their natural behavior of iterative search in favor of following a fresh information scent. This interrupts the finding flow on your site and dumps your customers directly into your competitors’ greedy hands.

Unfortunately, marketing folks often simply fail to grasp the iterative nature of the finding process and the critical role the no search results page plays in helping customers reformulate their queries. Some of them like to put ads on no search results pages, justifying their viewpoint by saying something like this: We are simply meeting the customer’s needs with our ads. The customer would leave our site anyway. We are just helping him and making some money along the way.

This thinking demonstrates a dangerous and fundamental lack of understanding of the differences between how search engines and ad-hosting services should work. Search engines that are internal to a Web site typically combine keywords, using an AND operator. Thus, they look for items or content that contain all of the keywords a customer provides. This practice often leads to over-constrained queries, limiting the finding conversation between a person and a Web site.

Now, let’s contrast search engine results with hosted advertising results. Search engines providing hosted ads have no requirements for fidelity or specificity—the rules by which a site’s own search engine usually abides. By using an OR operator instead of AND, hosted ad servers can cherry-pick ads, displaying only the most relevant ads or those matching the most profitable keywords in a customer’s query, and ensure the ad server never fails to produce results. Also, when an ad only approximately matches a query, you can replace the ad’s title with dynamic text that exactly matches the keywords in the query.

For example, in one particularly memorable usability study, a task asked participants to find a ticket to a performance of the Nutcracker in San Francisco, during the holidays. After reading the task, one participant typed the query Nutcracker Ballet in San Francisco December 1-31st, which, of course, matched no results on the site. However, the ad hosting engine produced a page similar to that shown in Figure 6.

Figure 6—Ads matching the query Nutcracker Ballet in San Francisco December 1-31st

Upon seeing the no search results page displaying these ads, the participant promptly clicked an ad and bought a ticket from a competing site, even though cheaper tickets were readily available on the site we were testing. If the participant had relaxed her query, she would have discovered this, but instead, the third-party advertising appearing on the no search results page led her down a different path. When there are no search results, that is the wrong place and time for displaying third-party ads.

As this story demonstrates, it is very easy to lose customers on no search results pages, which represent a critical juncture in a conversation between a customer and a Web site—an opportunity for the customer to reformulate a query and gain a deeper connection with your brand. Host third-party ads on your no search results pages only as a last resort. We recommend you carefully calculate the cost of doing so. Which click would generate more long-term revenue: the ad or the tools for query reformulation? Which click would ultimately engender more long-term loyalty and a better relationship with your customers?

In Conclusion

In his keynote speech at the 2008 Business Of Software conference, Seth Godin famously quipped, “Marketing is too important to be left to the marketing department.” The same goes for hosting advertising. UX professionals should be actively involved in ad-hosting decisions to make sure your company’s golden goose continues to thrive, well beyond the current quarter.

We want to encourage you to use these ideas as a starting point for developing your own comprehensive merchandising and ad-hosting strategies. Making the tough choices that are necessary and getting creative with ad hosting is not easy, but it is an absolute must if your company is to survive and thrive in the current economic environment.

Conflicting demands make many UX professionals think of ads as a necessary evil. Customers frequently go out of their way to say they hate ads, while marketers always seem to try their hardest to stuff as many of them as they can on each search results page on your site. This leaves many UX design professionals caught in the middle, trying to balance the ad equation—and frequently failing to fully satisfy either customers or marketers. For this 2-part column, I’ve teamed up with advertisement and eyetracking research guru Frank Guo to present real-world strategies for successfully integrating ads into your search results. The goal is making money without unduly turning off your customers.

Don’t Kill Your Golden Goose

Internet ads have gone full circle—from the Ads are great! Now, we can make money on anything! mania of the early 1990s to the Ads are dead post-dot-com bust philosophy. Currently, the notion that ads are a legitimate way of boosting revenues on an ecommerce site seems to be making a comeback—in part, because this is a tough economy for many etailers. Every penny counts, and every stream of potential revenue demands careful consideration. However, research and experience show that, for an online business to succeed and thrive, it is important to temper any temptation to load up on ads and get a quick revenue boost this quarter with broader, long-term considerations—like user experience, customer loyalty, and brand attributes. If you are not careful, trying to squeeze out a few more eggs can easily kill your golden goose.

However, following a few simple guidelines—while listening carefully to your customers—can help your business make money with ads, without compromising either long-term customer loyalty or your brand image. Here are the best practices we’ll discuss in Part I of this column:

• Integrate your ads’ appearance with the rest of your site.
• Make sure customers can easily distinguish ads from content.
• Keep ads relevant and appropriate.
• Understand how your customers interact with ads.

Integrate Your Ads’ Appearance with the Rest of Your Site

In recent years, Google has emerged as the industry-leading supplier of ads on the Web. Indeed, Google ads make it easy—and often profitable—for you to sign up and host ads on your Web site. Most of the time, the ad content is fairly well targeted, so many retailers, bloggers, and developers of social networking sites have taken advantage of the opportunity to host Google ads. Unfortunately, few have made the effort to customize the boxed ads to make them look like the rest of their site. Figure 1 shows the social networking site Fishing.net, which carries Google ads just as they come “out of the box,” without any customization.

Figure 1—On Fishing.net, Google ads have no customization.

Neglecting to customize third-party ads is, of course, easier, but there are consequences to this approach:

• Customers frequently perceive your entire site as cluttered and disorganized.
• Customers mostly ignore the ads, because they look different from the rest of your content.

This is a lose-lose situation. Your customers lose out, because their experience of your site is degraded. Your marketers lose out, because customers click their ads less often—if at all.

Experience shows that a small amount of visual design and programming effort that makes your ads look like the rest of your site can yield tremendously positive responses from your customers. They stop seeing ads as clutter and instead perceive them as content. Google, of course, provides a superb example of this strategy in practice, as shown in Figure 2.

Figure 2—Google.com sets the standard for ad placement and integration.

Google has carefully crafted their customer experience, paying strict attention to everything from page balance and spacing to tweaking even the smallest visual design elements. In a recent Q&A with G. Hotchkiss, Marissa Mayer, Google’s VP of Search Products & User Experience, described how replacing the box that used to contain ads on the right side of search results pages with a vertical line separator improved ad traffic, because of the ads’ closer integration with the content.

Make Sure Customers Can Easily Distinguish Ads from Content

When taken to the extreme, the guideline to better integrate ads and content becomes a design antipattern. Customers’ being unable to distinguish ads from content becomes especially painful and disruptive when a Web site carries a large number of ads. For example, on Fishing.com, as pictured in Figure 3, there are so many closely integrated ads that they become virtually indistinguishable from the content—to the extent that it becomes difficult to understand what service the site actually provides.

Figure 3—Ads on Fishing.com are overwhelming and confusing.

However, even if the number of ads on your site is not overwhelming, customers can have difficulty distinguishing them from content. One very common problem is providing different types of search results—some of which stay on the same site, while others take customers elsewhere. (We should note that surprising customers about where links go is never a problem for Google or other Web search engines, because both their ads and their search results take searchers to other sites.)

On the other hand, for destination sites that sell their own merchandise or provide a branded service, while also hosting third-party ads, it is important to positively differentiate between ads, legitimate content, and featured—that is, paid—results. Autotrader.com, which is pictured in Figure 4, mixes many different types of search results together, so it’s hard to tell what clicking each result might do. Orange results are actual, “organic” search results, while the slightly padded results in blue are really paid advertisements.

Figure 4—Autotrader.com mixes many different types of search results.

When clicking a search results link takes searchers somewhere surprising or, by mistake, they click a featured result, their reaction is often very negative. Yet, to the extent that a link satisfies a customer’s need, clicking it can be a very positive experience—finding exactly what the customer was looking for. The key to solving this problem is grouping the different kinds of content, while making it clear what are paid advertisements or featured results and clearly differentiating paid content from organic search results through subtle, but telling visual cues.

Colin Ware, in his book Information Visualization: Perception for Design describes so-called preattentive attributes, which involve the early stage of visual perception that occurs mostly below the level of conscious thought, at a very high speed. He distinguishes four categories of preattentive attributes: form, color, spatial position, and motion. We can apply the grouping strategies for all four categories of these attributes to ads, typically using line size, shape, hue, and enclosure to subtly differentiate ads from content. Motion applies mainly to animated ads, which can be an appropriate differentiation strategy, depending on their content. (We will discuss this further in Part 2.)

Google’s search results provide a great example of subtle preattentive differentiation. As shown in Figure 2, Google displays three kinds of results on search results pages:

• ads at the top—Their subtle, yellow background hue differentiates them from the organic results.
• organic search results in the middle—These are the actual content.
• ads in the column on the right—Their placement and narrow column format differentiate them from the organic results, plus a vertical line sets them off.

Overall, the results on this page fit together and flow really well, while the ads’ formats subtly, but unequivocally differentiate them from the content.

Customers’ ability to effectively differentiate various types of content diminishes as the numbers of different types and sources of content appearing on a page increase—even when the content is grouped appropriately and visually integrated, using the site’s colors and fonts. At some point, a search results page simply reaches its point of saturation, and it becomes impossible for customers to tell the different types of search results apart. When doing usability studies for a major etailer that provided ten different types of search results, Greg found that most test participants could not distinguish one type of result from another when scrolling through search results pages. Thus, the experience became overwhelming for people using the site, who were often frustrated, because they never knew where they were about to go when they clicked a search result.

Buy.com, pictured in Figure 5, is another site that is oversaturated with different types of search results and ad content. There are at least 13 different formats for third-party ads on a single page! No surprise, the site rating service Internet Retailer commented, “The site all but overwhelms its visitors with information and options.”

Figure 5—Buy.com’s different types of ads and third-party content are overwhelming.

For your search results pages to be effective, they must display only a limited number of different types of content and ad formats. Otherwise, with no clear guidance, your main content gets lost and customers become confused, because they are overwhelmed by the number of things competing for their attention. The key to integrating ads into your search results, without destroying the finding experience is becoming really clear about what generates the most site revenue. Is it the ads? Or the content? Does it depend on the query? Have you made the costly mistake of serving as many ads to your top customers as you do to window-shoppers and people who just happen to drop by? Once you’ve answered these key questions, you must remain disciplined and stay focused on your core earning potential. Though you can provide occasional, helpful third-party content on the side—particularly, if it helps a customer make a decision.

For example, can you tell whether Buy.com makes more money if people buy the headphones shown on the page—or instead investigate how to make their sandwiches moist and juicy with Best Foods Mayonnaise? Or maybe Buy.com really rolls in the dough when people buy their headphones from eBay or Dell instead? From their page content and layout, it is impossible to tell. One gets the impression that Buy.com may itself be somewhat confused about where the bulk of their revenue is coming from.

Keep Ads Relevant and Appropriate

While many people are, at best, only dimly aware of ads, some ads are so toxic that hosting them can damage your brand perception and destroy the entire user experience of your site. While everyone, no doubt, has his or her own list of the most annoying advertisements ever, one of Greg’s favorite examples is the infamous animated popup ads featuring the X10 spy cam that became popular in the early 2000s. The ad was, at once, so annoying and so ubiquitous, that X10 bears the dubious distinction of having been one of the first companies to get people to register on the company Web site just to opt out from seeing their ads for 30 days. Figure 6 shows one version of the X10 ad.

Figure 6—The infamous X10 popup ads were some of the most annoying ever.

Even though ads may not be utterly obnoxious, they can nevertheless be completely inappropriate. Ad placement is often the key. FoxNews.com, which is pictured in Figure 7, shows just how insensitive and inappropriate some ads can be.

Figure 7—The insensitivity of inappropriate ads on FoxNews.com is striking.

The news story on the page describes the accidental death of a Marine, yet ads for yellow teeth litter the page. Putting politics aside, how do you think the dead person’s family felt when viewing this story? How about all the other families who have fathers, husbands, sons, or daughters serving overseas? Please, make sure your ads are appropriate to the content on a page. One way of doing that is to conduct user research and learn how your customers react to particular ads.

Understand How Your Customers Interact with Ads

Although usability studies and field research can give us important clues about how people interact with ads, eyetracking research can be especially helpful. Eyetracking studies help us to understand how customers perceive and interact with ads, then, depending on our objectives, design ads intelligently—to either catch or not catch their attention. Let’s look at an ad user experience that is different from the typical user experience. Customers definitely don’t come to a site just to look at ads, so they pay attention to them only in a spontaneous, but not intentional way. Thus, it’s hard to ask them about whether they paid attention to ads, because they probably won’t be 100% sure. Eyetracking can fill this gap in our knowledge.

By meticulously tracking all eye movement during a test, you can tell whether and how a participant pays attention to the ads on a Web page, what visual search pattern he uses, and how he either skips or focuses on particular information on the page. (Note—When running eyetracking studies, spontaneity is key. If you interrupt a participant, ask questions, or have a broken prototype that doesn’t let a participant interact with it naturally, your eyetracking data gets contaminated with all sorts of noise.)

So, what does eyetracking methodology tell us about ads? First and foremost, we’ve observed the well-documented banner blindness—that is, customers typically ignore static banners—which are often located at the top of a Web page—even if they are of a ridiculously large size like those on Tutorialized.com, which are shown in Figure 8. Though research clearly shows people usually ignore banner ads, the banner ads on that site take up the entire area of the page above the fold, negatively impacting the user experience and forcing customers to scroll.

Figure 8—Banner ads on Tutorialized.com take up the entire window.

As Jakob Nielsen wrote in his August 20, 2007 Alertbox, “The most prominent result from the new eyetracking studies is not actually new. We simply confirmed for the umpteenth time that banner blindness is real.”

At this point, you may ask What makes a good advertisement? An excellent question we will answer—along with providing some other useful guidelines—in Part 2 of this column on UXmatters next month. Be sure to stay tuned for our thrilling conclusion!

Recently, Office Depot redesigned their search user interface, adding attribute-based filtering and creating a more dynamic, interactive user experience. Unfortunately, Office Depot’s interaction design misses some key points, making their new search user interface less usable and, therefore, less effective. That’s the bad news. The good news is that the Office Depot site presents us with an excellent case study for demonstrating some of the important best practices for designing filters for faceted search results, as follows:

1. Decide on your filter value-selection paradigm—either drill-down or parallel selection.
2. Provide an obvious and consistent way to undo filter selection.
3. Always make all filters easily available.
4. At every step in the search workflow, display only filter values that correspond to the available items, or inventory.
5. Provide filter values that encompass all items, or the complete inventory.

By following the attribute-based filtering design best practices this article describes, you can ensure your customers can take care of business without having to spend time struggling with your search user interface.

1. Choose either drill-down or parallel selection.

There are two basic ways of selecting values for filters: drill-down and parallel selection. Ignoring the various modalities of the many derivative mechanisms for these primary modes of selection, the two basic ways of specifying a value for a filter essentially boil down to two choices: links and check boxes.

A link is the simplest mode of filter selection. By clicking a link, a customer can either select a single value for a specific filter or drill down a level in a taxonomy, like a category or department hierarchy. Amazon.com, shown in Figure 1, provides one of the best examples of a search results user interface that uses links to indicate filter value selections. Links usually indicate a straightforward equals condition—for example, I want to narrow my search results to Department = Books—as they do on Amazon.

Figure 1—On Amazon, links let customers indicate a single filter value

In contrast to links, which let customers indicate a single filter value, check boxes let customers indicate parallel selections of multiple filter values, limiting the scope of search results to those that match them. The Kayak.com search user interface, shown in Figure 2, uses check boxes to limit search results to specific airlines. As on Kayak, check boxes typically indicate an additive OR condition—for example, I want to narrow the search results to: Airline = American OR Delta OR United.

Figure 2—Kayak airline selector uses check boxes to select multiple values

Links and check boxes complement one another very well. Links are great where customers need to display multiple levels in a hierarchy—for example, multilevel category drill-downs. Check boxes are great for selecting one or more options for attributes like brands or sizes. Unfortunately, not all Web sites use these two value-selection paradigms correctly.

As shown in Figure 3, the Office Depot user interface uses check boxes for indicating value selections, leading customers to expect a parallel selection paradigm, in which they could indicate they want to search multiple price ranges by clicking several check boxes. For example, to find chair mats that are priced from $0 to $100, you might expect to be able to select the price filter’s first two check boxes. Thus, after clicking the $50-$100 check box, you would expect to retain the ability to select the $50 and below check box.

Figure 3—Office Depot mixes the drill-down and parallel selection paradigms

However, once a customer selects the $50-$100 check box, the Office Depot search user interface does something completely unexpected. It drills down into the $50-$100 price range and removes the $50 and below check box, breaking the affordance of a group of check boxes that should let customers select multiple, parallel OR values for a single filter. Instead, the user interface now displays $50-$60 and $60-$70 ranges, while still displaying the range of $50-$100 the customer originally selected as one of the available OR values. Mixing the drill-down and parallel selection paradigms results in a very confusing search user interface.

Another confusing thing about the Office Depot user interface is that it does not provide any obvious way to deselect filters and return them to their original, pre-filtered state. This brings us to the next best practice: providing an obvious undo mechanism.

2. Provide undo for filter selections.

Both Amazon and Kayak—pictured in Figures 1 and 2, respectively—provide a clear and consistent way to undo a filter value selection and go back to All or Any for a specific filter. For example, Amazon provides a link to Any Department. Notice that the link is offset to the left, and a less than (<) symbol helps customers understand that this particular link goes back.

Under Airlines in Figure 2, you can see that Kayak solves the undo problem by letting customers click an All check box to view flights from all of the available airlines. Note that the Kayak undo is slightly less clear than that in the Amazon user interface, primarily because the All check box is neither highlighted in any way nor separated from the rest of the check box options. Instead Kayak highlights the airline that offers the best price. While not standard, this is hardly something that would cause undue confusion.

On the other hand, the Office Depot user interface, shown in Figure 3, does not, at first glance, provide an obvious way of undoing the selection of the $50-$100 price filter. There is actually a mechanism that lets customers do this, but it is not at all obvious. The way for a customer to return to Any price is to deselect all filter selections. While this design may be completely valid from a computer’s standpoint, deselecting all check boxes does not effectively communicate the concept of All Prices to a person using the user interface. It would be much more effective to provide an All or Any option.

There are sites that use the deselect-to-undo paradigm successfully—one fine example is Yelp.com. However, sites that use deselect-to-undo typically take special steps to ensure consistency in the filter value options that do not change based on other filter selections. For example, instead of removing options, the Yelp user interface makes certain filter values unavailable to indicate lack of inventory availability. Items that are unavailable appear dimmed, as shown in Figure 4.

Figure 4—Yelp ensures consistency in its deselect-to-undo paradigm

However, in cases where consistency is difficult to achieve, it is a far safer option to use Any or All to undo filter selections, particularly when a filter’s value options are dynamic. As shown in Figure 5, when a user deselects filters out of sequence on the Office Depot site, even more confusion results.

Figure 5—On Office Depot, the order in which filter options are deselected matters

The Office Depot user interface is particularly sensitive to the order in which a user deselects the check boxes for a single filter. For example, the price filter option $50-$100 simply disappears if a user deselects it before deselecting the $50-$60 filter option. A customer would have to click the browser’s Back button twice to undo both selections, because there is no other way to return to the default state. Typically, all of the available filters and options, plus the data that appears on a page can change after each click. The key is to avoid completely removing options in the same filter where a click took place.

3. Always make all filters easily available.

Please don’t misunderstand me. I am not saying all filters should be visible at the same time. It is perfectly acceptable to collapse filters to just a label, providing a single link like View All Filters, or to display previously selected filtering options in a unique way. However, if at different steps in the search workflow, filters start randomly disappearing from the search user interface with no way to bring them back, bad things start to happen very quickly.

Figure 6 shows what happens to the Office Depot search results when a user selects the option Red in the Color filter. As you can see, once a user selects a color, all of the other filters disappear entirely.

Figure 6—On Office Depot, filters disappear randomly

Does this system behavior make any sense? Let’s say I want to buy a red chair mat. Under Color, I can select the Red check box and view a great variety of red chair mats. Is there really a compelling reason for me to do anything else with the color filter? Will I, as a true connoisseur of all things red, perhaps drill down into shades of red for my chair mat, such as Burgundy, Farmhouse Red, or Ripe Tomato? Not very likely—at least not for chair mats. Instead, I am much more likely to say, okay, these are all of the red chair mats. Great! Now, can I select a mat from a known brand? Or perhaps I would like a budget-priced mat, so I need to further constrain my search by price. Or maybe I am a stupendous fan of the Ohio Buckeyes and want to find a chair mat with my team logo, leaving fans of the Michigan Wolverines green with envy. Using other filters to continue massaging my red chair mats query is a far more likely scenario.

Alternatively, let’s consider a different use case: I just selected Red, and now I can see a bunch of red chair mats. Almost immediately, I realize these red chair mats are almost, but not quite, entirely unlike the mat I actually want to buy. So, I immediately attempt to seek out two things:

1. a way to undo my most recent selection
2. a way to review what other filtering options I might want to use

Unfortunately for both these use cases, when I selected a color, the Office Depot search user interface unhelpfully removed all other filters. This action would be tantamount to Kayak’s removing all other filters like departure and arrival times, number of stops, price, and class whenever a customer selected a particular airline. How helpful do you think that would be to someone trying to find a flight? Removing any filters completely can be detrimental to customer success and is, therefore, not recommended. If you really feel compelled, for some reason, to minimize the screen real estate you devote to filters, you can collapse individual filters or hide their options temporarily under a More Options link.

4. Display only filter values that apply to the available inventory.

I touched on the idea of showing only options and links that actually point someplace useful in my first UXmatters column, “Starting from Zero: Winning Strategies for No Search Results Pages.” At every step in the search workflow, any visible filtering options should reflect only the inventory that is available. This is dependent on a customer’s previous actions—both the keywords in the original query and the other filtering selections. As shown in Figure 7, the chair mat color selections expand to reveal tasty color choices like Maple Pale and Cherry Spice. Selecting Maple Pale and Red together should give us both red and Maple Pale chair mats. Unfortunately, the user interface does not seem capable of supplying Maple Pale chair mats, because, when the page reloads, it displays the same five red chair mats.

Figure 7—Office Depot shows color selections that are not applicable to a query

This idea seems really obvious, but quite a few otherwise first-rate sites seem to miss it. Often, the underlying issue is technical. On many sites with Ajax search user interfaces, pages retrieve and recompute all the summary data for all filter values in response to each change a customer makes, so speed becomes a primary concern. For many legacy systems, the demands of just-in-time data delivery are simply too much. At the end of their development cycle, developers are often frustrated to find that, despite their best efforts, their aging systems simply cannot deliver the speed necessary to retrieve all the filtering options, plus the items themselves, plus the data for pagination, in a single Ajax call. In such an event, rather than redesigning the system from scratch to address the root cause of this problem, developers often end up taking shortcuts—either

• caching all of the filters or
• providing a generic list of values that may or may not be applicable to the specific query at hand—like the color values on Office Depot

5. Provide filter options that encompass the complete inventory.

An equally important point is that we must always strive to design every filter to include a list of options that encompasses the entire available inventory. Let me give you an example. As you can see in Figure 7, the color filter in the Office Depot search user interface does not display the most common chair mat color, which is clear. Therefore, once a user clicks any of the color options, the site can display only a fraction of the available inventory—in this case, only 16 of 23 available items. The seven chair mats that are clear have no color attribute, so customers can never choose to view only those items using the Office Depot user interface. Instead, by specifying a color value, they unknowingly remove clear mats from consideration. This system behavior is clearly not beneficial to someone looking for a chair mat—especially if they are looking for the most common variety.

I am not sure what Office Depot’s reason for omitting clear from their color options might be, but it is reasonable to assume that the clear chair mats have the color attribute of empty, or null. Therefore, it is not possible to group these items under a valid color filter attribute. Fortunately, one fairly straightforward way to resolve any missing or inconsistent data is to include Other or All Others as a filter option. The SQL condition corresponding to Other should specifically include any items for which the color field is empty, or null, enabling customers always to view the entire inventory by using some combination of the available filter selections.

Often, there are more options for a given filter than you can show at once. Typically, best practice tells us to show 4 to 7 options for each of the filters. If there are too many options to display, it is often desirable to hide the rest of the options in a More Choices panel or popup. In this case, Other might be an option that shows up only once a customer is viewing the additional options. Specific queries dictate where to show each particular option. In our example, given the popularity of clear chair mats, I think Other should be at or very near the top. However, regardless of their order, it is critical to include all of the options that encompass the complete inventory. This is especially important in cases where there is no All or Any undo option—as in the Office Depot search user interface. Otherwise, we risk making some of the most popular items on a site unfindable.

In Conclusion

Faceted search user interfaces are fairly new, and potential design pitfalls abound. Fortunately, there are a few relatively simple and straightforward design best practices that should help designers to minimize cognitive friction and create search user interfaces that are easy to understand and use.

In this column, I’ve presented five best practices for designing filters for faceted search results. Of these, I think the first—choosing either drill-down or parallel selection—is the most important. If you choose your filter value-selection paradigm correctly, you are already half way there.

Whichever filtering paradigm you decide to implement, be sure to test your search results user interfaces thoroughly with both your potential and existing customers. I cannot overemphasize the importance of testing search user interfaces using realistic tasks. If your budget allows, it is best to avoid predefined search tasks. Instead, study how real people find items they are actually interested in—preferably in an environment where they would normally do the kind of searches you want to study. When designing search user interfaces, field studies are a crucial tool for continually making and validating design improvements—not just something you do as a formality at the end of a project.

When the Web began, pages were mostly text, but today, everywhere we look, it seems like image content is taking over the Web. The ubiquitous use of digital cameras and improvements in the picture quality of mobile phone cameras has likely helped this phenomenon along. The shift toward content that is primarily visual introduces new challenges and opportunities for developing intuitive and powerful user interfaces for browsing, searching, and filtering visual content. To help me cover this important topic, I’ve asked Ahmed Riaz—Interaction Designer at eBay and physical interaction design enthusiast—to contribute his insights and ideas.

Introducing Visual Browsing

What, you may ask, is visual browsing? Loosely defined, visual browsing user interfaces are those that let people navigate visual content—that is, search for content using pictures. We will discuss four types of visual browsing:

1. browsing images or photos using text attributes or tags—for example, on Google Images and Flickr
2. using images in queries to describe attributes that are hard to describe with text—as on Like.com and Zappos
3. using images to facilitate wayfinding and navigation in the real world—such as on Google Maps and Photosynth
4. collecting and browsing images on mobile devices—like on RedLaser and NeoReader

Each of these approaches to visual browsing supports different goals for searchers and introduces its own unique challenges, design directions, and best practices, which we’ll explore next.

Browsing Images Using Text Attributes or Tags

While there are many image-centric sites, the majority of the Web’s image content is on either Google Images or Flickr. No discussion of visual browsing would be complete without mentioning these two sites. When it comes to finding images that are published somewhere on the Web, Google Images is the site most people go to. Google uses a proprietary algorithm to assign keywords describing each image, then index and rank all of the images in their enormous database. They also capture a thumbnail of each image and store it along with the source URL.

While Google Images is undeniably a very useful site, it exemplifies some of the challenges of automated image finding. The primary mechanism for finding images is a text-only query, which perfectly demonstrates the impedance mismatch that exists between text and image content. Because no single keyword can adequately describe all of the pixels that make up an image—and, most of the time, the keywords that surround an image on a page provide an incomplete description at best—Google Images interprets search queries very loosely. Typically, it interprets multiple keywords using an OR instead of the usual and expected AND operator. As a consequence, it omits some keywords from the query without letting the searcher know explicitly what is happening.

On the one hand, because Google Images interprets search queries in a such a loose fashion, it rarely returns zero search results. On the other hand, image searches sometimes give us strange and unpredictable results, while making it challenging to understand what went wrong with our queries and how to adjust them to find the images we’re really looking for. Figure 1 shows an example—the search results for Africa Thailand elbow Indian.

Figure 1—Google Images search results for Africa Thailand elbow Indian

Judging from the search results, Google Images seems to have omitted keywords randomly from the query, without indicating to the searcher it has done so. Of course, one could argue that getting such poor results for a query is an example of garbage in/garbage out. However, we think this example demonstrates that the keywords Google takes from the surrounding content on a Web page—and possibly some Web pages that link to it—do not always match an image. While an image may technically be located at the intersection of Africa, Thailand, elbow, and Indian, this by no means guarantees that the image would actually show Indian warriors demonstrating the perfect Mai Thai elbow-strike technique on the plains of the Serengeti.

Plus, a computer-generated index usually makes no distinction between the different possible meanings of an image’s surrounding keywords—for example, a search engine can’t tell when an image appears in juxtaposition to text to add humor. Nor do automated indexing engines handle images well when they have purposes other than providing content—such as images that add visual appeal or bling or those for marketing or advertising. Such image content is, at best, orthogonal to its surrounding text.

Image search filters that are based on text are, at the moment, not particularly robust. For example, Figure 2 shows Google Images search results for iPhone with some filters applied. As you can see, the resulting images have little to do with the iPhone.

Figure 2—Google Images filtered search results for iPhone

As a photo-hosting site, Flickr enjoys the distinct advantage of having people rather than computers look at, interpret, and tag the images they create and view. Users can search for images by their tags. While Flickr is not a perfect image-searching system, it provides an entirely subjective, human-driven method of interpreting and describing images. Why is that valuable?

According to Jung’s conception of the collective unconscious, we can assume all individuals experience the same universal, archetypal patterns. These archetypes influence our innate psychic predispositions and aptitudes, as well as our basic patterns of human behavior and responses to life experience. Likewise, at the root of all folksonomies is the inherent assumption that people—when we look at them collectively—tend to respond in similar ways when presented with the same stimuli. Simply put, people looking for images of cats would be quite happy to find the images that a multitude of other people have taken the time to tag with the word cat.

As Gene Smith describes in his book, Tagging: People-Powered Metadata for the Social Web, folksonomies and tagging usually work quite well—in most cases, nicely resolving the kinds of issues that occur in Google Images due to auto-indexing, which we described earlier. Unfortunately, the human element also introduces its own quirks along the way. Take a look at the example of a Flickr image shown in Figure 3.

Figure 3—One of the most interesting Flicker images tagged with iPhone

Among almost 2 million Flickr images tagged iPhone, Flickr considers this image one of the most interesting, with 785 favorites and 37 comments. As you can see at the bottom of the page, one of the problems with having a community-driven popularity algorithm is that some people post nonsense comments just to be able to say they’ve added a comment to a popular item.

Another problem has to do with tags that are strictly personal. For example, tags like GOAL:EXPLORE! and hawaalrayyanfav have meaning only to the person who actually tagged the item—and little or nothing to the population at large. On the other hand, adding tags like funny and humor is actually quite useful, because computers don’t recognize humor at all. In fact, humor is the aspect of image description where algorithm-based search results fail most spectacularly. We’ll talk more about folksonomies, controlled vocabularies, and searching for tags in future columns. For now, we’ll just say that neither way of finding images—through text tags or associated surrounding text—is perfect, and there is a great deal of room for improvement, perhaps through combining both search approaches, using a single algorithm and a user interface that does not yet exist.

Using Images in Queries

We think one of the most intriguing aspects of visual browsing user interfaces has to do with the mismatch between images and the text algorithms or people use to describe them. They highlight the inherent limitations of finding and managing images through text-based queries. Typically, a person sees an object, determines what to call it, then tries a few keyword searches based on that interpretation. Adding a photo to a query lets a searcher bypass the interpretation step and just let the computer see what the query is about.

On the Web, one of the more successful examples of image-based search is the shopping site Like.com, which lets searchers use images as part of their queries to describe the attributes of items that are difficult, if not impossible, to describe precisely using text alone.

While Like.com has some issues when it comes to presenting good entry points into their enormous inventory for people who like to browse, once you find an item you’re interested in, the search engine is absolutely phenomenal at picking up visually similar items. For example, Figure 4 shows the results for an image-based search that is based on a picture of a striped shirt. The algorithm is smart enough to explore useful and popular degrees of freedom, deviating from the starting image, while keeping the zeitgeist of the original image intact.

Figure 4—Like.com image-based search results

In addition to doing a great job of exploring the possible degrees of freedom from the original image, the system lets shoppers fine-tune their queries to match a specific part of an image by zooming in on the part they’re interested in. This is very similar to selecting part of an image on Flicker and tagging only that part of the image. For example, a user could tag a person in an image as friend and a snake in the image as giant anaconda, while naming the entire image The last-known picture of Billy—hopefully adding the tag humor, while he is at it. As images become more ubiquitous on the Web—and generate evermore layers of meaning—being able to select and annotate only part of an image will become more and more important in a visual browsing user interface.

The Like.com search algorithm can relax different attributes, including color, pattern, and shape, as well combine image searching with text attributes, demonstrating the potential of image-based searching and browsing.

While government sources can neither confirm nor deny this, the Department of Homeland Security is allegedly interested in image-based search technologies, mainly in regard to facial recognition. However, as Munjal Shah, CEO and Cofounder of Riya told the audience at the MIT/Stanford Venture Lab’s Next Generation Search Symposium in 2007, commercial-grade facial recognition technology is not quite there yet. Such search algorithms can generally recognize which part of an image is a human face and venture a fairly accurate guess at a person’s gender, but recognizing a specific face requires matching the precise expression and angle of the face in a photograph with the sample, which can often be difficult. Despite these limitations, the potential of image-based search technology is enormous.

Does a user interface need to have a true visual-search algorithm to perform image-based search? Although technology helps, it does not afford the only way of performing a visual browse. One alternative is to have a human being tag items, using abstract visual attributes—such as a range of icons or outlines of shapes, using limited screen real estate—that, in turn, would let customers describe their visual queries. This kind of a pseudo-image-based search could, for example, successfully complement the existing people who shopped for X, also shopped for Y metrics-based algorithm.

Using Images for Navigating in the Real World

By now, most people are familiar with street view in Google Maps, which is pictured in Figure 5.

Figure 5—Google Maps street view

Using this phenomenal technology is almost like being there, with virtual-reality controls for navigating through nearly continuous photos of real space. Google Maps is an excellent example of automated sense-making of images, using precise GPS coordinates. Visual browsing controls let users walk, turn around, and jump through real space, using virtual projection. However, as anyone who has looked up their street and their house can tell you, these pictures are frozen in time. Some people can even tell you when the Google Maps picture of their house was taken, because a friend’s car was captured in the satellite photo.

In contrast—or perhaps as a complement to Google Maps—Photosynth, Microsoft’s new acquisition, uses crowd-sourced photographs to construct and let users navigate through 3D street-level space. It can also add the fourth dimension of navigating through time, letting users cross-reference multiple images. The feel of the interaction evokes looking into vignettes and being able to shift your point of view through space and time. One of the best examples available of this new capability is the Photosynth of the Obama inauguration shown in Figure 6.

Figure 6—Photosynth of President Obama’s inauguration

What is really interesting about this emerging technology is that it sidesteps—quite literally—the issue of a finite density of information within a single photograph. With systems like this, we are moving into a world where—rather than relying on just keywords and tags—we can use deep image analysis to sort and restructure sets of images and make sense of what we are looking at. The computer can truly see a real 3D space.

Collecting and Browsing Images on Mobile Devices

Using images as input for searches on a hand-held device is reminiscent of using barcode readers to enter inventory counts into a computer. Historically, barcode recognition required the use of a specialized device that was available only to governments and larger businesses. However, recent developments in mobile technology are rapidly changing this paradigm. An increasing number of mobile phone applications like RedLaser, LifeScan, and NeoReader, which is shown in Figure 7, now let customers snap photos of barcodes, using their mobile phones, then immediately use the photos as input for a search algorithm.

Figure 7—NeoReader iPhone application

Another tool called Semapedia offers an interesting way of using mobile barcode recognition. Semapedia makes it possible to tag any item in the physical world with a barcode that contains a link to the appropriate Wikipedia Web page. A person can use a mobile phone with the free NeoReader application to read the link’s URL and navigate to the Web page providing Wikipedia content about the object or place a person is looking at. Figure 8 shows a historical site that is tagged with a barcode linking it to a Wikipedia page.

Figure 8—Historical site tagged with a Semapedia barcode

Another example of the capability of mobile barcode-recognition technology comes from the Suntory Company in Japan, which tagged their beer cans with a special 2D barcode. Scanning this barcode with the application NeoReader takes a consumer to a mobile Web site that lets visitors register to offset 100g of CO2 emissions once per day and get tips on mitigating their own greenhouse gas emissions.

Both of these examples demonstrate that we are moving ever faster toward a world that is populated by smart objects, which Bruce Sterling dubbed spimes. We can track spimes’ history of use and interact with them through the mesh of real and virtual worlds that pervasive RFID and GPS tracking create. Mobile picture search is certainly emerging as the input device of choice for connecting the real and the virtual worlds to create the Internet of Objects.

Peter Morville, in his book Ambient Findability, talks about the sensory overload, trust issues, and bad decisions that are sure to result from our interactions with the Internet of Objects. However, it is hard to elude the siren’s call of such technology. It is now almost possible to use technology similar to that of Like.com to analyze every image and frame of video a mobile phone captures and tag them with text, GPS coordinates, time, and author. At the same time, it is possible to cross-reference each image with other images of the same place or similar things, along with all the tags, content, and links an entire world of users has added to this collection of images. The technology to collect all of these images in a massive 3D collage that users can navigate along a time axis, Photosynth-style, is also just around the corner.

Interactions of this kind make it possible to establish a unique and natural connection between the real world and the Internet computing cloud. The key is the added meaning the computer provides, based on its understanding and interpretation of the content of an image. Imagine, for example, taking a picture of a CD cover with your mobile phone’s camera and sharing it with your friends to learn what they think about the music; using the same picture to get more information about the artist on the Web—for example, pictures, videos of recent concerts, and a biography; finding out where you can buy the CD at the best price online; or using GPS to find coupons to purchase the CD from a local merchant. Imagine being able to do all of this with only a few screen taps and without typing a single character, while listening to a sampling of the CD’s tracks, playing through headphones connected to your mobile phone.

We are just starting to scratch the surface of these types of massively distributed, image-based, human-to-mobile interactions. Although image-processing speeds are getting much better—witness the smooth, fast scrolling of iPhone photo album images versus the slow shuffle of images on a Treo—the time it takes to process images on a mobile device and send them back and forth between that device and a server is still the biggest barrier to this type of technology. Once the image feedback loop is nearly instantaneous, this new paradigm for mobile input will truly come into its own, bringing the digital world ever closer to the real world and ushering in the arrival of a brave new world of visual browsing.

References

Jung, Carl Gustav. Man and His Symbols. New York: Dell, 1968.

Morville, Peter. Ambient Findability. Sebastopol, California: O’Reilly, 2005.

Nudelman, Greg. “Making $10,000 a Pixel: Optimizing Thumbnail Images in Search Results.” UXmatters, May 11, 2009. Retrieved July 26, 2009.

Riflet, Guillaume. “Semapedia, or How to Link the Real World to Wikipedia.” Webtop Mania, August 26, 2008. Retrieved July 26, 2009.

Smith, Gene. Tagging: People-Powered Metadata for the Social Web. Berkeley: Peachpit Press, 2008.

Smolski, Roger. “Beer, Carbon Offset, and a QR Code Campaign.” 2d code, July 27, 2009. Retrieved July 26, 2009.

Sterling, Bruce. Shaping Things. Cambridge, Massachusetts: MIT Press, 2005.