Has site search lost its way?

Why is it that site search is implemented in such a poor way on many sites out there? To prove my point I will take you through the following example about David:

David has just moved to Rotterdam. His new house has been renovated and David is left with a truckload of rubble and garbage that he wants to dump at the closest disposal site. He's heard that there are self-service disposal sites in the area. All David needs to do is figure out which site is closest and what its opening hours are. So David sets out to gather all the information he needs to achieve his goal. He visits the Rotterdam city website but thinks the menu is too cumbersome and can't find what he needs, so finally decides to try search.

David finds the searchbox in het conventional upper right-hand corner and enters the Dutch word for garbage/waste. The following results appear:

None of these results do the trick. The results are a mixture of links related to waste management policy and general guidelines. David tries a couple of other search terms but even though the information that David needs does exist, he can not get to it. Although David is initially put off, he is slightly more determined than the average visitor and tries a different approach. David recalls that the waste disposal sites have a rather special name and calls one of his friends to figure out what the exact used term is. Thus he finds out that the sites are called 'environmental parks' or 'milieuparken' in Dutch. He enters the term 'mileupark' in the searchbox, thereby unknowingly misspelling the word. Misspelling the word 'milieu' is not uncommon.

 

This time the search doesn't return any results and David finally decides to give up.

Does this paint an all too familiar picture? I think unfortunately it does. In this post I will go through some of the reasons why site search is often poorly implemented and try to give some useful tips for improvement.

Why search?
I know talking about search is a bit of a no go area ever since Jared Spool et al conspired to tell us that most users are not search dominant. Supposedly a recent research study by Forrester tells a different story (Dutch link)   I personally would have to side with Jared and find these research conclusions hard to believe.

Even though search might be considered to be a last resort from a user design point of view, I still feel that search will always have an import role to play on most sites. I can therefore not understand why search has been implemented so badly on so many sites out there.

Why is it that the content in most sites is structured and the search results are not? Why do we adhere to the notion that site search should be implemented as a Google clone? The prevailing view has to be that if it works in one area it should also work in another.

Fortunately most sites with extensive product catalogues have long ago embraced the concept of site search. Let's hope that more sites will follow their example.

Not like Google internet search
Site search isn't like Google. Google has to resort to complex algorithms to properly order its search results. Google has to do this since it searches the entire internet. It can simply not rely on an editorial board to figure out what is interesting and what not.

However, the amount of content within most sites is fairly limited and should allow site-owners to influence the way search results are shown. Site-owners can easily identify all the good stuff on their sites. Search should tap into this knowledge.

Focused on half the problem
Naturally what you would want to do when you set out to build your site is use the most of what is already out there. Let's face it, so would anybody. Unfortunately almost all search products that are in the market today have focused on half the problem: How to determine the relevancy of the results through basic generic algorithms. A number of techniques are used for this like:

- number of occurrences of search terms in content
- stemming 
- fuzzy search 

If you're interested you can read more about them here. Most of these search products use a full text index to come up with their results. Since the algorithms tend to be generic and do not discriminate in the types of content that is being scanned, they don't take into account the context of the specific site and the goals of its users. 

Getting back to our earlier example with poor old David, we can see how a traditional approach to search leads to unsatisfying results. I am sure that the search algorithm did its job wonderfully and that the results that were listed on top probably contained the best (mathematical) matches to the entered search term. Still, search failed miserably in its purpose. What a shame right?

The city of Rotterdam has carefully thought about its primary target group, namely the inhabitants of Rotterdam. They have even realised that inhabitants are primarily interested in the products and services that the city provides. (and not all the lengthy policy documents) They have made a comprehensive city catalogue of its products and services. So why then isn't the catalogue being pushed in the search function? If the catalogue is one of the pillars of the site strategy, why doesn't this extend into search?

The custom relevancy ladder: let the site-owner determine what is relevant



As suggested before, the trick is to not ignore the human factor when designing a proper search function. By combining the relevancy score of a standard search product with the knowledge that is already present within the organisation you get the best of both worlds. All the stuff that can be used to positively influence the results, but which is specific to the site can be included in what I call the custom relevancy ladder.

The cool thing is that all the components within the ladder are waged and can be changed over time. In this way the site-owner can fine-tune the workings of search.

I will not bore you with the exact detailed workings, but will name a few important components of the ladder.

Important site sections or clusters
For most site-owners it should be possible to identify the parts of the site which are the most important. In David's example the product and services catalogue of the city would have been an important section of the site to promote in the search results. Another classic example would be the whoiswho address book of any large corporate intranet. Isn't it annoying when you try to find someone on the intranet and the first page of search results only contains pages of meetings and content that this person has published? Wouldn't it be great if any proper hit in the address book would be placed at the very top of the results? By listing all the important site sections in the customer relevancy ladder and waging them individually, site-owners can really score quickly with the users.

Differentiation in the type of content matched
It's important to place value on the type of content that is found. For example, when a match is found within one of the labels of the sites' navigation structure, it could be seen as more important than a match found in the body of an attached document. A typical order of importance could look something like this:

1) navigation / taxonomy
2) tag / label
3) title
4) abstract
5) author
6) text body
7) attachments like word documents and PDFs

Leverage user input
The best user information is always data that can be derived from actual behaviour so use it! Examples of user input include:

• most viewed
• best rated
• user tagged content
• user bookmarked content
• learning engine: relates the results that users click on to the search terms that they entered. In this way dynamic associations are build up that move relevant results up the list

Not all these examples are applicable in all environments but they should give you a good idea of the possibilities.

Use a proper thesaurus
Next to the components of the custom relevancy ladder, a properly maintained thesaurus can really help to sort out your search results. No matter how carefully you conduct your user research, your sites navigation structure can never completely satisfy all your visitors. As with any classification scheme, choices will have to be made that will seem less logical to others. This means that users will not always know which exact term to use. With the help of a thesaurus, the user can more quickly find the information that he needs, even when he doesn't exactly know how to formulate his search.

A great thesaurus can be partly maintained by studying the common terms that are entered in the search box. Looking at David's example taking up popular misspelled words in the thesaurus would be a real winner. Also, if possible, one could use the thesaurus in combination with an autocomplete function to make things easier up front.

Use faceted search / navigation where appropriate
Generally when there is a lot of content to be searched faceted search can come in handy. My advice would be to stay away from the extended search option as much as possible. Users mostly don't like complicated search filters as they have learned too often that filter combinations will result in no hits.

Therefore strive to:
- show results first
- then offer ways to narrow and refine

Faceted navigation buys into this line of thinking. All relevant filters are shown next to the results. There is real strength in the combination of filters and results. The results give the user plenty of context instantly while the navigation filters offer great suggestions on how to further narrow search using the terminology used in the site. Especially that last bit is so important. For example, even though I have a bit of a multilingual background, I couldn't figure out what the English word for SLR cameras was. (perhaps I just wasn't into cameras when I was a child) I figured it out through Amazon's faceted search function.



About Facets
Take a simple apple: you can describe it according to foodcategory (fruit), color, weight, point of origin and so on. This comes natural to us, as this is quite similar to how our minds work. We make sense of the world through mental models in which we build up associations between things and thoughts around us. The way these models are organised is not hierarchal but networked or distributed. That's why so many people have a hard time in trying to figure out sites that use a strict top-down hierarchal navigation structure. Let's just say we are all used to looking at the different facets of things around us. So why shouldn't we let search benefit from this?

As you can see above, Amazon has chosen facets like category, brand, seller and so forth. The cool thing about this filter navigation is that you can combine all these different filters in a very easy way just by clicking on them. One of the rules that I always follow is never show any filters that do not result in any hits. Another tip is to show the number of hits that will be shown when clicked on a filter. (displayed above between the brackets)

Zooming in and out



Another important point is that you should always provide a mechanism by which the user can easily narrow and broaden the search. I tend to use a horizontal display like the ones that are shown above and below to indicate the combination of filters that have been used. I have also seen other designs where the chosen filters are added vertically. This also works but has the disadvantage that it takes up valuable vertical real-estate in a place where you would really like to place your filters.

Every design has its specific advantages and disadvantages. Like in all design work, one has to match the workings to the specific goals of the sites' user community. Amazon's approach is great for its simplicity. All you have to do to lose the filters and broaden the search is click on a filter to the left of the current one.

The design we used in the picture above was focused on allowing the user to switch between search selections. The user can switch between selections using the tab structure without loosing the selected filters. Of course when desirable, filters can be removed by clicking the 'X' in each tab.

The next example comes from the well known Dutch real-estate site Funda. Again, the selected filters are displayed in a horizontal fashion, but they have added an interesting feature. When a selected filter is clicked, all the possible values within the facet of the filter are displayed, so that the user can quickly change the value of the filter. For example, the user selects a different type of housing in the example above and switches from a family home to an apartment.

Don't forget to give the selected filters enough exposure on the page. It should be clear to the user what is happening while combinations between filters are made.

Browsing
Most traditional search design efforts focus on getting you the results you want through the use of filters. However this is only part of the search behaviour we see with users. Another important behaviour is the combination of search and browsing. Apart from the major eCommerce stores with extensive catalogues, too little attention is given to the notion of browsing. Although not fit for every situation, I think incorporating browsing techniques could improve search dramatically for most sites.

Browsing is especially good when the user in an exploratory search mode. In this case, the user perhaps doesn't exactly know how to formulate his search or is not familiar with the offerings. When browsing is implemented successfully it gives the user a lot of context to go on. Getting back to my point with the SLR cameras; you recognize one if you spot one right? So wouldn't it have been great that if I had spotted a SLR camera within the large mix of other cameras, I would have had a way of finding similar cameras like it? 

Last year I found this site Gettyimages that does exactly that. Its a stockphoto site that just works brilliantly. At that time we were building some personas for a client and just quickly needed some pictures to go with the them. What we needed were almost iconic pictures of people without too much background fluff as I didn't feel like cutting them out by hand. On top of that it would be really cool if all the pictures were taken in the same photographic 'style', so to say. I had no way of knowing how to formulate such a search so I just gave it my best shot, typed in 'salesperson' and hoped for the best. After browsing the results for what seemed like an hour I finally found a picture that I liked.

It could have looked something like the above. After I had overcome my initial wave of joy over this first success, I started to realise my predicament as I was about to spend the rest of the afternoon browsing for the remaining 4 personas. With panic just about to strike I noticed this little feature tucked away at the top of the page called 'find similar images' Didnt this just seem like a gift from heaven?

The feature opens a page that looks like the one below that shows you all the meta information about the photo. You can then the select the combination of properties that you want to find similar images with.

 

In this way I was able to quickly come up with a whole bunch of similar photos as you can see above. So if you have a lot of annotated content, browsing can help users to properly sift through it.

Make sure the search results show enough scent
It should be clear to the user what content will be shown when they click on any given search result. In other words the search results should have enough information scent The context in which the result was found should also be clear. If search doesn't facilitate this enough, users will hesitate to click on the results. Repeatedly clicking on links that lead to the wrong content just takes too much effort. So always provide ample information scent and context where needed.

For example let's say you're designing search for the city of Paris. The search term 'eiffel tower' will find a lot of results in different categories or content types. Suppose search just shows you the titles of the following three results:

- Paris, remaining on top
- Directions
- The Eiffel tower

This doesn't give you a lot to go on. Now let's change the way the results are shown applying the theory of information scent into the examples below:

	City attractions: The Eiffel tower All the practical information that you need for a visit of a lifetime - Fees - getting there - Pictures
	City directions: How to get to the Eiffel tower - maps for download or print - public transport info - taxi companies
	City program: Paris remaining on top Paris' plans to remain the fashion capital of the world ...new Fashion board presented in the Eiffel tower by the head...child Jean was intrigued by the design of the Eiffel tower. Something that has never...the implicit effects of a city's architecture on fashion. Take the Eiffel tower for example...

You can see how this makes a world of difference and allows the user to get an idea of the content that the result links are pointing to. Looking at the third result we can see how important it is to show the context in which the search term is found. This is especially true if the term is found in a body of text and not the content title.

Have a go at it!
I've shown some suggestions for improving search. Naturally one should not go overboard and blindly implement all these suggestions. Some features will work in one environment but be uncalled for in the next. It's still up to the design team to make the right specific choices.

A proper search implementation will probably require a greater effort and investment. Still, only add features to your site if they offer true added value to the user. Therefore: why bother implementing a search function that doesn't work? So just have a go at it!