Show me the

UIE Conducts E-Commerce Comparative Analysis With Morae

The Project

User Interface Engineering (UIE) was hired by a large retailer to create a set of e-commerce best practices based on an in-depth study of the country's leading consumer Web sites, including: Amazon.com, BarnesandNoble.com, BestBuy.com, CDW.com, CircuitCity.com, Crutchfield.com, Dell.com, EBGames.com, HP.com, JR.com, Target.com, and Walmart.com.

The Scope

  • 72 shoppers
  • 4 Web sites visited per person
  • Over 220 hours of usability testing

Background

As most usability professionals know, testing comes down to being able to collect, organize, and reduce the enormous amount of data gathered into manageable repositories that can be reviewed, referenced, and analyzed.

With the help of one or more technicians, usability professionals would watch and listen to a picture-in-picture view of a user and a Web site. This wouldn't be in real time. It would be after heavy editing and the quality of the video wouldn't be ideal (think of recording your favorite TV program by holding a camcorder in front of the screen). From this mass of videotape, consultants would spend hours in front of televisions and video machines, wearing out the fast-forward and rewind buttons, trying to find specific points of interest or trying to count mouse clicks and other repetitive tasks.

Morae is changing all that. Based on patent-pending Rich-Recording Technology (RRT), Morae records and synchronizes video and audio of the user and the Web site, along with system data (e.g., mouse clicks, keystrokes) into a single digital, indexed and searchable file. It doesn't require a technician to operate it, and it supports remote usability tests so that test subjects do not need to be in the same location as test administrators.

The User Task

Each participant was asked, in advance, to come prepared to buy items they were already planning to purchase. The items could be for the participants themselves, or for other people. The participants were instructed to do enough advance research so that they would be ready to make purchases during the test. The purpose of this preparation was to minimize the role of outside factors (e.g., lack of product knowledge,spousal disagreement about whether a purchase is necessary, etc.) in the buying decisions, and maximize the role of each Web site's usability in winning a customer's business.

Morae Integrates With UIE's Testing Cycle

During the forty days of testing, Morae's Recorder component was installed on the test machine to record all interaction. Each time a participant clicked on a link anywhere on a Web site, the action was recorded. If a dialog box opened, or if an error message, Pop-Up window, or Flash video ad appeared, it was recorded. And if the participant sneezed, that was recorded, too.

All this happens automatically, nothing is missed, and it's done with no disruption to the participant's shopping session. UIE saved five test sessions at a time to a portable drive. Morae's flexibility provides several options for saving recordings. In fixed locations, recordings can be saved to the test computer's hard drive or to a connected network drive. For field research, consultants can use other popular media for saving session files, including Flash Thumb Drives, CD-ROMs or DVDs.

In usability testing projects, getting senior executives and decision makers involved is often nearly impossible due to scheduling and location issues. Morae's Observer component eliminates this challenge by allowing usability professionals to conduct Collaborative User Experience Testing (CUET). In other words, multiple observers can view and give input on the same usability session over a corporate network, VPN, or broadband connection. They can be sitting in an office across the hall or thousands of miles away.

The heart of Morae is the Manager component. This is where TechSmith's RRT synchronizes, consolidates, organizes, indexes, and makes searchable all the video and data collected during a test. For maximum flexibility, Morae Manager can be installed on a laptop, making it easy to conduct field tests, analyze results anywhere and anytime, or show potential clients or stakeholders a sample of what they will receive.

Morae Makes Transcription for Clickstream Analysis Easy

After testing, Will Schroeder was able to quickly and easily review his data digitally - a process that had been manual and slow in the past. Since Morae automatically arranges all data from a test session chronologically, the analysts could search by mouse clicks, Web page changes, keyboard activity, text entries, Windows events, and other points of interest (or a combination of events).

Morae exported the raw clickstream data into UIE's Excel-based classification system. This functionality allowed analysts to focus on clickstream analysis, and calculate key variables, such as time-on-task. Clickstream analysis enables a company to predict which Web page a person is most likely to visit next, and with Morae they also know what prompted the user to leave the page in the first place. This allows UIE clients to prevent costly mistakes before they launch new Web sites or redesign existing sites.

This also makes it easy for UIE to answer common client questions, such as:

  • How often is the back button used in successful and unsuccessful tasks?
  • At what point in the task are lateral links used?
  • Are the lateral links associated with successful or failed transactions?
  • What was the average time spent reading product reviews?

Generally speaking, the e-commerce sites that performed best in these tests did one of two things: either they pointed the way clearly to a product the shopper wanted and offered a strong value statement, or they allowed users to put together lists of all the items they were interested in and then whittle down those lists in an intelligent way. However, no site was completely successful at this. It seems that most shoppers thought they knew what the popular items should cost, and were more confused than enticed by rebates and other special offers that obscured the actual price.

Making the Business Case for Usability Testing With Morae

UIE consultants wear many hats, and are skilled at scoping projects, selecting and scheduling usability participants, budgeting and bidding for potential projects, and jumping from one problem set to the next. Their accounts range from one-time projects for a small business to multi-million dollar Fortune 500 projects.

One of the biggest challenges for any consulting firm is to be competitive during the bidding process. Inevitably, bids come down to anticipated man-hours. And success comes down to delivering meaningful, accurate results on time. Morae helps UIE package and price their services more accurately.

Moving forward, Morae will continue to be a critical tool in UIE's testing arsenal for analyzing how users interact with Web sites. Having had great success to date, UIE plans to explore more of the options and features Morae offers to maximize efficiency and streamline testing processes.

About User Interface Engineering

User Interface Engineering (UIE) is a leading research, training, and consulting firm specializing in web site and product usability. Well-known usability expert and recognized authority Jared M. Spool founded the company in 1988 and has built UIE into the largest organization of its kind in the world. With in-depth research findings based on user observation, UIE empowers development teams to create usable Web sites that increase customer satisfaction and loyalty.

Obviously, what got our attention was the affordable price. But the reason we're committed to Morae is that it provides a superior feature set. It's easy to learn and use. And it enables us to accelerate testing projects dramatically.

- Will Schroeder,
Principal Consultant, User Interface Engineering

A big reason why we got up to speed so quickly is that Morae didn't force us to change our style. We have always transcribed and imported our data into Excel, where we have our own classification system. With Morae, we did it much faster, without sacrificing accuracy or detail. We cut transcription and importation time by 200 percent.

- Will Schroeder,
Principal Consultant, User Interface Engineering

Morae makes us extremely competitive and allows us to be flexible when bidding on new projects. After using Morae for only a few months, we know for a fact how much time we can and will save on any given project. This translates into a significant competitive advantage for UIE in terms of winning projects, predicting project profitability, and taking on more projects. That's hugely important for the health and future of our business.

- Jared Spool,
founder of User Interface Engineering
More customer stories