Scoreberlin Raises the Bar With Usability Testing and Eye Tracking With Morae
A Report from: Marcus Voelkel, Managing Partner of Scoreberlin GmbH Usability Consulting, Berlin
In our Berlin based usability lab, consultants analyze and optimize the usability of web sites, software, designs, and their effects on users. For these purposes they apply agile user-centered methods and modern technologies like remote high-speed eye tracking. Morae plays a key role in the usability labs.
In a common usability test setup we have three “usabiliteers” run a test. The facilitator is testing the participant, while the other two are observing - one is taking Quotes-n-Notes (ad-hoc findings) directly into a web-based application, and the other is setting markers and notes via Morae Observer for later analysis in Morae Manager. Usually customers, developers, designers, or project managers are also observing tests, taking notes and drinking lots of coffee.
Total User Experience DVD
While the test is running, the complete user experience is recorded simultaneously into a big WMV file by the Observer – including the participant's video and audio as PiP (picture in picture), highlighted mouse movements and clicks, and the screen recordings. When the last session is completed, the developers or project managers drink a last cup of coffee while we burn DVDs with the WMV files and export the Quotes-n-Notes into PDF or MS Excel. We also offer the following formats as well: AVI, Quicktime MOV, MPEG or DV.
The advantage is obvious. Directly after the test, customers are able to evaluate our first findings and analyze the Morae videos. And only a few days later they receive a full size usability report in CIF (Common Industry Format) with entire transcriptions and highlight videos. (Think back two or three years, remember the old times, and laugh about yourself having offered Rapid Prototyping back then.)
UCEE – a Hybrid Usability Methodology
One of the great advantages of Morae is certainly the opportunity of delivering and communicating the results quickly. But Morae also allows us to share our insight into the total user experience when we additionally use our eye tracking system and our UCEE methodology (User/Customer Experience Eyetrack Evaluation).
UCEE (speak: You See) is a hybrid usability methodology which combines quantitative eye tracking and qualitative interviews into one session. This was not possible with common eye tracking systems before due to their low head motion tolerance (which is one of the reasons for using eye tracking as a single method for pure quantitative testing). With modern eye trackers this key limitation belongs to the past. Our eye tracker (a 120 MHz Eyegaze Analysis System with binocular option) allows participants an enormous free head range while calibration happens automatically and lasts only seconds. We can now test eye gaze and qualitative user experience together within one session.
In brief, UCEE lets the participant focus on the stimulus and task without having to talk or think aloud. After each Eyetrack Walkthrough we repeat the scenario with the think-aloud method by replaying the eye gaze video with the visualized scan paths. We have found participants have considerably fewer problems to reflect on events that just happened and express their thoughts in their own words more precisely when the scenario is instantly replayed (in comparison to conducting both methods separately). This is especially true in cases with shy and analytical thinking participants (Lewis/Mack, Evidence from thinking aloud protocols; Nisbett/Wilson, Verbal Reports on Mental Processe).
Results To Go
The complete walkthrough is recorded by Morae and synchronized into one file. Morae sends this file (including video, audio, mouse movements and clicks, system events and eye tracking video) as a live video feed to the Observer in the observation room. During the Eyetrack Walkthrough the observers have insight into the whole user experience - on the canvas they see the actual screen recording, in the lower left side they can watch the video of the test participant and listen to him. The mouse cursor is highlighted by a yellow dot, clicks are visible by a red triangle. The eye gaze overlay is visualized by a green dot. In this way the observers are able to track the participant's mouse and eye movements simultaneously – the complete interaction at a glance. This is really an advantage because observers have a live view on what participants are really looking at or what they ignore.
How to Gain The Total User Experience
Again, Morae helps us to build this quick'n'dirty setup. In the usability lab we have computer A with the eye tracker running VNC server and computer B with mouse, keyboard, Morae Recorder and a VNC client connected to computer A. Computer C with Observer is located in the observation room, connected to computer B.
Computer A (eye tracker) broadcasts the gaze positions together with the screen contents to the Morae computer B. On computer B Morae is recording the synchronized screen contents, system events, and the test participant via webcam. Whenever computer C connects to computer B over Observer, it receives a fully synchronized video together with video, audio, mouse and eyegaze highlighting.
The green dot visualizes the eye gaze (eye tracking), the yellow dot highlights the mouse (generated by Morae). A red triangle shows clicks.
As a relevant evaluation factor eye tracking is undoubtedly closing the big gap in the usability method's mix, especially in combination with classical methods like user tests and expert evaluations. For example Web2.0, AJAX, and web based dynamic applications lead to new hypes as to new questions concerning de facto standards, conventions, and common sense, also concerning users and their experiences. Our pragmatic usability approach gives the answers to these urging questions.
Beneath traditional methods usability needs user centered approaches and methods which consider the requirements that new technologies and products demand from their users. More than ever aspects like user acceptance, aesthetics, fun, joy of use, and hedonistic quality come to the fore. The combination of qualitative and quantitative methods like UCEE bears these aspects in mind.
Morae is especially great for qualitative user testing. Problems occur when you have to explain why the navigation has been difficult to understand for users or how they can be better supported by better working designs. At this point the combination of Morae and eye tracking really makes sense.
Questions you can't answer without eye tracking methods are typically those referring to attention, contact time, or orientation:
- Which elements attract attention first?
- Were elements or objects perceived?
- For how long has an object been perceived?
- Does the user find important elements?
- Which positions, which designs work better?
- What do they really read, what do they ignore?
Morae revolutionizes the usability lab. Especially in processing and analyzing the tests and presenting the results Usability testing has never been this efficient. Our customers are completely hooked.
By using Morae we save time. Almost immediately after ending the sessions the customer receives the videos on DVD and first results they can work with. Afterwards, we open the project in Morae Manager with the markers being set via Observer. Now we are able to easily analyze the scenarios and cut and edit highlight videos – all in one application.
Morae allows us to develop newer, simpler and faster methods. Without Morae we would still work with methodologies and technologies from the last century.
Scoreberlin GmbH Usability Consulting belongs to the established usability companies in Germany. Since 1999 Scoreberlin stands for high-quality usability solutions. Scoreberlin GmbH is an exclusive partner of TechSmith Corp. and a Morae Value Added Reseller in Germany. For more information visit: scoreberlin.de