We specialise in online usability testing.

Return of Investment for User Experience made simple – A Workshop.

June 20th 2013 by Posted In: Events, Research methods and approaches

Tags: , , , ,


There is the challenge that many UX or Usability issues go unfixed. One of the biggest causes is that there is often no financial argument or Return on Investment (ROI) case being made by the UX team.  ROI enables teams to overcome arguments of resource limitations, and opposing opinions of other stakeholders by putting hard financial numbers to the argument and show the benefit that would be achieved if issues were fixed.


This workshop will explain how to create a Return on Investment Argument for User Experience, quickly and easily. We will share our methods to simply calculate a Return on Investment. We will give examples for both web applications in competitive markets, and for in-house applications behind firewalls.


The workshop will cover both the methods, and how to collect the data to act as the inputs to the calculation. We will go through practical examples of both calculating the ROI as well as defending the calculation.  Being able to make an ROI argument makes your UX and Usability findings more likely to be implemented as well as increases the audience for your work.


At the end of this one day workshop you will walk away with the knowledge of:-

  • Benefits of a Return on Investment argument.
  • How to make a Return on Investment Case for making UX and Usability improvements.
  • Which methods to use under which circumstance to calculate the ROI.
  • How to build a simple model to illustrate your case.
  • What data you need to collect, and how to collect it


The workshop has been designed for the UX practitioner who has little finance or maths knowledge. You will need to have some experience of User Experience and be able to do basic arithmetic in Excel.


The workshop will be given by James Page and Sabrina Mach. Both of them have many years of experience quantitatively analyzing websites from a User Experience point of view. They have helped clients ranging from British Telecom to Vodafone, Millenium Bank and Sky, and worked in conjunction with agencies from Ogilvy, SapientNitro, LBI, Flow, and Head London.  They are both founders of Webnographer, which is the world’s leading Remote Usability Specialists.


When: 26th July 2013.
Where: At Webnographer’s in Lisbon office, Portugal.
Cost: 150€ + (€34.50 IVA/VAT) until 8th July and then 250€ + (€57.50 IVA/VAT) after. (Includes lunches and coffee). There is a student and start-up discount. Contact us to find out more.
Getting there: We have deals with hotels. Contact us, with the sort of Hotel you are looking for and she can put you in touch with the right place.


We are looking forward to seeing you in Lisbon!

Buy your ticket here: http://roilisbon.eventbrite.com

Image credit: Miguel Vieira http://www.flickr.com/photos/miguelvieira/

Leave a Comment

Information Visualization for Knowledge Discovery

March 10th 2009 by Posted In: Design, Events, Information Visualisation

Tags: , , , , , , ,

Treemap developed by Jean-Daniel Fekete

Treemap developed by Jean-Daniel Fekete

Ben Schneiderman from the University of Maryland, gave a fascinating talk in Cambridge on 5th March 2009 about the topic of “Information Visualization for Knowledge Discovery.”

Ben has authored many books and papers on human computer interaction, and was the founder of the Human Computer Interaction Laboratory at the University of Maryland. His keen interest is the field of information visualisation.

During his talk, Ben pointed out that in contrast to scientific visualization, information visualization is a relatively young field, as information visualization conferences have only been going for about 15 years. He added that, the challenge with information visualization is that the information keeps changing over time.

Ben presented his conceptual break down of information visualization tasks: “Overview -> zoom and filter -> details on demand.” What he means is that one should provide an overview first, showing all the information, for example complex graphs, diagrams and maps. This allows the user to orientate themselves and get the big picture. Then allow the user to zoom into more detail and filter out any unwanted information. Finally, allow the user to select an item and get more detail about it when required.

The most enlightening point that Ben made during his talk was that: “Information visualization gives you answers to questions you didn’t even know.” He went on to argue that “there should be a move from opportunistic discovery and to a more systematic discovery of knowledge.”

Ben illustrated his argument with a number of demonstrations and screenshots of projects that he and his students have developed over the years. Each guides knowledge discovery thought the visualization of different patterns in the data. Ben emphasised that for information visualization “the interest is not in a particular value, but an overall view and patterns in the data”. Yet, he also emphazised the importance “trying to see the violations in the data that are contrary to your expectations”.

Ben demonstrated his famous treemap that has now been modified by many commercial companies. It has been modified for example  to visually show the constantly changing landscape of the google news aggregator, and even the New York Times has used it to show changes in truck and car sales.

Other tools which Ben showed were the ShapeSearcher which finds spikes in the data, Scattergrams which provide the opportunity of hunting for stuff, the alignment tool which can filter by event and show what happened before and after this event, and another tool which identifies gaps in the data.

The most intriguing example of finding patterns in data was Ben’s demonstration of the SocialAction tool, which uncovers hidden structures in social networks over time. The visualization presented the correlation between US senators voting the same way. It showed a strong correlation that democrats and republicans vote the same way. Only four republicans sometimes voted similar to the democrats. Yet the most surprising finding through this visualization was that the correlation for democrats voting the same way was far stronger then republican voting the same way.

“The social network of the U.S. Senators voting patterns. Here, the threshold is raised to 290 votes. The Democrats’ relationships are much more intact than the Republicans. Details-on-demand are provided for Senator Whitehouse, the senator with the highest degree at this threshold.” (by Ben Schneiderman)

Ben concluded his talk with three key points for information visualization to guide knowledge discovery:
1.    Rank-by-Feature Framework, i.e. rank by what people want to know
2.    Decomposition of complex problems into multiple simpler problems
3.    Ranking guides discovery. It is important to provide systematic
approaches for discovery.

Challenges of visual literacy

A theme that kept popping up in the talk and particular in the questions afterwards, was the challenge of visual literacy. Words can help to clarify matters of information visualization, but Ben explained that textual information is only good for simple queries (such as a rank list in Google search results). Visual tools on the other hand are better for complex queries.

For anyone who is interested in finding out more about the challenges of visual literacy, Ben recommended the work of Colin Ware, a perceptional psychologist, who looks at the challenges of understanding visual information.

Interesting reads about information visualisation:

Bederson, B. and Shneiderman, B. (2003) The Craft of Information Visualization: Readings and Reflections, Morgan Kaufmann Publ., San Francisco, CA. Amazon UK, Amazon US

Card, S., Mackinlay, J., and Shneiderman, B. (1999) Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann Publ., San Francisco, CA. Amazon UK, Amazon US

Tufte, Edward (1983) The Visual Display of Quantitative Information, Graphics Press, Cheshire, CT. Amazon UK, Amazon US

Tufte, Edward (1990) Envisioning Information, Graphics Press, Cheshire, CT. Amazon UK, Amazon US

Tufte, Edward (1997) Visual Explanations: Images and Quantities, Evidence and Narrative, Graphics Press, Cheshire, CT. Amazon UK, Amazon US

Ware, Colin (2004) Information Visualization, Second Edition: Perception for Design (Interactive Technologies), Morgan Kaufmann Publ., San Francisco, CA. Amazon UK, Amazon US

Ware, Colin (2008) Visual Thinking for Design, Morgan Kaufman, Burlington, MA. Amazon UK, Amazon US

Designers Dilemma: visual convention vs. breaking new ground

January 21st 2009 by Posted In: Design, Research methods and approaches

Tags: , ,

Without innovation the internet would still look like this. (Gutenberg's printing press. Photograph by Matthias Kabel)

Without innovation the internet would still look like this. (Gutenberg’s printing press. Photograph by Matthias Kabel)

The dilemma of visual convention vs. ground breaking new design seems to be a fearsome concern for usability specialists.  In a recent blog post on the Concept 7 blog, Stefan Wobben quotes a paper by Luis Santa-Maria and Mary C. Dyson form the University of Reading that investigated the impact of violating visual conventions on user’s performance and orientation. Santa-Maria and Dyson explain:

“Although initially violating visual conventions might hinder user performance and leave users disoriented this experiment indicates their experiment indicates that these problems can be short-lived and users can adapt reasonably fast to a new set of visual conventions.”

This is good to hear, yet its no news as such. If design had always only followed convention we would not have progressed from the written word to the printing press to computers and the internet.

“So the decision to whether conform or violate visual conventions when designing a website should ponder that although users might adapt quickly to novelty there is an initial performance hindrance and disorientation.”

They have a good point there in encouraging those violations. Too many studies focus on first time use, but not repeat users, how behaviour changes over time, and the experience and use of the system by expert users. A single lab study as a Q+A exercise just before the launch of your website is not going to do the trick in gaining this understanding. Usability is an ongoing process, not a one of label of approval. As Harry Brignal pointed out on his blog: “A UX designer’s job is never done.”

The one great thing about remote usability testing is that it is cost efficient and can therefore be carried out more often, than a lab study. Webnographer, as a remote usability testing tool, makes ongoing testing simple and affordable. It makes it easy to test the learning curve and behaviour and satisfaction of experienced users.

As Jared Spool explained back in 2003 small ongoing changes carry far less risk, then a major relaunch and re-design, which is very likely going to fail. As with the printing press, changes in improvements were small, stretched out over a length of time and we ended up with the internet, which makes the spreading of ideas and information easier than ever. For website design, the small changes allow you to measure the effect of that change on your users, and you will find out whether the change has made your site better or just different, and how it affects your users over time.