As previously mentioned, I am involved in a very interesting project at the Concours Group, called tools and techniques for business experimentation. We had a great teleconference yesterday, discussing the use of experimentation as a business tool, particularly focusing on the role of IT.
One issue that came up towards the end of the conference was the question of justification – how do you get funding for running experiments? Experiments, in our view at least, are not happenstance “lets try a few things and see what happens”, but real, planned activities with predefined learning objectives. The costs of those are substantial and relatively easily determined. But what insights do you gain from business experimentation, and what is the value of those insights?
Thinking about this, it struck me that the learnings from business experimentation most likely will be boring, obvious and – somewhat implausibly – very valuable. They are boring because they generally lead you to design offerings that the regular customers wants, not what designers dream up. And they are obvious because you really should have thought about them yourself. They are valuable because, while boring and obvious, you really need the experimentation not just to discover them, but also to document that this is what the customers really want.
But still – boring and obivous?
Let’s start with the boring part. Check out Fidelity’s homepage. While it will never win any awards for fancy design, it is in daily use by millions of customers who use it as a tool for managing their finances. The page – as are all of Fidelity’s pages – is carefully designed after extensive usability testing in Fidelity’s usability lab. In this lab, they use eye tracking, videorecording and careful monitoring of user behavior (and they use regular users, not whiz kids) and have come up with a page that is boring – but highly usable. The most used features – customer login, access to customized page, most used services) are up in the left corner, where people look first. The choices up on the top, providing access to the services offered, stay the same through all web pages. I could go on and on …
In fact, I think I will. Search is prominently featured, as is stock quotes. The main space, where user-defined information such as bank transactions would go, is used for advertising new services and features, until the customer specifies something else. Every choice you make takes you to a page where you can do something (as opposed to many financial web pages, where, if you jump from banking to insurance, will take you to the top level of the insurance subsidiary.) Corporate information and contact info is available at the bottom of the page, easily found when needed but not in the way for daily use. The colors are understated but easily recognizable.
This is an excellent web site – and rather boring. But in this sense, boring is good – as seen in the disappearance of the brochure-oriented financial website, high on flashiness and very low on actually getting the job done. Insisting that web pages be subject to usability tests – that is, to highly structured experimentation – means that customers will like them. Not to mention that it is a great way to fend off the fancy web designers, still wearing black turtlenecks, who insist on Flash animations and font coloring straight from WiRED Magazine. Fidelity did this early, and it was experimentation that got them there.
As for obvious – in the late 90s, I learned about the English retail chain Sainsbury‘s, who was about to design a service where customers could order groceries and have it delivered at home. The company was well underway with the design when it occurred to them that perhaps they should test this out with the customers first. They ran a few experiments – and discovered that not only was it extremely expensive to deliver groceries at home – the customers did not want it! What customers wanted, was either to have the groceries delivered at work (so you could walk out the front door of the office, pick up the groceries from a Sainsbury’s truck parked outside, dump them in the back of your car and go home) or to pick them up outside the store (which meant that, if they had forgotten something or wanted to touch and feel the merchandize, they could run into the store for those items). Delivering at home with any kind of precision is hard in the densely populated England, and customers can’t be bothered to be at home to receive the goods at a precise time anyway.
With hindsight, the fact that most people don’t want groceries delivered is fairly obvious – but it took experimentation to discover it and the documentation provided by the experimentation to convince the organization that home delivery would not be something the customers wanted.
The very valuable part has to do with how we think. As pointed out in Clayton Christensen‘s excellent The Innovator’s Solution, companies designing products and service have a tendency to think in boxes. Either they become product focused – that is, they look at what the competition is offering and then design something that imitates successful features of the competitors’ offerings. Or they become segment focused, obsessing over what they should do in the youth market, the retiree market, or whatever. The Norwegian radio channel Kanal24, for instance, decided that their average customer was a 32 year old working woman who liked Elton John and easy listening music, not realizing that just because companies have a need to simplify their picture of the customer, the average customer not only may not exist, but also don’t want to be boxed in with “appropriate” offerings.
What companies should do, says Christensen, is ask themselves what job the customer is hiring your product or service to do – realizing that it might differ with time, place, and mood. Most of all, it differs by preference, as anyone involved in a discussion of what computer tool to use for almost any job quickly will find out.
Doc Searls has stated that “markets are conversations” – you have to find out what to do in a dialogue with your customers (as Hugh McLeod says, smarter conversations). Having a facility to do experimentation, not to mention a culture that allows for it, is crucial in establishing and nurturing that communication.
And what you will find, is boring, obvious – and very valuable.
Antony Beevor: 
Applied Abstractions 