Chances are you have used Wikipedia for something in the last few months. And if so, chances are you have seen one of their fundraising pleas, such as these:

What’s really interesting is that Wikipedia is publishing the results of the ads, in something we in the biz call an “A/B” or multivariate test.  The idea is this: Create a series of different ads – with different pictures, headlines, buttons and links – then alternate them across the site, and see which combinations work best. You can measure “version A”  versus “version B” (that’s called an A/B test), or you can do a more sophisticated mix and match of the elements, called a multivariate test.  The great thing is you get data from 10,000 or 100,000 or 1,000,000+ interactions with your web site creative content, and from that can see what visitors are actually interested in, based on their behavior.

Commercial web sites do this all day every day, but the difference is that Wikipedia is posting their results in real time publicly, to the world.  There’s a ton of raw data about click behavior in multiple languages.

In their link to their A/B tests data updates you see a lot of interesting and counter-intuitive behaviors.  For instance, I thought “Admit it- without Wikipedia, you never could have finished that report.” is a fantastic headline, but it got less than a 1% click-through rate, whereas the “personal appeal from Jimmy Wales” with his picture got almost 3% clickthrough on a recent week.

The Wikipedia team are also posting regular summaries of their findings, in case you don’t want to slog through their detailed testing page or download their spreadsheets of raw data.  Some findings from last week emphasize that the total experience (from clicks through donation) are important to measure, and not just the initial click-through rate:

• “The original two-step payment form we’ve been using is the most effective, it performed better than the new one-step process.”
• “Adding an editor’s image to the landing page did not significantly affect donations.”
• “The click-through rates on the editor banners continue to be on par with the winning Jimmy banner, but bring in fewer donations.”
• “Many donors appear to relate better to letters which focus on readers showing support instead of individuals editing.”
• “There has been a positive response to the new editor banners, the variety keeps our campaign interesting.”

From the commercial web world, there are a couple of other sites that post results of A/B and multivariate tests: WhichTestWon.com and MarketingExperiments.com both show recent design tests and let you guess which ones were most effective.

By the way, you don’t have to use A/B tests just for advertising. In fact, many companies use them as part of their design regimen to test for which designs actually are most effective for users to complete a task.  The A/B test tells you which design works better, but of course not why it works better; you still need usability testing for that.

P.S. Shoutout  to the folks at digitaloptimizer and others for pointing out the Wikipedia tests to me – very interesting.

Tags: design, usability, webexperience

This past weekend, I enjoyed a couple of very straightforward and simple online shopping experiences. But, unfortunately, I also endured some really bumpy ones. Which led me to wonder: Doesn’t everyone in the digital world know about doing walkthroughs before a launch?

Walkthroughs are a very simple way of previewing experiences so you know what they’ll be like when you go live. They’re like a dry-run of the experience that you can do early in the development process.  Here is how it works:

1. Figure out the common paths through an experience. (For instance, one that I saw this week on a consumer site was to sign up for a promotional program: It started with an ad, led to a landing page, which went to a Facebook page and on to a sign-up form [which turns out is too many steps, but that was the flow, so that is what you would walk through].)
2. Mock up a placeholder page for each step. If you have a small team that’s in one place, you can just draw with pencil or pen; if your team is more distributed, pop some fake screens into PowerPoint or another online tool so you can share via WebEx.  If parts of the web site or mobile screens are already built, use those.
3. Then, in a small meeting, have someone pretend to be the site visitor, and step through the experience. Make sure someone takes notes about missing steps, messages you want to get across on specific screens, what the main calls to action should be, etc.

The picture below shows what it’s like if you walk through the steps posted on a wall. We sometimes do it this way at Cisco, although more often we set up WebEx Meeting sessions so and walk through web pages or PowerPoint mockups. But the idea of walking through sequential steps is the same.

A walkthrough is a great focusing exercise because it gets everyone to focus as a team on what the end customer will see, and a lot of petty or political issues melt away once everyone is focused on the end experience. On a big project, you should walk through many key scenarios and repeat until you get it right.

I can’t remember a walkthrough I’ve done where something wasn’t caught. And catching problems before launch is much better than having thousands or millions of people experiencing them live.

Enjoy!

Tags: design, web design, webexperience

The folks who run our online programs for our Partners team showed me some interesting charts the other day. They show that Cisco Partners are savings lots of time using My Cisco and My Cisco Workspace.  If you don’t know, My Cisco is kind of like a central dashboard that follows you around (as a menu) on Cisco.com, and that also has a full workspace page.

My Cisco is especially popular with Partners, who can access their deals and quotes, find events, check training and certifications and check Partner news – and access all manner of information that previously was locked up in separate tools.  Instead of navigating to individual tools for each new task, Partners can go directly to My Cisco Workspace to get this information at a glance on one page.

Almost half of Partners surveyed say they are saving at least 20% of the time they used to spend with online tools – and some save as much as 50% of their time!

My Cisco also has many functions for Customers, too, including an array of support-related capabilities:

• Recent service requests
• Alerts for updates on software previously downloaded
• Products recently viewed or visited in the support area (so you can get back quickly)
• Products and devices that you own or manage (so you can see one-stop documentation, support and alert information)
• Active support notifications
• Network bookmarks available from any system you use

There’s a convenient catalog you can use to choose and customize the functions that you want to keep in your workspace and follow-along My Cisco menu.

To get to My Cisco, you can access the My Cisco menu at the bottom of most every Cisco.com page. And at the top of most pages there’s a convenient shortcut link to My Cisco Workspace.

Enjoy!

Tags: design, usability, webexperience

I was chatting with a colleague recently who was recounting her experience as a first-year engineer just out of school. It was a role that required designing and troubleshooting complicated networks—MPLS or ATM with intricate VPI/VCIs. Not being a technical person, the acronyms alone seemed daunting to me. But what became painfully obvious was the time and resource drain that is inherent when supporting a vast number of customers with needs that change on an ongoing basis. Without a standardized reference or blueprint, they were forced each time to create—and re-create—the wheel, over and over. It was clearly a problem in need of a solution—and an architectural one at that.

At Cisco, we’ve been talking about how Borderless Networks can transform your business—from the IT management side of things, and from the end-user experience perspective. But what helps make that a reality is the underlying architectural blueprint.

Read More »

Tags: architecture, Borderless Networks, configuration, deployment, design, Networks, SBA, Smart Business Architecture

At Cisco’s Financial Analyst Conference Paul Mountford, President Emerging Markets, was joined in his theatre breakout session by two organisations pivotal to redefining ICT in and for Africa, Brian Herlihy, CEO of SEACOM and Mark Schneider, Group Chairman of Wananchi. 

Read More »

Tags: design, home, webexperience