This content is part of the Essential Guide: Want satisfaction guaranteed? Add user experience to the design process

Drunk user testing changed HubSpot's approach to UX research

Giving a drunken UI designer a user test changed HubSpot's UX research and UI design practices. The results were stunning, said UX designer Austin Knight at the Fluent Conference.

What do you do with a drunken user? Put him through a user test, and watch and listen carefully. That's what UX designer Austin Knight convinced his company, HubSpot Inc., to do. The results from drunk user testing led to a change in user experience research and UI design practices, as well as a sizeable improvement in the stickiness, conversion rates and navigation effectiveness of HubSpot's websites. HubSpot is a cloud provider of customer relationship management, sales and marketing software.

What led Knight to drunk user testing? In this post, I share that story and the advice Knight gave in his session, "UX Insights from a Drunk Guy," at the 2016 O'Reilly Fluent Conference in San Francisco this week.

Knight came across Will Dayble's user interface testing tutorial, called "The user is Drunk," on YouTube. In it, Dayble posited that a user interface design should be so simple that even a drunk person could use it. Remember, he said, a person with an IQ of 160 still has that IQ when drunk. "They just get more pedantic and more annoyed." Dayble is co-founder of Squareweave, an Australian CRM and Web application development services provider.

Dayble's insights inspired Knight, and he got funding to hire a drunken consultant to evaluate HubSpot's website. Yes, there are consultants who will get drunk and evaluate a client's website. Knight chose Richard Littauer, a veteran designer and founder of

The HubSpot site had been created by internal UX and UI experts, and was, to their thinking, damn good. The drunken user didn't agree. In the conference session, Knight shared some of Littauer's criticisms, which included:

  • There was too much happening on the homepage. He stared at it for a long time, and couldn't figure out what the company did and what it was selling.
  • He couldn't figure out how to find the product sections.
  • The homepage evoked little emotional response, especially not the "warm, friendly feeling" HubSpot wanted to convey.
  • He couldn't understand the site's descriptive term for HubSpot's products, which was "inbound marketing and sales." Looking dazedly at the homepage, he said, "It sounds like you're meant for large-scale clients who are willing to spend money on buzzwords."

These and other criticisms "sent shockwaves through our company," Knight said. Obviously, "none of the magic was coming out" of the company's website. "We looked like just another [software as a service] company."

Knight and his team began re-evaluating their user interface user testing, seeking means to draw similarly honest feedback. Customers' reviews of HubSpot products on social media were positive. They described the business value of the products in glowing terms. One customer proposed marriage. "We could see the community impact in hashtags," he said.

How, then, could the UI team bring that community magic to the homepage? UI designers found a number of means to that end.

  • Communicate the "magic" as described by unsolicited feedback found in social media.
  • Emulate the presentation of HubSpot's salespeople: Show what the products do. Don't tell people about them.
  • Put multiple qualifying questions in user tests to gather information about how to create navigation paths for users with various levels of technical experience.
  • Design so that even a drunk user can immediately deduce where to click next. Organize information intuitively, so the site visitors know where they are and where they're going.
  • When testing the effectiveness of user tests, start out with pilot test runs, with three to five users. Larger samples deliver data that is too complex.

Knight said he considers the rainbow spreadsheet to be the best tool for viewing user test and other UX research results. It represents all the data collected during a UX study in a color-coded format.

Other types of user test tools Knight called out favorably in his session include:

  • Site interaction tools, such as Behavior Flow in Google Analytics or the Retention Analysis in Mixpanel, to track user behavior;
  • An A/B test tool to reveal which version of a webpage or other user interface best delivers a user's business goals; and
  • Crazy Egg to see where users click.

As for specific tools, Knight recommended the following:

  • All-in-one platforms, such as UserTesting, UserZoom and TryMyUI;
  • Screen recorders, such as Silverback, Screencastify, Loop11, WebEx and Google Hangouts;
  • Participant recruiting, such as uTest, Ethnio or Hotjar, Amazon Mechanical Turk and Knowbilly; and
  • Blended data, such as Usabilla, UsabilityHub, Optimal Workshop and Inspectlet.

What was the end result of testing a drunken user? Focusing on the wants and needs of the user, as determined by improved user-testing techniques, Knight's team redesigned HubSpot's homepage and website. "We saw conversion rates go up to a 22% increase in clicks to product page," Knight said. The user exit rate went down 18%. Also, user pain point behaviors went down, as fewer clicked to the FAQ, did search queries or made misguided clicks.

Overall, Knight recommended drunk user testing, but he warned that it's not for the faint of heart. "It is easy to dismiss harsh criticism," he said. "We turned it into something with business impact."

Next Steps

Bringing together UX testing and software testing

Boston Heart Diagnostics confides in UX design and UI development

Developers should prioritize UX development in 2016

Can prototyping avoid an unnavigable UI?

Dig Deeper on Topics Archive