How to coach design experiments

Leaders know that design experiments are critical for innovation and a company’s success. But just asking teams to run experiments isn’t enough.

This week, we hosted a webinar with Alissa Briggs, Director of UX at Brigade and formerly of Intuit, about how real culture change—particularly in Enterprise environments—requires a new type of embedded design leader: the Experiment Coach.

“Real culture change requires an Experiment Coach.”

Watch the full recording below, or read on for our short recap.

Why experiments matter

At its core, experimentation is an approach to problem solving. You begin with an idea, move toward identifying the underlying assumptions of that idea, and then eventually validate those assumptions.

The process of experimentation can also be a great way to approach product design, because it allows you to explore a variety of ideas quickly and cheaply—and scientifically.

“Experimentation is an approach to problem solving.”

The word “experiment” might make some folks uncomfortable—it brings up ideas of uncertainty and a risk of failure. So, expect that you might need to rally your team—or even use a different term, like “trial” or “observation.”

At the end of the day, facing uncertainty gets you to certainty quicker, and failing quickly helps you avoid those big, spectacular failures that could get you fired.

The relationship between experimentation and innovation

Experimentation is also a big part of innovation, something many companies strive for. Alissa gave an example from her time at Intuit. Execs wanted the company to be seen as an 8,000-person startup, not a 30-year-old company. Knowing they needed to innovate, they tried bringing in speakers periodically and hosting hackathons. But once people went back to their day jobs, they returned to old habits.

So they tried another idea: they trained individuals throughout the company to make experimentation a part of how everyone worked. Known as Experiment Coaches, these people had a tough job—they had to change the behaviors and mindsets of fellow team members.

As it turns out, UX practitioners make exceptional Experiment Coaches because they’re used to talking with users and iterating until they find a solution.

How to become an Experiment Coach

So, how can you lead your team to run experiments together? Alissa showed us the experiment grid she uses that can help you plan, run, and track experiments.

It’s best to get your team together and on board as early as possible during the planning stages (the first 3 steps below). An added bonus: this will brand you as a leader/facilitator in your company.

Once everyone is on board, you can run through the process of experimentation.

1. Start off with an idea.
2. Dive into assumptions. This is the trickiest part of experiment planning. Determine the implicit assumptions that underlie your idea. Try asking: if our idea works, what must be true about customer behavior? The answers are your assumptions.
3. Translate those assumptions into hypotheses. Use the consistent structure of “If we do __, then __% of people will do __”. That structure allows you to clearly validate (or fail to validate) your hypothesis. Alissa’s advice if you have multiple assumptions? Translate the riskiest one—the one that keeps you up at night—into your hypothesis.
4. Run the experiment. As a group, decide on the quickest, cheapest way to validate your hypothesis. Remember that it’s not about creating one perfect experiment. As Alissa said, “In the world of product design, there’s no such thing as a perfect experiment leading to some perfect truth.”
5. Track the results and share publicly.

Ultimately, we fully agree with how Matt put it: