How to measure the impact of your work

As designers, we know our work has an impact. We see the power of design in every aspect of our lives. But not everyone is able to connect the dots between design and business growth. It is up to us, as designers, to help tell our own success stories. When we’re able to demonstrate the value of our work, design becomes a critical contributor to the business. We can advocate for bigger design teams, higher salaries, and more integrated design systems.

This article is for anyone who wants to get more involved in product analytics. Far too often, designers don’t have direct access to the tools they need to measure the impact of their work. If this is the case on your team, I hope this post can help you overcome those barriers.

“We can only elevate the role of design if there’s a proven track record of its business value.”

Define your impact

Teams are often chasing conversion goals that aren’t well defined or understood. It’s important to build a shared understanding of what you’re going after. Start with the goal of the project you’re working on. Break that goal down into units you can measure.

Related: Measure UX for better understanding

For example, say your project goal is to increase hotel room bookings. How will you know when you hit your target? A few ways to make this goal more clear would be to know:

• Unique number of booking views
• Average number of views (per customer) before booking
• Unique number of people who started the booking process
• Unique number of people who put in their payment info
• Unique number of people who submitted their booking

Consider including qualitative success criteria like booking satisfaction scores or review ratings. You can also note sentiments you expect to hear from customers like “made it a lot easier to book” and “my trip went smoothly.”

Make sure you document the goal and its success criteria in a common place the team can access. My team uses a “Project Brief” format which states these 2 key elements at the top.

Once you have clear success criteria, you can figure out what’s already tracked and what needs to be implemented. Continuing with our hotel example: Are you already tracking number of bookings? Who is in charge of that metric? Where is it tracked?

These may seem like obvious questions, but I often speak with designers who have never seen their company’s analytics tools. Airbnb takes this so seriously that they developed an internal educational program to train everyone on how to use their tools. While I don’t think everyone needs to learn to write SQL, I do think everyone should know what’s available and how they can get more involved. If your team makes decisions from data, data must be accessible to everyone on the team.

Measure quantitatively

Now that you’ve defined your metrics, you’ll want to get your analytics tool setup to support your project. Hopefully, your team has an analytics tool in place. If not, check out this post from Amplitude on setting up your analytics. I recommend looking at Amplitude, Mixpanel, Heap and Kissmetrics if you are starting from scratch.

If you have not yet been involved in analytics, here are 3 key concepts to know:

1. Event-based analytics

Most product analytics tools are event-based (in contrast to page-based tools like Google Analytics). An event is a discrete interaction someone has within your product. For example, “viewed_booking” or “booked_room.” When you use events instead of screen or page views, you are able to better understand the details of an interaction.

2. Segmentation and funnel views

Anyone who has ever seen an infographic knows data can be presented in a plethora of ways. I use 2 key views when I’m tracking a project: segmentation and funnel views. A segmentation view shows you how many of one event fired in a given time frame. For example, how many times did people fire “booked_room” in the last 7 days? A funnel view shows 2 or more events in succession. Each tool handles each of these a little differently and it’s helpful to read their specific documentation.

3. Path view

Event-based analytics tools will often include a way to view an individual user’s path. This is a designer’s dream tool—you can see how people navigate through your product moment to moment. I guarantee it will surprise you.

The first thing you want to do with a new project is verify that existing events are firing correctly.

To do this, you can look up your user ID in your analytics tool to pull up your live event view. Then, use your app and watch the events as you go through an activity. You’ll see an event stream like the one above and the events should be in order based on how you interact with the app.

Next, you’ll want to capture your baseline metrics. On your project brief, note the current percentages and absolute numbers for all your metrics. For example, what’s the current rate of conversion for your key metric? How many total people are booking rooms? What’s the total revenue of the bookings? I typically go 3 to 6 months back to capture numbers, depending on how much has changed in the product. I also recommend using unique numbers (number of users firing an event) rather than totals (number of events fired overall).

Once you’ve verified your baseline metrics, you can define any new events you’ll need to instrument to track your success. Here are some guidelines for creating new events:

Follow your existing event naming pattern

If your events are present tense with underscores, keep that style. For example “view_booking” rather than “viewed_booking.”

Make the name of the event as clear as possible

Naming an event so it’s exactly what someone is doing in the app is ideal. Data is confusing and the simpler you make it, the faster your team can learn.

Document your events

I always document in 2 places: on our project brief and in our global event list (available to our whole company). Include the event, its properties, and when it fires.

I recommend only defining events for core parts of your product. Don’t measure everything. Only measure what you’re planning to take action on.

Finally, set up reports to track your success criteria. Many analytics tools will let you create dashboards or shortcuts to important reports. If you set them up ahead of time, they’ll be ready when data starts coming in. Make sure to include a link in your project brief and share it with your team.

If your team wants to run experiments, you’ll want to use the same metrics you defined as your project goals. Some A/B testing tools, like Apptimize, will pull in your analytics events without having to send custom events. This means you can incrementally measure your experiments the same way you track project success.

Related: How Netflix does A/B testing

If analytics aren’t your cup of tea, I highly recommend partnering with someone on your team who can fill this role. Designers should be aware of what the data is saying about the design. Whether or not you take the lead on this, you can bring a unique lens to understanding and taking action on data.

Measure qualitatively

Data can tell us what’s happening, but it doesn’t tell us why. You’ll want to gauge how customers are responding to your product or initiative. When company leaders hear direct customer feedback, it can make the quantitative data much more meaningful.

Method 1—Run a beta program

Both Google Play and iTunes offer beta testing programs. It’s easy to recruit beta testers on the web by putting up an announcement bar or an unobtrusive alert. Beta testers often give detailed feedback. You can also reach many people much faster than if you interviewed them one on one. Before you release your beta versions, make sure you’ve defined how you’re going to gather feedback. Google Play has a built-in mechanism, but for other platforms, you’ll need to create a feedback channel. I recommend setting up at beta@[yourapp].com email for receiving feedback or providing an in-app way for people to weigh in like Intercom.

Method 2—Review customer support emails, reviews, and social posts

You need to hear from people using the product to gain insight into why something is effective or not. Download a batch of support cases and reviews. Then, sort them to find top complaints and usability issues. It is an incredibly humbling exercise.

Our customer support team does an excellent job of tagging cases so I can download them by theme. I’ll combine reviews and support cases into one spreadsheet. Then I go through it with a stack of Post-its, writing down the gist of the message. As I continue, I’ll just add case numbers so I can easily see how many people are talking about that topic.

Sort them by the volume. What received the most praise? What were the biggest issues? Bring the top 5 in each category back to your team to decide what to take action on.

Method 3—Interview customers

Recruit customers who are using the feature or product and talk with them about it. How is it solving their problem? What are they using most? What gaps is it not solving? Ash Maurya’s solution interview template from Running Lean can be a great starting place for this. This article from his blog also provides helpful interviewing guidelines. If you’re relying on this method to track the impact of your work, I would interview as many people as possible (15-20). If that’s not feasible at your scale, you can try using a survey.

After you’ve interviewed people, hold a workshop to brief the team and any company leaders on what you learned. You can use the rainbow spreadsheet to share the main points. I also like to cut together an audio highlight reel (5-8 mins long) of the customers speaking directly. It’s incredibly motivating for the team to hear a customer’s feedback in their own voice. Your team will want to get back to work right away improving the product.

Start anywhere

We can only elevate the role of design if there’s a proven track record of its business value. It’s important that you get involved in telling your success stories no matter what method you use.

When you’re embarking on your next project, choose a measurement method that will help you do your best work. Find a partner on your team who can help you implement it and share your progress along the way. Decision making will become clearer and iterations will move faster. Above all, you’ll be able to show how design led the way to growth.