Editor’s note: We’ve asked a handful of design leaders to respond to prompts each week. This week’s prompt was “What key metrics do you use to track your design’s effectiveness?” Below, Matthew Smith and Mohan Ramaswamy answer. Read responses from Chris Thelwell and Jeff Gothelf, and check out Nir Eyal’s answer here.
Design should get business done
Art is my background, but I choose design because we can objectively measure the effect of our our work. I’m a fine artist by training. I used visual language to speak through both ceramics and conceptual art. A diagonal for activity. Thin edges for delicacy and refinement. Balanced elements to create a sense of rest. In art, I was looking for emotional reaction.
Design, too, uses visual language, but the desired response isn’t solely emotional—it should also inspire an action.
I see a well-designed coffee cup on the store shelf. I appreciate its form and color. I have a sense of how it’ll make me feel and how well it’ll work, and that convinces me to purchase it. I take it home and fill it with coffee.
“Design should inspire action.”
Image from Inside Design: Trunk Club.
This is the moment design is tested. How does it feel in my hand? Does the weight of the cup complement the look and feel of it? Most important, can I drink from it? Can I sip my coffee without it dripping? Does it keep my coffee warm? Does it do the job I hired it to do?
From the beginning of a project at Fathom & Draft, we’re figuring out what problems or needs our clients have and why they care about them. We get to know their obstacles and opportunities. We research, sketch, layout, prototype, iterate, and test our hypotheses until we’re getting some results that look promising, and then we double down.
Throughout our work, we look at 2 measurements of effectiveness. First, does the client feel that we’ve listened to them, and do they feel we’ve executed on what we all agreed was the best hypothesis to test? Second, and most important, we watch closely for return on investment.
We can measure the ROI based on objectives we set at the beginning of our work, which may include:
- Increase audience engagement
- Reduce friction in an experience
- Create tangible lasting connection with customers
We choose the tools to measure the effect of each objective based on the kind of data we’re looking for. We test comps and interaction in InVision or in live prototypes. We interview customers, watch them use the product, and get to know what matters to them. The scope of testing and experimenting we do is a ratio of the project size and budget—it increases with complexity. These tools are gauges to help us measure tangible outcomes of our work. Design must get business done.
Image from Inside Design: Google Ventures.
Some ROI is measured in love. Like the day I got a call from Phil and Louise at Blue Sky Resumes 4 years after we’d launched a redesign of BlueSkyResumes.com. The project was a big investment. The work was innovative—a risky site redesign. Phil and Louise were calling from their new homestead in England and their restful happiness was pouring through the phone. They’d called to let me know the last 4 years had been the most successful business years in the history of the company. They attributed the bulk of that success directly to our work.
“Design must get business done.”
That just feels good. It’s the kind of ROI that a design studio isn’t always privy to.
Thankfully, it doesn’t take 4 years to know what’s working anymore. We improve our tests and experiments so that we meet and exceed our objectives on products, sites, experiences, installations, systems, and brands more expediently and with greater effect. We lead with intuition and decades of combined experience, and track it with a wide range of metrics.
We design business, and we design products. And then we celebrate our returns.
Dig deeper to see where to measure
Good design has the ability to solve tough business challenges (increasing revenue or loyalty, for example) while also making a product, service, or experience more delightful and enjoyable to use. In fact, it’s been shown that design-focused organizations are also better businesses.
We’ve always been metrics-driven at Work & Co. At the start of any project, we partner with our clients to align on 2 or 3 core, measurable goals. The types of goals range from increasing revenue on an ecommerce site to lowering exit rates on article pages for a content site, or even driving user adoption of a new product or feature.
“Good design can solve business challenges and make a product more enjoyable.”
Starting with a few specific goals helps focus our strategy and define the key design challenges, in addition to helping us prioritize templates, features, and ideas during design concepting. It also allows us to align with our clients on what success means post-launch.
We set goals for ourselves, too, so that we’re motivated to create new design standards and launch new, better digital paradigms that push the medium forward.
The Virgin America website redesign is a great example of a project where we tried to meet both business and team goals. The project began with a simple, yet broad brief: increase online revenue in a category that has deeply competitive shopping behavior and significant margin pressures.
“Design-focused organizations are also better businesses.”
But this could mean many things for an airline: upsells of premium products and fare classes, more direct bookings on the site, or more hotel/vacation package sales. Our first task was to understand where the best opportunities existed to increase revenue, so we partnered with the Virgin America team to assess their overall revenue mix and identify the key drivers.
We discovered that the main opportunity was to focus on increasing the site’s conversion rate for flight purchases, rather than further fragmenting our efforts by aggressively integrating packages and bundles.
Our mission became clear: ensure people who started the booking process also finalized their ticket purchases on VirginAmerica.com. Instead of over-extending design and development time or fragmenting the team’s time on a number of different parts of the experience, we were able to clearly focus on booking conversion.
We spent the majority of our design time prototyping, testing, and optimizing a unique, single-page booking flow. Each design and development iteration helped streamline the booking process, generating improvements to the conversion rate. We tested and implemented everything from speeding up page load times by minimizing imagery and optimizing error-handling through real-time messaging, to simplifying decision-making by reducing the number of fares presented on a single screen.
“Design takes more than just creative genius.”
By taking this focused approach, we improved VirginAmerica.com’s conversion rate by more than 10%, while also making the site easier to use and more fun to book on. It also met the key goal we set for ourselves: introducing what Wired has called “the first radical rethinking of the flight booking experience in a decade.”
Design can have a great impact for users and a business, but it takes more than creative genius. You have to define key goals that you continually prioritize, test, and iterate. Focusing your approach—and validating it along the way—are essential for product success.
Join the conversation
Write your own response to the prompt “What key metrics do you use to track your design’s effectiveness?” on Medium, and submit it to our publication.
Mohan works with Work & Co’s clients in leadership to help them systematically plan and build their digital products, capabilities, and organizations. Most recently, Mohan has led Work & Co's engagements for Virgin America, Aldo, and Chase. Prior to founding Work & Co, Mohan was an Engagement Manager at McKinsey and helped build Huge's strategy practice.
Matthew Smith is Director the design studio Fathom & Draft. He’s worked clients like Gates Foundation, MailChimp, Seth Godin, and MIT. He’s spent years building high fidelity brands and teams around marketplace and ecommerce startups. Now his well honed opinions are hard at work with a new breed of client that demands design expand beyond interface to include installations, devices, and data—getting business done. On a personal note, Matthew has a love affair with his 1980 Honda SP400, though married to a beautiful human woman and father to three wonderful surprises. Matthew lives in Greenville, SC where he co-owns CoWork Greenville, the workspace he founded in 2007. He believes his city is better than yours.
