Design debt

I have always been a major proponent of iterative design (namely Lean UX) and at this point, I’d be hard-pressed to name a successful SaaS company that isn’t leveraging experimentation, rapid iteration, and data-backed decision making in their design process.

Derivatives of agile methodologies can be found all over the place, and when coupled with tactics like A/B testing and hypothesis-driven experiments, they’re pretty powerful. But when applied over time, they can yield an important side effect that’s rarely discussed.

The side effect that I’m referring to is something called design debt. You’ve probably heard of technical debt, and this is a lot like that. But where technical debt affects the integrity of the codebase, design debt affects the integrity of the user experience. This is what happens when a bunch of incremental changes collect over time and yield a disjointed, inconsistent, and patched-together experience. With each successful experiment, a small amount of design debt is incurred as the design moves slightly away from its original, cohesive roots.

“Design debt affects the integrity of the user experience.”

Design debt and interest

When a new design is born, it’s debt free. Think of a website or a product after a full design refresh. Everything is cohesive, intentional, and well-thought out. It’s as the designer originally intended, and as such, each element in the design was created with a certain awareness of its surrounding elements. The navigation is aware of the CTA, the CTA is aware of the video, the video is aware of the social feed, and so on and so forth.

This sort of reciprocal awareness between elements is what creates a notion of consistency and cohesiveness in the design. But while new designs excel in consistency, they also leave a ton of room for optimization. We collect data on the new design’s performance, determine where areas of weakness or opportunity may exist, and release an experiment. When a new variation of an element wins, we integrate it into the design, bringing us one step away from the original. This is good because we’re ultimately improving the design. But it comes at a bit of a price.

While these new elements may be created with an awareness of their surrounding elements (sometimes they aren’t), no matter what we do, the original elements simply won’t share that critical awareness of the new elements. The original elements weren’t created within the context of the new elements. This is where we start to incur design debt.

As design debt builds, it accrues interest that compounds on itself over time. The more new elements that we introduce into the design, the less elements were created within the context of each other, and the more disjointed the design becomes (despite the fact that it may be performing better quantitatively). Eventually, we have to break the cycle of interest and pay down the debt by refactoring the design.

Common causes

When designs aren’t regularly refactored, they start to collectively form what may look like a digital version of Times Square (an entertaining example of this would be Ling’s Cars). Numerous disjointed and unrelated elements in a single field of view, all of which are competing for the user’s attention and may even be solving for different goals. Projects that are especially at risk for this commonly include a few of the following characteristics:

• Fire drills. When teams are dealing with aggressive goals and tight deadlines (sometimes compounded by a lack of true direction), they’re prone to hack together elements and test them in pursuit of quick, short-term wins. This comes at the expense of long-term viability and it incurs substantial design debt. Making sure that you set reasonable deadlines and expectations (coupled with a clear direction) can help remedy this.
• Competing teams. Especially as organizations scale, they may have multiple teams working on different areas of a single product. When these teams suffer from competing goals, conflicting opinions, or poor communication, they’re very prone to build designs that are inconsistent and lack cohesion. Having clear alignment on design goals, strong leadership, and healthy communication can help solve this.
• Feature testing. When a product doesn’t have a clear direction or “north star,” teams are more prone to test multitudes of features in an aim to “see what works.” This is great for closing in on product/market fit, but as more features are tested, more debt is incurred. For early products, this is somewhat unavoidable and the best remedy is regular refactoring. For more mature products, there is still room for this (pivots can still play a role), but it should be more deliberate.
• Technical Debt. Skimping on the codebase can also have effects on the integrity of the design. This is especially the case when organizations try to solve infrastructure problems through design. Whenever a design change is proposed, it should first be determined whether the problem at hand is best served by a design solution.

Balancing data and intuition

An interesting phenomenon in data-backed design is that while data can take a good design and make it great, it can’t take a bad design and make it good. Data isn’t a means for creative direction; it’s a means for optimization. And any great design is going to need a solid amount of both in order to be successful.

This is why, in recent months, many designers have publicly shared an angst for data-driven design, and expressed the importance of intuition in design. They’re emphasizing the need for a balance between data and creative direction; machine and man.

“More data does not necessarily equate to better design.”

Just as design used to be run exclusively by intuition, the dawn of big data caused it to be run exclusively by numbers. Neither of these approaches are healthy or responsible. More data does not necessarily equate to better design, and neither does more ego or more intuition. The best approach exists somewhere in between, where we focus on creating designs that are both data-informed and human-centered. In the iterative or experiment-driven process, the best way to do this is through multiple small iterations followed by a refactoring.

Preventing debt

During the small iterations that lead up to a refactoring, there are several things that we can do to extend the life of a design and maintain its health along the way. Practically speaking, these preventative measures just ensure that we’re always doing our best to produce high-quality designs and only take on debt when it’s absolutely necessary.

• The Boy Scout Rule. In Boy Scouts, they have a rule that says “always leave the campground cleaner than you found it.” When applied to design, we should always strive to leave the design in better shape than it was originally. This increases the longevity of a design and maintains a higher standard of quality throughout its lifetime. Matt Rheault, one of my co-hosts on the UX and Growth podcast, discussed this concept in-depth during our episode on technical debt and design debt.
• Strong design leadership. Debt is most easily incurred when there isn’t a clear vision for the design and key goals (or “north stars”) change regularly. Design leaders should know where the design is going, what is being tested (and why), what standard of quality needs to be upheld, and when refactoring needs to occur. In order to do this, they need to be provided with enough authority to fundamentally drive the direction of the design, while still affording a deal of autonomy to the designers that work for them.

“Let’s focus on creating designs that are both data-informed and human-centered.”

Eliminating debt

When a design has taken on a substantial amount of debt and it needs to be paid down, the best approach is to initiate a refactoring as a large iteration in a process that otherwise contains small iterations.

• Refactoring. I mention “refactoring” instead of “redesigning” because this shouldn’t be viewed as a ground-up creation of a new design (that would defeat the purpose of an iterative process), but rather as a full integration and reorganization of the new structure that we’ve experimentally developed. Throughout this process, we’ll universally implement the learnings from our experiments, consolidate our design patterns, and restore context and consistency.
• Small iterations, followed by a large iteration. The refactoring process can be viewed as a large iteration that ties together the changes made by the multiple small iterations that came before it. While the small iterations were focused on quickly learning (and incurring debt), the larger iteration should be focused on universally applying those learnings (and paying down debt). This is critical for maintaining the integrity of a user experience and it should happen on a semi-regular basis, depending of course on how frequently we’re running experiments and implementing changes.

Adopting an iterative and experiment-driven process is undoubtedly one of the best ways to learn about an audience and develop a design that truly works. By all measures, designers should be doing more of it. But it’s important to always consider the debt that can be taken on by a design over time, and plan for refactoring as necessary.

This stuff doesn’t come in the “A/B testing manual,” but it’s a critical part of the experimentation process.

This was originally posted on Austin Knight’s blog.