After historically running the Iowa Caucuses reporting via paper score sheets, the Democratic Party tried a different approach earlier this week: A new vendor-made app designed to reduce the time it takes to announce results from the nearly 1,700 precincts. And this digital disruption would have speeded up the process, had the app not been fraught with coding, connectivity, and usability issues. While currently gaining new national attention, these issues aren’t anything new to those in the tech world. But this moment in the political zeitgeist offers a great opportunity for designers to reflect on how the everyday issues they face can create chaos on a national scale.
InVision’s Aarron Walter, VP of Design Education, and Liz Steelman, editor of Inside Design, spoke with Dana Chisnell, an expert with over 20 years of experience in election and civic engagement design, about what went wrong in Iowa and what we can learn from the situation.
A bit more about Dana: In 2013, she cofounded the Center for Civic Design and from 2014 to 2016, she worked as a “generalist problem solver” in the Obama White House. In 2019, Apolitical named her one of the world’s most influential people in digital government. Earlier this month, Dana joined the National Conference on Citizenship (NCOC), a 75-year-old, federally-chartered non-profit started by Eisenhower and Truman to harness patriotism after the end of World War Two. She’s partnering on a new incubator focused on bringing user-centered design back into policy-making processes and promote civic engagement in a time when democracy needs strengthening.
Aarron Walter: I gotta say, as a person born and raised in Iowa, this makes my stomach churn. What happened Monday night and what’s your general perspective?
Dana Chisnell: Both of the major political parties held caucuses: An in-person meetup to show which delegates the party prefers in the presidential campaign. The Iowa Caucuses are run by the political parties, which is unlike other elections we’re familiar with. Republicans also had a caucus, but nobody’s talking about that. The Democrats had the one that turned into the dumpster fire.
The Democratic National Committee had decided six months ago that they wanted to make some changes to the approaches they’d been using. They hired a vendor to make an app to make it faster to count the results, so they could announce them more quickly. Normally, precinct captains fill out score sheets on paper and then they call in those numbers. There are [nearly 1,700] precincts—that seems like kind of a lot. This is always a problem in elections and it’s going to continue to be a problem in elections. [The DNC] for whatever reason decided that was not an efficient way to do it. You could imagine a world in which everybody sending in their results through an app could speed things up if the backend works well, if connectivity works well everywhere, and the usability is also in place. Iowa was the first place to try some of these changes, and turns out, none of those things actually worked.
AW: It’s my understanding that what happened was a mix of usability issues and dodgy connections, but also a lack of testing in the real conditions that it would be used.
DC: And there are ways to simulate that. Large development teams in both private and public sector organizations have been doing this for a while. There’s a role called system reliability engineer, that is their job to do that stuff. The evidence would suggest that had not taken place.
AW: Do you have any insights on the usability shortcomings and what happened?
DC: It’s hard to know if users having difficulty downloading and then logging into the app were true usability problems. While reliability and response time is part of the experience, I don’t know if you could take apart the interaction design as a specific element. One thing that is driving me nuts in the coverage is that reporters point out a lot of the precinct captains are older people. I am very close to being in that category, and and I just want to say for the record: A bunch of us designed the internet. Any user issues aren’t because there are older users, it is because it is not designed inclusively and accessibly enough.
AW: My mother and father have been precinct captains multiple times, and I’m pretty sure my dad was last night. I learned DOS from those people.
DC: Right! That’s what I’m saying. I’ve only seen a little bit of the UI. It looks like standard iOS stuff that implemented typical interaction design conventions. But a thing that we know is problematic in all kinds of apps is on-boarding and app authentication. Imagine that you’re in a room: Let’s say it’s a middle school gym. It’s eight o’clock on caucus night, and you’re supposed to be reporting the numbers. For whatever reason, you haven’t had time to download the app and get started. But you’re out in some rural place where there’s not much in the way of cell signal. There might not be WiFi in the middle school gym for whatever reason. And if there was, if you probably need a password to get onto it as a student or as faculty, but you don’t have it because you’re neither of those things. Now, you are dealing with the cognitive load of all those stressors. I can only imagine that you’re just going to give up because the important thing is to capture the numbers from the caucus. You’re going to find the paper worksheet, and the phone number, and fill out the paperwork sheet and try to call it the results.
Liz Steelman: From reading the coverage, it seems that the app was designed for a user, but not the users in mind.
DC: That seems like a reasonable take away from this. It’s not clear that there was any usability testing done. And while you might start with best practices (e.g. conventional interaction design, maybe a borrowed design system from the field), there is nothing like observing people interacting with it to learn whether it’s going to work in real life.
AW: I know you’ve done a lot of work on election design. Are there core principles that might have helped design a better system?
DC: Again, it’s not clear what happened, but as Liz was saying, it seems like it was designed for a user, but maybe not for the user. I don’t know how much the team knew about how caucuses worked. As far as election design goes, caucuses are rare and unusual. Some of the principles that apply when you go to a polling place and you get a paper artifact that you then put a mark on and then put into a box, really don’t apply to a caucus situation. But there are classic things about visual and document design that are important and have proved to work well for all kinds of users in all kinds of situations for decades. Simple things like “use mix case,” “align everything on the left, not in the center,” “make sure the targets for interaction are clear and easy to hit,” “make sure they’re good enough contrast between the type and the background,” “use the thing in whatever lighting conditions there’s going to be.” Anybody who’s reading this would probably say the same thing.
To their credit, most government, local, and state-elected officials work really hard at delivering a great voter experience. They actually see security as their number one job and understand that if it’s not usable by all the people, it’s not secure. They pay a lot of attention to training poll workers, for example, to make sure that they not only care about the custody of the ballot, but they also care about conveying a good election and interacting with voters. One of the great things about elections is that they’re naturally iterative and they’re most of us don’t really notice, but that the election officials work on all the time. For example, the presidential primaries that start in March (most of them are on March 3), election officials started last November to update their poll worker training and their websites with the presidential primary-specific processes. They built-in evaluations of issues they saw in their November elections to make improvements for March in all different ways, from vote by mail envelopes, inserts to ballot design itself, and what the processes are like in polling places and in the election offices.
LS: Do you think there was a miscommunication around expectations—not only where technology is, but how it can be implemented?
DC: I don’t know what the catalyst was for this company making this particular app, but I’ve heard the general atmosphere in Democratic political campaigns is that they have terrible tools that don’t last from one campaign to the next. There is a lot of interest in creating products that prove that digital can do a better job than what already exists. You probably saw from the media coverage that Iowa had used an app from Microsoft that didn’t work either. It’s not clear what problem was trying to be solved here. If the problem really was that they wanted to report the results quicker, it is not clear that the method they chose was actually faster than [almost 1,700] people calling in to phone lines and having the tally board at HQ. There are times when digital is not the answer.
But outside the campaign world, my experience has been that when things like this happen, it’s not because there’s bad intention or bad ideas. It’s usually because there is the combination of insurmountable constraints and leadership or stakeholders who aren’t given a clear status about what’s actually happening. Anything else I would say about this would be pure speculation, but I believe that the team that made this tried to do their absolute best with what they had to work with. My understanding is that the vendor had only about two months to make this thing and launch it—crazy. I feel bad for some of the folks who might have designed the front end or were responsible for network resiliency and reliability. They did not sign up for the kind of coverage that happened. I’m just going to guess that they didn’t want to release the thing on the schedule they had either. I’m going to assume the best intentions on everybody’s part.
LS: Do you feel like when small start ups bite off more than they can chew, it sets back the larger public trust in technology or makes it harder for companies to have an opportunity to build a more perfect system in the future?
DC: There’s this mystique around, “We can have a hack-a-thon and we’ll make a thing over a weekend and it’ll work and it’ll be great.” That is a sketch from my point of view. I don’t know if that’s what they did, and I don’t know anything about their internal process, but it feels like they shortcut a lot of the process. I don’t know what their inputs were: Did they just get business requirements and data flow diagrams, or have they been active observers of this process before? That would have gone a long way and made a big difference. If they had started maybe six months ago, when the DNC decided this, it might have been enough time for discovery work, user research, and even a mock caucus. That would have given them the opportunity to rehearse, role play, and experimentally train—not all, but some—precinct captains.
One of the things I don’t think we talk about enough is this “Move fast and break things” attitude. I think we need to slow down a little bit and actually go through the steps. There’s this misunderstanding of design as kind of a “one size fits all.” If I’m designing a paper airplane, I could do that really fast with a little bit of testing and get great results. But if the stakes are higher and it’s a passenger airplane, I probably need to slow down and double check even triple check to make sure that I’m following the process really well because I can’t afford to have major mistakes. We see in the high stakes situations when you don’t do all the testing that you need to do, that people die. As far as I know in this country, people have not died because of [tabulating] election results in this country, but they do in other countries. I don’t want that ever to happen here.
LS: A lot of the time, the technology works, but it’s underutilized because users don’t trust it. In this case, there were reports that people chose not to use the app at all.
DC: A bunch of them learned about [the app] the day before. This is not a simple straightforward process that they take on. Being a precinct captain and running the caucus in your middle school gym is a big job. If I found out about a new app the day before, I’m going to say, “No, thank you. I will go analog.”
We’re in a political atmosphere where a lot of people are talking about how you shouldn’t trust anything digital in elections. That we should all have hand-marked paper ballots and they should all be hand-counted. That is just a non-starter for a lot of reasons. First, let’s take a large jurisdiction like Los Angeles County that has 5 million registered voters. They’re not going to have 100% turnout, but even if they have 60% turnout, that’s still a lot. They’re not going to hand count because nobody will stand for that and it’s not accurate. There’s a bunch of science around hand counting, how it’s not accurate. It’s not practical. Local election offices do a lot of work to secure their systems. When you have a completely manual process, this cuts out a lot of people. It’s not inclusive and universally-accessible. Nobody who is talking about paper ballots has a good answer for that.
LS: Do you have any recommendations for the individual designer or design leader looking to build markers of trust in their designs when building the next voting app?
DC: Everybody’s doing this after the 2016 election. At the Center for Civic Design, we got probably 100 queries from people saying, “The system is broken, and I’m going to fix it. Let me show you my app!”
But most of them were approaching problems that the person wanted to fix for themselves, not actual electoral issues. They’re great for student council or the neighborhood association. They might even work for a caucus, but they’re not going to scale very well to a larger general election.
If you’re a designer who wants to get involved in election design and you’re frustrated by what’s happening: Work the polls on Election Day (if you work in a state that isn’t all vote-by-mail, which four states are now). You’ll meet all kinds of people, learn about how elections work from the inside—what some of the constraints and considerations are. You’ll learn about what operations are like, at least for that small window of time. There’s a lot that happens upstream and downstream of Election Day, but working polls is a great bit of civic duty that you can use as field research.
This interview has been edited and condensed for length and clarity.
