Chatbots: Your ultimate prototyping tool

Conversational UI is getting a lot of buzz out in the world, and for good reason. We’re in an exciting moment as we observe existing technology, design zeitgeist, and user needs aligning around leaner, more immediate, and human ways to engage with digital services.

SMS is going through a renaissance, and designers are looking at this medium and interaction model to create new services and experiences. The immediacy and accessibility of SMS make it an ideal channel for building services in a world where systems continue to converge and apps are no longer the medium for digital experiences. As technology becomes more complex, users will need even more simplicity to engage with digital experiences.

This article was originally published on Medium’s IDEO Stories.

Beyond the buzz, what conversational UI shows us is the idea that everything in design should be considered a conversation. All elements in an experience are exchanges in a conversation amongst people, a brand, a business, a service, and, increasingly, a machine-powered intelligence. In this sense, conversation is the primary design material for designers—and its general purpose is to impact or inspire people’s behaviors and emotions.

Designing conversations

Innovation is not rocket science. On the contrary, innovation begins simply by talking to real people, in real contexts. By putting ourselves in their shoes, we begin to truly understand their needs, motivations, and desires.

Chatbots are exciting, but we’re also excited about what they unlock for us as a learning medium, not just as the next big thing. While these days there’s a strong focus on chatbots and using conversations as end products, in our IDEO New York studio, we’ve been working for the past few years on using conversations as a prototyping tool to test assumptions, learn about a topic, and to inspire the design.

Here, we share our experiences designing conversations to learn from real people, and to design real products.

Stories of 3 chatbots

Below are 3 experiences prototyping with chatbots: to learn about user behaviors and mindset around health and fitness, to learn about culture and knowledge related to a health issue with local populations, and to test an early messaging platform to gain insights for the design of a messaging platform for social workers.

Walkbot: Prototyping content and engagement

How might we better understand the nuances of what our participants need?

In 2014, together with a Japanese electronics manufacturer, IDEO designed a device and smartphone app that captures a woman’s activities throughout the day in order to generate thoughtful, actionable insights about her individual fitness choices. While we were working on this product, we had to tackle a brief that was quite broad: what are the unmet fitness needs of American women, between age 35 and 54?

“If I’m going to interface with something that’s recreational, I want it to be entertaining. I don’t want it to hold me so accountable that I want to throw it against a wall.” –Anonymous user

To explore this opportunity, we crafted several prototyping sprints to better understand the needs of these women. At the time, the landscape of wearable devices was very heavily marketed towards men: data-heavy, competitive, and physically unappealing for women. The women we were addressing wanted meaningful content, were far more focused on competing with themselves, and wanted something more discreet. Once we landed on a product, we took a step back to learn whether or not the product was meeting those needs. Turned out our product was also too data-heavy.

“I like that it’s there if I need it. It’s kind of like a little friend.” –Anonymous user

We decided to pivot in order to uncover what meaningful content looks like for these women. Is it a conversation? Is it one-directional? What type of tone or personality works best? What do they consider to be helpful or meaningful? Is content about being a sounding board or offering ways to push fitness?

We wanted to test this out in real-time without building the real thing. Our constraints were clear—use only the information that we’d normally get from an app’s on-boarding experience to craft parameters like gender, preferences around activities and metrics, and keep the mechanics simple so we could get to the meat of the problem and not get lost in the weeds.

A software designer on the team built a custom tool leveraging Twilio’s SMS API, and created an SMS platform that a team of people could use to manage and direct messages. The tool itself would send out messages from one source/brand. We recruited a panel of women, and during their workouts, a team member would send messages such as “great workout, keep going” or “you’ve walked _____ steps today, add a 5 minute walk to reach 10,000.”

Lisa: “Ugh, he wants me to pick up my intensity and burn more calories.”
IDEO: “How do you know it’s a man?”
Lisa: “Because I’m mad at it right now.”

What was interesting about the prototype was uncovering unexpected learnings. Women who participated in the design research would project gender and personality to our bot based on their interactions with it. What participants assumed to be a pushy male coach as the bot was our IDEO female colleague sending messages in real-time while walking side by side with the participants.

The responses from our users were also far more nuanced than we expected. Women wanted to share not only their activities, but moments in which they failed or were simply being human, such as the times in which they caved and went ahead and ate a chocolate bar. Those moments mattered just as much for telling a holistic fitness story. In other moments, women assigned different genders to the bot depending on the response. In moments of frustration, the bot was referred to as a “he.” To these women, fitness wasn’t purely about achieving a goal, but also the process and dialogue in getting there.

Considerations and learnings

• Learn by doing. The best way to understand what a conversation needs to be is by starting the conversation, and figuring out tone of voice, pacing, and interactions—as the dialogue unfolds.
• Conversations are about people. Making conversations feel more personal is possible with only a few data points. It implies being intentional and creative around what we know around the interlocutor, and progressively build and evolve from there.
• Bots are personas, whether or not it’s intended. Every participant will project an identity of the bot, its gender and personality—whether or not it has been created intentionally by the design team.
• Be willing to pivot from happy accidents. Bots allow for a nuanced interaction that might bring surprising elements to your product. Be ready to shift, in case you uncover an unexpected direction that seem promising for your goals.

Milton: Prototyping a service and product

How might we build a more effective service for connecting New Yorkers to public benefits and services?

Milton was a bot we created to test out a text-driven call center—a kind of “integrated-and-everywhere” support meant to arm people with the knowledge they need to navigate the public benefit system, whenever they need it and wherever they are.

The service would answer questions when possible, and direct clients to additional resources and other channels when appropriate. SMS felt like the obvious solution for the following reasons:

• It’s immediate and accessible
• It’s responsive and meets people where they are
• It can be automated and human
• It’s once removed, allowing for anonymity
• It’s asynchronous, which allows for graceful stops and easy exits
• Its lower-touch interactions can be more frequent, and more frequent means more data

“[The prototype]—it was just a text message. It was an answer at my fingertips while I was cooking or while I was taking care of my kids.” –Anonymous user

Our team of 4 designers had limited knowledge in how to advise people going through the public benefit experience, so we hired two Single Stop case workers as advisors, created our custom bot, and set out on a 2-week texting experiment to simulate an SMS call center experience. We weren’t sure what kinds of questions or cases we’d receive, let alone what the expectations would be.

Could we really support people remotely as they sought out a particular benefit or service? What should be the appropriate tone for people in crisis mode? Can we establish trust and rapport through a medium like SMS? And how easily is a problem resolved over text?

The requirement for each of our participants was that they go through all of the steps necessary to apply for a benefit, and that they would text us their questions, concerns, and feelings throughout.

“I don’t want to be treated that way.” –Anonymous user

During our 2-week prototype, we stayed connected to our participants and even managed to get one to successfully apply for a benefit. We learned that people have different modes for conversation. Some were looking for logistical answers in which case an automated/impersonal response was fine. Others were looking for a sounding board and we discovered that a small, personable gesture of support can have a huge impact. And still others were trying to simply solve a problem and needed clear directives. We also learned the art of keeping the conversation open-ended—if you’re intentional about “leaving the door open,” people are more willing to come back. And finally, through our prototype, we learned the vital lesson that support is just as much about the right timing as it is about the right resources.

Considerations and learnings

• 24/7 is not a requirement. You have a lot more permission than you realize and can design for that with automated responses that communicate hours, waiting time, and set expectations with users around your service.
• Treat people like people. People are smart, sophisticated beings: Treat them with respect, responsibility, and honesty.
• Talk like a real person. Design the SMS service like a real conversation—simple things like starting the dialogue with a friendly “hello” and ending it with a proper goodbye, establish respectable boundaries, and have a clear follow-up.
• Adapt for different modes of engagement. People have different modes for conversation and the service should adapt to that.

Zikabot: Prototyping a service

How might we listen to and build empathy with the communities we want to help?

In May 2016, IDEO embarked on a pro-bono design sprint with a public health organization that was 90 days into the ongoing emergency response to the Zika virus in Puerto Rico. The organization needed a way to quickly evolve their hypotheses about engaging Puerto Ricans in protecting pregnant women from the virus.

After an initial round of in-person interviews revealed that many in Puerto Rico lacked accurate health information about Zika and were skeptical about the gravity of the virus, the team sought to prototype engagement channels that could increase awareness of the virus’ potential impact on the population. One of those prototypes was Zikabot, a chatbot service for Puerto Ricans to ask anonymous questions about Zika and receive accurate health information in return. To advertise it, we plastered signs on the side of a car that was re-branded as Zikamobile, and handed out about 150 cards in a mall. Users texted their questions to the Zikabot, while on the back-end a team of subject matter experts and IDEOers answered as the bot, in certain cases debating internally on appropriate answers around the inquiries.

“From the point of view of the bot, which I played briefly this afternoon, I felt such responsibility in a way I hadn’t felt when speaking to people. It was interesting. People can be very casual when texting. I worried about the nuances of how I worded my responses—bad writing could put a person in danger!” –Anonymous user

The questions people texted confirmed the team’s insight that many people in Puerto Rico lacked basic knowledge about the Zika virus. Furthermore, the team quickly learned that tone was vitally important. When some users expressed distrust in the formally worded responses they received, the team was able to immediately adjust their writing style to test a more conversational approach. Finally, it became apparent that the anonymity of the chatbot emboldened users to ask far more personal and revealing questions than were asked during the in person interviews. “Why is the Zika virus in Puerto Rico?” “How is it possible that the same mosquito can spread three different viruses?” “Does Zika affect pets?” “I’ve had a lot of intercourse lately—what should I do?”

Overall, the Zikabot service proved a useful tool for the health organization to distribute accurate information about the virus in a resonant way, as well as to keep their finger on the pulse of a population from which they felt largely disconnected. However, uncovering the ways in which people might want to be spoken to in this kind of conversation was just as meaningful.

More broadly, during our time on the ground, it became clear that when people were given all the facts about the virus, it dramatically changed how they viewed Zika and its impact on public health. Our team learned that these communities crave the why as well as the what. We found that agile, iterative, human-centered design methods consistently won over the Puerto Ricans we met, and made them want to join the fight against the Zika virus.

Considerations and learnings

• Being accountable to real people means taking on real responsibility. This prototype had considerable implications in terms of ethics and impact. We had to put extra awareness and intention in making sure that the service was reliable, protected, and that the messages were truthful and validated scientifically.
• SMS can be a safe space for tough topics. Have a clear objective, especially with sensitive topics.

Chatbots to learn from people, and to design real products

During the past years the interaction design and design research teams at IDEO New York worked closely to find opportunities to engage with real people early on in the process to test value propositions and the desirability for digital products and services. As part of this collaboration, we built ad-hoc tools that helped us designing conversations with users, and engage with them in real-life contexts, to unlock empathy, and to individuate opportunities for the design process.

In this article we covered the experiences we made through custom-made SMS chatbots—Milton, Walkbot, and Zikabot—and we dove into our considerations and learnings for each one of those. With Milton we prototyped a communication service for social workers. With Walkbot we engage with women engaging with fitness to test language and messaging. Finally, with Zikabot we immersed ourselves and our partners in a local context to learn more about the level of preparedness and shared knowledge around the issue of Zika and its risks. This insight helped us iterate all messaging and content to a more accessible and straightforward language with the intention to demystify misconceptions and rumors regarding Zika.

Although these 3 contexts are radically different from each other, for all of them we went though a set of principles that we followed when we planned and ran these chatbot-based prototypes.

Start with a question.

A prototype is a question, embodied. What are you trying to learn? Who are you learning from? What’s the ask of your users?

Establish a set of parameters.

Test your assumptions. It’s not about getting it perfect and figured out, but setting up just enough to get the conversation going.

Build an initial content library.

When we say content, we mean: think about the tone, the types of content, when and how you exchange content, the frequency. Is it manual or automated?

Hack existing tools.

There’s no need to build the real thing from the get-go or reinvent the wheel.

Iterate in real-time.

The beauty of SMS is that you can quickly adapt in real-time to build on your learnings.

Be mindful.

Even though you’re working on an early prototype, the interactions might occur with real people that might not be completely aware of your intentions. Be explicit about the context, and make sure that your actions are in line with the ethics of field research.

Let go.

Let the conversations evolve naturally, and be open to learn about the unexpected.

Look back, look forward.

Once you’re done with your prototype, look back at the stories, anecdotes, and insights you gathered through your chatbot, to inspire the design of the service, content, or product.

Meet Stanley

So at this point you might ask yourself this: How can I get started with my own chatbot to prototype and learn? There are 2 ways to get started. 

There are several services that can help you setting up an SMS-based platform with little effort. Alternatively, you can create your own conversation bot following the instructions we shared for Stanley, the SMS-based platform that we have been using for the stories we shared in this article.

You can find the source code, and the instructions to setup Stanley here.

For some more examples on projects using bots for design research, you can read an article on IDEO Labs by some of our friends and colleagues here.

The Little Book of Design Research Ethics we made at IDEO aims to provide practical guidance for user research. The book is a guide on how to seek and share insights about people’s lives in an ethical way.

Thanks
To Amy Wang for the illustrations; to Anna Silverstein, Jackie Cooksey, Peter Olson for helping with this story; and to everyone at IDEO and our partners involved in the projects that were mentioned in this article.