Getting started with AR and VR design

We’re in a particularly big moment for tech. Artificial intelligence and machine learning are changing the ways we conduct business, interact with each other, and understand the world around us.

Conversational bots and voice-activated assistants are becoming an expected part of our lives. Search powered by computer vision is changing how we find immediate information on topics of all kinds. And augmented reality (AR) and virtual reality (VR) are making inroads into our workplaces at a shocking pace.

Related: What to know before you start a VR design project

It seems like every day a new, game-changing piece of technology enters our lives for good. When your mission is to improve everyday experiences, as it is for us at Myplanet, these rapid tech shifts are pretty exciting.

But they also present new and complicated challenges. With every new technology comes new skills to learn and new parameters to comprehend. And as AR and VR move out of the novelty game-play space and into mainstream use, those new skills and understandings have to come about as fast and furious as the technology itself.

Adjusting to a tech-enabled future is something most of us are doing all the time, whether we or not we realize it. But as designers, we can’t afford to be so cavalier—we need to gain an understanding of how to work with these new technologies quickly so we can create meaningful user experiences that add value and minimize the potential downsides of disruption at this scale.

Whether you’re being asked to theorize and prototype possible applications for AR and VR, or you’re being tasked with a specific implementation, having a thorough understanding of the technology itself as well as a strong foundation of the skills needed to create experiences with it is essential.

I’ve worked on several AR and VR projects recently, so I’d like to share a list of tips for getting started with AR and VR design.

Get to know AR and VR

It should be obvious, but there’s nothing like first-hand exposure to something to help inform your designs.

This is actually a two-step process, but the first step is to simply experience AR and VR as a user. Expose yourself to the new technologies and devices as soon as possible—preferably before a project comes across your desk. By immersing yourself in a VR experience or interacting in real-time with an AR application before you’ve got the added concerns of a specific project, you’ll get a sense of what your users will experience the first time they interact with these new platforms.

Be open-minded to experience, but also be attentive to it. What does it feel like to wear the headset? How comfortable are you with the controls and how easy was it to figure them out? How did it feel to disappear into the experience and what would make it easier, less stressful, or more enjoyable? Take note of the good, the bad, and the confusing so that when you’re asked to design for the experience, you’re already aware of some of the potential pitfalls and major wins.

Things like user onboarding will be a crucial part of designing AR and VR experiences. Myplanet Visual Designer Yueyao Wang notes that because these experiences are new and without established norms, people will need more instruction than they would with more established situations.

“Since there isn’t a universal set of interactions that apply across all platforms/devices yet, and the majority of people are not currently VR consumers, an onboarding session at the beginning of the experience is necessary. It provides users with necessary information and training on how to interact and navigate through the content,” says Yueyao.

By paying attention to your own initial responses, reactions, and assumptions, you’ll start building a mental framework that will allow you to create a more enjoyable and meaningful experience for your users when the time comes.

Think holistically

The second step is to start playing around with it as a designer—and as a developer. Get to know the equipment that’s available now, and even if you’re not planning to pick up the hard-coding skills these new tech pieces require, gain a familiarity with how their programs work.

Andrew Semuschak, Associate Director of Design at Myplanet, recently crafted a Hololens experience for a client in the manufacturing sector, and his recommendation for first-time AR designers is simple: “Any designer working on this platform would benefit from downloading Unity and experimenting with 3D modelling.”

Yueyao seconds that opinion. “If you’re going to be doing more intensive VR design, it will be beneficial and necessary to get yourself familiar with Unity or other development tools.”

Whatever the device and whatever the platform used to enable the experience, gain as much of an understanding of it as you can. Design and development always work better when they work together, so understand not just how this new tech is reshaping your design work, but how it’s reshaping the dev work as well.

And don’t forget, the parameters for these experiences will change from device to device. As Yueyao notes, “You need to decide what you’re designing for before you begin designing, because that influences what interactions there will be in your system.”

Related: Designers with these 3 skills work better with developers

Every system will have different requirements, so narrow the field down and start to familiarize yourself with the specific hardware. It’ll make things easier for both you and the development team you’re working with.

There will be new skills on both sides of the equation, and inevitable road blocks and speed bumps and detours along the way. But building empathy for your development peers in the process starts with gaining at least a solid overview-level understanding of how it all comes together.

Consider everything

The best designers consider various external and environmental factors. We think about how sunlight impacts a wearable device’s interface, for example. But this becomes even more important when we’re designing an experience that either interacts directly with the environment (as in the case of AR) or blocks out the surrounding environment entirely (as in the case of VR).

It’s just not factoring in things like obstacle identification and alert notifications. It’s understanding that wearing a lightweight watch all day is easy—but wearing a bulky, heavy headset is not. How would a headset device impact someone who is on their feet in a factory versus an individual in an office? The road map for best practices is almost completely blank at the moment, so the level of detailed thinking required to ensure we create something worthwhile is much higher. What kind of experience makes the most sense for the role, the need, the environment?

And though it’s probably obvious, it’s still worth mentioning: View the experience in the proper format! “You can’t really expect your VR project to look like your website, or only view it on the web and expect the wow experience,” says Yueyao. Too often, people try to understand X in a format designed for Y.

“You have to view it in your headset. Just like any other design process you’re used to, use the format it’s meant for. Mobile is not the best for viewing a web app designed for desktop, and a VR experience won’t come across without the headset either,” she adds.

Consider the full experience, test with the full experience, and you’ll ensure that you’re creating a full experience for your users.

Get creative

We’re entering uncharted territory for the most part. AR and VR have certain expectations when it comes to game-play, but outside of that the rules are unwritten. This is your chance to write them.

What kind of experiences are possible? Already we’re seeing AR, in particular, make incredible inroads with retail and commerce experiences.

Pinterest has been beta testing an AR application called Lens for about a year now, which makes their pinboards more searchable and usable.

“I really believe that the camera will be the next keyboard,” says Pinterest CEO Ben Silbermannin this Fast Company article. “It will be a fundamental tool you use to query the world around you, discover things around you, or visualize how something might fit into your life.”

IKEA has an AR app that allows you to envision a room with their furniture in it, so you can get a real sense of how a piece would look in situ. And Amazon has has done the same, including a 360 degree view—only they have the full spectrum of Amazon available goods. There are no boundaries, as of yet, on what is and isn’t possible.

But it’s not just the use cases that can push boundaries, it’s the ways we interact with the devices that enable them, too. Yueyao even suggests throwing out the rule book—at least for a little while. “Don’t borrow existing interactions, if possible. Instead, try to think of how we usually interact with objects and people in real world, and start from there.”

You spent all that time carefully observing what it was like to experience the device for the first time. So now put your knowledge of that experience into practice and start testing the waters on what could be done.

Don’t throw the baby out with the bathwater

If the first four steps seemed daunting and even overwhelming, this final one should set you at ease. Yes, these experiences are brand new. Yes, there’s going to be a lot to learn. And no, we can’t know what we don’t know. At least not yet.

But there are also lots of things we already do know about designing great experiences. And there are lots of programs that can still be used to great effect to accomplish that task.

“While applications for virtual reality and augmented reality can definitely go way beyond the traditional processes, workflows, and tools of digital product design, there is still a lot that can be done with Sketch when designing for VR and AR,” says Andrew. “The thinking that goes into these new experiences is definitely more complex and sensory-driven than a static application or feature-driven site, but a lot of the same principles driving experience design and user centrism remain as supports to the overall process.”

Great designers put their users first, regardless of the experience they’re designing. We already know how to conduct research and ask meaningful questions to figure out the heart of our users’ needs. We know how to balance stakeholder excitement, budgetary restrictions, and project uncertainty. We know how to test and validate to be sure we’re on the right track—just because the things we’re testing for change, doesn’t mean we suddenly lack the basic foundational skills to do what needs to be done.

So buck up! Designers have built new roads into uncharted territory before. We’ve carved out paths that we thought made sense and backtracked to forge new ones when it was necessary. We will undoubtedly make mistakes, but we’re also going to get a lot of things right, because we’ve been building up to this for a long time already.

Trust your instincts and keep your head about you, and you’ll be just fine. After all, isn’t this why we became designers?