In this episode of the True North Podcast, we find out more about the revolutionary dance program that’s been liberating Parkinson’s disease
(PD) sufferers from their symptoms through movement and music.
So what does this have to do with UX? Well, the program received a grant from Google to create an augmented reality (AR) application to bring their dance classes out of the studio and into the world of their students.
How exactly do you design an AR product that caters to the many and varying symptoms those with Parkinson’s experience? We find out.
Transcript
Ben Newton: Hi, guys. This is the first episode in a mini-series that revolves around the UX Awards, which this year will be held in Palo Alto, California—the heart of Silicon Valley—on November 8th and 9th. Now, here’s the episode.
JM Tolani: Traveling and taking photography has set me up with this life in so many ways. I was so happy to find something that I love doing and feel that that’s been taken away from me. The kind of photography I did was photo journalism. I photograph people who worked in very difficult conditions and very remote parts of the world like the Amazon, the mountains, jungles, and areas where it’s very difficult to get to. There were photographs of people working in the fields and in the coal mines, and all that stuff.
And what I didn’t want to do was photograph people showing how depressed they were, how they were exploited. That was not what I was trying to capture. But in spite of the situation, they did have something positive in their life, something… their expressions, their smiles, their happiness, and that’s what I try to capture.
In order to do that, you have to have a very positive look at things and not always concentrate on the negatives. They always manage to smile and be grateful for what they had in life, actually, and that’s what I was starting to capture in my photographs, and that’s the way I approach my life, too, having Parkinson’s as well. All I started to see are the positive side of life rather than the negative.
Ben: That’s JM Tolani, a New York-based photographer. In case you missed it, he said he has Parkinson’s disease. He was diagnosed in his 40s.
JM: And the first time I noticed it was in 2007. I felt a tingling feeling in my… these 2 fingers. I felt a lot of stiffness in my upper arm. And when I walk, I kind of stumbled a little bit.
So, I go from one doctor to another, and somebody told me that I had carpal tunnel. Another one wanted to perform a surgery. I went to a neurologist who said, “Why don’t you take this walk from one end to another in the hallway?” He looked at me right away and said, “You have Parkinson’s.”
I said, “How do you know that?” He said, “You’re not swinging your arm at all and you’re stumbling.”
Well, I was one of the lucky ones that it only took 6 months to diagnose it. Some people have gone through many years without being diagnosed and have suffered tremendously. I’m not suffering at all. I’m struggling to get through every day, which is understandable in my condition. But compared to a lot of people, I’m doing so much better. I ain’t sitting it out doing nothing. Sometimes I wonder how I can manage all this, but I can. And while this world doesn’t break is what it comes down to, actually.
Ben: JM was a part of a trial which aimed to help Parkinson’s sufferers tackle their illness and elevate their quality of life. The challenge the designers were faced with was how do you start with a revolutionary dance program and then combine it with Google Glass to create an experience that can be used and enjoyed by users with significant accessibility needs?
Related: The ultimate guide to web content accessibility
Hi, and welcome to True North. My name is Ben Newton and I’m from Loop11. This is the podcast where we share stories of discovery and innovation. Parkinson’s is a progressive neurological condition that affects people from all walks of life. The average age of diagnosis is 65 years and its symptoms can be debilitating.
JM: Your vision gets very weak. Your voice gets very soft as well. It’s like your whole system breaks down, basically. The medicine helps in that way, but it’s every 2-and-a-half hours to 3 hours—it wears off and you have to wait for it to kick in again. It’s like rebooting a computer every 3 hours. The computer breaks down and you have to reboot it again to get it started, except the computer can reboot itself in minutes. In this case, it can take 15 minutes to an hour, actually.
Ben: There are a range of terrible symptoms, but perhaps the most terrifying is the freezing.
JM: Oh, my goodness. It’s the worst feeling you can ever have, actually, to know that you can’t move an inch and you’re totally frozen. And then, sobriety sets in and the fear sets in that you’re about to fall any minute on the floor and hurt yourself. You feel absolutely helpless, actually. You basically feel you’re not… It’s not worthwhile to live in this world in many ways when that happens, actually. But to be able to unfreeze yourself and know that you can do that gives you tremendous power and energy to know that you can move on in life.
Ben: Being able to unfreeze himself is something that JM learns not from a doctor, but a dance instructor.
David Leventhal: Dance for PD began in 2001 when we here at the Mark Morris Dance Group were approached by a woman named Olie Westheimer.
Ben: That’s David Leventhal, the Program Director of Dance for PD and an instructor at the Mark Morris Dance Center in New York.
David: Olie is the Founder and former Executive Director of the Brooklyn Parkinson Group, which is a support group and activity group for people with Parkinson’s here in Brooklyn.
Ben: Olie observed a lot of anxiety from the members of their group who’d become circumscribed by the medicalized existence. They thought of themselves as patients and felt trapped.
David: Olie really wanted to create and think about an opportunity for people to connect on a different level, to connect not on the topic of Parkinson’s, but to connect as human beings with their own needs and desires, and interests.
Ben: Olie had a dance background and started thinking about all the things that happened in a dance class. The social element, the connection, the physical balance, and the strategies that dancers use to execute movements with fluent grace, they’re all really useful for people with Parkinson’s. She thought it could give them strategies to execute movement with greater ease.
David: So, she pitched this idea to us. We had just opened a new dance center here in Brooklyn and we said, “Yeah, let’s try it. We don’t know anything about Parkinson’s, but we wanna give it a try. Do you think that anyone would show up?”
Ben: And with that, the classes began. The very first class was in the fall of 2001 and ran once a month. David and his colleague, John Higginbotham, incorporated elements of ballet and modern tap jazz in improvisation.
David: And so, the curriculum of Dance for PD developed around legitimate dance training with adaptations and translations for people who might need to go a little bit slower or need things broken down a little bit more. But what we didn’t change was the overall thrust of the class. The class was a dance class—it was not a therapy session.
"I don't look at my disability as good or bad or indifferent. It just is. I have to dance in this body." Kitty Lunn https://t.co/ZD56UtIOvD pic.twitter.com/0oWdnYLdad
— Dance for PD (@danceforpd) March 23, 2017
And because of that, we really were able to create a world that was not about Parkinson’s. This is a world that was about dance. It was about being an artist, about being a dance student, about sort of coming back to a more holistic identity as a creative person, not as a patient. We really wanted to start a different conversation about the value of movement. The value of movement is not just about the scores on the standard Parkinson’s disease rating system that’s used, but actually, about quality of life and sense of one’s self as a person first, and someone living with Parkinson’s second, or third or fourth.
Ben: As the classes progressed, the patients quickly began to see startling results. Here’s JM again. I asked him if he had been reluctant to start the dance classes.
JM: Yes. Absolutely—yes, yes. I hadn’t done ballet previously and I wasn’t sure if I could dance at all. I mean, having… considering that you move so slowly, I couldn’t understand how one could dance. What’s amazing is when you go to this class, and the instructor like David Leventhal would come and show us some of the moves, and you go, “There’s no way I could do this. It’s impossible.” And then, the music comes on and everything just moves and glides smoothly. It’s just that then you wonder, “What on the earth happened over here? A few moments ago I couldn’t walk down the street. And now, here I am moving my hands and legs as if I’m a future ballet dancer.” And that’s what absolutely amazed me and shocked me. And I thought, “Well, if this can happen, then I can… I should keep on trying and see what the end result would be.”
Ben: And so, Dance for PD started making a terrific impact on the lives of many Parkinson sufferers. And over the years, Dance for PD spread across the globe to a point where classes were being offered in over 20 countries.
“A few moments ago I couldn’t walk down the street. And now, here I am moving my hands and legs as if I’m a future ballet dancer.”
But the one issue with the dance classes was that participants couldn’t attend as frequently as they might like and ensure the benefits were sustained. DVDs were available, but the dexterity required to load them into machines provided their own set of challenges. Then, in 2014, David received a suggestion.
David: Somebody sent me a notice from Google for Nonprofits, but there was a contest called Giving through Class, and that nonprofits were being asked to propose a project based on the Glass platform that would further their missions. And I remember the first time someone sent this to me, I kind of ignored it. I just let it sit in my email for a while.
And then, they resent it and they said, “Hey, you know, the deadline is tomorrow.” And I said, “Oh, yeah. You know, I’ve really been thinking about it too much.” But I sat down and I said, “You know, let me let me learn a little bit more about what this platform is. I didn’t know a lot about Google Glass at this point.” So, I looked into it and did some research, and started thinking about how we might leverage some of the elements of that platform to basically extend the approaches that we use in class.
Ben: So, David compiled the application and sent it off. Then, he waited and waited until he finally heard something.
David: It’s actually a very funny story. I got a note, an email saying, “Thank you so much for applying. We’ve had many applications, but unfortunately, we decided not to go forward with yours, and we appreciate your interest.” So, I wrote them back a thank you note saying just “Thank you for your message and for considering our application. We wish you all the best.” And at that point, they sent me an email back saying, “Oh, my gosh. We’re so sorry. You should not have received that first message. It was an error. You’ve actually received the grant.” So, you know, so much for big data.
Ben: Out of 1,300 applications, Dance for PD was one of 5 grants awarded. Each of the winners received $25,000 as well as access to developers who could work with the Google Glass platform. Then, through connections, David was able to reach out to the agency, SSNK, to partner with them. Their goal was to merge the Dance for PD class experience with the first ever augmented reality application for providing around-the-clock aid for people living with Parkinson’s. The man who was in charge of figuring out what this grand baking experience should be like was this guy.
Amos: My name is Amos Bloomberg. I’m a Clinical Assistant Professor at New York University and User Experience Designer at SSNK.
Ben: Normally, the first step for a designer would be to take a deep dive into understanding the uses they were about to design for. However, due to a Bachelor of Brain and Cognitive Sciences, Amos had somewhat of a head start.
Amos: I had a basic understanding of what Parkinson’s was about neurologically. So, for me, the new thing I had never touched before was, actually, Google Glass. You know, the only interaction I ever had with Google Glass was walking around downtown Manhattan with some creep with an eye piece on taking pictures of you, and you’re just like down the street, right? That’s my understanding of Google Glass. And so, I had to get a set. So SSNK was, you know, given many sets of Google Glass and I had to basically learn the paradigms that they had developed of how they expect people to be able to run applications and use applications using that hardware.
So, they have this idea of the timeline where you have cards that you can flip through to go back in time in your history of using that device. It’s actually very interesting, but not very easily usable operating system. So, I had to learn how to get around in that environment, in that operating system, and that was the first challenge for me. And then, the second challenge, of course, being, well, how do we make an app that’s for this particular user group?
Ben: As Amos and his team developed and tested different experiences, they started to realize the fundamental limitations of Google Glass from an accessibility perspective. This meant they’d have to go outside the existing design architect to find something that would fit the needs of their users.
Amos: To go through some of the common symptoms, their movement is quite shaky. So, we’re talking early stage, people who can still move but need that sort of push. So, their symptoms will be shaky movement and they can’t have very fine, smooth motor control. They often have stilted speech. They often get stuck or frozen. They might start moving, but then suddenly stop and have difficulty restarting, and they may have mild cognitive impairments, but that’s usually later stage.
So, with the Google Glass, you have an idea of you… essentially like a phone, you swipe in various directions to get some and use to show up, and so on. There’s a little touch bar you can tap or swipe on the actual glasses. If you think about somebody with shaky movement swiping and tapping, you’re not thinking about a very good success rate, right? So, somebody who’s actually not able to easily control their arms, little on their fingers, will have difficulty swiping and tapping in this tiny little frame of a glass.
Glass'ım şekil ?? #googleglass #atwork #google #glass
A post shared by Serpil Özer (@sserpilozerr) on
And then, Google Glass is also voice activated. You can say things like “Okay, Glass.” You know, “Start me up,” or something like that. Now, if they have stilted speech which is a common symptom of Parkinson’s, the chances, again, of success of that instruction will be relatively low. And if they’re frozen, how can we unfreeze them if they can’t speak or touch? So, there are some, you know, very big challenges in this particular merger of people and technology. And I think that this is indicative of kind of the future of augmented reality, the future of invented devices. You’re gonna see more and more of this kind of Utopian vision of technology, I believe, where technology is designed assuming relatively able-bodied people are using it. But, of course, as we know and as accessibility is becoming more of an awareness in the public, you’re gonna have more of these conversations of, well, how do you design these things to work for not just the perfectly able-bodied beautiful people of the world, but the people who actually have, you know, very serious problems—not just movement?
Related: We’re all just temporarily abled
Ben: So, there were all manner of challenges Amos and his team faced when designing Moving through Glass. At its core, the product was supposed to bring up a video of an instructor for the wearer of the glasses to watch, hear, and follow. A problem they soon uncovered though was that the glasses would overheat and shut down after only 30 or 40 seconds of video.
Amos: So, how do you have somebody who’s, let’s say stuck, who has been able to activate the software to ask them to get moving again as a dance instructor show up who starts working through some exercises, and that progresses to get them moving more fluidly, and then suddenly, that shuts down, overeats, and then they have to restart the system? So, we devised workarounds where we would start off with video, for example, to get the initial movement going. And then, after the dance had progressed to a reasonable state of movement, we would leave the audio track running but shut down the video for a while to let the device cool down a bit.
Ben: What they had learned through their research was that for the students who had already experienced the instruction, music itself, would be enough to help trigger their movements. Once enough time had elapsed on the video, that would turn it off, leaving only the audio running. Then, when the next class was initiated, the video would turn back on. So, what was the response from the students who tried the glasses? Here’s David again.
David: They loved being able to carry dance with them in their daily lives. This this is a fairly novel concept because, obviously, the live classes that we offer are transformative for people. They’re highly enjoyable, but they’re in a specific time and space, and they are in a dance studio at a certain time of the week. When people leave that studio, the question is how do the approaches and movements that we do in class travel with them? Do they travel with them? Are the participants able to recall what they did in class in their daily lives in a way that’s helpful? And the answer was yes to some degree, but what would really be helpful is some kind of queue and some kind of guide. The Moving through Glass platform provided that guide because you can call up a dancer at any time. And, I think, people really liked having a reminder of some of the activities of class with them wherever they were.
Actually, quite surprisingly, we created 2 sets of modules, 4 total. Two of the modules are based on choreography. So, they’re short dances based on choreography by Mark Morris who is the founder of Mark Morris dance group, and they are really centered on warming the body up, preparing the body for movements, working on balance through actual dances. So, they’re not exercises per se. They’re really short dances.
The other 2 modules are really based on pragmatic goals. How do we get somebody moving if they are frozen, if they’ve experienced what’s called freezing of gait where one feels like one’s feet are just stuck to the floor and you have difficulty initiating movement? And the other module is trying to answer the question how do we help somebody maintain a steady pace of walking once they start? So, those 2 modules, we thought, would be would be the more used modules because they address specific symptoms or, I should say, they address challenges that arise because of specific symptoms.
Actually, in the surveys, more people enjoyed the dancing activities than use the more practical exercises. And, of course, that could be self-selecting, you know. It may be that the people who chose the test glass are… they don’t need the queues to get moving. They don’t have freezing. That’s quite possible. But I also think that what people enjoyed about it was that it was an extension of the Dance for PD experience rather than a move into a more therapeutic or clinical model.
Ben: For those of us who have not experienced Parkinson’s, it’s easy to underestimate the impact of a project like this might have. But for those like JM, it’s been transformative.
JM: What I do when I wake up first thing in the morning is I use 2 modules. One is the balance and the warm-up modules, actually. I use these glasses to get myself going in the morning because it also helps me to unfreeze myself, which is very important. And the more active you are, the more exercise you do, I notice that my medicine is kicking a lot faster as well, as supposed to just sitting around and waiting for something to happen. It could take half an hour to an hour because your brain is not active and you’re waiting for something to happen. I mean, my basic philosophy is you have Parkinson’s, but the Parkinson’s doesn’t have you. So, that might have getting your body and you wanna make it really as comfortable as possible.
Ben: JM Tolani has dedicated much of his time to supporting and informing others with Parkinson’s. Check out his photography here. And you can learn more about Dance for PD here. Thanks again to the UX Awards, from whom Moving through Glass received an honorable mention.
For all the links relating to this episode, go to truenorthpodcast.com and see the show notes. While you’re there, we’d love for you to join the community and tell us what you think about the show. True North is available on Apple’s Podcasts where you can subscribe and leave a review. We’d love to hear what you think of the show. Our music is by the mysterious Breakmaster Cylinder. True North is produced by Loop11. We’ll see you next time.
Listen to more episodes of True North
by Ben Newton
Having ran a web design consultancy, started a national print and online magazine, and co-founded a international fitness website, Ben has seen design and development from many different angles. Now working on product and growth at Loop11, a user testing tool for UX professionals, he's enjoying the challenges of combining all of his experiences and focussing them on one product.
