the retainer is a next generation UI concept which, placed inside the mouth, exists on the physical and psychological boundaries between the self and the digital world, allowing intimate new interactions not possible through a screen.
The next great interface will be in your face.
MAYA Design Blog August 7, 2015
The next great user interface (UI) will be in your face. Literally. The technology is already in your pocket, trapped behind your smartphone’s screen. All screens have inherent limitations that cannot be escaped through incremental innovations like strapping it to your wrist or attaching it to a pair of spectacles. Next generation UI transcends the limitations of screens and reveals new interactions not possible through graphical user interfaces. In order to see what’s possible with next generation UI, let us first look into our own minds — where vision occurs — with the help of our imagination:
In front of you is a gorgeous billboard for an iPhone. Smooth, crisp glass with beautifully machined aluminum curves floating on a white background. It’s sleek, exciting and sexy. You want it.
Now, replace the iPhone with the retainer you wore in middle school. Custom-molded plastic body and hand-fitted stainless steel clasps floating on a white background. Still want it? Probably not, but appearances can be deceiving.
Visual bias
I’m not against aesthetics, but our visual biases severely limit our ability to see what’s possible when it comes to interface and product design. Functionality and interactions are frequently compromised in pursuit of our cultural obsession with aesthetic ideals such as being thinner or having the “right look.” Poor battery life and flat UI buttons that are indistinguishable from text are notable examples of the price we pay for “beauty,” but these sacrifices have more profound costs.
As it turns out, objectification works the same way with things as it does with people, and our bias toward visual interfaces comes at the expense of intimacy. Admit it, you are only interested in the way your iPhone looks and what it can do for you. And, because your eyes can’t focus comfortably at less than 25cm (the length of your forearm), you’ll always keep your phone at arm’s length unless you let go of your visual biases. So, let’s look at how our relationship with technology might be different without our visual biases in the context of a normal, everyday scenario.
Nothing to see here
Imagine you have been waiting at the bus stop for a long time. You mutter under your breath, “Where is that *&$% bus,” and kick a pebble. The virtual personal assistant on your smartphone chimes in with an abysmal ETA and asks if you’d like to catch up on your podcasts while you wait. Appreciative of the distraction, you listen to the episode of WNYC’s Radiolab you missed last week. Suddenly, your friend calls to confirm dinner plans. Since you are running late, you ask them to pick you up at a local bar on their way to the restaurant. In the bar, you grab a pint and catch up on social media and sports scores before exchanging emoji-ladened texts with your significant other. (The heart-eyed smiley always makes you blush). Eventually your friend arrives and you scurry off to dinner.
Pretty normal, right? Sure…except I forgot to mention that you never took that sexy iPhone out of your pocket and you aren’t wearing earbuds. Somehow, no one around you heard the podcast playing or your virtual assistant speaking, and even though you never looked at a screen, you were able to see the image of the heart smiley in your message!
How is this possible? You’re wearing a very special retainer.
Next generation UI
The retainer-like device I describe is a next generation UI that doesn’t need to live at arm’s length. Here are the high-level concepts and specifications for how this could very well become our reality:
Form Factor: next generation UI are very close to us — often against or inside our bodies. This retainer is custom fit from a digital scan and fits perfectly against the roof of your mouth. Invisalign already produces individualized products of a similar kind on a massive scale.
Voice Control: tiny wireless microphones inside your mouth can easily hear you talking under your breath, and relay the sound to the virtual assistant on your phone for processing.
Private Audio: bone conduction technology, like that used in AfterShokz’s Bluez 2 headset, transmits vibrations through the bones of your skull to produce crystal clear audio only you can hear — even in noisy bars. (If you’d rather make your own set, the necessary component sells for $8.95 on adafruit.com).
Navigating blind
Thankfully, the retainer is much more than a Bluetooth headset in your mouth. In the scenario, you checked social media and sports scores in the bar without saying a word to your virtual assistant. This requires you to somehow navigate the OS blind:
Visualization: VoiceOver is a built-in feature of Apple’s iOS that helps you to visualize the UI in your mind’s eye by reading what is on the screen out loud to you as you move your finger over it. My friend Greg at the Apple Store in Lincoln Park, IL is an accessibility expert and showed me how to enable VoiceOver on any iOS device simply by telling Siri, “VoiceOver on.”This is pretty handy for Greg, who is blind, because it means he can immediately use any iOS device in the world. And it’s going to be pretty handy for you too, because now you don’t need to look at a screen. However, you still need to control the OS, and the answer is right on the tip of your tongue.
Touch: the tongue has a lot of dexterity, and since the retainer covers the roof of your mouth, it’s a perfect place for a touch control surface. A flick of your tongue across the roof of your mouth, like the swipe of a finger across a screen, lets you navigate a full-featured “tongueOS” with the aid of VoiceOver cues you can hear through your bones.
See things differently
Your phone will now stay in your pocket while you launch apps and have sports scores read to you in a noisy bar. But what about the heart smiley? Having the ability to see it without using your eyes poses a significant challenge and requires a technology that encourages your brain to rewire itself.
Synesthesia: your brain can take input from one of your senses and interpret it with another. For example, some people associate certain colors with specific musical notes or keys. Thanks to the phenomenon of synesthesia, your retainer will help you see with your tongue. The BrainPort v100 is an FDA approved oral vision aid developed by Wicab, Inc. It provides electro-tactile stimulation to the tongue, which your brain learns to interpret as low-resolution gray scale video images that you can actually see in your mind’s eye. Our neighbors at UPMC conducted a study with this device, and for someone using the BrainPort v100, a smiling emoji with heart eyes appears as a 40x40 pixel greyscale image–over 6 times better than the 16x16 pixel image you would see on your phone's screen!
Paradigm shift
Some might find it unsettling to put a retainer-like device in their mouth and interact so intimately with it, but getting really close to technology is something we have to become more comfortable with. As the Internet of Things (IoT) surrounds us with computing, we will increasingly find ourselves interacting with technology in very intimate ways, whether we know it or not. It’s a new relationship for us all and it is possible we may get hurt, but if we stay at arm’s length we will never get close enough to see what is possible — and it is beautiful.