The stick phone
Essential is Andy Rubin’s hardware company that focuses on smart home products and smartphones. https://www.essential.com/
The "Stick Phone" is a new AI-based wearable smartphone concept. The phone is smaller in size but is more suitable in many outdoor scenarios that require less digital content consumption and more digital content creation as well as real-life content interaction.
image from essential.com
Design Challenge
No smartphones like this before existed. The blue picture of this phone is more than finding a small segment in the packed smartphone market and occupying it. Referred to as "a new device to reframe your perspective on mobile," the project was launched with a highly ambitious goal to change the lifestyle of an information-overloaded generation.
My role at the beginning was to study the problems, brainstorm with the team, and visualize the new phone concepts. Many of the concepts I proposed to the team were further developed later and even patented.
patent record from USPTO website
Kick Off
As a start, we listed several markets and scenarios for this new phone that we could look into. These included (may be different from final product definition) payment, light & instant social interaction, commuting, and small app platforms (operation system level).
Alipay & Apple pay screenshots taken 2017
In 2017, Wechat and Alipay officially stepped into the mobile payment market and rapidly changed the payment habits of billions of users permanently. In the US market, similar products like Apple pay also began to thrive. The booming mobile payment market made portable devices even more essential than before now that people could interact with the physical world via their mobile devices.
Snapchat screenshots and Spectable photos taken 2017
Mobile phones are perfect carriers for interpersonal interaction through current social media. However, we could still prospect a future of lighter social interaction with much less screen dependence. For example, Snapchat released Spectacle in 2017 which was more than just a pair of sunglasses with a small camera. It's designed for faster and more natural video taking and sharing which made perfect sense as it seamlessly connected with Snapchat's social network product line.
Screenshots of Uber and DiDi taken 2017
Using phone apps to call a taxi was not a new feature. However, to further improve users' commuting experience, mobile phones of the future should be smarter in many ways. With the purpose of reducing heavy VUI interaction, some steps in the user flow could be minimized. For example, the meeting details sent via email by your business partner should be applied automatically.
Screenshots of Wechat Mini Programs in 2017
For the purpose of making a phone that is not only physically smarter but also systematically lighter and faster, Wechat Mini Program was a very good inspiration. This mini operation system within the most popular social media app made it much easier for small businesses and service providers to add a client on users' smartphones. In the future world of mini programs, there's no installation/uninstallation of apps, nor subscription/membership, all you need to do is walk into a restaurant, scan a QR code, and use its mini program and forget about it the moment you leave.
Early Insights
People today are more or less becoming slaves instead of owners of their smart devices.
Only by making digital content consumption lighter and easier can we make room for real-life content.
Stick Phone and its AI focus should be more on real-life content consumption. This means the AI should be able to understand the world better than before and provide more compound and environment-responsive information.
My illustration in 2017
My illustration above explains how our digital tools are taking control of the world around us. Light red stands for a certain device's ability to present certain content. Dark red stands for the content that a certain device is very good at. They are good at presenting digital content from audio to video, and we interact with our own life through these digital media. However, current smart devices are not smart enough to help us interact with real-life content more directly and smoothly.
For example, we may spend lots of time watching videos on TV or reading articles on our laptops but we can't take them with us to consume geographical or temporal content. Smartphones have the potential to exchange all of this information but their screen size limitations make them less than ideal for videos. Current smartphones are not meeting users' expectations in higher-level features related to processing human identity information, object recognition, and even behavior learning & prediction.
Scenario Analysis
There are many cases in everyday life when people use their phones but get lost.
1. Social media: Content creating & sharing boundaries
In many social media use cases, there could be more than only one content creating tool and also more than one target from different social media apps to share the content with. A simple behavior of "sharing a picture" can be complicated because our phones are not smart enough.
2. Allodoxaphobia: Redundant apps.
Similar to social media use cases, many people have more than one app in their smartphone that serves the same purpose. Sometimes it’s because different apps have slightly different features and sometimes it's simply because users are influenced by competing promotions. For example, with the battle between Uber and Lyft, many users download both of these two similar apps. Then they make choices between them each time they want a car based on the price and wait time estimation.
3. Multi-task control over apps
Continuing with redundant apps, there are also many cases when multiple apps are needed to complete one task. In these cases, users have to split the task into smaller chunks and frequently switch between the different platforms to move forward with the steps. While making a giant full-stack multi-function app is impossible with current technology, to refine the user experience with these tasks, better and smarter multi-task control over apps should be designed in future phones.
4. Info overload: Quick search is never quick enough
Right now, searching technology has already made it much easier to search and look for specific information on the phone. Unfortunately, it’s still not quick enough though. Few operating systems are able to locate specific information from different apps, which could be a game-changing feature. Imagine if your phone knew you had a GAP coupon in your mailbox and automatically prompted it out once you stepped into a GAP store.
Re-framing the Problem
Current smartphone designs, like their form factor, operating system and the apps make them useful but complex. Applications are isolated from each other in terms of data, function and reaction to users' habit. Phones needs to be lighter in shape and in the user experience.
“How might we make a phone that is smarter and less intrusive?”
My approach toward answering this question started with imagining a few "happy moments" and telling stories through sketches. After a brief brainstorm with the team, I proposed a dozen concepts of the small stick-shaped smartphone that shares some common features of a regular smartphone and wearable products. Nobody had a clear definition for the product so sketches of use cases were helpful and inspiring.
Concept Sketches
Some faster and lighter mobile payment concepts.
For example, using ultrasound or haptic feedback for exchanging payment credentials.
Possible methods of interaction with a 360-degree paranormal fish eye camera which reduces the amount of steps compared to traditional smartphone cameras during outdoor photo shooting.
Concepts of using voice to greatly reduce the use of VUI, texting as one example.
The point of this concept is to automatically convert text messages between voice and text in different scenarios.
Deeper Insights
After the first round of brainstorming and researching, we had a few more user research & use case investigations internally. The research focused on getting feedback for the concepts & sketches, questions around some key terms we kept discussing around included:
1. What is "present" and how exactly can digital devices help people enjoy the "present"?
2. What are "heavy" smartphone users and what are "light" users? Are they both within our target user group?
3. Are we designing a "wearable smart-not-phone" or a "smart wearable phone"?
4. What kind of AI will be needed to make this phone a value-add to the current lifestyle instead of a reduction of features?
We designed a questionnaire and collected answers from within the company (approx. 20 employees who are not involved in the project).
_edited.jpg)
Research & questionaire designed together with Gabby H. and Alistair I.
We dug into the feedback we got from this research and brainstormed around "what is present" and "how can we help to enjoy the present."
In our most intuitive perception, the reduction of features itself could be a feature to make smart devices less intrusive. However, many features like live streaming, 360-degree cameras, voice command triggered navigation system, and an AI system that is smart enough, could be more powerful for the information overwhelmed generation.
Brainstorming with Gabby H. and Alistair I.
Illustration by me
Current technology is very good at collecting & throwing information at us. However, almost none of them know when, how, and what information to promote exactly as we expect. This causes a greater workload on the user side as more features are loaded into the phone. While simply reducing the size and features wouldn't be helpful in solving this problem, the future of all smartphones (not only stick-shaped phones) will be relying on the smartest-ever AI systems behind them. The special form factor of Stick Phone would be a perfect vessel for this AI in most outdoor scenarios and many swarm/minimalist scenarios.
The design challenge has, therefore, been re-framed as
“What is the natural evolution of the AI centered device?”
Use Cases
To answer this question, we explored a couple of the strongest use cases that could best present how an AI-centered smart device could change users' behavior. Some of them took the advantage of the small form factor of the stick-shaped phone while others didn't necessarily.
Below are some of my proposals and illustrations.
Concept design & illustration by me
One of my concept designs was called "Toast to New Friendship." In many social scenarios, people have nowhere to put their large, bulky smartphones but also can't do without them for reasons like exchanging contact information so they can keep in touch with people they meet. Stick Phone can do a better job within these occasions because it's easier to carry and has an interaction design that matches better with natural behavior. For example, a "toasting" behavior between two Stick Phones would easily trigger a name card exchange. No typing a password, no phone unlocking, no app launching, no QR code scanning, or even typing phone numbers.
Concept design & illustration by me
The illustration above presented two more social scenario concepts. Information exchanging steps between people are greatly reduced by the help of AI. Imagine you are busy preparing for a party. One of the guests, Mary, sends you a message saying, "Hi Liz, I'm on my way! Would you send me your address again please?". Instead of running to get a piece of paper towel to dry your wet hands, taking out your phone, unlocking it, and struggling to type your address, your AI assistant would read the message for you and ask you whether you want it to share your address with the message sender.
Content creating like capturing photos and live streaming would also become easier. You can forget about your different accounts on different social network platforms, forget about filters, post effects, and forget about the painful process of transferring images from one app to another. Just say "live stream this moment on my Facebook, Snapchat and share the video with my mother" to the phone and enjoy the present. Your phone will take care of the rest.
Concept design & illustration by me
A smarter, quicker, and funnier way for people to take selfies together. Similar to the "bump and exchange name card" concept, the "take a selfie together in a snap" promotes a natural behavior interaction based on the future AI system. It can be triggered by haptic motion, face recognition, and omnidirectional movement sensing.
Concept design & illustration by me
Two similar concepts of smarter ways to trigger photo shooting: Smile to the camera to take a photoshoot or a "magic wand" gesture control.
Concept design & illustration by me
Taking advantage of Stick Phone's small form factor, precise sensors & powerful AI, some interesting use cases become possible. Your smartphone can be an extension to your existing smart devices or even an extension to your physical accessories and surroundings. You can cast photos to your TV, write with it on your tablet, or even take notes in the air and it'll recognize your handwriting.
Concept design & illustration by me
Stick Phones can become a universal remote and even a central control hub for all of your smart home devices. It knows your habits and needs much better than any individual smart home system.
AI Features &
Card-based UI
One of the AI features that we spent lots of time working on was experimenting how a voice UI would work on a small display.
The voice-based UI was commonly seen at the time but not on a stick-shaped display. What's more, it shares many common problems with other smartphones, like information overwhelming. We proposed this "card-based UI" that would work perfectly with not only voice input but also with physical location, time of day, user habit, etc.
The idea was that with the help of powerful AI, most of the information we present to the users on current smartphones could be dramatically reduced so that a large screen will no longer be needed in some cases. Instead, only the most relevant information at the moment would be presented.
VUI based AI prototype made by me
Card-based UI prototype made by me