Control Smartphones and Wearables Under Water

Smartphones and wearables are now waterproof. This is great. It means we can shower without taking off the smartwatch; we can swim but won't miss messages/notifications; and we can surf and still pick up phone calls even if the device dropped in water.

All these sound great, but touch screens are water-unfriendly. Whenever there is some water on the screen, the touch screen just went nuts. Either it is unresponsive or it randomly clicks like there is a ghost. Water affects the screen capacity change caused by human skin and destroy the functionalities of the touch screen.

Let's look at some alternatives.

Audio seems like a great option. We can control devices using voice commands. But today's voice commands are limited and major apps still require a working touch screen. It might be great if we can scroll up/down and swipe left/right using voice commands while we are taking shower or surfing. But what if we are in water. How can we speak?

Maybe motion gestures using IMU sensors can be a better solution. It does not involve sound and we can use hand/wrist/arm gestures to mimic touching gestures. For example, rotating hand can be set as scrolling up/down. While sounding, the motion can be very sensitive to noises, or undesired motions. Moreover, it can be difficult for people to learn and memorize all these gestures since different people have different motion behaviors.

Camera can be another way towards the solution. When sound and motion are not available, capturing a sequence of photos and analyzing the intended action may help. For example, when sound is unavailable, we can use image processing to translate the shape of mouth into spoken words and perform certain actions. But this needs good lighting conditions, and image might be distorted by water.

Some may also try RF networking radios to use wireless signals to identify gestures and avoid the physical touch requirement. The fundamental problem of this is the mobility of wireless devices. To identify gestures, the radios have commonly assumed to be stationary.

From above, we see that each individual method has their own advantages and limitations. It is possible to combine these together to achieve this single functionality to control smartphone under water. Or, maybe someday in the future, using some wireless mechanisms, we can directly read minds from our brain to control the smartphone so no physical movement is required.

======= 05/24/2017 added =======

I thought of another possible way to add this interaction: physical keys. There might be a day that we need to return to physical keys to take control of the smart devices. A good example is Galaxy S2/S3 with bezel. The rotation provides the user a new way to interact with the device. The downside is that the apps and functionalities still require a finish action to touch the screen as a confirmation. It might be great if we use an additional physical button as confirmation.

Leave a comment

Your email address will not be published. Required fields are marked *