Menu Close

A fingertip pressed against a phone’s camera lens can measure a heart rate. The microphone, kept by the bedside, can screen for sleep apnea. Even the speaker is being tapped, to monitor breathing using sonar technology. In the best of this new world, the data is conveyed remotely to a medical professional for the convenience and comfort of the patient or, in some cases, to support a clinician without the need for costly hardware.

But using smartphones as diagnostic tools is a work in progress, experts say. Although doctors and their patients have found some real-world success in deploying the phone as a medical device, the overall potential remains unfulfilled and uncertain.

Smartphones come packed with sensors capable of monitoring a patient’s vital signs. They can help assess people for concussions, watch for atrial fibrillation, and conduct mental health wellness checks, to name the uses of a few nascent applications.

Companies and researchers eager to find medical applications for smartphone technology are tapping into modern phones’ built-in cameras and light sensors; microphones; accelerometers, which detect body movements; gyroscopes; and even speakers. The apps then use artificial intelligence software to analyze the collected sights and sounds to create an easy connection between patients and physicians. Earning potential and marketability are evidenced by the more than 350,000 digital health products available in app stores, according to a Grand View Research report.

“It’s very hard to put devices into the patient home or in the hospital, but everybody is just walking around with a cellphone that has a network connection,” said Dr. Andrew Gostine, CEO of the sensor network company Artisight. Most Americans own a smartphone, including more than 60% of people 65 and over, an increase from just 13% a decade ago, according the Pew Research Center. The covid-19 pandemic has also pushed people to become more comfortable with virtual care.

Some of these products have sought FDA clearance to be marketed as a medical device. That way, if patients must pay to use the software, health insurers are more likely to cover at least part of the cost. Other products are designated as exempt from this regulatory process, placed in the same clinical classification as a Band-Aid. But how the agency handles AI and machine learning-based medical devices is still being adjusted to reflect software’s adaptive nature.

Ensuring accuracy and clinical validation is crucial to securing buy-in from health care providers. And many tools still need fine-tuning, said Dr. Eugene Yang, a professor of medicine at the University of Washington. Currently, Yang is testing contactless measurement of blood pressure, heart rate, and oxygen saturation gleaned remotely via Zoom camera footage of a patient’s face.

Big tech companies like Google have heavily invested in researching this kind of technology, catering to clinicians and in-home caregivers, as well as consumers. Currently, in the Google Fit app, users can check their heart rate by placing their finger on the rear-facing camera lens or track their breathing rate using the front-facing camera.

“If you took the sensor out of the phone and out of a clinical device, they are probably the same thing,” said Shwetak Patel, director of health technologies at Google and a professor of electrical and computer engineering at the University of Washington.

Google’s research uses machine learning and computer vision, a field within AI based on information from visual inputs like videos or images. So instead of using a blood pressure cuff, for example, the algorithm can interpret slight visual changes to the body that serve as proxies and biosignals for a patient’s blood pressure, Patel said.

Google is also investigating the effectiveness of the built-in microphone for detecting heartbeats and murmurs and using the camera to preserve eyesight by screening for diabetic eye disease, according to information the company published last year.

The tech giant recently purchased Sound Life Sciences, a Seattle startup with an FDA-cleared sonar technology app. It uses a smart device’s speaker to bounce inaudible pulses off a patient’s body to identify movement and monitor breathing.

Binah.ai, based in Israel, is another company using the smartphone camera to calculate vital signs. Its software looks at the region around the eyes, where the skin is a bit thinner, and analyzes the light reflecting off blood vessels back to the lens. The company is wrapping up a U.S. clinical trial and marketing its wellness app directly to insurers and other health companies, said company spokesperson Mona Popilian-Yona.