Google is introducing features that will allow users to take vital health measurements using just the camera they already have on their smartphone, expanding health and fitness features typically only available on dedicated wearables to a whole new group of people. Beginning next month, and available initially on Google Pixel phones exclusively (but with plans to offer it for other Android devices in future), users will be able to measure both their heart rate and their respiratory rate using just their device’s camera.
Typically, taking these measurements has required specialized hardware, including red or green light-based heart rate monitors like those found on the Apple Watch or on fitness trackers like those made by Google-acquired Fitbit. Google’s hardware and software teams, including the Google Health unit led by Director of Health Technologies Shwetak Patel, have managed to develop computer vision-based methods for taking these measurements using only smartphone cameras, which it says can produce results that are comparable to clinical-grade measurement hardware (it has produced a study to validate these results, which it’s making available in pre-print format while it seeks peer review through an academic journal).
For respiratory rate, the technology relies on a technique known as ‘optical flow,’ which monitors movements in a person’s chest as they breathe and uses that to determine their breathing rate. In its clinical validation study, which covered both typical individuals in good health, and people with existing respiratory conditions, Google’s data indicates that it’s accurate to within 1 breath per minute across all participants.
For heart rate, Google is initially using the camera to detect “subtle color changes” in a user’s finger tip, which provide an indicator about when oxygenated blood flows from your heart through to the rest of your body. The company’s validation data (again, still subject to external review) has shown accuracy within 2% margin of error, on average, across people with a range of different skin types. Google is also working on making this same technology work using color changes in a person’s face, it says, though that work is still in the exploratory phase.
Google is going to make these measurement features available to users within the next month, it says, via the Google Fit app, and initially on currently available Pixel devices made by the company itself. The plan is then to expand the features to different Android devices running Android 6 or later, sometime “in the coming months.”
Image Credits: Google
“My team has been working on ways that we can unlock the potential of everyday smart devices,” Patel said in a press briefing regarding the new features. This would include smart devices in the home, or a mobile phone, and how we leverage the sensors that are starting to become more and more ubiquitous within those devices, to support health and wellness.”
Patel, who is also a computer science professor at the University of Washington and who has been recognized with an ACM Prize in Computing Award for his work in digital health, said that the availability of powerful sensors in ubiquitous consumer devices, combined with advances in AI, have meant that daily health monitoring can be much more accessible than ever before.
“I really think that’s going to be a really important area moving forward given that if you think about health care, the journey just doesn’t end at the hospital, the four walls of the hospital,” he said. “It’s really this continuous journey, as you’re living your daily life, and being able to give you feedback and be able to measure your general wellness is an important thing.”
It’s worth noting that Google is explicit about these features being intended for use in a person’s own tracking of their general wellbeing – meaning it’s not meant as a diagnostic or medical tool. That’s pretty standard for these kinds of features, since few of these companies want to take of the task of getting full FDA medical-grade device certification for tools that are meant for general consumer use. To that end, Google Fit also doesn’t provide any guidance or advise based on the results of these measurements; instead, the app provides a general disclaimer that the results aren’t intended for medical use, and also offers up some very high-level description of why you’d even want to track these stats at all.
Many of the existing dedicated wellness and health tracking products on the market, like the Oura ring, for instance, provide more guidance and actionable insight based on the measurements it takes. Google seems intent on steering well clear of that line with these features, instead leaving the use of this information fully within the hands of users. That said, it could be a valuable resource to share with your physician, particularly if you’re concerned about potential health issues already, in place of other less convenient and available continuous health monitoring.
Patel said that Google is interested in potentially exploring how sensor fusion could further enhance tracking capabilities on existing devices, and in response to a question about potentially offering this on iPhones, he said that while the focus is currently on Android, they ultimate goal is indeed to get it “to as many people as possible.”
