Google's PhotoScan estimates body fat from smartphone photos
Seeing beyond BMI: Estimating cardiometabolic risk with smartphone imagery
Google researchers have developed PhotoScan, a deep learning framework that estimates body composition metrics like body fat percentage and visceral-to-subcutaneous fat ratio from standard 2D smartphone photos. In an independent cohort, it achieved a mean absolute error of 2.13 for body fat percentage, outperforming smartwatch-based bioelectrical impedance analysis (MAE 2.91) and approaching the accuracy of DXA scans. The model also improved insulin resistance classification (AUROC 0.760) over demographics alone (0.692), nearly matching DXA (0.773).
Our PhotoScan approach offers a promising middle ground, estimating granular body composition from standard smartphone imagery with near-DXA accuracy.
- faangguyindia
Beating Smartwatch based Body Composition Analysis (using Bio impedance snalysis) approach isn't a great feat as its single point crude sensor is widely inaccurate compared to production grade multi segment multi frequency inbody machines using same technique
Google PhotoScan (~2.1 pp MAE) may put it between DXA and well performed multi site Bodyfat Caliper by an skilled operator and WAY ABOVE US Navy circumference, consumer BIA ( smart scale with handlebars), poorly performed calipers.
But i remember microsoft research paper:
>In 2022, a Microsoft Research associated team published “Smartphone camera based assessment of adiposity”. Their Visual Body Composition (VBC) system used ordinary smartphone photographs plus a CNN to predict body fat percentage against DXA. In a 134 person, two site validation cohort, it achieved 2.16 ± 1.54 percentage point mean absolute error, with concordance to DXA of CCC = 0.96.
So from 2022 to 2026, we still don't have a model which can provide benefit to people.
This can help so many people using apps which currently simply wrap Gemini Flash to detect "body fat."
Unfortunately, none of these models, code, or training data are available freely.
These companies are seriously withholding public benefit. Currently, a guy who wants to get a DXA scan has to pay $50 in the US and 1,500-4,000 INR in India.
In poor countries this DXA may not be affordable to vast majority, if models are released we can perhaps setup even a "photo booth" with better came […]
- jerlam
Amazon Halo did something similar five years ago:
https://www.amazon.science/latest-news/the-science-behind-th...
A recent clinical study, whose results haven’t been published yet, determined that [Amazon Halo's] Body is nearly twice as accurate as smart scales in measuring BFP when using DXA as the ground truth.
Some discussion on HN at the time: https://news.ycombinator.com/item?id=24295779
- kamranjon
I wonder if this could be used by insurance companies to determine premiums?
- sgallant
If this gets good enough, you could imagine your phone periodically tracking body composition and combining it with wearables, bloodwork, glucose, etc. Suddenly some measurements that currently require a clinic and specialized equipment become essentially free and continuous.
...at the same time, giving Google/Apple photos of your body is creepy
- CodeWriter23
Putting this out here in case anyone with metabolic disorders may find it useful. Labcorp has a recently added blood screening, the 'Metabolic Vulnerability Index' aka MVX. This looks outside the 'standard of care' cholesterol/glucose/bp triad which typically results in managed decline to identify underlying causes of said triad.
A metabolic specialist who understands how these work is essential to the recovery process.