Can a sleep tracker determine when you’re sleeping or scrolling? Despite knowing better, many people use a smartphone or other electronic devices in bed, whether before they go to sleep, when they wake up in the middle of the night, or before rising.
A new study aimed to determine how well different consumer sleep trackers, including Oura, and a research-grade actigraph, could detect relatively motionless smartphone use while participants were in bed.
“While sleep trackers are generally very good at detecting when we fall asleep, they have a weakness in detecting when we are awake but relatively motionless,” says Dr. Michael Chee, MBBS, FRCP (Edin), director at the National University of Singapore’s Centre for Sleep and Cognition, Oura advisor, and Principal Investigator of the study.
The measurements in the study were compared to gold-standard polysomnography (PSG), which is used in clinical sleep laboratories. This study accompanies a prior report from the same group that evaluated device performance in the same participants but without sleep interruptions and disallowing of in-bed smartphone use.
In the findings, researchers determined that Oura was more accurate at detecting sleep and wake than the ActiGraph and other lower-cost wearables.
Additionally, Oura was the least prone to misclassifying motionless wake as sleep.
Study Design
The study’s final sample size was 63 participants from three different age groups (young: 18-30; middle: 31-50; older: 51-70 years old). It took place in a sleep research laboratory.
Five wearable devices were used in the study: Oura Ring Gen3, the Fitbit Sense, the ActiGraph GT9X, the Xiaomi Mi Band 7, and the AXTRO Fit 3 Fitness Tracker. Four of the five devices were worn concurrently.
Polysomnography (PSG) was also used as a reference device.
Simulation of Bedtime Smartphone Use
Before sleep, the participants read news articles, watched nature videos, and texted with a chatbot for approximately 10 minutes.
After falling asleep, they were woken up by an alarm five minutes after the first appearance of deep sleep or 45 minutes after sleep onset and had to respond to pre-configured messages in order to simulate how a person might check and respond to messages in the middle of the night.
Participants were then allowed to fall back to sleep and were not disturbed further. However, persons with lower sleep quality could still wake up and fall asleep again spontaneously.
If they were not already awake at their usual wake-up time, an alarm sounded. Participants then read four news articles and completed a questionnaire, which took around 15 minutes before getting out of bed,
Participants could sit or lie in any comfortable position while they used their smartphones. Bedroom lights were kept off throughout.
Key Findings
- This is the first study where in-bed smartphone use was factored into device performance evaluation.
- Oura and Fitbit were the only two wearables that automatically detected the interval between the cessation of smartphone use and when the participant initially fell asleep. On average, this was 8.57 minutes later on Oura, and 9.75 minutes later on Fitbit compared to PSG. The research ActiGraph does not automatically detect this interval and requires a button press to signal the completion of the pre-sleep smartphone task and the intention to initiate sleep.
- Oura was highly sensitive to detecting when participants woke up during the interruption and in the morning, frequently detecting the wake interruption within 30 seconds of PSG.
- Accuracy did decrease slightly with individuals who took a long time to fall asleep or those who woke up more often during sleep, indicating that sleep trackers may be less accurate for people with insomnia or disturbed sleep.
Discussion of Findings
Well-developed sleep trackers outperform “research-grade” actigraphy that only measure movement, as they harness data from additional sensors, including photoplethysmography (PPG) and temperature sensors.
However, just having more sensors alone is not sufficient. The lower-quality devices that were tested also incorporate pulse rate sensing, but they performed poorly. Oura’s algorithms have been developed and refined using large and diverse (for age and ethnicity) training sets, which increases their accuracy.
Most importantly, the findings provide assurance that quality devices, whose sleep algorithms are well-trained, are not significantly fooled into mistaking modest amounts of night-time smartphone use during user’s habitual sleep time, Chee notes.
“This is the first study to evaluate how bedtime smartphone use can affect the accuracy of sleep trackers. Such behavior is pervasive in modern societies, so we hope that this feature will be built into all future sleep tracker performance evaluations,” says Adrian Willoughby, PhD, the first author of the work.
RELATED: Researchers Explore The Impact of Phone Usage on Sleep With Oura
