webinar
Watch the full webinar, available now.

Health Scores in Medical Devices

A recorded session on what changes when a wearable health score becomes software as a medical device. Using sleep apnea as the example, it covers how a medical-grade score is designed, tested against a clinical reference and validated for the people who will rely on it.

Intended useSensitivity and specificityROC curveClinical evaluation
Watch now
MDR Class IIa

What You'll Learn About Health Scores in Medical Devices

01

Why the Claim Changes Everything

The same code can be a wellness feature or software as a medical device (SaMD). What decides it is the intended use. "Helps you understand your sleep" needs no certification. "Flags possible disease to help with diagnosis" falls under MDR Rule 11 as at least Class IIa, requires clinical evidence and shifts development from features to requirements and risk analysis. Aleksander sets up the example used throughout the session: detecting sleep apnea with a wearable.

General wellness

"Helps you understand your sleep"

  • CertificationNone needed
  • Clinical evaluationLiterature review is enough
  • DevelopmentOutcomes and features
Medical device software

"Flags possible disease to help with diagnosis"

  • CertificationMDR Rule 11, at least Class IIa
  • Clinical evaluationClinical evidence required
  • DevelopmentRequirements and risk analysis

The code can be identical. The intended use decides the regulatory path.

02

Designing the Score: Data, Annotations, Algorithm

A medical-grade score starts with four inputs: raw signals from the device, a reference standard such as polysomnography or polygraphy, a target population that matches future users, and paired recordings of the same nights. Certified scorers annotate the reference, and patients are split into a development set and a locked test set. The section ends with the choice between a rule-based and a machine learning algorithm, and the extra documentation an AI model requires.

03

Testing: How Good Is Good Enough?

Every event the device detects is compared with the clinical reference in a confusion matrix. Aleksander explains sensitivity and specificity for diagnosis, why event-level detection has no true negatives, and how PPV and NPV answer the question a patient actually asks. He then uses the ROC curve to show how the alert threshold trades missed cases for false alarms.

04

Validation: Proving It Works for Patients

Clinical evaluation under MDCG 2020-1 rests on three pillars: valid clinical association, technical performance and clinical performance. A score built on a known metric such as AHI gets the first pillar from the literature. A new metric needs its own clinical investigation. Aleksander compares the cost and time of a literature review, a performance study and a clinical investigation.

The cost of clinical evidenceEU ballpark for a software or wearable device
Literature reviewEUR 10-40k1-3 months
Performance studyEUR 20-200kA couple of months
Clinical investigationEUR 300k-1.5M+1-3 years

Known metric (Path A): literature review plus a performance study. New metric (Path B): a clinical investigation on top. Ranges from the session, verify per project.

05

A Real-Life Lesson from Pulse Oximeters

Pulse oximeters calibrated mostly on people with light skin missed dangerously low oxygen in Black patients about three times as often as in white patients. Nobody had changed a line of code. The bias came from who was in the calibration data, which is why the test set has to look like the people who will use the device.

Built for Teams Turning Wearable Data into Medical Claims

Product teams deciding whether a health feature stays in wellness or becomes a medical device.

Founders planning the budget and timeline for clinical evidence before certification.

Data scientists and engineers building scoring algorithms that will be tested against a clinical reference.

Regulatory and QA leads preparing the clinical evaluation and the documentation behind a score.

Who's Presenting

Aleksander Cudny, Business Analyst at Momentum

Aleksander Cudny

Business Analyst at Momentum

Aleksander works through regulatory requirements with clients before a build starts, from intended use and device classification to the documentation a certification needs. He also presented our earlier sessions on MDR and FDA certification.

Where This Comes From

Momentum is a healthtech software company. We have been building health applications since 2016 and hold ISO 13485 certification. Our health science team designs health scores, and our business analysis team works through the regulatory requirements behind them with clients before a build starts.

This session picks up where our previous webinar, Health Scores: How Sleep Gets Measured and Scored, left off. In it, Anna Zych shows how the sleep score in Open Wearables is built. For more on the topic, read what it takes to build reliable health scores from wearable data and our guide to design controls for medical device software.

Sign Up to Watch the Recording

The link arrives by email right after you sign up.

First Name*
Last Name*
Business E-mail*
Company*

Health Scores in Medical Devices FAQ

What is software as a medical device (SaMD)?

Software as a medical device is software intended for a medical purpose, such as diagnosing, monitoring or treating a condition, that performs that purpose on its own rather than as part of a hardware device. A health score becomes SaMD when its intended use is medical, even if the code is the same as in a wellness app.

When does a health score become a medical device?

When its intended use is to help diagnose, monitor or treat a condition. A score that "helps you understand your sleep" is a general wellness feature. The same score presented as "flags possible disease to help with diagnosis" is medical device software and, under MDR Rule 11, at least Class IIa.

Can Apple Watch detect sleep apnea?

Apple Watch offers sleep apnea notifications, a feature that received FDA authorization in 2024. It flags possible signs of moderate to severe sleep apnea and does not diagnose it. It shows the line this session covers: once a wearable feature claims to flag a disease, it needs a regulatory pathway and clinical evidence behind the number.

What is the apnea-hypopnea index (AHI)?

AHI is the number of apnea and hypopnea episodes per hour of sleep. An AHI above 5 indicates mild sleep apnea, and severity grows as the number increases. Because medicine already trusts AHI, a device that estimates it can rely on published literature for the clinical association.

What do you need to build a medical-grade health score?

Raw signals from your device, a reference standard to compare against (for sleep apnea, polysomnography or polygraphy), data from your target population and paired recordings of the same nights. Paired data is usually the most expensive part, since sleep labs charge per night and a credible dataset needs hundreds of nights.

Why do you need more than one annotator?

Experts don't fully agree with each other. Two certified sleep scorers agree on roughly 83% of epochs, so the ideal setup uses three, with documented instructions and a rule for resolving disagreements.

Should a health score be rule-based or machine learning?

A rule-based score uses weighted components with thresholds. It is easier to explain to an auditor, and every weight traces back to published research. Machine learning is usually more accurate but needs a lot of annotated data, and retraining counts as a new device unless a Predetermined Change Control Plan (PCCP) was approved upfront.

What is the difference between sensitivity and specificity?

Sensitivity is the share of people with the condition that the device flags. Specificity is the share of people without it that the device correctly leaves alone. Both describe the algorithm itself and don't depend on how common the condition is.

Why use PPV and NPV for event detection?

At event level there are no true negatives, because you can't count non-events. PPV answers "my device raised an alert, how likely is it true?" and NPV answers "no alert, can I stop worrying?"