Real-life based quality improvements
When the first version of the FibriCheck application was released as a medical device on the market, we encountered several challenges that were indicated out of the post-market surveillance process. The quality system, which is a crucial component of the FibriCheck algorithm, was very sensitive in classifying the majority of the Android-based measurements as Bad Quality to prevent inaccurate analysis and diagnosis to take place. These quality-related components arise from device and signal-related parameters.
It is nearly impossible to compensate for all of the device variability. So, instead of enabling a step-up program, where we first approve a device and release it to the market, we opted for a step-down process. You define the technical requirements for a device to be accepted on purely technical specifications and when the quality system kicks in you engage into a pro-active mode to detect and identify the problem related to that specific device and decide the corrective actions. Just to give an anecdote, there are less then 20 different iOS devices on the market, which seems an acceptable number of devices that can be technically supported. On the other side, there are over 16,000 different Android devices, which makes it impossible to screen each device.
Based on the real-life data we collected the normalised majority of bad-quality data came from Samsung and Huawei devices. Bad quality in PPG data can arise from 3 different sources: (1) Either the device is causing the problem or (2) the signal itself contains a problem or (3) the user is causing the quality related parameters.
This was a crucial input for our technical and mobile development team to come up with solutions that could support and drive the quality issues for these devices. After several iterations and market-based feedback, we were capable of successfully reducing the amount of bad-quality data to an absolute minimum. This means improvements were made in how the signal acquisition took place, how the signals were processed and how we could extract the most accurate and detailed information out of the smartphone. This had a drastic effect on the data received from Android devices where the amount of bad-quality data dropped from 46% to a staggering 7%. A similar but smaller effect was observed for iOS devices where the current quality standard is at 5%. This means that iOS and Android do not have a significant difference in generating bad-quality data and that the remaining quality issues are related to the user. For this, ongoing research is taking place to improve UX/UI features to support users as much as possible in optimising the quality of the data.
These percentages of bad-quality data are comparable to single-lead ECG devices, which is remarkable for smartphone-derived PPG data.