Shen.ai SDK Documentation
Shen.AI SDK (Software Development Kit) by MX Labs (opens in a new tab) is a B2B platform with precise and easy-to-use camera-based diagnostics of vital signs and wellness.
Evaluating the SDK
The easiest way to get an impression of the SDK's capabilities is by trying out our official app - Heart Monitor. The video measurement functionality is built using the Flutter version of the SDK.
You can also try a live demo of the Web version of the SDK: beta.webheartmonitor.com (opens in a new tab)
If you already have an API key, you can access the Web Playground (opens in a new tab) which displays all the SDK methods in context.
What metrics are currently available?
Our SDK currently exposes the following metrics based on real-time video analysis:
- Heart Rate (real-time and aggregated)
- Interbeat Intervals (real-time and aggregated)
- Heart Rate Variability (HRV) - SDNN, lnRMSSD
- Breathing Rate
- Stress Index
- Blood Pressure (systolic and diastolic) - beta version
See Results for more details on the video-based metrics.
The SDK also computes Cardiovascular health risks based on provided risk factors.
What platforms are supported?
The current version of the SDK supports mobile and web platforms.
- Android - Flutter and Native (Java/Kotlin)
- iOS - Flutter and Native (Objective-C++)
See system requirements for more details.
We are open to expanding our platform support based on customer demand - feel free to contact us at mxlabs.ai (opens in a new tab)
How does it work?
The SDK connects to the camera of a mobile/desktop device and uses real-time Computer Vision to isolate a human face in the video stream. A 3d model of the face is constructed, tracked and used to extract a high-quality, dense signal of blood pulsations using remote photoplethysmography (rPPG). Based on that dense signal, the SDK accurately determines the timings and shapes of all observed heartbeats and provides the user with precise heart-related metrics.
What does it consist of?
The SDK consists of:
- compiled machine code (native shared library on Android/iOS, webassembly on Web) that contains high-performance real-time Computer Vision algorithms and neural networks
- platform/framework-specific components for ease of integration and camera access
- example app code (opens in a new tab) for each supported platform