VitalLens API

The VitalLens API is a web-based service that estimates a person's vital signs from a video of their face and upper body.

TLDR: Heart rate (HR), respiratory rate (RR), pulse waveform, and respiratory waveform.

The VitalLens API provides an estimate of the pulse waveform and respiratory waveform at the same sampling rate as the provided video. From this, heart rate (HR) and respiratory rate (RR) are derived.

It is theoretically possible to also derive heart rate variability (HRV) from the pulse waveform, however we have not yet validated the accuracy of this - such a study is on our roadmap. Our roadmap also includes additional vital signs such as blood oxygen (SpO2) and blood pressure (BP).

TLDR: Estimates have a high degree of accuracy for high-quality video. Each estimate includes an estimation confidence.

We have conducted a study on the Vital Videos dataset with 289 unique participants, showing a mean absolute error for HR at 0.71 bpm and for RR at 0.76 bpm.

For each vital sign estimate, the VitalLens API also produces an estimation confidence between 0% and 100%. When these confidences are lower, this means that accuracies are not as high as the above results - this can be mitigated by ensuring high-quality videos with our guidelines outlined below.

To get access to the VitalLens API, please first use the button above to register for an API Key. You will receive an email with a link to verify your account. Once verified, we will automatically generate your unique API Key associated with the free plan. You can then log in on this page to view your API Key and start using the VitalLens API.

For each request, the VitalLens API accepts a maximum of 900 frames of spatially downsampled video depicting a single person's face and upper body at resolution 40x40px. This video is passed through our estimation engine, which produces estimates for several vital signs. These estimates are then sent back, after which both the video frames and estimates are disposed of.

Please consult our API Documentation to learn how to use the VitalLens API.

As of now, the best way to use the VitalLens API is with our Python Client. Install the client:

  pip install vitallens
                

Get vital signs estimates from a video in a few lines of code:

  from vitallens import VitalLens, Method

  vl = VitalLens(method=Method.VITALLENS, api_key="YOUR_API_KEY")
  result = vl("video.mp4")
  print(result)
                

You can also directly call the API. For this, you need to first parse the video file using ffmpeg:

  input_video="video.mp4"
  api_key="YOUR_API_KEY"

  fps=$(ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate -of default=nw=1:nk=1 "$input_video" | awk '{printf "%.2f", $1}')
  video=$(ffmpeg -i "$input_video" -vf "scale=40:40" -pix_fmt rgb24 -f rawvideo - | base64)
  echo "{\"video\": \"$video\", \"fps\": \"$fps\"}" > payload.json
  curl -X POST -H "x-api-key: $api_key" -H "Content-Type: application/json" --data-binary @payload.json https://api.rouast.com/vitallens-v2
                

Note that this only works well if the video is cropped to the face and shoulders.

In the future, we will also provide SDKs for iOS and Android.

The returned object for a request to the VitalLens API looks like this:

  {
    'face': {
      'confidence': Face live confidence for each frame,
      'note': Explanatory note
    },
    'vital_signs': {
      'heart_rate': {
          'value': Estimated value as float scalar,
          'unit': Value unit,
          'confidence': Estimation confidence as float scalar,
          'note': Explanatory note
        },
      'respiratory_rate': {
        'value': Estimated value as float scalar,
        'unit': Value unit,
        'confidence': Estimation confidence as float scalar,
        'note': Explanatory note
      },
      'ppg_waveform': {
        'data': Estimated waveform value for each frame,
        'unit': Data unit,
        'confidence': Estimation confidence for each frame,
        'note': Explanatory note
      },
      'respiratory_waveform': {
        'data': Estimated waveform value for each frame,
        'unit': Data unit,
        'confidence': Estimation confidence for each frame,
        'note': Explanatory note
      },
    },
    'message': Message about estimates
  }
              

Our Python Client and SDKs are provided to handle face detection, video cropping and spatial downsampling, splitting of longer videos, as well as API interaction for you.

To ensure accurate vital signs estimation, follow these guidelines:

• Minimize camera movement.
• Keep the subject still and facing the camera.
• Ensure a bright and steady light source.
• Video compression can destroy the vitals signal. If you need to encode the video before using the VitalLens API, use as little video compression as possible.

These guidelines are informed by our study in which we investigated how environmental factors such as the above impact estimation accuracy of VitalLens.

Additionally, a good internet connection and sufficient bandwidth are required for optimal performance of the API.

The accounting unit is the number of video frames analyzed. Each frame is assumed to have only one face present. For example, if you have a one-minute video depicting one person at a frame rate of 30 fps, that use 60 seconds x 30 fps = 1,800 frames.

If you are using our Python Client with a longer video of N frames, it will be split into overlapping segments and use slightly more frames.

Please contact us at vitallens@rouast.com.

No, we do not collect or store any of your video or vital sign estimates data. After returning the API request result, we immediately dispose of this data - hence it is your responsibility to store the returned estimates. Please also see our Terms of Service for more information.

No, VitalLens is not a medical device and its estimates are not intended for any medical purposes. Please also see our Terms of Service for more information.