Behavioral Considerations

Individual behavior is one of the more important aspects to consider when calculating digital biomarkers. OpenDBM is blind to the different behaviors the individual is participating in within the video or audio it is processing. For example, if in the same video, the individual is demonstrating spontaneous facial behavior (e.g. responding to an open-ended question) and then later in the video are asked to make a face (e.g. being asked to purse their lips, as some patients are during clinical assessments of facial tremor), OpenDBM is going to make its measurements across both behaviors. So, if the user is trying to measure spontaneous emotional expressivity, they really only want to do that in the former case; not the latter. Hence, when processing markers from data using OpenDBM, the user needs to split data by behavior.

Splitting behaviors

The user is most knowledgeable about the data being processed, the experimental paradigm that was used to collect it (or lack thereof), and the different clinically meaningful behaviors that may be present in it. An exhaustive list of different kinds of clinically meaningful behavior is not within the scope of this section; it depends of course on the disease population and the literature that may or may not exist regarding relevant behaviors. Below are examples of some behaviors we’ve learned to analyze separately:

• Facial behaviors
  • Spontaneous facial expressivity while resting
  • Spontaneous facial expressivity while talking
    • In response to neutral stimuli
    • In response to valenced stimuli (e.g. talking about positively or negatively valenced images or videos, questions about symptomatology)
  • Expressions made on cue (e.g. when asked to make a face such as a happy face, sad face, pursed lips, shut eyes)
  • Facial expressions evoked naturally in response to stimuli (e.g. immediate visual responses such as ‘micro-expressions’ to images, videos, or other stimuli)
• Vocal acoustic behaviors
  • Acoustics of voice during sustained vowel sounds (e.g. say aah for a few seconds, say eee for a few seconds––this is super prevalent in the literature).
  • Acoustics of voice during free speech (e.g. responding to an open-ended question or just generally talking or in conversation)
• Speech behaviors
  • Free speech as part of general, neutrally valenced conversation
  • Free speech as part of positively or negatively valenced conversation or responding to valenced stimuli (e.g. being asked to speak about a past traumatic experience, image, or video)
  • Evoked speech when asked to say something (e.g. saying the names of the days of the week or months)
  • Evoked speech when asked to read something (e.g. reading a passage)
• Movement behaviors
  • Free head movement when resting or speaking
  • Eye gaze behaviors when looking at social vs. non-social stimuli
  • Eye gaze behaviors in contexts where saccades could be measured

As can be deduced from the list above, there are a lot of different kinds of behaviors––and each of them may or may not be relevant depending on the clinical population being studied. For example, we find that measurements of blunted affect in individuals with schizophrenia are much stronger when acquired from elicited expressions (e.g. being asked to make an expressive face) compared to during spontaneous behaviors (e.g. when responding to a question), but that the case is the opposite in individuals with Major Depressive Disorder (please note that this is not a universal truth; just something we’ve observed in our experiments). We can’t comment on which behaviors are best for the user to be able to measure the symptomatology they’re looking for––all we can suggest is diving into past literature to see if there are clues as to what behaviors are best for eliciting the disease’s symptomatology.

Analyzing behavioral data

If the user is working with data that contains different patient behaviors i.e., let’s say they split up a video of a patient participating in free speech and also sustained vowel sounds into two separate videos: a free speech video and a sustained vowel sound video. The amount of digital biomarker variables the user has access to multiplies by the number of behaviors. So––let’s say the user is interested in measuring the mean fundamental frequency (Section 5.2.1) of voice. Now they have two fundamental frequency mean variables: one for free speech, one for sustained vowel sounds. In the data analysis that follows, these can essentially be treated as separate variables.

There’s an additional point to be made here: Some variables only make sense for certain behaviors. For example, the vocal tremor variable in Section 5.4.3 is only useful in the context of sustained vowel sounds. Even in clinical examinations of vocal tremor, patients are asked to make sustained vowel sounds (i.e. say aah out loud for a few seconds) and the tremor in their voice is then assessed subjectively by the interviewer/clinician from that sound. Hence, in some cases, a variable may not be informative if it is not collected from the right kind of behavior. Referencing the example from the previous paragraph, mean fundamental frequency is also more relevant when measured from sustained vowel sounds, whereas its standard deviation may be more relevant during free speech. The user must take such factors into consideration when analyzing digital biomarker data.