Validating the efficacy and value proposition of Mental Fitness Vocal Biomarkers in a psychiatric population: prospective cohort study

Mental Fitness study app

Vocal biomarker analysis and visualization was accomplished using a smartphone app developed by Sonde Health, described in detail in the Supplemental Material. Briefly, the app implements a voice journaling capability where users can record 30 second voice samples in response to a range of predefined prompts or talk about their own thoughts and feelings. The response is automatically transcribed and stored in a journaling section, while the recorded audio is used for vocal biomarker analysis using methods similar to those described in OpenSMILE (43) and Praat (44). A “Mental Fitness” Vocal Biomarker (MFVB) score and its feature components are presented to the user after each journaling session and stored in a history section.

The MFVB score components were selected based on a literature review of well supported vocal features relevant to mental health, in particular depression. The features measure different systems involved in vocal production – briefly, they include jitter and shimmer (45–47), pitch variability (45,47–50), energy variability (45,51), vowel space (52,53), phonation duration (37), speech rate (29,50,47,54), and pause duration (50,29,55,47,56). More information on these features is provided in Table 1 and the Supplemental Material.

These features were used as inputs to the MFVB summary score using an averaging approach, resulting in an output range of 0-100 with higher scores indicating higher mental fitness (lower likelihood of elevated mental health symptoms). Before averaging, the vocal feature values were normalized using distributions (see Table 1) calculated from a large proprietary digital biobank previously acquired through clinical studies across a large number of hospital sites in India. The resulting MFVB score distribution for the India-based reference population was used to create score categories indicating relatively high, medium, or low MFVB scores. The score ranges were labeled to allow easier interpretation for the user as “Excellent”, “Good”, and “Pay Attention”.

Study design

This study was designed and reported in accordance with STARD (57), and aimed to demonstrate that MFVB-based mental health symptom severity information could be obtained in consumer-grade products from users that are not guided or trained by a coordinator or clinician. Participants received a 1-page study instruction document that included details on how to install and set up the Mental Fitness app and how to get best results from the recordings. An online video with explanation of how to properly use the app and interpret the results was also made available. Product and study support was available to participants via email if needed, although few participants required assistance.

Participants were recruited from Cognitive Behavior Institute (Pittsburgh, PA), a provider of mental health counseling to people with a range of mental disorders, including depression, anxiety, trauma and stress, obsessive compulsive disorder, and others. The study was advertised via an email campaign that included all patients with either a recent or upcoming counseling appointment. The advertisement included a link to an online consent form where patients were able to review study details, including potential benefits and risk, incentives, and participant expectations. Patients that wished to participate could provide electronic consent and leave a contact email address, which was used by the study team to email the study instruction document described above.

The study flow is further illustrated in Fig. 1. Consent and onboarding questionnaires are hosted online through the SurveyLex platform (58). Completion of the onboarding survey was used to define Day 1 for each participant. At the end of week 4, participants received a second online questionnaire as their final study activity. Participants were informed that the study app was intended for daily use in voice journaling, although app use was self-guided.

• Download figure
• Open in new tab

Figure 1.

Study design and participant flow. Patients under treatment for mental health disorders at the study site receive an email invitation for the study with a link to the online consent form. After consenting and providing contact information, they receive study instructions containing a link to the onboarding survey and guidance for how to use the Mental Fitness study app. Participants use the study app for voice journaling and Mental Fitness tracking on their own for 4 weeks and receive weekly gift card incentives if they use the app at least 4 times per week. At the end of the study, they receive an offboarding survey.

Participants received a $50 Amazon gift card upon completion of the onboarding and also the final questionnaire. To incentive app use, a $15 gift card was provided on a weekly basis to participants that used the app at least 4 times during the preceding week. Participants that did not meet this requirement did not receive any reminders to promote app usage during the study. App usage was monitored by the study team using dashboards hosted on the Sonde Health infrastructure.

The 4-week per-participant duration of the study was chosen to support the study goals by allowing participants sufficient time to use the app so that they could provide meaningful feedback. It also provides a reasonable amount of time to study engagement and retention. Finally, it provides the opportunity to obtain two self-reported mental health questionnaires with sufficient separation in time, each of which can be associated with vocal biomarker results from voice recordings conducted around the time of the questionnaires.

Participants that completed both onboarding and final survey and also used the study app at least 8 times during the study were invited to participate with an extension phase of similar design but reduced incentive structure. The findings from this extension phase will be presented in future reports.

Endpoints

MFVB scores were compared to self-reported mental health symptoms using the M3 Checklist. The M3 results include a score and symptom severity category, including normal, mild, moderate and severe; these scores and categories are provided for overall mental health and in each of the four mental health categories of depression, anxiety, post-traumatic stress disorder (PTSD), and bipolar. MFVB scores are compared to M3 severity categories. Severity categories of moderate and severe will be referred to as “elevated” vs. normal and mild as “lower”.

The primary endpoint is the Relative Risk Ratio (RR Ratio) based on relative risk (RR) estimates for elevated overall M3 symptom severity in MFVB ranges of 0-69 (“pay attention”) vs. 80-100 (“excellent”). RR is calculated as the prevalence of elevated symptom severity within a specified MFVB range divided by the analogous prevalence for the reference cohort, which will in general be the analysis cohort (defined in the Statistical section below), unless otherwise specified. An RR Ratio of 1.0 indicates that the MFVB scores have no ability to differentiate mental symptom severity risk. The primary endpoint is also computed for the four mental health domains mentioned above and also for subgroups based on demographic, clinical, and engagement-related factors¹.

RR Ratio based on relative risk was selected as the primary endpoint instead of other measures like accuracy, sensitivity, specificity, positive/ negative predictive value as these other measures are more typically applied to diagnostic or screening instruments, which is not consistent with the intended use of the MFVB scores. Rather than providing a prediction about the likely absence or presence of mental health symptoms with a binary outcome, RR provides a more nuanced way to convey increasing or decreasing likelihood of elevated symptoms. A lower RR is not a prediction that the user does not have such symptoms, but rather that their vocal characteristics are more consistent with a reduced likelihood, and vice versa for a higher RR result. The middle MFVB score range is anticipated to display RR estimates near 1, conveying that such MFVB scores imply the users voice contains no evidence to suggest a higher or lower risk of elevated symptoms relative to the reference population.

Secondary endpoints are the engagement levels with the study app, measured as number of app sessions in study week 1, 2, 3 and 4; as well as retention in week 4, defined as the percentage of participants that use the study app at least once in week 4. Engagement groups were defined based on total app sessions: high engagement was defined as 16 or more sessions (average of 4 per week, required for receiving the weekly app use incentive), medium engagement as 8-15 sessions, low engagement as fewer than 8 sessions.

Secondary endpoints also include participant feedback as collected via the end of study questionnaire at the end of week 4, which includes multiple choice questions regarding usability and helpfulness of the app and free-text responses, which will be summarized by theme. Both engagement and feedback will be assessed in the same subgroups as mentioned in the primary endpoint description to uncover potential variations in performance or feasibility of the tool within these subgroups.

Eligibility criteria

Patients treated at the study site were eligible for the study if they had at least one clinician-verified symptom of depression according to DSM-5 (59), 14 years or older, and have English as their first language or have conversational proficiency in English as judged from the audio recordings. Participants must own a smartphone device and willing to install and use the Mental Fitness app.

Potential participants were excluded if having a diagnosis of severe cognitive disability precluding informed consent, diagnosis of dementia, schizophrenia, or a speech disorder (e.g., apraxia, dysarthria), or use of certain medications. Although most medications are not formally evaluated for effects on voice and speech, psychiatric, metabolic or other physiologic side effects can occur (10–13). In the context of this study, a short list of psychiatric medication that may have such side effects was excluded, including first-generation antipsychotics (chlorpromazine, haloperidol, loxapine, thioridazine), two or more second-generation antipsychotics (amisulpride, aripiprazole, clozapine, iloperidone, lurasidone, olanzapine, paliperidone, quetiapine, risperidone, ziprasidone, or use of high-dose benzodiazepines (diazepine>=30 mg/day, clonazepam>=2 mg/day, lorazepam>=4 mg/day, alprazolam>=2 mg/day). Additional exclusion criteria were substance abuse, defined as any of the following behaviors in the prior 12 months: 5+ alcoholic drinks in a single day, use of prescription drugs for non-medical use, use of illicit drugs, to avoid potential confounding effects on vocal production (60,61).

Because participants were recruited via email advertisements, we relied on participant self-report in the onboarding survey to verify the exclusion criteria, in particular the criteria involving medical diagnoses and medication use. Language ability and speech disorders were judged from manual quality control on the voice recordings from the study app. Any participant who met any of the exclusion criteria was not removed from the study, but their data was omitted from the analysis cohort (described in the Statistical section below).

Assessments and data collection

Voice recordings were collected through the Sonde Mental Fitness app on participants personal smartphone device. These were 30-second recordings of responses to predefined randomized prompts or the participant’s own topic. The voice journals were transcribed and logged in the app for the user, but transcriptions were not part of the study data set and not considered in any analysis. Acoustic features were extracted from the voice recordings and used to calculate MFVB scores and categories (see Supplemental Material). The number of voice recordings per participant varied as app use was self-directed.

SurveyLex, an online questionnaire and voice recording platform for research use (58), was used to collected various questionnaires:

• Onboarding questionnaire including demographic (gender, age, race, ethnicity), medical (excluded medications, other medication use, diagnosed medical conditions) and behavior (smoking, vaping, substance abuse, use of mental health focused apps) questions.

• M3 Checklist, collected once at onboarding and once at the end of the study in week 4.

• Feedback on the Mental Fitness app and the study. Feedback questions covered level of agreement with liking to use the study app, understanding of the results, helpfulness of the results, helpfulness as addition to treatment, and desire to keep using the study app after the end of the study. Free text responses were provided to questions about how the study app was used and how it was or was not helpful and the best and worst aspects about the app and the study.

The M3 Checklist was used to gather patient-reported mental health symptoms and includes 27 questions related to depression, anxiety, post-traumatic stress disorder (PTSD) and bipolar disorder, as well as questions related to substance abuse and impact of symptoms on work, school, social, and home life. The M3 has been validated vs. MINI mental states diagnoses and compared with existing screening instruments for mental disorders (41,42). We used the M3 due to the convenience of covering multiple mental disorders in one instrument and the ability to assess mental health symptom severity using established score ranges (normal, mild, moderate, severe) for overall transdiagnostic mental health as well for each of the 4 subdomains.

The following data was collected from patient medical records at the study site:

• Diagnosis from treating clinician, reflecting the primary focus of treatment at the time of the study. Typically, this also reflects the condition having the most significant impact on the participant’s mental health, if other conditions are present.

• Treatment duration at the study site at the time of onboarding.

• Treatment elsewhere immediately prior to treatment the study site.

• Date and results of most recent PHQ-9 administered as part of the participant’s treatment.

• Initial (start of treatment) and most recent depressive symptoms noted by the treating clinician.

Data were linked across platforms by matching patient name and email as reported in the informed consent document to medical records at the study site, survey responses, and study app account information.

Statistical considerations

Statistical analyses

The association between MFVB scores and M3 results was conducted in two ways, illustrated in Fig. 2:

1. Closest-MFVB: The single MFVB result that was nearest in time (before or after) to the completion of the M3, but not more than 2 weeks before or after.

2. Time-weighted MFVB: Using all MFVB scores that were available from 2 weeks before to 2 weeks after the M3 was completed. Each MFVB result was weighted as function of its temporal proximity to the M3 with a linear decreasing function of absolute time separation (weighting of 1 for no separation, weighting of 0 for 2 weeks separation). After the initial weights were thus assigned, they were all scaled by a constant so that the weights would sum to 1.

• Download figure
• Open in new tab

Figure 2.

Illustration of time-weighted method to link M3 assessments (right-hand scale, square symbols) to Mental Fitness Vocal Biomarker (MFVB) scores (left-hand scale, round symbols). M3 assessments are obtained on Day 1 (blue) and Day 28 (red) and are each paired with weighted averages of MFVB scores obtained in a period from 2 weeks prior to 2 weeks after the M3 timepoint. Weighting emphasizes MFVB scores closer to the M3 time point, indicated by the size of the MFVB symbol and the decreasing shading applied around each M3 time point. Weights are normalized to sum to 1. Note that in this example, the M3 total score declines over time, paired with a positive trending MFVB score, as intended (recreation of actual participant data).

The rationale and interpretation for the closest-MFVB approach is that this demonstrates the association between a single voice recording analysis to the M3 result, which is useful for one-time or screening-like approaches.

The time-weighted approach evaluates if the association between MFVB scores and M3 results can be enhanced through repeated app usage over time. This is particularly relevant for scenarios like fitness tracking, where users engage with the application repeatedly. The chosen 2-week window aligns with the M3 survey’s duration, mirroring similar time frames in mental health questionnaires like PHQ-9 and GAD-7. This window encourages participants to consider symptoms over time, reducing the risk of momentary biases during assessments. Aggregating vocal characteristics over a 2-week period enhances alignment with survey results, offering a more comprehensive evaluation compared to a single time point measure.

The implementation of decaying weights in the time-weighted MFVB approach introduces a recency bias, assuming that M3 responses are more influenced by recent feelings and moods during the 2-week window. The linear weighting function was selected for its simplicity in incorporating this recency bias into the MFVB aggregation. Associating MFVB scores from both before and after the M3 questionnaire is suitable under the assumption that symptoms change gradually, allowing mental states after the M3 completion to remain correlated with the result for some duration. This ensures that vocal characteristics during this time window remain pertinent to the mental health assessment.

Ethical considerations

The study was conducted in accordance with the principles of Good Clinical Practice as described by the study designs and controls previously. Furthermore, the research protocol was reviewed and approved by WCG IRB (Protocol #20220961) prior to study initiation, and the Hamilton Integrated Research Ethics Board (Project #14494) for the pilot phase at St Joseph’s Healthcare Hamilton. All participants provided electronic informed consent in the online SurveyLex platform before participating in the study. The study app and databases were hosted on Amazon Web Services cloud servers with various security mechanisms in place, described in more detail in the Supplementary Material. Participants were made aware in the informed consent information about financial incentives in the form of electronic gift vouchers upon completion of onboarding and final questionnaire ($50 in each case) and $15 for any of the 4 study weeks if they used the study app at least 4 times during that week. Total potential incentive amount was $160. Incentives were provided solely based on the completion of these study activities and were not contingent on specific outcomes.

Enrollment

The participant cohorts are illustrated in Fig. 3: a total of 147 participants from the study site provided electronic informed consent on the web application between May and August 2023. All these participants were emailed study instructions and a link to the onboarding questionnaire, which was completed by 120 participants. Of these, 115 participants met eligibility criteria, and of those 104 participants used the study app at least once during the study. This set of 104 participants forms the analysis cohort and will be used to report results, unless otherwise indicated. The offboarding survey at the end of week 4 was completed by 81 participants, forming the completer cohort.

• Download figure
• Open in new tab

Figure 3.

Cohort definitions and participant counts.

Demographics and health characteristics

Demographic and health characteristics of the analysis cohort are provided in Table 2. The analysis cohort was majority female (73%); skewed to younger adults with proportion in age ranges below 30, 30-39 and over 40 of 41%, 27% and 32% (age range 16-80); mostly white (93%) and non-Hispanic (99%). Most participants have been in longer-term treatment (66% for >1 or 2 years) and were not treated elsewhere prior to their current therapy (75%). Most participants (73%) reported using prescription medications for their mental health treatment (main reported medications being fluoxetine, sertraline, escitalopram, bupropion, lamotrigine, buspirone, lisdexamfetamine, methylphenidate, venlafaxine, and duloxetine). About half of the participants (47%) reported one or more diagnosed health conditions, including obesity (19%), asthma (15%), hypertension (12%), diabetes (3%) and other conditions (19%).

Clinical presentation

Psychiatric diagnoses were extracted from participant medical records at the study site and grouped into high-level categories as indicated in Table 3. Anxiety-related diagnoses were the most common (38% of the analysis cohort), followed by trauma and stress-related disorders (31%), and depression disorders (20%). The most prevalent specific diagnoses were generalized anxiety disorder (22% of the analysis cohort), major depressive disorder (16%), adjustment disorder (19%), obsessive-compulsive disorder (11%), and post-traumatic stress disorder (PTSD, 10%). Various other conditions were also present in smaller numbers, e.g., attention-deficit hyperactivity disorder, bipolar disorder, and borderline personality disorder.

Among consented participants that failed to onboard onto the study, a relatively large number included participants with diagnoses of generalized anxiety disorder (11, or 30% of the consented group with this diagnosis) and attention-deficit hyperactivity disorder (4, or 50%). Onboarded participants that never used the study app or were not eligible and therefore not included in the analysis cohort were not concentrated in any particular diagnosis, being somewhat high only in major depressive disorder (4, or 19% of the onboarded group with this diagnosis). Most participants (80%) also completed the offboarding questionnaire (completer cohort) and no major differences between diagnostic categories in this regard were noted.

Table 4 offers insights into mental health symptom severity through M3 Checklists, encompassing 104 participants and 185 M3 assessments (5 M3 assessments could not be linked to any participant and were excluded from analysis). Results are presented at onboarding for both the analysis and completer cohorts, allowing for direct comparisons with offboarding outcomes available exclusively from the completer cohort. At onboarding, the analysis cohort presented the following severity distribution: normal (0%), mild (31%), moderate (48%), and severe (21%). The cumulative prevalence of elevated (moderate-to-severe) symptoms was 69%. Examining mental health categories, elevated symptoms were prevalent in 53% for depression, 30% for anxiety, 27% for PTSD, and 11% for bipolar.

In approximately one-third of participants, elevated symptoms were observed across multiple mental health categories, indicating a high overall symptom burden (100% of these with elevated total symptom severity). The most prevalent comorbid symptom categories involved depression, anxiety, and PTSD, with additional occurrences of elevated bipolar symptoms in certain cases. Depression with anxiety and depression with PTSD was also observed. Notably, elevated depressive symptoms were widespread, while elevated bipolar symptoms were relatively uncommon, except for a small subset exhibiting elevated symptoms across all categories. Consequently, the presence of elevated bipolar symptoms emerged as the primary predictor for elevated overall mental symptom burden and exhibited the highest average number of comorbid symptom categories. These findings underscore the importance of comprehensive mental health assessment, enabled by transdiagnostic tools like the M3.

Symptom severity at onboarding was correlated with the likelihood to complete the offboarding survey: completers vs. non-completer (participants from the analysis cohort that are not included in the completer cohort) demonstrated the following elevated symptom severity distribution: Total (65% vs. 81%), depression (47% vs. 71%), anxiety (29% vs. 33%), PTSD (27% vs. 29%), and bipolar (9% vs. 14%).

Symptom severity reported in the offboarding survey at the end of week 4 indicates a notable reduction in elevated symptom severity within the completer cohort over the study period. The prevalence decreased from 65% at onboarding to 53% at offboarding, reflecting a 13% reduction—suggestive of the potential impact of treatment provided at the study site. The average reduction in elevated symptom severities across the four subcategories was 5%, with the most substantial improvement observed in depression (13%), while the lowest reductions were noted in PTSD (2%) and anxiety (1%). These improvements were also noted on functional impairment outcomes of the M3: 25-30% of participants reported improvements in functioning at work or school and relationships with friends or family. These numbers rose to 35-40% for those participants that reported significant impairment in functioning on their initial M3.

Voice sample recordings

Participants conducted 1,336 app sessions with voice recordings during their 4-week study period, resulting in an average 12.8 sessions per participant, or 3.2 per week – nearly exactly once every other day. Additional detail regarding app use will be provided in the Participant Engagement section below. Vocal feature distributions from the study data set were compared with those from the reference dataset that was used to develop the MFVB scoring algorithm and were found to be in close alignment (see Table 1).

Relationship of vocal biomarker to mental health symptom severity

Out of 185 M3 assessments, 177 were included in analysis of the primary endpoint due to having 1 more associated MFVB results within the 2-week time window. The prevalence or risk of elevated mental health symptom severity across this dataset was 62% (110 out of 177), defining the Relative Risk (RR) level of 1.00.

Visual representation of the alignment between MFVB score categories and total symptom scores from the M3 assessment is shown in Fig. 4, using the closest-MFVB and time-weighted MFVB approach. Both approaches indicate that M3 scores and thus symptom severity distributions increase from MFVB categories of “Excellent” to “Good” to “Pay Attention”, as intended.

• Download figure
• Open in new tab

Figure 4.

M3 total risk score distributions by MFVB score category, indicating the correlation between vocal analysis results and participant reported mental health symptom severity. The left-hand side of the panel associated the MFVB result closest in time to the M3, whereas the right-hand side of the panel uses a time-weighted average of MFVB results within 2 weeks of the M3 (see Methods). Results in Table 5 are calculated on the same data as displayed here.

RR was determined within MFVB score categories of Excellent, Good, and Pay Attention. Table 5 outlines the RR estimates and 95% confidence intervals (omitted here for brevity) for these MFVB score categories as 0.82, 1.02, and 1.25 when using the closest-MFVB method and 0.62, 1.07, and 1.23 when using the time-weighted method.

Notably, both methods show that the “Excellent” and “Pay Attention” MFVB categories reflect RR below and above 1.00 respectively, as originally intended, as well as an RR near 1 for the MFVB score category of “Good”. The RR Ratios between Pay Attention and Excellent MFVB score categories for these two methods are statistically significant at 1.53 (1.09-2.14, p=0.0138) and 2.00 (1.21-3.30, p=0.0068), respectively, meeting the primary endpoint of the study. A principal effect of the time-weighted versus the closest-MFVB method is an increased distinction in RR away from 1, presumably benefiting from “multiple looks” at the MFVB results across the 2-week aggregation period.

The same principles observed for the total M3 can be extended to the subcategories for depression, anxiety, PTSD, and bipolar (Table 5). MFVB efficacy for depression is more pronounced compared to total symptoms: this heightened distinction may be attributed to the selection of vocal features, which were primarily chosen for their established association with depression. RR Ratios for closest and time-weighted methods are both statistically significant at 1.78 (1.08-2.93, p=0.0237) and 2.60 (1.45-4.66, p=0.0013), respectively. Results for anxiety and PTSD indicate consistent RR trends despite wider confidence intervals and non-significant RR Ratio due to a lower prevalence of elevated symptoms in these categories. It is particularly challenging to accurately assess RR for bipolar due to the limited total number of reported cases of elevated symptom severity (only 14 M3 results fall into this category).

Additionally, hypomanic patients may manifest a mixture of vocal qualities, some of which are normally associated with strong mental health.

Participant engagement with the Mental Fitness study app

As indicated in Table 6, participants engaged with the study app an average of 12.8 total times over the course of the 4-week study period, with 70% of users remaining active in week 4. These average usage patterns can be further examined by categorizing engagement into three levels, as described in the Methods section.

The high engagement category included 38% of participants, with a consistent 5-6 app sessions each week, 100% retention in week 4 and an average of 21.1 sessions in total (approximately 10 minutes of voice recordings over the study period). The medium engagement group (8-15 sessions) comprised 33% of participants. This group typically utilized the app 3-4 times per week with a gradual decrease over time, had 71% retention rate in week 4, and an average of 11.6 sessions overall. The remaining 29% of participants exhibited low engagement and had minimal app usage, even in the initial study week; only 14% of these users were retained in week 4, with an average of 3.5 total app uses. The engagement groups thus differed both on total app usage but also persistence of use over time.

To find potential reasons for these very different levels of engagement, cohorts defined by the engagement groups were constructed and assessed for variations in demographics, health characteristics, and clinical presentation. Statistical significance was assessed in each case using chi-square test and 5% significance level. The only statistically significant factor was age (p=0.0131), with older age groups displaying higher levels of engagement. The oldest age group (40+ years) comprised 45% of the high engagement group, 26% of the medium engagement group, and 20% of the low engagement. While the middle age group (30-39 years) trended similarly, for the youngest age group (<30 years) the analogous proportions were 20%, 53%, and 57%, revealing an opposite trend.

Other factors, including male gender (p=0.1273), absence of prescription medication use in mental health treatment (p=0.3157), lower symptom severity (p=0.4425, comparing completer and non-completer participants), and having a psychiatric diagnosis other than anxiety-related, exhibited positive trends toward increased engagement, although statistical significance was not reached. Notably, age and these non-significant factors, which trended positively with higher engagement, were also linked to lower symptom severity. This suggests that symptom severity may be a primary determinant of engagement, with higher symptom levels associated with reduced engagement and vice versa. This aligns with the expectation that individuals experiencing more severe symptoms might be less engaged.

Participant feedback

Participant agreement with five statements presented in the offboarding questionnaire at the conclusion of week 4 is summarized in Table 7. This questionnaire was completed by 80 participants, 77% of the analysis cohort. The high response rate reduces potential responder bias in the response analysis.

In general, a substantial majority of participants expressed partial or full agreement with all statements. The highest level of agreement (81%) was in response to the statement expressing contentment with the capability to assess Mental Fitness through voice recordings on their smartphone, underscoring the favorable reception of this activity and application among participants. The lowest level of agreement (55%) was noted in response to the statement asserting that the Mental Fitness app constituted a beneficial adjunct to their treatment at CBI. Here, a relatively large proportion of participants indicated a neutral opinion. It is noteworthy that neither the design of the Mental Fitness app nor the study itself inherently positioned it as a supplement to mental health treatment. Consequently, the observed level of agreement can in fact be considered rather high. Evidently, about half of the study participants organically discovered meaningful ways to incorporate the app into their treatment regimens. The remaining three statements (pertaining to comprehension of MFVB scores, the perceived utility of MFVB scores for self-assessment, and the intention to continue using the Mental Fitness app in the future) received partial or full agreement from approximately 70% of participants.

Participant responses to the statements were also analyzed based on level of engagement to determine if satisfaction correlated with usage (Table 7). Results showed that participants in the high engagement group agreed more frequently, with 82% somewhat or completely agreeing on average across all statements. Using the same calculation, the medium engagement group had a 61% agreement rate, while the low engagement group had a 57% rate. Interestingly, the statement about understanding the MFVB scores showed a modest 9% difference between the high and low engagement groups, indicating that level of understanding of the scores wasn’t a significant factor driving app usage. Conversely, the statement about the app being a helpful addition to treatment at CBI had the most substantial differences between engagement groups, ranging from 71% agreement in the high engagement group to 50% in the medium engagement group and 29% in the low engagement group. This suggests that the perceived value of the app as an addition to treatment was closely tied to consistent app usage. Overall, engagement and satisfaction were strongly correlated.

Free response feedback

The offboarding survey at the end of week 4 also included several free response questions where participants could provide more detail about how they used the app, whether it was a helpful addition to their treatment at the study site, and what was perceived as the best and worst things about the study app. The combined response set from these questions provided a rich source of insight on how participants used the app and what sources of benefits they perceived, with approximately 40% of participants reported making some change in their behavior or lifestyle as a result of using the MFVB tool and approximately 30% mentioned perceived benefits to their wellbeing.

Primary endpoint: subgroup analyses

MFVB scores stratify participants into mental symptom severity groups

This study demonstrates the practical deployment of vocal biomarkers in a prospective cohort study, providing participants with real-time insights into their mental health symptom severity. Despite advancements in identifying voice-related features and predictive analytics, translating these approaches into practical technology had remained a challenge, and relatively few published works combine all the aspects tested in our study. To our knowledge, this is the first demonstration of a vocal biomarker tool in any health or wellness application that has delivered meaningful health-related information to actual users in real-time, with a statistically significant level of efficacy.

One recent smartphone app-based speech analysis approach using semantic information for depression and anxiety screening in a mostly older population and compared 5-min voice recordings with PHQ-8 and GAD-7 as reference as a validation study and showed good performance (63). The analysis approach differed from ours in not using acoustic features and providing only binary classifications, without feedback to participants, making it less suited as a personalized mental wellbeing tool. Each session required 5 minutes of voice recording time, significantly more than is required for purely acoustic and prosodic feature extraction. A uniquely long-term (6-month) study in bipolar patients tested whether a machine-learning based vocal biomarker approach could make use of repeated observations over time to identify euthymia, hypomania, and manic episodes in participants via weekly telephone interactions (64), and reported good performance for differentiating hypomania and depression from euthymia. As we found in our study, continuous observations over time proved essential, especially for bipolar disorder, characterized by fluctuating mood over time.

Our method, averaging normalized values of eight vocal features from a 30-second speech sample (see Supplemental Material), facilitates measurement of moderate-to-severe (elevated) mental health symptom severity in terms of Relative Risk (RR), laying a foundation for application-specific products that offer objective and quantitative mental fitness information. RR offers comparative understanding of the likelihood of elevated symptom severities being present within a range of Mental Fitness Vocal Biomarker (MFVB) scores compared to an overall population average.

The use of RR instead of accuracy or sensitivity and specificity, typically employed as outcome measures in vocal biomarker studies, is appropriate given the likely intended uses, which would not primarily involve the identification or differentiation of mental health conditions. The underlying vocal features were selected based on reported correlations with depression but do not have well-supported threshold values that separate clinical categories. Interpretation of MFVB score in terms of RR avoids potentially incorrect interpretation by users that a high (or low) MFVB score rules out (or in) the presence of a mental health condition, whereas the risk concept more appropriately conveys a change in likelihood of underlying symptoms.

We found that MFVB scores also stratified symptom severities within subdomains of depression, anxiety, and PTSD. The RR Ratio was greatest for depression, perhaps explained by the fact the underlying vocal features were primarily selected based on pre-existing evidence for that condition. What is unique about the current study is the transdiagnostic participant sample, applying a uniform vocal processing approach to demonstrate MFVB efficacy for multiple and potentially comorbid mental health conditions. Another differentiating feature of our work is the inclusion of diagnosed individuals only, without healthy controls, aiming to differentiate symptom severity rather than diagnosis status.

Optimal MFVB performance requires frequent observations over time

We employed two methods for linking MFVB scores to M3 results: one associating a single MFVB score closest in time to the M3, and the other averaging all MFVB results within 2 weeks before or after the M3 (applying weights to diminish the contributions of more distant MFVB scores). Both approaches affirm that MFVB scores provide valuable information about symptom severity. Notably, the time-weighted method exhibits superior risk stratification ability, capitalizing on the accumulation of vocal information over time and aligning more closely with the M3 time window. This distinction, pointing toward the significance of longitudinal observations, is not merely an accidental statistical outcome but an expected behavior. Analogous to understanding a region’s climate, reliable insights into overall mental fitness—the backdrop against which day-to-day mood fluctuations occur—require frequent observation over time. Vocal biomarker tools are inherently well position to gather and refine insights over time, emphasizing the need for future studies and product designs to leverage this intrinsic quality.

Our results confirmed the utility of frequent measurement over time in another way, by revealing significantly enhanced RR estimates in participants that were more highly engaged with the study app. We were able to determine that the enhancement was primarily the result of more frequent “observation” (voice sample recordings). This is good news, because it implies that improvements in MFVB performance can be achieved by boosting engagement for users that would otherwise have low engagement levels. Achieving this goal will require research into strategies that can increase engagement, perhaps tailored to specific user characteristics. An alternative option is to develop passive voice monitoring tools that can continuously observe the user’s voice in the background, once consent has been provided. This would remove the engagement element completely and allow a maximum of observation to occur for all users.

Participant engagement

As we found above, achieving and sustaining high engagement is essential for many health apps, but often proves challenging in practice. Although the duration of this study is relatively brief, the observed engagement levels are promising. This is evident from the proportion of participants engaged, which combined the mid and high engagement groups, reaching 71%, and the persistence of that engagement, with an observed 87% retention in week 4 averaged across these two groups.

These metrics compare favorably to most other app-based or mHealth studies (65,66). Study design factors that have previously been found to promote retention that were present in this study are the recruitment method (study invitation sent from participant’s clinical treatment organization), financial compensation for participation, and having a relevant condition to the app design (mental health concerns). Perhaps the most important success factor was delivering real-time useful information (MFVB scores) back to users through the study app. The combination of insight and engagement that this approach fosters could be a natural way to enhance the perceived value and success of other digital therapeutic products.

Although participants were offered a financial incentive of $15 per week to encourage app usage of 4 or more times weekly, the incentive did not drive high levels of engagement among all participants. The proportion of participants receiving gift cards in each week was, not surprisingly, correlated with engagement level. That proportion was about 80% in the high engagement group, about 40% in the medium engagement group, and near zero in the low engagement group. Further, a potential small positive effect on app usage in weeks where a gift card was not received for the prior week was observed for participants in the low and medium engagement groups.

Engagement was negatively correlated with mental health symptom severity, both for total symptoms and within symptom categories. In particular, the prevalence of elevated depressive symptoms was relatively high in the low engagement subgroup, although the same trends were observed for bipolar, PTSD, and anxiety. Individuals experiencing more severe symptoms may find it challenging to engage consistently with the study app due to the potential impact of their mental health condition on motivation, energy levels, and overall well-being. Higher symptom severity often correlates with increased psychological distress and reduced functional capacity, making it difficult for individuals to sustain regular and active participation in the study. Therefore, the observed trend of lower engagement among those with higher symptom severity aligns with expectations based on the potential influence of mental health symptoms on individuals’ ability and willingness to engage in additional activities. Targeted and higher-intensity engagement and retention strategies may therefore be beneficial for participants with high symptom burden in future studies.

Engagement was positively correlated with age, an effect that has been observed previously in digital health app studies (65,66); although it is contrary to conventional expectations that it is more difficult to engage older populations with digital tools. In this study, age was also negatively correlated with symptom severity; it is therefore possible that age and symptom severity both reinforced the observed engagement trends.

Study app perception differed by engagement groups

Comments made by participants from the high engagement group indicate a strong commitment to the Mental Fitness app, appreciating the daily check-ins and addition to their daily routines. They found the app beneficial for tracking their mental wellbeing and for self-expression. Participants in the medium engagement group displayed more mixed sentiments, with concerns about notifications and scoring accuracy affecting their overall engagement. The low engagement group participants exhibited the least enthusiasm in their feedback, with limited app use due to forgetfulness or perceived inconvenience, although some participants still acknowledged potential benefits from more frequent use. This seems consistent with the 53% level of agreement stated by the low engagement group in the end of study feedback, when asked whether they intend or would like to keep using the app to track their future wellbeing – a number well above their actual level of use (even the subset of low engaged users that completed the feedback).

Perceived accuracy of the MFVB scores was expected to have an impact on engagement, but this was not evident from the feedback. The low-engagement group made no comments about lack of accuracy, while high- and medium-engagement groups commented similarly on perceived accuracy (or lack thereof) of the MFVB scores. Interestingly, Table 7 indicates that the medium engagement group has the lowest level of agreement with the statement that the MFVB scores are helpful in understanding how they are doing, even less than the low engagement group. This appears consistent with the summarized free response feedback described above, and the perception of limited benefits for personal insight may be a key factor holding these participants back from more consistent app use.

When considering whether the MFVB tool is a beneficial addition to mental health treatment, this study has revealed how digital health apps may provide complementary benefits to therapy sessions. Participants that frequently used the app say that they generally found the app to be a valuable addition to their treatment as a tool for self-reflection, providing reminders to relax and a way to practice the skills they worked on in therapy. It helped them focus on positive thoughts and stay in a mental health routine. Less engaged users also appreciate the tool as a platform to voice thoughts and feelings but do not perceive it as contributing to their treatment as much, either because they do not use it as frequently or because they view it more as a mental exercise.

Limitations

Potential applications and future work