Abstract
This study introduces a novel transformer-based ensemble framework for the multi-label detection of mental health disorders from social media posts. Unlike traditional multi-class approaches that often struggle with comorbidity, the proposed method employs a binary relevance strategy using fine-tuned DistilBERT models to identify co-occurring conditions, including depression, anxiety, and narcissistic personality disorder. To address class imbalance and optimize decision boundaries, the framework integrates a composite loss function (focal, dice, and log loss) and utilizes Youden’s J statistic for threshold calibration. Validation on textual datasets demonstrates the efficacy of this approach, with an overall F1-score of 0.930 and AUC values exceeding 0.89. Comparative analysis suggests that decomposing complex diagnostic tasks into independent binary problems significantly reduces inter-class confusion relative to standard multi-class baselines. Furthermore, a qualitative error analysis highlights specific linguistic challenges, such as contextual polarity shifting, metaphorical ambiguity, and colloquial usage, that impact model specificity. The findings demonstrate the potential of the proposed framework as a robust screening tool for online mental health monitoring, while underscoring the necessity of human oversight to mitigate linguistic misinterpretations.
Author summary Mental health disorders such as depression, anxiety, and narcissistic personality disorder represent a major global health challenge. This work proposes a method that employs transformer-based deep learning models to analyze social media posts for mental health assessment. A significant hurdle in automated diagnosis is that these conditions often occur together (comorbidity), whereas many existing Artificial Intelligence (AI) systems are designed to detect only a single disorder at a time. This study proposes a solution using a “multi-label” deep learning framework. Rather than relying on a single multi-class classifier, the approach utilizes an ensemble of specialized binary models, each trained to detect indicators of a specific disorder.
This design reduces classification confusion between clinically similar conditions, such as depression and anxiety. The method was evaluated on publicly available datasets, had an F1-score of 0.930 which outperformed the existing approaches. The presented approach demonstrated high effectiveness, achieving better separation between clinically similar disorders compared to traditional methods. Crucially, the detailed investigation beyond the standard statistical metrics was performed which looked into specific models mistakes. It was found that, while the presented AI model is highly sensitive, it can be confused by the specifics of the language such as metaphors (e.g., “feeling like a pressure cooker”), negations (e.g., “I am not worried”), and the colloquial clinical terms. These results highlight that AI is a powerful tool which can be used for early screening and continuous monitoring on social media, while it still requires careful calibration and human oversight to distinguish between genuine symptoms and everyday emotional expression.
The findings demonstrate that analyzing social media texts with advanced machine learning techniques can serve as a powerful complementary tool to clinical diagnostics. While not intended to completely replace professional evaluation, the proposed approach can help identify potential risks, promote earlier detection of mental health disorders, support preventive interventions, and ultimately improve access to care.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This study did not receive any funding
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
All data are fully available without restriction: The datasets underlying the results of this study are available on Kaggle: the Text Classification dataset (https://www.kaggle.com/datasets/comsys/text-classification) and the Depression: Reddit Dataset (Cleaned) (https://www.kaggle.com/datasets/infamouscoder/depression-reddit-cleaned). The source code and pre-processing scripts are available on GitHub at https://github.com/oovcharuk/MentalHealthTextClassifier.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data presented in this paper are contained in the manuscript and in references provided in the manuscript.
https://www.kaggle.com/datasets/comsys/text-classification
https://www.kaggle.com/datasets/infamouscoder/depression-reddit-cleaned
