Detecting Mental Disorders in Social Media Using a Transformer-Based Ensemble of Binary Classifiers =================================================================================================== * Oleksandr Ovcharuk * Olexander Mazurets * Maryna Molchanova * Alexander Kirpich * Pavel Skums * Olena Sobko * Olexander Barmak * Iurii Krak * Sergiy Yakovlev ## Abstract This study introduces a novel transformer-based ensemble framework for the multi-label detection of mental health disorders from social media posts. Unlike traditional multi-class approaches that often struggle with comorbidity, the proposed method employs a binary relevance strategy using fine-tuned DistilBERT models to identify co-occurring conditions, including depression, anxiety, and narcissistic personality disorder. To address class imbalance and optimize decision boundaries, the framework integrates a composite loss function (focal, dice, and log loss) and utilizes Youden’s *J* statistic for threshold calibration. Validation on textual datasets demonstrates the efficacy of this approach, with an overall *F*1-score of 0.930 and AUC values exceeding 0.89. Comparative analysis suggests that decomposing complex diagnostic tasks into independent binary problems significantly reduces inter-class confusion relative to standard multi-class baselines. Furthermore, a qualitative error analysis highlights specific linguistic challenges, such as contextual polarity shifting, metaphorical ambiguity, and colloquial usage, that impact model specificity. The findings demonstrate the potential of the proposed framework as a robust screening tool for online mental health monitoring, while underscoring the necessity of human oversight to mitigate linguistic misinterpretations. **Author summary** Mental health disorders such as depression, anxiety, and narcissistic personality disorder represent a major global health challenge. This work proposes a method that employs transformer-based deep learning models to analyze social media posts for mental health assessment. A significant hurdle in automated diagnosis is that these conditions often occur together (comorbidity), whereas many existing Artificial Intelligence (AI) systems are designed to detect only a single disorder at a time. This study proposes a solution using a “multi-label” deep learning framework. Rather than relying on a single multi-class classifier, the approach utilizes an ensemble of specialized binary models, each trained to detect indicators of a specific disorder. This design reduces classification confusion between clinically similar conditions, such as depression and anxiety. The method was evaluated on publicly available datasets, had an *F*1-score of 0.930 which outperformed the existing approaches. The presented approach demonstrated high effectiveness, achieving better separation between clinically similar disorders compared to traditional methods. Crucially, the detailed investigation beyond the standard statistical metrics was performed which looked into specific models mistakes. It was found that, while the presented AI model is highly sensitive, it can be confused by the specifics of the language such as metaphors (e.g., “feeling like a pressure cooker”), negations (e.g., “I am not worried”), and the colloquial clinical terms. These results highlight that AI is a powerful tool which can be used for early screening and continuous monitoring on social media, while it still requires careful calibration and human oversight to distinguish between genuine symptoms and everyday emotional expression. The findings demonstrate that analyzing social media texts with advanced machine learning techniques can serve as a powerful complementary tool to clinical diagnostics. While not intended to completely replace professional evaluation, the proposed approach can help identify potential risks, promote earlier detection of mental health disorders, support preventive interventions, and ultimately improve access to care. ## Introduction Mental health conditions such as depression, anxiety, and narcissistic personality disorder represent a major global health challenge, affecting millions of individuals worldwide. However, many cases remain undiagnosed by conventional assessment methods [1–4]. Over the years, various strategies have been developed to diagnose certain mental disorders and risky behaviors, including depression, eating disorders, gambling, and suicidal ideation [5]. One such strategy involves the monitoring of social media content [6]. This approach enables the identification of behavioral trends by analyzing the lexical, semantic, and stylistic characteristics of individuals’ posts. Social networks function as platforms for communication and as environments where people express their experiences, emotional states, and attitudes toward various events, either explicitly or implicitly. Consequently, analyzing textual content to detect signs of psycho-emotional changes can serve as a valuable tool for the early detection of potential mental health disorders [7]. Conventional methods for diagnosing mental disorders, such as clinical interviews and psychometric assessments, are effective but require direct personal interaction with patients. These approaches are often time- and labor-intensive, have limited throughput, and are typically not proactive [8, 9]. In contrast, the analysis of user-generated content in online environments allows for real-time data collection from large and diverse populations, facilitating earlier and more scalable detection of mental health disorders [10]. The other notable advantage of this approach is its ability to reveal hidden behavioral patterns that might be overlooked by traditional assessments. By utilizing natural language processing algorithms and deep learning techniques, it is possible to analyze vast amounts of text data, pinpointing distinctive linguistic features associated with specific mental disorders. This methodology, therefore, paves the way for the development of automated systems for the early detection of mental health issues, which can enhance clinical methods and improve the overall effectiveness of psychodiagnostics [11]. The study of early diagnostic methods for mental disorders using social media text analysis has a well-established history. Previous research has confirmed the existence of statistically significant lexical and semantic patterns that distinguish social media posts associated with various mental health conditions [12]. It has also demonstrated the feasibility of machine learning approaches, supported by psychological theories, to enhance the understanding of cognitive styles and emotional expressions in identifying conditions such as depression, bipolar disorder, anorexia, self-harm and suicidal tendencies, attention deficit hyperactivity disorder (ADHD), and post-traumatic stress disorder (PTSD) [13–16]. A wide range of machine learning techniques and tools has been employed in the development of diagnostic systems for mental health analysis. These include classical methods such as *n*-gram analysis, Support Vector Machines (SVM), and Random Forests [17], as well as established tools like SentiStrength, the NRC Affect Intensity Lexicon [15], and Linguistic Inquiry and Word Count (LIWC) [18]. In the spectrum of modern computational methods, deep learning architectures play a dominant role, particularly Long Short-Term Memory (LSTM) networks, Convolutional Neural Networks (CNN), Extreme Learning Machines (ELM), and Transformers [16, 18, 19]. A summary of key studies is presented in Table 1. View this table: [Table 1.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/T1) Table 1. Overview of deep learning approaches in mental health classification Analysis of the presented data illustrates substantial progress in the application of AI to mental health tasks. Contemporary models often achieve accuracy exceeding 90% when classifying specific, isolated disorders. Furthermore, the integration of textual data with demographic information, clinical assessments [17], and visual modalities [21] demonstrates the potential to further enhance the predictive capability of these systems. However, performance tends to decline significantly in more complex classification scenarios. For instance, the classification of eight mental disorders [19] (major depressive disorder, generalized anxiety disorder, PTSD, obsessive-compulsive disorder, borderline personality disorder, schizophrenia, eating disorders, and bipolar disorder) based on a large Reddit corpus using a BERT model yielded *F*1-scores ranging from 0.57 to 0.73 (mean *F*1 = 0.645). These findings highlight the inherent difficulty of simultaneously identifying multiple diagnoses and the presence of significant inter-class confusion. In another study [16], a hybrid architecture combining pre-trained transformer models (RoBERTa, BERT, XLNet, ALBERT) with deep learning methods (CNN, LSTM, BiLSTM) was proposed for the multi-class classification of five mental disorders (ADHD, Anxiety, Bipolar Disorder, Depression, and PTSD) using Reddit posts. The best performance based on *F*1-score was by the Transformer model with late fusion, reaching an *F*1-score of 0.8965 and outperforming the baseline RoBERTa model (*F*1 = 0.8441). Despite the strong performance, this study also reported notable inter-class confusion, particularly among Anxiety, Depression, and PTSD. Despite these summarized achievements, several critical limitations persist in the field. These include the lack of publicly available datasets suitable for modeling comorbidity and the continued dominance of traditional multi-class classification approaches. When the presented study was conducted, the only open resource enabling multi-label classification of five different disorders (including anxiety, depression, and personality disorders) is the “Text Classification” dataset [22, 23]. Consequently, this dataset was selected as the empirical basis for the present work. The persisting challenge for studies of mental disorders is the scarcity of data for external validation. In particular, the only available resource, the “Depression Reddit Cleaned Dataset” dataset [24], is limited exclusively to data regarding depression, allowing for only partial verification of the proposed models. The absence of published benchmarks for these new datasets precludes a direct comparison of results. Consequently, the validation of the proposed method’s efficacy is performed by benchmarking against statistical metrics reported in the review of relevant scientific literature. The main fundamental issue with existing approaches, which is addressed in this work, is their focus on detecting a single disorder, which contradicts the clinical nature of mental illnesses where comorbidity is often inherent. Traditional multi-class models demonstrate limited effectiveness in identifying co-occurring conditions [25]. Therefore, a transition to a multi-label classification paradigm is a necessary step to adequately reflect the complexity of mental disorders. Particular attention must be paid to the problem of defining the optimal classification threshold to ensure a balance between sensitivity and specificity, which is a critical parameter in medical diagnostics. In this work, instead of empirically selecting thresholds, the presented approach proposes a mathematically grounded method for their optimization. In summary, the proposed approach “shifts away” from the concept of a single diagnosis in favor of detecting a spectrum of conditions. The main contributions of this work are as follows: * **Multi-label Classification Architecture:** A Binary Relevance strategy based on an ensemble of fine-tuned DistilBERT models is applied. This allows for the independent modeling of each disorder, effectively identifying linguistic markers of comorbidity. * **Decision Threshold Optimization:** To minimize classification errors between phenotypically similar disorders, Youden’s J statistic is integrated, ensuring an optimal balance of sensitivity and specificity. * **Sampling Strategy:** A method for creating balanced training subsets for each classifier in the ensemble was developed and applied, mitigating the issue of class imbalance. * **Loss Function Analysis:** A comparative assessment of the efficacy of Focal Loss, Dice Loss, and Log Loss was conducted, resulting in formulated recommendations for training models on imbalanced medical texts. ## Materials and methods ### Background DistilBERT is an optimized version of the BERT model [26], created through knowledge distillation, which reduces its size and speeds up its operation while preserving the main properties of the original architecture. In particular, the training process uses BERT as a teacher model, ensuring consistency between the output probabilities [27]. More specifically, DistilBERT undergoes pre-training using masked language modeling (MLM) approach, when some of the tokens in the input text are hidden and the model has to restore them. This approach contributes to the formation of context-dependent representations of words and allows for a more generalized understanding of language structure [28]. From the modeling perspective, one of the key objectives of optimization is to maintain the semantic properties of the hidden states produced by the model [29]. To achieve this a loss function based on cosine proximity is employed, which guarantees a similarity between the vector representations of DistilBERT and its original counterpart. This approach facilitates the development of a compact and efficient model that remains effective across various natural language processing tasks, including text classification. This fine-tuning of DistilBERT is an essential step in adapting the model to classify mental disorders based on the textual content produced by users of social media. For that purpose, the model is trained on a specific selected dataset to better generalize information in the relevant subject area. In the end, the fine-tuning approach for each model involves adding a classification layer to the DistilBERT output layer, allowing the model to determine the probability of one of the studied classes of mental disorders in the text. To quantify the model performance, loss functions are utilized to evaluate the calibration of DistilBERT models as well as the model performance on training and test datasets. More specifically, the loss function for the DistilBERT model is set to its default when using the Trainer class from the Hugging Face library i.e. the model automatically uses the cross-entropy loss function for binary classification tasks, where the goal is to minimize the difference between actual labels and model predictions. Tokenization, i.e., the process of converting raw text into a format suitable for the model in the form of tokens, in DistilBERT is performed using the WordPiece tokenizer used in the original BERT model [30]. The tokenization process includes the following steps: i) raw text pre-processing (i.e. normalization, lowercase conversion, removal of extra spaces), ii) breaking the text into sub-tokens according to the WordPiece dictionary, and iii) adding special tokens necessary for the model to function correctly. During the tokenization process each token is assigned a unique identifier (ID) from the corresponding dictionary and the tokenizer also reduces the text to a specified maximum length (128 tokens) to ensure a unified input data representation. The other essential element of the performed tokenization is the attention mask i.e. marking tokens as important (1) and filler tokens (0), which tells the model which tokens to consider during processing and which to ignore. From the perspective of mental disorders, the model’s task is the accurate classification of the corresponding disorders. At the same time, there exist three types of potential classifications: single-label (i.e. binary and multi-class) and multi-label. Among the first two, binary classification assigns an object to one of the two mutually exclusive classes, while multi-class classification assumes that there are more than two classes, but the classes are still mutually exclusive, i.e., an object can only be assigned to one of the classes at a time [31]. These two types of classification are often referred to as single-label classifications (SLC) because only one class label is used when training the classifier. Multi-label classification (MLC) assigns an object to several classes simultaneously, which are not mutually exclusive. At the same time, MLC can be treated as a special case of SLC. To illustrate that the general classification problem should be considered. More specifically, let *X* be a set of *n* objects to be classified where *X* = {*x*1, *x*2, …, *x**n*} and *Y* be a set of *m* class labels where *Y* = {*y*1, *y*2, …, *y**m*}. There is an unknown mapping of the set of objects to the set of classes (classifier): *f* *∗* : *X → Y*, whose pairs of values are already known on the finite training sample of size *k*: (*X, Y*)*k* = {(*x*1, *y*(1)), (*x*2, *y*(2)), …, (*x**k*, *y*(*k*))} where *y*(*i*) is the known label from *Y* which corresponds to the object *x**i* *∈ X* for *i ∈* 1, 2, …, *k*. The goal is to (re)construct this classifier *f* *∗* : *X → Y* which assigns *y**i* *Y* for each object on *x**i* *∈ X* for *i ∈* 1, 2, …, *k*. If *m* = 2, then the classification is binary, and if *m >* 2, then it is multi-class. The difference for MLC is that the training set will contain several class label fields and an ensemble of classifiers is required: ![Graphic][1], whose dimension is equal to the number of possible class labels (*m*). There are two main approaches to building MLC classification models: data transformation and algorithm adaptation. The first involves transforming the training dataset, while the second involves adapting conventional machine learning algorithms to create an MLC classifier. The known data transformation methods include binary relevance (BR), classifier chains (CC) and label-powerset (LP). The BR method transforms the MLC problem into several binary classification tasks, each focused on a single label. For each potential class that an object may belong to, a distinct field is created in the training set, where a value of 1 indicates membership in that class, and 0 signifies non-membership. The BR method, however, has two notable drawbacks. The first drawback is the requirement to train multiple classifiers, which can lead to significant computational costs. The second drawback is its failure to consider the interdependence between labels. An alternative approach CC can be used to overcome this label dependence issue of the BR method. The CC technique constructs binary classifiers in a sequential chain, whereby the inputs for each classifier consist of the original objects and the outputs from the preceding classifier in the chain which accounts for interdependence. The LP technique treats every possible combination of labels as a single label in a multi-class problem. The choice of the metrics for evaluating the accuracy of MLC is crucial. In particular, the selected metrics will vary between conventional classifiers, which predict a single class label, and MLC classifiers, which produce a vector of labels as output. The following specific metrics are often employed to assess the accuracy of MLC classifiers: Hamming loss, which is based on the Hamming distance and defined as the relative frequency of incorrect classifications as well as 0/1-loss, which represents the proportion of training examples for which at least one label has been misclassified. It’s important to note that the Hamming loss metric is typically applied to the BR method while the 0/1-loss metric suits the CC and LP methods. The Hamming Loss metric, however, is not applicable for this study due to the nature of the datasets utilized in the analysis, which are labeled according to SLC tasks. This annotation structure allows for the potential presence of multiple diagnoses for each patient, but only guarantees the presence of a specific disorder associated with the corresponding label. Consequently, using metrics aimed at label completeness, such as Hamming Loss, is unjustified, since the labeling process does not account for the multiplicity of possible labels. The use of different loss functions such as focal loss, dice loss, and log loss in the presented study is justified by the need to ensure optimal model training in conditions of class imbalance, fuzzy class boundaries, and different sensitivity to classification errors. In particular, the focal loss is a modification of the log loss that inhibits the influence of easily classified examples and focuses training on complex samples [32]. This is crucial in tasks where classes have different expressiveness or are unevenly represented, as in this study. The dice loss function is used in segmentation tasks, but it is effective in classification when the number of examples of the target class is limited [33], since it directly optimizes the *F*1- score, which makes it relevant when the balance between precision and recall is important, rather than just prediction accuracy. The log loss (cross-entropy) function remains the standard in binary classification tasks, providing stable convergence and a single benchmark for comparing the performance of alternative loss functions [34]. In summary, the incorporation of all three functions allows evaluating the adaptability of models to different training scenarios and selecting the approaches that best suit the characteristics of the models. In addition to the three discussed losses, it is also useful to evaluate the binary classifiers underlying the BR method using metrics specific to binary classification, which include accuracy – the proportion of correctly classified examples out of the total number; precision – the proportion of correct positive predictions among all predicted positive cases; recall – the proportion of correct positive predictions among all actual positive cases (completeness); as well as the *F*1-score – the harmonic mean of precision and recall [35]. The other summaries which were used to evaluate the classifier’s ability to extract hidden dependencies from the training dataset include the visual summaries, such as Receiver Operating Characteristic (ROC) curves and the area under the curve (AUC). In particular, the above indicators are calculated for visual summaries based on the error matrix parameters: True Positive (TP) – the number of correctly predicted positive cases; True Negative (TN) – the number of correctly predicted negative cases; False Positive (FP) – the number of incorrectly predicted positive cases; False Negative (FN) – the number of incorrectly predicted negative instances. Overall, finding an appropriate balance between sensitivity and specificity is particularly relevant for binary classification, where reductions of the FP and FN proportions are necessary. This is especially relevant for medical and psychological research, where correct identification is critical. In this context, the determination of the balance based on the Youden index [36] helps minimize the risk of erroneously assigning text to the mental-disorder category or, conversely, failing to detect it. ### Proposed approach and algorithms The following notations are used in the method formulations: * *T* – the input text for analysis. Note that the analysis focuses on user-generated short content from social networks, and therefore, there is a constraint on text length: the number of tokens must not exceed 128. * *f*1 – WordPiece tokenizer [37], which implements the mapping *f*1: ![Formula][2] where *tkn* is the tokenised text *T* in the form of a set of tokens; * *D* – a set of mental disorders identified by *tkn*: ![Formula][3] The number *N**d* denotes the number of mental disorders considered; *id**Dso* is the identifier of a mental disorder *Dso* (with *id**Dso* = 0 indicating the absence of any disorder); and *Name**Dso* represents the name of the corresponding mental disorder. * *DS* – a dataset with texts are already marked according to their affiliation with a specific mental disorder *d**i* for *i* = 1, 2, …, *N**d*, which will be used to create datasets for fine-tuning deep learning models: ![Formula][4] The number *N**t* is the number of samples in the affiliation *DS*. * *Df* – a set of datasets for fine-tuning deep learning models for binary classification; each dataset from *Df* contains a set of labeled tokens *tkn* for mapping (1) a specific mental disorder *Cls* = *id**Dso* or other disorders *Cls* = 0; *Df* is obtained by mapping *f*2: ![Formula][5] ![Formula][6] In this notation, *K* denotes the number of samples in the dataset *Df**i*. Each dataset *Df**i* is constructed for binary classification, where the target class consists of samples related to a specific disorder *d**i* *∈ D*, and the non-target class includes samples associated with other disorders or with no disorder. To avoid confusion between mental disorders, and considering that a single text may contain manifestations of multiple disorders from the set *D*, the non-target category is formed according to specific rules [38]: (a) the number of samples in the non-target group is equal to, or approximately equal to, the number of samples in the target group, (b) the non-target group consists of an equal proportion of the remaining samples from *DS* that either represent other mental disorders or show no signs of mental disorders. * *M* – a set of deep learning models obtained using the mapping *f*3, which implements fine-tuning of *DLM* using *Df* : ![Formula][7] where *DLM* is the DistilBert deep learning model [39] and *M* is a set of finely tuned deep learning models for each mental disorder *d**i* *∈ D*: ![Formula][8] where *mod* is a finely tuned deep learning model *DLM* using the dataset *Df**i* *∈ Df* for a specific mental disorder *d**i* *∈ D*; * *τ* – is a threshold for determining mental disorder *d**i*, which is determined by the Youden index – the value of the point on the ROC curve that corresponds to the best ratio of sensitivity and specificity [40]; * *R* – the result of applying *m**i* *∈ M, i* = 1, 2, …, *N**d* to an arbitrary tokenized text *tkn*: ![Formula][9] where *val* is the numerical value of the influence of disorder *d**i* on tokenized text *tkn* with the range of values *r**i* *∈* [0,1] which indicates the severity of mental disorder *d**i* *∈ D* [41]. The value of *val* is the output value of the last layer of the model *mod* *i* which is normalized by the *softmax* activation function; More precisely, *R* is obtained through the mapping *f*4: ![Formula][10] where *R* contains numerical values representing the severity of each disorder *d**i* *∈ D*. Note that the presence or absence of a mental disorder is determined by comparing the resulting numerical value *val* with the threshold value *τ*. To provide a complete workflow, Figure 1 presents a diagram of the data processing sequence used to obtain the sets *Df* and *M*. In Figure 1, the first step employs Algorithm 1 to generate a set of datasets *Df* (via the mapping *f*2), which are used for fine-tuning a set of deep learning models *M*. ![Fig 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F1.medium.gif) [Fig 1.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F1) Fig 1. Data processing scheme for obtaining the set of datasets *Df*, which are used to generate deep learning models *M*. Algorithm 1 Implementation of the *f*2 ![Figure2](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F2.medium.gif) [Figure2](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F2) At Step 2 (see Fig. 1), Algorithm 2 is used to fine-tune the set of deep learning models *M* via the mapping *f*3. Algorithm 2 Fine-tuning models *M* ![Figure3](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F3.medium.gif) [Figure3](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F3) An example of constructing the training dataset *Df*1, targeting the mental disorder group *d*1, is shown in Figure 2. ![Fig 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F4.medium.gif) [Fig 2.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F4) Fig 2. Example of forming dataset *Df*1 for disorder *d*1 (*id**DsO* = 1). Figure 3 presents a data processing diagram for determining the presence or absence of mental disorders. ![Fig 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F5.medium.gif) [Fig 3.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F5) Fig 3. Data processing diagram for determining the presence or absence of a specific mental disorder. At step 1 (see Fig. 3), the tokenized text *tkn* is obtained from the input text *T* using the mapping *f*1. At step 2 (see Fig. 3), Algorithm 3 is used to apply the model *M* to the tokenized text *tkn*, resulting in a decision *R* regarding the presence or absence of mental disorders (mapping *f*4). Algorithm 3 Obtaining solution *R* (mapping *f*4) ![Figure6](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F6.medium.gif) [Figure6](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F6) ### Datasets To implement and test the proposed approach, two publicly available datasets from Kaggle were utilized: “Text Classification” [22], [23], and “Depression: Reddit Dataset (Cleaned)” [24]. The study focused on the following five mental disorders: Anger/Intermittent Explosive Disorder (*d*1), Anxiety Disorder (*d*2), Depression (*d*3), Narcissistic Disorder (*d*4), and Panic Disorder (*d*5). Samples representing these specific disorders were extracted from the “Text Classification” dataset, while samples exhibiting no disorder manifestations (*d*) were sourced from the “Depression: Reddit Dataset (Cleaned)”. The “Text Classification” dataset consists of 740 tweets referencing potential mental disorders. The data is distributed as follows: 208 samples for *d*3 (Depression), 158 for *d*4 (Narcissistic Disorder), 154 for *d*1 (Anger), 153 for *d*2 (Anxiety), and 112 for *d*5 (Panic Disorder). The “Depression: Reddit Dataset (Cleaned)” dataset is a refined collection of 7,650 Reddit posts related to depression. Of these, 3,900 samples show no signs of depression or other mental health issues. This dataset is released under the CC0: Public Domain license, which permits unrestricted public use. Both datasets contain secondary data which are freely and publicly available, requiring no prior approval. They do not include any personally identifiable or private information. Ethical approval for this research was obtained from the review board of the Faculty of Information Technologies at Khmelnytskyi National University. The research exclusively focused on diagnostic efficacy based on text, with no inclusion of clinical metadata (e.g., age, gender, comorbidities) to ensure confidentiality. ### Experimental Setup and Evaluation This section details the experimental design, model architecture, training protocol, and evaluation metrics used to validate the proposed approach. To demonstrate the effectiveness of our approach, we conducted a series of experiments with two primary objectives: 1. To evaluate the capacity of the proposed models to identify textual patterns related to mental disorders and accurately classify those mental disorders based on those patterns. 2. To compare the performance of the proposed approach against other established methods in the field. The core of our approach is a set of models, *M*, based on the DistilBERT architecture. We chose DistilBERT because standard BERT models can be prone to overfitting on specific words or markers, which may lead to biased conclusions, especially in emotionally charged texts. DistilBERT, with its reduced depth and training via distillation, offers more stable generalization, which is crucial for analyzing texts from diverse individuals. The datasets utilized in this study present certain limitations, including moderate sample sizes, class imbalance, and potential stylistic and contextual biases inherent to social media data. To mitigate these constraints, we employed a fine-tuning strategy in conjunction with cross-validation. We adopted a 4-fold cross-validation scheme, as research [42] has demonstrated this to be a well-established and robust practice for similar classification tasks. The training and evaluation protocol for each model *m ∈ M* on its corresponding dataset *D**f* was as follows: 1. *Data Splitting* : The dataset *Df* was randomly split into training and test sets in the proportion of 80% for training and 20% for test, respectively. 2. *Model Fine-Tuning* : The pre-trained DistilBERT model m was fine-tuned on the training set using the following hyperparameters: a batch size of 4, 4 training epochs, and a learning rate of 2e-5. These hyperparameters were selected based on established best practices for transformer-based models on similarly sized datasets. During this process, the fine-tuning loss functions (e.g., focal loss, dice loss, log loss) were monitored. 3. *Evaluation*: The performance of the fine-tuned model was evaluated on both the training and test sets. For the test set, we generated a confusion matrix, calculated performance metrics, and plotted the ROC curve. 4. *Repetition for Robustness*: To ensure the stability and reliability of the results, Steps 1-3 were repeated four times using different random splits of the data. 5. *Result Aggregation*: The final performance was reported as the average of the metrics obtained across all four runs. We also calculated the maximum deviation from the mean to estimate the variance of the results for the *F*1-Score metric. Since the model relies on multi-label classification and accurate mental health diagnosis is critical for human well-being, a comprehensive set of metrics was used to evaluate its performance. These metrics included: * Classification Metrics: Accuracy, Precision, Recall, and *F*1-Score. * Diagnostic Tools: Confusion Matrices and Receiver Operating Characteristic (ROC) curves. * Loss Functions: Focal Loss, Dice Loss, and Log Loss were used during training to handle class imbalance and guide the optimization process. Additionally, the Youden’s *J* Index was employed to determine the optimal threshold for classification, minimizing the risk of both false positives and false negatives, which is crucial in a clinical context. The experiments were implemented in an IPython Notebook environment. For training the five individual disorder models, CPU execution environments were used, while a TPU v2-8 accelerator was utilized for the multi-class classifiers. The complete source code has been made publicly available on GitHub to ensure reproducibility: [https://github.com/oovcharuk/MentalHealthTextClassifier](https://github.com/oovcharuk/MentalHealthTextClassifier). Per our methodology, Algorithm 1 was used to preprocess and structure the corresponding dataset *D**f* for each model in *M*. ## Results After applying Algorithm 1, the following data distribution was obtained in the datasets: *Df*1 contains 154 entries for “Anger/Intermittent Explosive Disorder” and 150 entries for the category “non-target”; *Df*2 includes 153 entries for “Anxiety Disorder” and 150 entries for “non-target”; *Df*3 contains 208 entries for “Depression” and 205 entries for “non-target”; *Df*4 includes 158 entries for “Narcissistic Disorder” and 155 entries for “non-target”; *Df*5 consists of 112 entries for “Panic Disorder” and 110 entries for “non-target”. After completing step 1 (Experiment for evaluating *M* models), the following distributions of training and test data were obtained: for *d*1, the training set contained 243 samples (80%) and the test set 61 (20%); for *d*2, training: 242 (80%), test: 61 (20%); for *d*3, training: 330 (80%), test: 83 (20%); for *d*4, training: 250 (80%), test: 63 (20%); for *d*5, training: 177 (80%), test: 45 (20%). At step 2, a set of models *M* was experimentally trained for each of the 5 types of psychological disorders. In step 3, the loss function values were obtained for each model. In step 4, the aforementioned procedures were replicated using four random 80/20 splits. The rationale for selecting four splits is detailed in the “Experimental Setup and Evaluation” section. In step 5, the average values of all metrics were obtained, and the maximum deviation from the average value was calculated for the *F*1-score metric. Table 2 presents the mean values for Focal Loss, Dice Loss, and Log Loss across both the training and test datasets. The integration of these loss functions allows the model to simultaneously address class imbalance, optimize the *F*1-score, and ensure training stability. Consequently, the values reported in Table 2 provide a representative assessment of the model’s behavior under various scenarios. View this table: [Table 2.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/T2) Table 2. The value of loss functions for binary models with *M* Table 3 shows the average values of the statistical metrics Accuracy, Precision, Recall, and *F*1-score for the training and test data. The Delta *F*1 column contains the maximum deviation from the average value for the *F*1-score metric. View this table: [Table 3.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/T3) Table 3. Metrics of binary models with *M* on training and test data The ROC curves and confusion matrices for each disorder are presented below. It is important to note that these visualizations depict a single, representative experimental run, whereas the metrics provided in Table 3 are averaged across four data splits. For “Anxiety Disorder” (*d*2), the optimal threshold – determined via Youden’s index as described in the “Materials and methods” section – is 0.7401; the corresponding confusion matrix and ROC curve are shown in Figure 4. ![Fig 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F7.medium.gif) [Fig 4.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F7) Fig 4. Anxiety disorder class diagrams: A: ROC curve for “Anxiety disorder” classification. B: Confusion matrix on test set. ![Fig 5.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F8.medium.gif) [Fig 5.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F8) Fig 5. Anger/Intermittent explosive disorder class diagrams: A: ROC curve for “Anger/Intermittent explosive disorder” classification. B: Confusion matrix on test set. The optimal threshold for “Depression” (*d*3) is 0.501. The confusion matrix and ROC curve are shown in Figure 6. ![Fig 6.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F9.medium.gif) [Fig 6.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F9) Fig 6. Depression class diagrams: A: ROC curve for “Depression” classification. B: Confusion matrix on test set. For “Panic Disorder” (*d*5), the optimal threshold is 0.6094. The confusion matrix and ROC curve are shown in Figure 7. ![Fig 7.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F10.medium.gif) [Fig 7.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F10) Fig 7. Panic Disorder class diagrams: A: ROC curve for “Panic Disorder” classification. B: Confusion matrix on test set. For “Narcissistic Disorder” (*d*4), the optimal threshold is 0.8978. The confusion matrix and ROC curve are shown in Figure 8. ![Fig 8.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2025/12/18/2025.12.16.25342390/F11.medium.gif) [Fig 8.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/F11) Fig 8. Narcissistic Disorder class diagrams: A: ROC curve for “Narcissistic Disorder” classification. B: Confusion matrix on test set. Table 4 contextualizes the performance of the proposed ensemble framework against benchmarks reported in recent state-of-the-art studies. While a direct side-by-side comparison is precluded by the unavailability of specific external datasets used in [16] and [19], this overview highlights the efficacy of the proposed binary relevance strategy relative to traditional multi-class architectures employed in similar mental health monitoring tasks. View this table: [Table 4.](http://medrxiv.org/content/early/2025/12/18/2025.12.16.25342390/T4) Table 4. Performance overview of the proposed method versus reported baselines ## Discussion In this study, we presented a transformer-based ensemble framework for the multi-label detection of mental health disorders. The proposed approach achieved a high overall *F*1-score of 0.930, with AUC values consistently exceeding 0.89 across all modeled conditions. These results suggest that decomposing the multi-class problem into independent binary relevance tasks, combined with threshold optimization via Youden’s *J* statistic, effectively captures the linguistic nuances of co-occurring mental health conditions. ### Impact of Loss Functions The choice of loss function proved critical for model performance. Our experiments indicate that Focal Loss is particularly effective for hard-to-classify or minority classes, such as “Panic Disorder”, as it penalizes difficult examples more heavily than standard Cross-Entropy. Conversely, Dice Loss demonstrated strong generalization capabilities for classes with ambiguous boundaries, such as “Anxiety Disorder”. The combination of these functions allowed the ensemble to mitigate the effects of class imbalance inherent in social media datasets, yielding stable convergence where “standard” Log Loss underperformed. ### Linguistic Interpretation and Error Analysis A qualitative analysis of misclassifications reveals that while the models exhibit high sensitivity, they are prone to specific linguistic pitfalls that vary by disorder: * **Contextual Polarity Shifting (Anxiety):** False negative errors in the “Anxiety Disorder” model often stemmed from polarity shifts induced by negations and modals. Statements such as *“I am paralyzed by fear and cannot make decisions”* or *“I am worried that I cannot cope with my anxiety”* were misclassified because the negation markers (“cannot”, “not”) locally inverted the sentiment intensity in the embedding space. Conversely, false positives were triggered by “panic spectrum” vocabulary in non-target contexts (e.g., *“I isolate myself so as not to cause panic”*), where the combination of worry and avoidance amplified the predictive signal despite the lack of clinical intent. * **Metaphorical Ambiguity (Anger/IED):** The “Anger” model showed a slight tendency toward over-prediction driven by metaphorical descriptions. Non-target instances like *“my body feels like a pressure cooker, ready to explode”* contain aggregated keywords (“pressure”, “explode”) that semantically align with explosive anger, generating false positives. On the other hand, euphemistic phrasing (e.g., *“Trying to find ways to manage emotions without them managing me”*) attenuated the signal, leading to false negatives. * **Colloquial vs. Clinical Usage (Depression):** While the “Depression” model achieved near-perfect sensitivity (0 False Negatives in the representative run), specificity was impacted by the colloquial use of depressive terminology. False positives were concentrated in statements where keywords like “sad” or “depressed” referred to external events or third parties (e.g., *“fadyanwar is sad because it was the last GSM company”* or references to celebrities like *“David Archuleta”*). In these cases, the model acted on superficial triggers without capturing the lack of personal symptomatic distress. * **Dataset Constraints (Narcissistic Disorder):** The perfect classification of “Narcissistic Disorder” (0 FP, 0 FN) at a high threshold (*τ* =0.8978) likely indicates distinct linguistic markers in this subset, but may also suggest limited variability in the training data, posing a risk of overfitting to specific keywords. ### Comparison with Benchmarks Our binary ensemble approach compares favorably with existing benchmarks. Our mean *F*1-score (0.930) surpasses the results reported in related studies using standard multi-class BERT architectures (*F*1*≈*0.65–0.73) [19] and hybrid deep learning models (*F*1*≈*0.89) [16]. Unlike traditional multi-class classifiers that force a single label per text, our multi-label framework acknowledges the comorbidity of mental disorders, providing a more realistic modeling of user states. The comparison suggests that shifting from a multi-class paradigm to a set of independent binary tasks significantly reduces inter-class confusion, leading to higher *F*1-scores on targeted datasets. ### Limitations and Future Work Several limitations must be acknowledged to contextualize these findings. First, the relatively small training corpus raises concerns about potential overfitting, particularly for classes with high performance like Narcissistic Disorder. Second, the ground truth labels are derived from user-generated content and self-reports rather than clinical diagnoses. Consequently, the model detects discourse patterns associated with disorders, rather than the disorders themselves. Finally, the available data lacks demographic metadata (such as age, gender, or location), which prevents the assessment of fairness or potential bias across different population groups. The deployment of AI in mental health requires strict ethical oversight. The identified potential for false positives, driven by colloquialisms and metaphors, underscores that such systems should function solely as decision support tools for professionals, not as autonomous diagnostic agents. Future research will focus on expanding the dataset to include more diverse sources, integrating explainer algorithms (e.g., SHAP) to improve interpretability, and refining the handling of negations and irony to reduce linguistic misclassifications. ## Conclusion This study presented a novel multi-label classification framework for mental health monitoring, integrating an ensemble of fine-tuned DistilBERT models with a calibrated binary relevance strategy. By decomposing the diagnostic task into independent detection problems, the proposed approach had a mean *F*1-score of 0.930. These results indicate that replacing traditional multi-class architectures with specialized binary classifiers significantly reduces confusion between clinically overlapping conditions, such as depression and anxiety, thereby offering a more effective method for modeling comorbidity in social media text. Beyond quantitative metrics, this research provided a critical qualitative analysis of the linguistic boundaries of transformer-based models. We have identified that while the proposed approach has high sensitivity, specificity has been challenged by contextual polarity shifting (e.g., negations), metaphorical ambiguity, and the colloquial usage of clinical terms. These findings highlight that it can still be challenging for high-performing neural networks to capture all the semantic nuances required for precise diagnosis, which highlights the existing gap between statistical correlation and language specifics. In summary, while the proposed framework demonstrates significant potential as a screening and monitoring tool, it must be deployed and used with strict human oversight to mitigate the risks of false positives driven by linguistic artifacts. Potential future research directions will focus on enhancing model interpretability through explainer algorithms, expanding datasets to include diverse demographic and cross-cultural samples, and developing architectures capable of better resolving syntactic dependencies to minimize errors caused by negation, irony, and colloquial language. ## Data Availability All data presented in this paper are contained in the manuscript and in references provided in the manuscript. [https://www.kaggle.com/datasets/comsys/text-classification](https://www.kaggle.com/datasets/comsys/text-classification) [https://www.kaggle.com/datasets/infamouscoder/depression-reddit-cleaned](https://www.kaggle.com/datasets/infamouscoder/depression-reddit-cleaned) [https://github.com/oovcharuk/MentalHealthTextClassifier](https://github.com/oovcharuk/MentalHealthTextClassifier) ## Author contributions **Conceptualization:** Olexander Barmak, Iurii Krak, Pavel Skums, Sergiy Yakovlev. **Data curation:** Oleksandr Ovcharuk, Maryna Molchanova. **Formal analysis:** Oleksandr Ovcharuk, Maryna Molchanova. **Project administration:** Olexander Barmak, Sergiy Yakovlev. **Investigation:** Oleksandr Ovcharuk, Olexander Mazurets. **Methodology:** Oleksandr Ovcharuk, Olexander Mazurets. **Software:** Oleksandr Ovcharuk, Maryna Molchanova. **Supervision:** Olexander Mazurets. **Validation:** Olexander Mazurets, Alexander Kirpich, Pavel Skums. **Writing – original draft:** Oleksandr Ovcharuk, Maryna Molchanova. **Writing – review & editing:** Olexander Mazurets, Alexander Kirpich, Pavel Skums, Olena Sobko, Sergiy Yakovlev. * Received December 16, 2025. * Revision received December 16, 2025. * Accepted December 18, 2025. * © 2025, Posted by openRxiv This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/) ## References 1. 1.Pathak Y, Makk-Frid E. Population Estimates of Self-Reported Depression and Anxiety in the US From a National Survey: Cross-Sectional Survey Study. Interactive Journal of Medical Research. 2025;14(1):e70626. doi:10.2196/70626. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/70626&link_type=DOI) 2. 2.Tarren-Sweeney M. Do adolescents in care systematically under-report their mental health difficulties in population studies? A narrative review. Developmental Child Welfare. 2019;1(3):251–72. doi:10.1177/2516103219829. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/2516103219829&link_type=DOI) 3. 3.Ma S, Kang L, Guo X, Liu H, Yao L, Bai H, et al. Discrepancies between self-rated depression and observed depression severity: The effects of personality and dysfunctional attitudes. General Hospital Psychiatry. 2021;70:25–30. doi:10.1016/j.genhosppsych.2020.11.016. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.genhosppsych.2020.11.016&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33689981&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2025%2F12%2F18%2F2025.12.16.25342390.atom) 4. 4.Fekadu A, Demissie M, Birhane R, Medhin G, Bitew T, Hailemariam M, et al. Under detection of depression in primary care settings in low and middle-income countries: a systematic review and meta-analysis. Systematic Reviews. 2022;11(1):21. doi:10.1186/s13643-022-01893-9. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s13643-022-01893-9&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35123556&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2025%2F12%2F18%2F2025.12.16.25342390.atom) 5. 5.Underhill R, Foulkes L. Self-diagnosis of mental disorders: A qualitative study of attitudes on Reddit. Qualitative Health Research. 2025;35(7):779–92. doi:10.1177/10497323241288785. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/10497323241288785&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=39422576&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2025%2F12%2F18%2F2025.12.16.25342390.atom) 6. 6.Montejo-Raez A, Molina-Gonzalez MD, Jimenez-Zafra SM, Garcia-Cumbreras MA, Garcia-Lopez LJ. A survey on detecting mental disorders with natural language processing: Literature review, trends and challenges. Computer Science Review. 2024;53:100654. doi:10.1016/j.cosrev.2024.100654. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.cosrev.2024.100654&link_type=DOI) 7. 7.Shetty NP, Singh Y, Hegde V, Cenitta D K D. Exploring emotional patterns in social media through NLP models to unravel mental health insights. Healthcare Technology Letters. 2025;12(1):e12096. doi:10.1049/htl2.12096. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1049/htl2.12096&link_type=DOI) 8. 8.Olawade DB, Wada OZ, Odetayo A, David-Olawade AC, Asaolu F, Eberhardt J. Enhancing Mental Health with Artificial Intelligence: Current Trends and Future Prospects. Journal of Medicine, Surgery, and Public Health. 2024;3:100099. doi:10.1016/j.glmedi.2024.100099. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.glmedi.2024.100099&link_type=DOI) 9. 9.Singh J, Sharma D. Automated detection of mental disorders using physiological signals and machine learning: A systematic review and scientometric analysis. Multimedia Tools and Applications. 2024;83:73329–61. doi:10.1007/s11042-023-17504-1. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11042-023-17504-1&link_type=DOI) 10. 10.Radwan A, Amarneh M, Alawneh H, Ashqar HI, AlSobeh A, Magableh AAAR. Predictive Analytics in Mental Health Leveraging LLM Embeddings and Machine Learning Models for Social Media Analysis. International Journal of Web Services Research. 2024;21(1):1–22. doi:10.4018/ijwsr.338222. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.4018/ijwsr.338222&link_type=DOI) 11. 11.Scherbakov D, Hubig N, Lenert LA, Alekseyenko AV, Obeid JS. Natural Language Processing and Social Determinants of Health in Mental Health Research: An Artificial Intelligence-Assisted Scoping Review (Preprint). JMIR Mental Health. 2024;12(1):e67192. doi:10.2196/67192. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/67192&link_type=DOI) 12. 12.Kim S, Cha J, Kim D, Park E. Understanding mental health issues in different subdomains in social networking services: computational analysis of text-based reddit posts (Preprint). Journal of Medical Internet Research. 2023;25:e49074. doi:10.2196/49074. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/49074&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=38032730&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2025%2F12%2F18%2F2025.12.16.25342390.atom) 13. 13.Uban AS, Chulvi B, Rosso P. An emotion and cognitive based analysis of mental health disorders from social media data. Future Generation Computer Systems. 2021;124:480–94. doi:10.1016/j.future.2021.05.032. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.future.2021.05.032&link_type=DOI) 14. 14.Dhelim S, Chen L, Das SK, Ning H, Nugent C, Leavey G, et al. Detecting Mental Distresses Using Social Behavior Analysis in the Context of COVID-19: A Survey. ACM Computing Surveys. 2023;55(14s):1–30. doi:10.1145/3589784. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1145/3561048&link_type=DOI) 15. 15.Sarsam SM, Al-Samarraie H, Alzahrani AI, Alnumay W, Smith AP. A lexicon-based approach to detecting suicide-related messages on Twitter. Biomedical Signal Processing and Control. 2021;65:102355. doi:10.1016/j.bspc.2020.102355. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.bspc.2020.102355&link_type=DOI) 16. 16.Arif M, Ameer I, Bölücü N, Sidorov G, Gelbukh A, Elangovan V. Mental Illness Classification on Social Media Texts Using Deep Learning and Transfer Learning. Computación y Sistemas. 2024;28(2):451–64. Available from: [https://www.cys.cic.ipn.mx/ojs/index.php/CyS/article/view/4873](https://www.cys.cic.ipn.mx/ojs/index.php/CyS/article/view/4873). xdoi:10.13053/cys-28-2-4873. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.13053/cys-28-2-4873&link_type=DOI) 17. 17.Madububambachu U, Ukpebor A, Ihezue U. Machine Learning Techniques to Predict Mental Health Diagnoses: A Systematic Literature Review. Clinical Practice & Epidemiology in Mental Health. 2024;20(1):e17450179315688. doi:10.2174/0117450179315688240607052117. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2174/0117450179315688240607052117&link_type=DOI) 18. 18.Schoene AM, Turner A, De Mel GR, Dethlefs N. Hierarchical Multiscale Recurrent Neural Networks for Detecting Suicide Notes. IEEE Transactions on Affective Computing. 2021;1:153–64. doi:10.1109/taffc.2021.3057105. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1109/taffc.2021.3057105&link_type=DOI) 19. 19.Jiang Z, Levitan SI, Zomick J, Hirschberg J. Detection of Mental Health from Reddit via Deep Contextualized Representations. In: Proceedings of the 11th International Workshop on Health Text Mining and Information Analysis, Online. Stroudsburg, PA, USA: Association for Computational Linguistics; 2020. p. 147–56. doi:10.18653/v1/2020.louhi-1.16. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.18653/v1/2020.louhi-1.16&link_type=DOI) 20. 20.Tadesse MM, Lin H, Xu B, Yang L. Detection of Suicide Ideation in Social Media Forums Using Deep Learning. Algorithms. 2019;13(1):7. doi:10.3390/a13010007. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/a13010007&link_type=DOI) 21. 21.Kamatala S, Jonnalagadda AK, Naayini P. Transformers beyond nlp: Expanding horizons in machine learning. SSRN Electronic Journal. 2025;8(7). doi:10.2139/ssrn.5112305. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2139/ssrn.5112305&link_type=DOI) 22. 22.Comsys. Kaggle – Text Classification Dataset [Internet]; [cited 2025 Sep 6]. Available from: [https://www.kaggle.com/datasets/comsys/text-classification](https://www.kaggle.com/datasets/comsys/text-classification). 23. 23.1. Kole DK, 2. Roy Chowdhury S, 3. Basu S, 4. Plewczynski D, 5. Bhattacharjee D Paul A, Mittal O, Ghosh S, Dasgupta S, Bhattacharjee D, Sarkar R. COMSYS Hackathon-1 2023: Igniting Machine Learning Marvels. In: Kole DK, Roy Chowdhury S, Basu S, Plewczynski D, Bhattacharjee D, editors. Proceedings of 4th International Conference on Frontiers in Computing and Systems. COMSYS 2023. vol. 974 of Lecture Notes in Networks and Systems. Springer, Singapore; 2024. p. 29. doi:10.1007/978-981-97-2611-029. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/978-981-97-2611-029&link_type=DOI) 24. 24.Dataset DRC. Kaggle – Datasets for data science projects [Internet]; [cited 2025 Sep 6]. Available from: [https://www.kaggle.com/datasets/infamouscoder/depression-reddit-cleaned](https://www.kaggle.com/datasets/infamouscoder/depression-reddit-cleaned). 25. 25.Karamat A, Imran M, Yaseen MU, Bukhsh R, Aslam S, Ashraf N. A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction using Social Media Text. IEEE Access. 2025;13:12148–67. doi:10.1109/access.2024.3519308. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1109/access.2024.3519308&link_type=DOI) 26. 26.Zhou C, Li Q, Li C, Yu J, Liu Y, Wang G, et al. A comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT. International Journal of Machine Learning and Cybernetics. 2024:1–65. doi:10.1007/s13042-024-02443-6. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s13042-024-02443-6&link_type=DOI) 27. 27.Albladi A, Islam M, Das A, Bigonah M, Zhang Z, Jamshidi F, et al. Hate Speech Detection using Large Language Models: A Comprehensive Review. IEEE Access. 2025;13:20871–92. doi:10.1109/access.2025.3532397. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1109/access.2025.3532397&link_type=DOI) 28. 28.Distilbert/distilbert-base-uncased. Hugging Face – The AI community building the future; 2025. Available from: [https://huggingface.co/distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) x[cited 2025 Sep 6]. 29. 29.Zalutska O, Molchanova M, Sobko O, Mazurets O, Pasichnyk O, Barmak O, et al. Method for Sentiment Analysis of Ukrainian-Language Reviews in E-Commerce Using RoBERTa Neural Network. In: CEUR Workshop Proceedings. vol. 3387; 2023. p. 344–56. Accessed 2025-05-17. Available from: [https://ceur-ws.org/Vol-3387/paper26.pdf](https://ceur-ws.org/Vol-3387/paper26.pdf). 30. 30.DistilBERT. Hugging Face – The AI community building the future [Internet]; [cited 2025 Sep 6]. Available from: [https://huggingface.co/docs/transformers/model](https://huggingface.co/docs/transformers/model) doc/distilbert. accessed 2025-05-17. 31. 31.Krak I, Didur V, Molchanova M, Mazurets O, Sobko O, Zalutska O, et al. Method for Political Propaganda Detection in Internet Content Using Recurrent Neural Network Models Ensemble. In: CEUR Workshop Proceedings. vol. 3806; 2024. p. 312–24. Accessed 2025-05-17. Available from: [https://ceur-ws.org/Vol-3806/S36Krak.pdf](https://ceur-ws.org/Vol-3806/S36Krak.pdf). 32. 32.El Koshiry AM, Eliwa EHI, Abd El-Hafeez T, Khairy M. Detecting cyberbullying using deep learning techniques: a pre-trained glove and focal loss technique. PeerJ Computer Science. 2024;10(e1961). doi:10.7717/peerj-cs.1961. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7717/peerj-cs.1961&link_type=DOI) 33. 33.Liu B, Dolz J, Galdran A, Kobbi R, Ben Ayed I. Do we really need dice? The hidden region-size biases of segmentation losses. Medical Image Analysis. 2023;91:103015. doi:10.1016/j.media.2023.103015. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.media.2023.103015&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=37918314&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2025%2F12%2F18%2F2025.12.16.25342390.atom) 34. 34.Mushava J, Murray M. Flexible loss functions for binary classification in Gradient-Boosted Decision Trees: an application to credit scoring. Expert Systems with Applications. 2023;238:121876. doi:10.1016/j.eswa.2023.121876. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.eswa.2023.121876&link_type=DOI) 35. 35.Rainio O, Teuho J, Klén R. Evaluation metrics and statistical tests for machine learning. Scientific Reports. 2024;14(1):6086. doi:10.1038/s41598-024-56706-x. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41598-024-56706-x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=38480847&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2025%2F12%2F18%2F2025.12.16.25342390.atom) 36. 36.Liu S, Tian Q, Liu Y, Li P. Joint Statistical Inference for the Area under the ROC Curve and Youden Index under a Density Ratio Model. Mathematics. 2024;12(13):2118. doi:10.3390/math12132118. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/math12132118&link_type=DOI) 37. 37.Schönle D, Reich C, Abdeslam DO. Linguistic-Aware WordPiece Tokenization: Semantic Enrichment and OOV Mitigation. In: 2024 6th International Conference on Natural Language Processing (ICNLP). IEEE; 2024. p. 134–42. doi:10.1109/icnlp60986.2024.10692355. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1109/icnlp60986.2024.10692355&link_type=DOI) 38. 38.Sobko O, Mazurets O, Molchanova M, Krak I, Barmak O. Method for Analysis and Formation of Representative Text Datasets. In: CEUR Workshop Proceedings. vol. 3899; 2025. p. 84–98. Accessed 2025-05-17. Available from: [https://ceur-ws.org/Vol-3899/paper9.pdf](https://ceur-ws.org/Vol-3899/paper9.pdf). 39. 39.Nabiilah GZ. Effectiveness Analysis of RoBERTa and DistilBERT in Emotion Classification Task on Social Media Text Data. Engineering, Mathematics and Computer Science Journal (EMACS). 2025;7(1):45–50. doi:10.21512/emacsjournal.v7i1.12618. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.21512/emacsjournal.v7i1.12618&link_type=DOI) 40. 40.Wang J, Yin J, Tian L. Evaluating joint confidence region of hypervolume under ROC manifold and generalized Youden index. Statistics in Medicine. 2023;43(5):869–89. doi:10.1002/sim.9998. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.9998&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=38115806&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2025%2F12%2F18%2F2025.12.16.25342390.atom) 41. 41.Krak I, Zalutska O, Molchanova M, Mazurets O, Bahrii R, Sobko O, et al. Abusive Speech Detection Method for Ukrainian Language Used Recurrent Neural Network. In: CEUR Workshop Proceedings. vol. 3; 2024. p. 16–28. Accessed 2025-05-17. Available from: [https://ceur-ws.org/Vol-3688/paper2.pdf](https://ceur-ws.org/Vol-3688/paper2.pdf). 42. 42.Heidari P, Milan A. Combining K-fold cross validation with bayesian hyperparameter optimization for accuracy enhancement of land cover and land use classification. Scientific Reports. 2025;15(1):39758. doi:10.1038/s41598-025-23336-w. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41598-025-23336-w&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=41233377&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2025%2F12%2F18%2F2025.12.16.25342390.atom) [1]: /embed/inline-graphic-1.gif [2]: /embed/graphic-2.gif [3]: /embed/graphic-3.gif [4]: /embed/graphic-4.gif [5]: /embed/graphic-5.gif [6]: /embed/graphic-6.gif [7]: /embed/graphic-7.gif [8]: /embed/graphic-8.gif [9]: /embed/graphic-9.gif [10]: /embed/graphic-10.gif