TY - JOUR T1 - Clinicians’ ratings of the Health of the Nation Outcome Scales (HoNOS) show sensitivity/bias that may affect patients’ progress: analyses of routine administrative data JF - medRxiv DO - 10.1101/2020.11.19.20234674 SP - 2020.11.19.20234674 AU - Jonathan Williams Y1 - 2020/01/01 UR - http://medrxiv.org/content/early/2020/11/20/2020.11.19.20234674.abstract N2 - Objectives (1) To estimate clinician sensitivity/bias in rating the HoNOS. (2) To test if high or low clinician sensitivity determines slower resolution of patients’ problems or earlier inpatient admission.Design The primary analysis used many-facet Item Response Theory to construct a multi-level Graded Response Model that teased apart clinician sensitivity/bias from the severity of patients’ problems in routine HoNOS records. Secondary analyses then tested if patients’ outcomes depend on their clinicians’ sensitivity/bias.Outcome measures The outcome measures were (1) overall differences in sensitivity/bias between (a) individual clinicians and (b) different Community Mental Health Teams (CMHTs); (2) clinical outcomes, comprising (a) the rate of resolution of patients’ problems and (b) the dependence of the time to inpatient admission on clinician sensitivity/bias.Setting All archival electronic HoNOS records for all new referrals to all CMHTs providing mental health services in secondary care in a New Zealand District Health Board during 2007-2015.Participants The initial sample comprised 2170 adults of working age who received 5459 HoNOS assessments from 186 clinicians. From these initial data, I derived an opportunistic, connected, bipartite, longitudinal network, in which (i) every patient received HoNOS ratings from 2 or more clinicians and (ii) every clinician assessed more than 5 patients. The bipartite network comprised 88 clinicians and 778 patients; 112 patients underwent later inpatient admission.Results Sensitivity/bias differed importantly between individual clinicians and CMHTs. Patients whose clinicians had more extreme sensitivity/bias showed slower resolution of their problems and earlier inpatient admission.Conclusions Raw HoNOS ratings reflect the sensitivity/bias of clinicians almost as much as the severity of patients’ problems. Additionally, low or high clinician sensitivity can adversely affect patients’ outcomes. Hence, the HoNOS’s main value may be to measure clinician sensitivity. Accounting for clinician sensitivity could enable the HoNOS to fulfil its goal of improving mental health services.Strengths and limitations of the studyThe study derived a connected network of clinicians and patients that approximates a rational design for estimating clinicians’ sensitivity/bias.The opportunistic network sample was atypical, with chronic patients and experienced clinicians – so the study may under-estimate clinician bias.The study’s statistical methods were appropriate to the ordinal nature of HoNOS ratings.The study used earlier estimates of clinician sensitivity/bias to predict later outcomes – so that effects of clinician sensitivity/bias on outcomes may be causalThe study assumed that all HoNOS items tap a single dimension of the severity of patients’ problems.Competing Interest StatementThe authors have declared no competing interest.Clinical TrialI did not pre-register the protocol for these analyses. However, they follow from previous work that I presented at the Royal Australia and New Zealand College of Psychiatrists Annual Congress and I outlined the idea for the present work to Dr Arran Culver at the New Zealand Ministry of Health in Wellington in February 2020.Funding StatementThis research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:Health and Disability Ethics Committees, Ministry of Health, 133 Molesworth Street, PO Box 5013 Wellington 6011, New ZealandAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesData will be freely available from the internet https://doi.org/10.7910/DVN/NSSZT2 ER -