TY - JOUR T1 - A prospective evaluation of inter-rater agreement of routine medical records audits at a large general hospital in São Paulo, Brazil JF - medRxiv DO - 10.1101/2020.01.21.20018325 SP - 2020.01.21.20018325 AU - Ana Carolina Cintra Nunes Mafra AU - João Luiz Miraglia AU - Fernando Antonio Basile Colugnati AU - Gilberto Soares Lourenço Padilha AU - Renata Rafaella Santos Tadeucci AU - Ederson Almeida AU - Mario Maia Bracco Y1 - 2020/01/01 UR - http://medrxiv.org/content/early/2020/02/21/2020.01.21.20018325.abstract N2 - Background The quality of the patient’s medical records is strictly related to patient safety. Besides, its data are widely used in observational studies. However, the reliability of the information extracted from them is a matter of concern in audit processes to ensure inter-rater agreement (IRA). Thus, the objective of this study is to evaluate the IRA among members of the Patient’s Health Record Review Board (PHRRB), in routine auditing of medical records, and the impact of periodic discussions of results with raters.Methods Prospective longitudinal study conducted between July of 2015 and April of 2016 at Hospital Municipal Dr. Moysés Deutsch, a large public hospital in São Paulo. The PHRRB was composed of 12 physicians, 9 nurses and 3 physiotherapists, who audited medical records, monthly, with the number of raters changing throughout the study. It was carried out PHRRB meetings to reach a consensus on criteria that the members have to rate in the auditing process. It was created a review chart that raters should verify the registry of patient’s secondary diagnosis, chief complaint, history of presenting complaint, past medical history, medication history, physical exam and diagnostic testing. It was obtained the IRA every three months. The Gwet’s AC1 coefficient and Proportion of Agreement (PA) were calculated to evaluate the IRA for each item over time.Results The study included 1884 items from 239 records with an overall full agreement among raters of 71.2%. A significant IRA increase by 16.5% (OR=1.17; 95% CI=1.03—1.32; p=0.014) was found in the routine PHRRB auditing, with no significant differences between the PA and the Gwet’s AC1, that showed a similar evolution over time. The PA decreased by 27.1% when at least one of the raters was absent from the review meeting (OR=0.73; 95% CI=0.53—1.00; p=0.048).Conclusions Medical record quality has been associated with the quality of care and could be optimized and improved by targeted interventions. The PA and the Gwet’s AC1 are suitable agreement coefficients that are feasible to be incorporated in the routine of PHRRB evaluation process.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis work was funded by the Brazilian Ministry of Health and Sao Paulo State Research Foundation (FAPESP) through Research Program for the Unified Health System-PPSUS grant 2012/51228-9.Author DeclarationsAll relevant ethical guidelines have been followed; any necessary IRB and/or ethics committee approvals have been obtained and details of the IRB/oversight body are included in the manuscript.YesAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe dataset supporting the conclusions of this article is available to researchers who want to explore the data. To request, please send an email to ana.mafra{at}einstein.br.AC1agreement coefficientCIconfidence intervalGEEgeneralized estimating equationsHMMDHospital Municipal Dr. Moysés DeutschIRAinter-rater agreementIRRinter-rater reliabilityORodds ratioPAproportion of agreementPHRRBpatient’s health record review board ER -