Protocol for a realist review of the influence of cultural factors on understanding the role of feedback in developing clinical competencies of health professional students in Asia ==================================================================================================================================================================================== * Paul Fullerton * Wendy McKenzie * Mahbub Sarkar * Shamsul Haque ## Abstract **Background** Clinical education has moved to a “competency-based” model with an emphasis on workplace-based learning and assessment which, in turn, depends on feedback to be effective. Further, the understanding of feedback has changed from information about a performance directed to the learner performing the task, to a dialogue, which enables the learner to act and develop. In health professional education, feedback is a complex interaction between trainee, supervisor and the healthcare system. Most published research on feedback in health professional education originates in Europe and North America. Our interest is on the impact of culture on this process, particularly in the context of Asian cultures. A realist approach looks at complex interventions in social situations, and so would seem appropriate lens to use to examine the influence of cultural factors on utilising feedback. **Methods** An initial search has been performed to define the scope of the review question and develop our candidate / “best guess” program theory. The formal electronic search was carried out in February 2020 and included: CINAHL, ERIC, MEDLINE, and PsycInfo, and repeated in October 2020. Retrieved articles were imported into Covidence for screening and data extraction, after which components of the Context – Mechanisms – Outcomes configurations will be sought to refine the initial program theory. **Discussion** Feedback has been recognised as critically important in competency-based health professional education, yet feedback is a complex, socially based “intervention”. Most of the published literature on feedback originates from “Western” cultures. This protocol aims to provide further information that may lead to improving the usefulness of feedback in the South East Asian region. **Systematic Review Registration** Registration was sought with PROSPERO and advice given was that this review was not eligible for registration as it did not have a “direct and clinically-relevant health-related outcome”. **Strengths and limitations of this study** * A Realist approach potentially best explains the complexities of Culture’s impact on feedback. * To our knowledge, there are few studies of feedback seeking and provision to health professional trainees in Asia. * In addition to formal literature database searches, we will need to conduct citation mining to locate other relevant resources. * The typical assessment of “Risk of bias” does not apply to the Realist approach. Keywords * Realist review * Realist Synthesis * Health professional students * Medical students * Feedback * Asia ## BACKGROUND Clinical education has moved to a “competency-based” model with an emphasis on workplace-based learning and assessment which, in turn, depends on feedback to be effective(1,2). Indeed Ramani *et al* (2019) (p744) describe feedback as *“a vital cog in the wheel of competency-based medical education*.*”* Embarking on workplace learning will require understanding of the tools used, and acceptance of the feedback process, by both supervisors and trainees. ### Complexity of Feedback Early definitions of feedback emphasised information giving to change behaviour(3–5). In a widely quoted paper, Ende (1983) described feedback as ***information*** given to trainees about a particular activity which was meant to guide performance of that or a similar activity in the future. The emphasis was that feedback was something supervisors directed at trainees, preferably after observation of the activity in question. University students commonly complain that they do not receive enough feedback, or that it is done poorly, such that academic staff are advised to “signpost” when feedback was being given(6). This dissatisfaction is still experienced particularly in the lead-up to visits by course accreditation bodies. Ajjawi and Regehr (2018) suggest that perhaps learners and teachers define feedback quite differently(7). Over time, feedback has been understood as more than simply providing information – information is only feedback when it is ***used*** to improve work or learning and is part of a sociocultural interaction. Furthermore, feedback value is influenced by the credibility of the feedback source(8–10). As it is essential to close the feedback loop, we can think of feedback as sense-making in the context of information provided from many sources to improve work and learning(11). The importance of relationships and trust between the supervisor and trainee, especially when the feedback relates to assessment has been emphasised (12). Many factors are recognised to impact the effect of feedback including context (e.g. the workplace – hospital or ambulatory settings, teaching a skill, formative assessment, summative assessment) (13,14) regulatory focus(15), self-efficacy(16,17); the person’s “theory of intelligence” (their understanding of whether intelligence is “fixed” or “improvable”), and therefore whether effort is worthwhile(18,19). If a person’s belief is that intelligence is fixed, effort may not seem to be worthwhile, whereas if they feel there is opportunity for improvement, effort becomes worthwhile. This raises the question of whether cultural factors influence acceptance and engagement with “dialogic feedback” in a non-Western or Asian context, and how does it compare with the “Western” situation? An initial literature review of feedback within health professional education suggests that the literature is heavy with a North American and European focus, but is limited from South East Asia or Asia-Pacific perspective, except notably in Indonesia(20,21). As such, a number of questions remain unanswered. For example, which cultural background of students / trainees and their teachers / supervisors influences their engagement with feedback, and how does it do so? Is it the “ethnic culture” (e.g. Confucian heritage, Malay / Muslim, or Indian cultures, in the Malaysian context), the national culture (e.g. Malaysia, Singapore, Indonesia) (22), or the education system culture (e.g. school education, university, or even discipline cultures) that have the predominant effect – or is there no dominant effect? We will return to this later. Feedback within the clinical learning environment is particularly complex and influenced by such things as the workload of providing patient care, hierarchies, time constraints and limited opportunities to observe a student’s performance, the supervisor’s experience of feedback during their training and therefore understanding of feedback, as well as the student’s expectations and engagement with feedback provided. These complexities will be recognised by clinical teachers in Western environments but are potentially magnified within the Asian setting. ### Complexity of Culture When we consider culture in the context of health professional education there are three predominant and interdependent cultures – the “big-Culture”, the Workplace culture and the Education culture. A useful description of **C**ulture comes from Edward Hall, an early cultural anthropologist, in a book first published in 1959: > *“Culture is a mould in which we are all cast, and it controls our daily lives in many unsuspected ways. It is part of a person’s behaviour in which he takes for granted - the part he doesn’t think about, since he assumes it as universal or regards it as idiosyncratic…. Culture hides much more than it reveals and strangely enough, what it hides it hides most effectively from its own participants*.*”* (23) (p. 29) **C**ulture (sometimes referred to as “big-Culture”) in this context is defined by Hofstede as > *“The collective programming of the mind that distinguishes one group or category of people from another culture is (a) a collective, not individual, attribute; (b) not directly visible but manifested in behaviours; and (c) common to some but not all people*.*”* (24) (p58). While there are several classifications of characteristics of Culture, Hofstede’s typology is widely used and can help in our understanding of the issues. Initially there were four dimensions described(25): 1. Individualism – Collectivism 2. Power Distance 3. Uncertainty Avoidance 4. Masculinity – Femininity. Subsequently, two further dimensions were added(26): 5. Long versus Short-term Orientation 6. Indulgence versus Restraint Of these dimensions, Individualism-Collectivism and Power Distance appear to be the most significant big-Culture influences in the clinical learning environment in South-East Asia(21) In focussing on the Asian region, we can recognise several broad cultural groups – the “Confucian Heritage Culture”, Indonesian-Malay / Muslim cultures, the Indian subcontinent cultures, then overlaid by the cultural impacts of colonialism. Workplace culture can be viewed as another cultural layer that interacts with the “big-Culture”, particularly in clinical teaching and hospital environments. Medicine around the world tends to be hierarchical and paternalistic. However, within the Asian region teaching by humiliation is common, and in Malaysia the term “scolding” is commonly used to describe teaching in the clinical environment. Another term frequently heard is kiasu – particularly applied to students of Chinese ethnicity. According to the Oxford Dictionary(27), kiasu refers to a person who is *“governed by self-interest, typically manifesting as a selfish, grasping attitude arising from a fear of missing out on something*.*”. Kiasu* is a Hokkien word meaning “fear of loss”. There are two aspects to *kiasu*, the negative side of being selfish and grasping, as seen in the Oxford dictionary definition, and a positive aspect of being successful through hard work – not evident in that dictionary definition(28). (*Kiasu* is related to the concept of “face”, which western stereotypes frequently regard as a characteristic of Asian Culture). The education system clearly has an overarching culture which influenced the students’ experiences of school. Over the past decade, Singapore has been modifying the approach to assessment in pre-school and early primary school. Even at that early age high-stakes examinations have been a feature and there has been significant push-back against reducing that emphasis. The following quotes from teachers in an article reviewing the reception of the changes to early primary school education are informative(22): > *“Doing away with SA1/2 is not a good idea. We still need to maintain the competitive edge and to instil that habit of doing so in our society. Our young are already too pampered, and I do not agree to taking away the SAs to reduce stress. I’ll agree to more pathways to success but not taking away tests. Life is a test in itself*.*”* > > And … > > *“parents do not read the comments or look at the stars given. Instead, they flip the pages in the paper and start to calculate how many marks their child had scored. Imagine the amount of time and effort the teachers have to go through writing feedback, which is hardly read. This is especially so because their parents were educated and brought up to look for a mark to their examination paper. They get better feedback through these marks they see on their children’s script*.*”* There has been much written about the influence of the “Confucian Heritage Culture” of learning (CHC) on students from South East Asia as well as from China, Japan and Korea. In his writings Confucius saw learning as a means of social change and to overcome social differences, but also placed much emphasis on personal effort(29). Wang (2006) sees parallels in Confucius’ writings about education with Plato’s “philosopher king”. The Chinese philosophy of education also highlighted a mutually respectful relationship between teacher and learner, with the teacher guiding the learner, rather than pulling the learner along (30). This parallels the role of *guru* seen in the Indian culture of education – with the *guru* (teacher) nurturing the learner(31,32). Malaysia is a predominantly Islamic country, and a former British colony. Clearly both these aspects of its history have shaped the education system of the country and are as important factors as the Confucian heritage. In an editorial for a special edition of Comparative Education Review in 2006, Kadi highlights the great diversity in Islamic education and gives a perspective to the colonial experiences(33). The school culture clearly responds to the education system’s overarching culture but adds its own layer. An examination-oriented curriculum was seen as a legacy of the colonial era. > *“A great deal of teaching preparation at this time involved equipping students for the examination. An exam-oriented curriculum had been in place in the Malaysian education system since it was introduced during the colonial era by the British. Although the curriculum had been revised many times over a period of more than five decades, assessment had been an integral part not only in schools but also at tertiary level. Teachers realised that they needed to gear their students towards the exam as pressure from every corner awaits them; the Ministry of Education demands schools produce good, intelligent students; the local education department pressurises heads of schools to produce excellent results, who then order teachers to do their best to at least beat the results of neighbouring schools, not to mention being under pressure to meet the expectations of parents who want their children to do well under the guidance of the teachers*.*”* (34)(p59) Tertiary education culture varies enormously across the region, from hierarchical approaches to being more collegial (especially in the later stages of the degrees). The high school experiences clearly impact students transitioning to university, coming with an expectation that university would simply be an extension of school – first year medical students in Malaysia clearly started university with the idea that knowledge was fixed and largely unchanging, and that their teachers or lecturers functioned as sources of knowledge who were not to be questioned. Knowledge was facts, and facts were immutable(35). As school had emphasised rote learning of fixed knowledge, and an important part of those students’ adaptation to university was coming to terms with thinking for themselves. As they move into workplace-based learning, the culture of the medical workplace is likely to have an impact. ### Realist Synthesis methodology The (scientific) Realist approach is a methodology that is useful for researching complex interventions in the social environment, such as healthcare and education, and asks: *“What is it about this intervention that works, for whom, in what circumstances, in what respects and how?”* (36,37). It seeks to find “mechanisms (M)” that fire in particular “contexts (C)” to produce “Outcomes (O)” in question – so called CMO Configurations. Realist synthesis or realist review (the terms are used interchangeably) is a theory driven, iterative and explanation-building approach, that usually starts with a “best guess” or candidate program theory and uses findings from primary sources to understand how and why the outcomes have occurred, and therefore refine the initial program theory(38). Interpretation involves looking for both confirming and negating data and explanations. The specific research questions for this synthesis are as follows: In the Asian or non-Western health professional education environment: 1. How, why and in what circumstances do ***health professional trainees*** (e.g. students and junior doctors) seek, respond to, and use feedback given in the clinical environment? 2. ***Supervisors*** (e.g. consultants, clinical tutors, preceptors): What do supervisors feel about providing feedback? How do they provide feedback, in what circumstances? Do they see their feedback being used? 3. ***Both trainees and supervisors***: How do they perceive feedback? The review is following the five steps of a realist review as enunciated by Pawson and colleagues(36), namely 1. Clarify the scope and purpose of the review question 2. Search for evidence – commencing with an exploratory search, with subsequent focussing and purposive and “snowball” sampling 3. Appraise primary studies and extract data 4. Synthesise the evidence to obtain conclusions, and … 5. Disseminate. In appraising studies, ***Relevance*** is assessed by whether it can contribute to theory building or testing, while ***Rigor*** assessment is based on whether the methods which generated a *particular piece of data* is trustworthy(39). Pawson argues that the methodological quality of a study is not appropriate grounds for excluding a study in realist reviews – *“There are often nuggets of wisdom in methodologically weak studies”* (40). ## METHODS AND ANALYSIS The protocol for this review was judged to be ineligible for registration with the International Prospective Register for Systematic Reviews (PROSPERO), as it did not *“have a direct and clinically-relevant health-related outcome”*. A PRISMA-P checklist has been completed and available as an additional file(41). ### Search Strategy An initial search has been performed to define the scope of the review question and develop our candidate / “best guess” program theory. This initial search utilised MEDLINE and PsycInfo, searching “Trainee” (and variations), Feedback (and debrief) and Culture (including cross-cultural, ethnic differences, anthropology). The formal electronic search was carried out in February 2020 and included: CINAHL, ERIC, MEDLINE, and PsycInfo. Search terms were developed in discussion with a librarian and the research team, with the same broad categories as before, although only articles available in English were retained. An example of the search strategy is given in *Table 1*. Both MeSH (medical subject headings) and free text were employed to ensure sufficiently wide article coverage. This search was repeated in October 2020, for articles published since the February search. A hand search will also be made of the following journals: Academic Medicine, Medical Education, Medical Teacher, BMC Medical Education, Education for Health, Teaching and Learning in Medicine, Perspectives on Medical Education, Medical Journal of Malaysia, Annals of the Academy of Medicine Singapore, and Singapore Medical Journal. Citation mining (Snowball or Pearl growing) searches of the reference list in included articles and searching for articles that cite these articles will occur. Although dissertations were initially excluded, relevant published articles arising from the excluded dissertations will be sought by hand-searching. View this table: [Table 1:](http://medrxiv.org/content/early/2021/01/25/2021.01.24.21250413/T1) Table 1: Example of Search strategy used in Ovid MEDLINE ### Selection criteria Following the searches as outlined, the citations were imported into Covidence(42) for “Title and Abstract Screening”. Duplicates were removed before title and abstract screening began with two team members reviewing a sample of the articles retrieved to ensure that criteria are agreed upon. The rest of this phase was carried out by any one of the team members but with the intention to err on retaining studies for closer evaluation at the full text screening stage. Core inclusion criteria sought studies relating to * Workplace-based learning and assessment, * Feedback giving, seeking and acceptance, * Culture (Ethnic and institutional), * Post-secondary and vocational education involving health professional training. Exclusion criteria centre around: * Biological cultures – microbiologic, tissue culture * Workplace safety / Organisational culture / Organisational development * Feedback in physiology or therapeutic situations - e.g. Psychiatry * Describing and/or feedback on new teaching methods, including technology * Teaching cultural competency * Reviews: Literature reviews, Book reviews * Primary and secondary school, learners with a disability, Community health education * Clinical trials * Patient satisfaction Following title and abstract screening, the full texts of the retained articles were imported into Covidence for further screening. From here, all full text articles in the process of being screened by two members of the team, and any discrepancies will be discussed to resolve the disagreements. Notes will be made to justify inclusion or exclusion and will assist with both resolving discrepancies and providing transparency. Selecting papers for the review will be guided by the research study questions – Does the study involve students in health professional courses (especially in their clinical training) or their supervisors? Does the study pertain to students in Asian countries (noting the possible need to expand to include non-Western countries if insufficient studies pertain to Asian countries)? Therefore, exclusion criteria at this stage will be expanded to exclude studies not related to health professional education, and “Western studies”. Studies remaining after the full text screen will be assessed for Quality and Rigour using the Critical Appraisal Skills Programme (CASP-Qualitative) checklist(43) and the Medical Education Research Study Quality Instrument (MERSQI) (44). Each study will be appraised by at least two team members, and any discrepancies will be resolved by the full team. ### Extracting Data Data extraction will follow, with data entered into a table within Covidence 2.0. Data extracted will include citation details, country or region where study was performed, population studied, methodology used, and an empirical judgement of Realist Relevance will also be made at this stage. Comments of potential context, mechanism and outcome will be recorded. Data extracted in Covidence will be exported as a .csv file into Excel, to be used in the synthesis phase. Finally, included articles will be entered into NVivo software(45) for further data extraction, coding and identification of CMO configurations. ### Synthesise findings to draw conclusions NVivo software in conjunction with the Excel spreadsheet exported from Covidence will be used to synthesise findings and modify our original “best guess” program theory. Discussion among the team members with the use of realist logic will aim to identify Contexts (C), Mechanisms (M) and Outcomes (O) and infer CMO configurations. The process will look for confirming and contradictory findings and will be iterative. ### Ethics and Dissemination As this study is a literature review, ethics approval is not required. The findings will be documented in line with the RAMESES publications standards for Realist syntheses(39), and we plan to disseminate the findings by means of a peer-reviewed journal article and conference presentation(s). ## DISCUSSION Feedback has been recognised as critically important in competency-based health professional education, yet feedback is a complex, socially based “intervention”. Most of the published literature on feedback originates from “Western” cultures. There is reason to expect that components of Culture – “big-Culture”, workplace culture and education system cultures will impact the provision, acceptability and use of feedback. ***Again***, complex interactions come into play. The realist approach is a relevant way to examine these processes. This protocol aims to provide further information that may lead to improving the usefulness of feedback in the South East Asian region. ### Systematic Review Registration Registration was sought with PROSPERO and advice given was that this review was not eligible for registration as it did not have a “direct and clinically-relevant health-related outcome”. ## Data Availability Protocol - no data availability ## Author Contributions PF developed the original idea for this review protocol in collaboration with MS, WMcK and SH. PF wrote the original draft of the protocol, which was subsequently refined by MS, WMcK, SH, and PF. All authors agree to be accountable for all aspects of this protocol. PF is guarantor of the review. ## Competing interests All authors declare that they have no competing interests. ## Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. ## Ethics Not applicable. ## Patient and Public Involvement None – Literature review. ## Footnotes * E-mail: paul.fullerton{at}monash.edu * E-mail: wendy.mckenzie{at}monash.edu * E-mail: mahbub.sarkar{at}monash.edu * E-mail: shamsul{at}monash.edu * Received January 24, 2021. * Revision received January 24, 2021. * Accepted January 25, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. 1.Norcini J. The power of feedback. Med Educ. 2010 Jan;44(1):16–7. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-2923.2009.03542.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20078751&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 2. 2.Ramani S, Könings KD, Ginsburg S, van der Vleuten CPM. Feedback Redefined: Principles and Practice. J Gen Intern Med. 2019 May 19;34(5):744–9. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 3. 3.Ende J. Feedback in Clinical Medical Education. JAMA J Am Med Assoc. 1983 Aug 12;250(6):777. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.1983.03340060055026&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=6876333&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1983RC16100019&link_type=ISI) 4. 4.Butler DL, Winne PH. Feedback and Self-Regulated Learning: A Theoretical Synthesis. Rev Educ Res. 1995 Sep 30;65(3):245–81. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3102/00346543065003245&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1995RX65500003&link_type=ISI) 5. 5.Hattie J, Timperley H. The Power of Feedback. Rev Educ Res. 2007 Mar 16;77(1):81–112. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3102/003465430298487&link_type=DOI) 6. 6.Boud D, Molloy E. What is the problem with feedback? In: Feedback in Higher and Professional Education. 2012. p. 1–10. 7. 7.Ajjawi R, Regehr G. When I say … feedback. Med Educ. 2018;3–5. 8. 8.Watling CJ, Driessen E, Van der Vleuten CPM, Vanstone M, Lingard LA. Beyond individualism: professional culture and its influence on feedback. Med Educ. 2013 Jun;47(6):585–94. 9. 9.Watling CJ. Unfulfilled promise, untapped potential: Feedback at the crossroads. Med Teach. 2014 Aug 5;36(8):692–7. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24597625&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 10. 10.Wilbur K, BenSmail N, Ahkter S. Student feedback experiences in a cross-border medical education curriculum. Int J Med Educ. 2019 May 24;10:98–105. 11. 11.Carless D, Boud D. The development of student feedback literacy: enabling uptake of feedback. Assess Eval High Educ. 2018 Nov 17;43(8):1315–25. 12. 12.Carless D. Trust, distrust and their impact on assessment reform. Assess Eval High Educ. 2009 Feb;34(1):79–89. 13. 13.Tekian A, Watling CJ, Roberts TE, Steinert Y, Norcini J. Qualitative and quantitative feedback in the context of competency-based education. Med Teach. 2017 Dec 2;39(12):1245–9. 14. 14.Harrison CJ, Könings KD, Schuwirth L, Wass V, van der Vleuten C. Barriers to the uptake and use of feedback in the context of summative assessment. Adv Heal Sci Educ. 2015 Mar 7;20(1):229–45. 15. 15.Watling CJ, Driessen EW, van der Vleuten CPM, Vanstone M, Lingard LA. Understanding responses to feedback: the potential and limitations of regulatory focus theory. Med Educ. 2012 Jun;46(6):593–603. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22626051&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 16. 16.Ramani S, Könings KD, Mann K V., Pisarski EE, van der Vleuten CPM. About Politeness, Face, and Feedback: Exploring Resident and Faculty Perceptions of How Institutional Feedback Culture Influences Feedback Practices. Acad Med. 2018 Sep;93(9):1348–58. 17. 17.van de Ridder JMM, Peters CMM, Stokking KM, de Ru JA, ten Cate OTJ. Framing of feedback impacts student’s satisfaction, self-efficacy and performance. Adv Heal Sci Educ. 2015 Aug 27;20(3):803–16. 18. 18.Nussbaum AD, Dweck CS. Defensiveness Versus Remediation: Self-Theories and Modes of Self-Esteem Maintenance. Personal Soc Psychol Bull. 2008 May 5;34(5):599–612. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0146167207312960&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18276895&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000255167900002&link_type=ISI) 19. 19.Tweed RG, Lehman DR. Learning Considered within a Cultural Context: Confucian and Socratic Approaches. Am Psychol. 2002;57(2):89–99. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1037/0003-066X.57.2.89&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11899565&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000174256000001&link_type=ISI) 20. 20.Suhoyo Y, Van Hell EA, Kerdijk W, Emilia O, Schönrock-Adema J, Kuks JBM, et al. Influence of feedback characteristics on perceived learning value of feedback in clerkships: does culture matter? BMC Med Educ. 2017 Dec 5;17(1):69. 21. 21.Suhoyo Y, van Hell EA, Prihatiningsih TS, Kuks JBM, Cohen-Schotanus J. Exploring cultural differences in feedback processes and perceived instructiveness during clerkships: Replicating a Dutch study in Indonesia. Med Teach. 2014 Mar 2;36(3):223–9. 22. 22.Ratnam-Lim CTL, Tan KHK. Large-scale implementation of formative assessment practices in an examination-oriented culture. Assess Educ Princ Policy Pract. 2015 Jan 2;22(1):61–78. 23. 23.Hall ET. The silent language. New York, NY: Anchor Books, Doubleday; 1990. 1–209 p. 24. 24.Hofstede G, McCrae RR. Personality and Culture Revisited: Linking Traits and Dimensions of Culture. Cross-Cultural Res. 2004 Feb 25;38(1):52–88. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1069397103259443&link_type=DOI) 25. 25.Hofstede G. Cultural differences in teaching and learning. Int J Intercult Relations. 1986 Jan;10(3):301–20. 26. 26.Hofstede G, Hofstede GJ, Minkov M. Part II: Dimensions of national cultures. In: Cultures and Organizations: Software of the Mind. Third. Mc-Graw-Hill Education; 2010. p. 199. 27. 27.Oxford English Dictionary. ‘culture, n.’ [Internet]. OED Online. June 2014. 2014 [cited 2014 Aug 6]. Available from: [http://www.oed.com/view/Entry/45746?](http://www.oed.com/view/Entry/45746?) 28. 28.Hwang A, Ang S, Francesco AM. The silent Chinese: The influence of face and kiasuism on student feedback-seeking behaviors. J Manag Educ. 2002 Feb;26(1):70–98. 29. 29.Wang T. Understanding Chinese Culture and Learning. In: Australian Association for Research in Education. 2006. p. 1–14. 30. 30.Jin L, Cortazzi M. Changing Practices in Chinese Cultures of Learning. Lang Cult Curric. 2006;19(1):5–20. 31. 31.Marambe KN, Vermunt JD, Boshuizen HPA. A cross-cultural comparison of student learning patterns in higher education. High Educ. 2011 Dec 13;64(3):299–316. 32. 32.Crozet C. The core tenets of education in ancient India, inspirations for modern times. Int J Pedagog Learn. 2012;7(3):262–5. 33. 33.Kadi W. Education in Islam—Myths and Truths. Comp Educ Rev. 2006;50(3):311–24. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1086/504818&link_type=DOI) 34. 34.Idrus F. Initiating Culturally Responsive Teaching for Identity Construction in the Malaysian Classrooms. English Lang Teach. 2014 Mar 6;7(4):53–64. 35. 35.Fullerton PD. ‘That’s the way we learn’: Exploring influences of culture on medical student learning in Malaysia and Australia (MHPE Thesis). Monash University, Melbourne, Australia; 2014. 36. 36.Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review - a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005 Jul 4;10(1_suppl):21–34. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1258/1355819054308530&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16053581&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 37. 37.Greenhalgh T, Wong G, Westhorp G, Pawson R. Protocol - realist and meta-narrative evidence synthesis: Evolving Standards (RAMESES). BMC Med Res Methodol. 2011 Dec 16;11(1):115. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1471-2288-11-115&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21843376&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 38. 38.Wong G, Greenhalgh T, Westhorp G, Pawson R. Realist methods in medical education research: what are they and what can they contribute? Med Educ. 2012 Jan;46(1):89–96. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-2923.2011.04045.x&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22150200&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 39. 39.Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: Realist syntheses. J Adv Nurs. 2013;69(5):1005–22. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/jan.12095&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23356726&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 40. 40.Pawson R. Digging for nuggets: How ‘bad’ research can yield ‘good’ evidence. Int J Soc Res Methodol Theory Pract. 2006 Apr;9(2):127–42. 41. 41.Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015 Dec 1;4(1):1. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/2046-4053-4-1&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25554246&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 42. 42.Veritas Health Innovation. Covidence systematic review software. Melbourne, Australia: Veritas Health Innovation; 2020. 43. 43.Critical Appraisal Skills Programme. CASP Qualitative Checklist [Internet]. 2018 [cited 2020 Nov 22]. Available from: [http://www.casp-uk.net/casp-tools-checklists](http://www.casp-uk.net/casp-tools-checklists) 44. 44.Cook DA, Reed DA. Appraising the Quality of Medical Education Research Methods: The Medical Education Research Study Quality Instrument and the Newcastle-Ottawa Scale-Education. Acad Med. 2015;90(8):1067–76. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/ACM.0000000000000786&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26107881&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F25%2F2021.01.24.21250413.atom) 45. 45.QSR International. NVIVO Software. Melbourne Australia: QSR International Pty Ltd, Second Floor, 651 Doncaster Road, Doncaster 3108; 2020.