Culture and understanding the role of feedback for health professions students: Realist synthesis protocol

Complexity of Feedback

Early definitions of feedback emphasised information giving to change behaviour.[3–5] In a widely quoted paper, Ende[3] described feedback as information given to trainees about a particular activity which was meant to guide performance of that or a similar activity in the future. The emphasis was that feedback was something supervisors directed at trainees, preferably after observation of the activity in question. University students commonly complain that they do not receive enough feedback, or that it is done poorly, such that academic staff are advised to “signpost” when feedback was being given.[6] Ajjawi and Regehr[7] suggest that perhaps learners and teachers define feedback quite differently.

Over time, feedback has been understood as more than simply providing information – information is only feedback when it is used to improve work or learning and is part of a sociocultural interaction. Furthermore, feedback value is influenced by the credibility of the feedback source.[8–10] As it is essential to close the feedback loop, we can think of feedback as sense-making in the context of information provided from many sources to improve work and learning.[11] The importance of relationships and trust between the supervisor and trainee, especially when the feedback relates to assessment, has been emphasised.[12] Many factors influence the effect of feedback including context (e.g. the workplace – hospital or ambulatory settings, teaching a skill, formative assessment, summative assessment)[13,14] regulatory focus,[15] and self-efficacy.[16,17] A person’s “theory of intelligence” (their understanding of whether intelligence is “fixed” or “improvable”) will also impact – if a person’s belief is that intelligence is fixed, effort may not seem to be worthwhile, whereas if they feel there is opportunity for improvement, effort becomes worthwhile.[18,19]

Feedback within the clinical learning environment is particularly complex and influenced by such things as the workload of providing patient care, hierarchies, time constraints and limited opportunities to observe a student’s performance, the student’s expectations and engagement with feedback provided,[8,16,20] as well as the supervisor’s experience of feedback during their training and therefore understanding of feedback. These complexities will be recognised by clinical teachers in Western environments, but we suspect are magnified within the Asian setting.[21]

Complexity of Culture

When we consider culture in the context of health professional education there are three prominent and interdependent cultures – the “big-Culture”, the Workplace culture and the Education culture.

Culture (sometimes referred to as “big-Culture”) in this context is defined by Hofstede as “The collective programming of the mind that distinguishes one group or category of people from another … culture is (a) a collective, not individual, attribute; (b) not directly visible but manifested in behaviours; and (c) common to some but not all people.”. [22](p58)

While there are several classifications of characteristics of Culture, Hofstede’s typology is widely used and can help in our understanding of the issues. Initially four dimensions were described,[23,24] with two further dimensions added later:[22,25]

1. Individualism – Collectivism

2. Power Distance

3. Uncertainty Avoidance

4. Masculinity – Femininity.

5. Long-term versus Short-term Orientation

6. Indulgence versus Restraint

Of these dimensions, Individualism-Collectivism and Power Distance appear to be the most significant big-Culture influences in the clinical learning environment in South-East Asia.[21]

All authors have experience teaching in health professions education in Asia and Australia in a University with campuses in both Malaysia and Australia. This experience raised the question whether cultural differences may influence our students’ learning. A preliminary literature review of feedback within health professional education showed a heavy North American and European focus, and few studies from a South East Asian perspective, except notably from Indonesia.[21,26] This heightened our questioning of whether cultural factors influence acceptance and engagement with “dialogic feedback” in an Asian context, and how does it compare with the “Western” situation? (While some Western studies have included international students / trainees, we contend that students who reached the clinical phase of training have had time and opportunity to adapt to their host country.) In focussing on the Asian region, we can recognise several broad Cultural groups – the “Confucian Heritage Culture”, Indonesian-Malaysian / Muslim cultures, and cultures of the Indian subcontinent, overlaid with the cultural impacts of colonialism. Given the limited literature found in our preliminary search focussing on Southeast Asia, we decided to look further afield to include the Middle East (the influence of Muslim learning culture) through to the “Far East” (the “Confucian Heritage Culture”). If the cultural background of students / trainees and their teachers / supervisors influences their engagement with feedback, how does it do so? Is it the “ethnic Culture”, the national cultures influenced by their colonial history (e.g. for Malaysia, Singapore, Indonesia)[27], or is it the education system culture (e.g. school education, university, or even discipline cultures) that have the predominant effect – or is there no dominant effect?

Workplace culture can be viewed as another cultural layer, particularly in clinical teaching and hospital environments[28] and potentially interacts with the “big-Culture”. Medicine around the world tends to be hierarchical and paternalistic. However, within the Asian region teaching by humiliation is common, and in Malaysia the term “scolding” is commonly used to describe teaching in the clinical environment. Another term often heard in the region is kiasu – particularly applied to students of Chinese ethnicity. According to the Oxford Dictionary,[29] kiasu refers to a person who is “governed by self-interest, typically manifesting as a selfish, grasping attitude arising from a fear of missing out on something.”. Kiasu (怕输) is a Hokkien word meaning “fear of loss”. There are two aspects to kiasu, the negative side of being selfish and grasping, as seen in the Oxford dictionary definition, but there is the positive aspect of being successful through hard work – not evident in that dictionary definition.[30] (Kiasu is related to the concept of “face”, which western stereotypes frequently regard as a characteristic of “Asian Culture”).

The education system clearly has its overarching culture which influenced the students’ experiences of school – a system that emphasises regular high-stakes examinations from an early stage of schooling. There has been much written about the influence of the “Confucian Heritage Culture” of learning (CHC) on students from South East Asia as well as from China, Japan and Korea. In his writings Confucius saw learning as a means of social change and to overcome social differences, but also placed much emphasis on personal effort.[31] The Chinese philosophy of education also highlighted a mutually respectful relationship between teacher and learner, with the teacher guiding the learner, rather than pulling the learner along.[32] This parallels the role of guru seen in the Indian culture of education – with the guru (teacher) nurturing the learner.[33,34]

Malaysia and Indonesia are predominantly Islamic countries; Malaysia is a former British colony, while Indonesia was formerly colonised by the Dutch. Clearly both these aspects of their history have shaped the education system of the respective country and are as important factors as the Confucian heritage. There is a great diversity in Islamic education which impacts the southeast Asian region and interacted with the colonial experiences.[35] The school culture clearly responds to the education system’s overarching culture but adds its own layer. An examination-oriented curriculum was seen as a legacy of the colonial era.[36]

Tertiary education culture varies enormously across the region, from hierarchical approaches to being more collegial (especially in the later stages of the degrees). Student experiences in high school impact their transition to university as they come with an expectation that university would simply be an extension of school – first year medical students in Malaysia clearly started university with the idea that knowledge was fixed and largely unchanging, and that their teachers or lecturers functioned as sources of knowledge who were not to be questioned. Knowledge was facts, and facts were immutable.[37] As school had emphasised rote learning of fixed knowledge, and an important part of those students’ adaptation to university was coming to terms with thinking for themselves. As they move into workplace-based learning, the culture of the medical workplace is likely to have an impact.

Realist Synthesis methodology

The (scientific) Realist approach was chosen as a methodology that is useful for researching complex interventions in the social environment, such as healthcare and education – interventions that frequently work differently in different environments.[38] The more “traditional” methods used in reviews in Medicine and other Health Sciences (such as Systematic Reviews) were felt to not capture or explain the complexities of feedback in the social and cultural environment. The realist paradigm asks: “What is it about this intervention that works, for whom, in what circumstances, in what respects and how?”. [39,40] It seeks to find “mechanisms (M)” that fire in particular “contexts (C)” to produce the “Outcomes (O)” in question – so called CMO Configurations. Realist synthesis or realist review (the terms are used interchangeably) is a theory driven, iterative and explanation-building approach, that usually starts with an initial program theory and uses findings from sources to understand how and why the outcomes have occurred, and therefore refine the initial program theory.[38] Interpretation involves looking for both confirming and negating data and explanations.

This synthesis asks what leads can Culture provide, in the Asian health professional education environment, to answer?

1. How, why and in what circumstances do health professional trainees (e.g. students and junior doctors) seek, respond to, and use feedback given in the clinical environment?

2. What do supervisors (e.g. consultants, clinical tutors, preceptors) feel about providing feedback? How do they provide feedback, in what circumstances? Do they see their feedback being used?

3. How do trainees and supervisors perceive feedback?

The review will follow the five steps of a realist review as enunciated by Pawson and colleagues,[39] namely

1. Clarify the scope and purpose of the review question

2. Search for evidence – commencing with an exploratory search, with subsequent focussing and purposive and “snowball” sampling

3. Appraise studies and extract data

4. Synthesise the evidence to obtain conclusions, and …

5. Disseminate.

In appraising studies, Relevance is assessed by whether it can contribute to theory building or testing, while Rigor assessment is based on whether the methods which generated a particular piece of data is trustworthy.[41] Pawson argues that the overall methodological quality of a study is not appropriate grounds for excluding a study in realist reviews – “There are often nuggets of wisdom in methodologically weak studies”.[42]

A PRISMA-P checklist has been completed and available as an additional file.[43]

Search Strategy

An preliminary search was performed to define the scope of the review question and develop our candidate initial program theory (IPT). This first search utilised MEDLINE and PsycInfo, searching “Learner” (and variations), Feedback (and debrief) and Culture (including cross-cultural, ethnic differences, anthropology). In terms of a modified PICo model[44] developed for Qualitative studies (Population, Intervention, Context) format: P: Learners, I: Feedback, Co: Culture.

The formal electronic search was carried out in February 2020 and included: CINAHL, ERIC, MEDLINE, and PsycInfo. Search terms were developed in discussion with a librarian and the research team, with the same broad categories as before, although only articles available in English were retained. Articles to be considered include qualitative, quantitative and mixed methods, as well as commentaries and review articles. An example of the search strategy is given in Table 1 and provided in more detail as a supplementary file. Both MeSH (medical subject headings) and free text were employed to ensure sufficiently wide article coverage. This search was repeated in October 2020, for articles published since the February search. A hand search will also be made of the following journals: Academic Medicine, Medical Education, Medical Teacher, BMC Medical Education, Education for Health, Teaching and Learning in Medicine, Perspectives on Medical Education, Medical Journal of Malaysia, Annals of the Academy of Medicine Singapore, and Singapore Medical Journal. These were chosen as leading health professional education journals and Southeast Asian medical journals published in English. Citation mining (“snowball”) searches of the reference list in included articles and searching for articles that cite these articles will occur. Although dissertations were initially excluded, relevant published articles arising from the dissertations will be sought by hand-searching for author and a related title.

Selection criteria

Following the searches as outlined, the citations were imported into Covidence[45] for Title and Abstract Screening. Duplicates were removed before title and abstract screening began with two team members (PDF and MS) reviewing a sample of the articles retrieved to ensure that criteria are agreed upon. Approximately 10% of retrieved articles were reviewed jointly by the two team members. The rest of this phase was carried out by either of those team members (predominantly PDF) but with the intention to err on retaining studies for closer evaluation at the full text screening stage.

Core inclusion criteria sought studies relating to

• Workplace-based learning and assessment,

• Feedback giving, seeking and acceptance,

• Culture (Ethnic and institutional),

• Post-secondary and vocational education involving health professional training.

Exclusion criteria centre around:

• Not related to health professional education

• Not related to feedback or culture

• Community health education

Following title and abstract screening, the full texts of the retained articles were imported into Covidence for further screening. All full text articles will be screened by two members of the team, and any discrepancies will be discussed to resolve the disagreements. Notes will be made to justify inclusion or exclusion and will assist with both resolving discrepancies and providing transparency. Selecting papers for the review will be guided by the research study questions – Does the study involve students in health professional courses (especially in their clinical training) or their supervisors? Does the study pertain to students in Asian countries?

Studies remaining after the full text screen will be assessed for Quality and Rigour using the Critical Appraisal Skills Programme (CASP-Qualitative) checklist for Qualitative studies[46] and the Medical Education Research Study Quality Instrument (MERSQI) for quantitative studies.[47] Each study will be appraised by at least two team members – usually PDF and one other. Any discrepancies will be resolved by the full team.

Extracting Data

Data extraction will follow, with data entered into a table within Covidence 2.0. Data extracted will include citation details, country or region where study was performed, population studied, methodology used, and an empirical judgement of Realist Relevance will also be made at this stage. Comments of potential context, mechanism and outcome will be recorded. Data extracted in Covidence will be exported as a .csv file into Excel, to be used in the synthesis phase. Finally, included articles will be entered into NVivo software[48] for further data extraction, coding and identification of CMO configurations.

Synthesise findings to draw conclusions

NVivo software in conjunction with the Excel spreadsheet exported from Covidence will be used to synthesise findings and modify our initial program theory. The realist approach involves looking for causal mechanisms and how they fire in particular contexts to produce their outcome(s). Data will be extracted from the included papers by the team of reviewers, with a minimum of 10% of papers being double checked. The data will be tabulated and potential Contexts (C), Mechanisms (M) and Outcomes (O) identified. Discussion among the team members, with the use of realist logic, will aim to further refine those Contexts (C), Mechanisms (M) and find Outcomes patterns (O) to refine our initial program theory and infer CMO configurations. This process will look for confirming and contradictory findings and will be iterative.

Potential limitations of the realist synthesis approach

We accept that there are possible limitations of this proposed realist synthesis. Firstly, we have confined our search to trainees in clinical training for health profession disciplines, thus limiting the generalisability outside this sphere of education. Secondly, our initial interest was in the Southeast Asian region but due to paucity of literature from SE Asia we expanded the geographical scope. However, the is significant overlay of Cultures between the Middle East with the impact of Islam, through to the Far East with the influence of the Confucian Heritage Culture, as well as the influence of the various colonising powers. Thirdly, we have decided to assess rigour in our screening of articles, but recognise that there is a debate among realist scholars as to the validity of assessing rigour of a whole study – Pawson emphasises that “nuggets of wisdom” may be found in studies that are methodologically weak.[42]

Ethics and Dissemination

As this study is a literature review, ethics approval is not required.

The findings will be documented in line with the RAMESES publications standards for Realist syntheses,[41] and we plan to disseminate the findings by means of a peer-reviewed journal article and conference presentation(s).

Systematic Review Registration

The protocol for this review was judged to be ineligible for registration with the International Prospective Register for Systematic Reviews (PROSPERO), as it did not” have a direct and clinically-relevant health-related outcome”.