A review of reviews exploring patient and public involvement in 4 population health research

22 Introduction : Patient and public involvement (PPI) is increasingly seen as something that is integral 23 to research and of importance to research funders. There is general recognition that PPI is the right 24 thing to do for both moral and practical reasons. The aim of this review of reviews is to examine how 25 PPI can be done ‘properly’ by looking at the evidence that exists from published reviews and 26 assessing it against the UK Standards for Public Involvement in Research, as well as examining the 27 specific features of population health research that can make PPI more challenging. 28 Methods : A review of reviews was carried out following the 5-stage Framework Synthesis method. 29 Results : In total 31 reviews were included. There is a lack of current research or clarity around 30 Governance and Impact when findings are mapped against UK Standards for Public Involvement in 31 Research. It was also clear that there is little knowledge around PPI with under-represented groups. 32 There are gaps in knowledge about how to ensure key specific attributes of population health research 33 are addressed for PPI team members – particularly around how to deal with complexity and the data- 34 driven nature of the research. Two tools were produced for researchers and PPI members to further 35 improve their PPI activity within population health research and health research more generally: A 36 framework of recommended actions to address PPI in population health research, and guidance on 37 integrating PPI based on the UK Standards for Public Involvement in Research. 38 Conclusions : Facilitating PPI in population health research is challenging due to the nature of this 39 type of research and there is far less evidence on how to do PPI well in this context. The tools can 40 help researchers identify key aspects of PPI that can be integrated when designing PPI within projects. 41 Findings also highlight specific areas where more research or discussion is needed. 42


Introduction 4 5 4 6
The focus of this review of reviews is on Public and Patient involvement (PPI) in population health 4 7 research. PPI is increasingly seen as something that is integral to research and of importance to 4 8 research funders. For our purposes, PPI is defined as: 4 • Mapping and interpretation (11) 2 0 8 Initial data extraction was carried out against a framework designed by the authors based on close 2 0 9 examination of background literature and initial review readings (Table 1) . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 16, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022

Study id
Problem Consequence Study id Solution Details dissemination and communication activities around the project so that they can eventually be more 2 6 1 accessible to a wider audience.

6 2
Duration was the only aspect that was not specifically discussed in the reviews and in finding 2 6 3 solutions. It is possible to postulate that building relationships and strong ways of working together 2 6 4 may help to address this issue. But also, that acknowledging upfront the changing requirements of a 2 6 5 long-term project will help researchers to plan accordingly -including planning for long term PPI. There are several aspects of PPI activity that are common across various types of health research, 2 6 8 including, but not exclusive to, population health research. Just over half of the reviews (18 out of 31 (14-31)) noted a range of potential challenges with PPI that 2 7 1 were reported to stand in the way of the successful development, implementation and evaluation of 2 7 2 patient and public involvement activity or models in health research in the UK and equivalent health 2 7 3 systems.

7 4
Consolidation of the challenges reported in the reviews suggested that the following (Table 3) were  2  7  5 the key issues. These have been grouped into appropriate headings. 2 7 6 2 7 7 Many of these challenges will be even more apparent in population health research where projects 2 8 0 tend to face the four challenges of: longer duration, involving more complex and varied processes, 2 8 1 alongside issues of big data, and finding appropriate representation to cover the project breadth and 2 8 2 length .  2  8  3   1   W  h  e  n  a  p  r  o  j  e  c  t  o  u  t  g  r  o  w  s  i  t  s  o  r  i  g  i  n  a  l  r  e  m  i  t  w  i  t  h  o  u  t  a  n  y  a  d  d  i  t  i  o  n  a  l  r  e  s  o  u  r  c  e  s  b  e  i  n  g  a  v  a  i  l  a  b  l  e  . . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint Solution: Offer public involvement opportunities that are accessible and that reach people and groups 2 9 0 according to research needs. Research also needs to be informed by a diversity of public experience 2 9 1 and insight, so that it leads to treatments and services which reflect these needs.

9 2
Eleven reviews mentioned inclusion (17-18, 20-21, 24, 29, 33-34, 36-38). Key themes emerged from 2 9 3 these studies and these are outlined in Table 4 below and explicitly addressed the problem area of 2 9 4 Representation.  Working Together 2 9 9 Solution: Work together in a way that values all contributions, and that builds and sustains mutually 3 0 0 respectful and productive relationships. Public involvement in research is better when people work 3 0 1 together towards a common purpose, and different perspectives are respected. 3 0 2 Twenty-one reviews (2, 16-18, 20-21, 23-29, 31-38) discussed aspects of this standard. The main 3 0 3 areas of discussion are outlined in Table 5 below and explicitly addresses the problem area of Conflict 3 0 4 and Control. 3 0 5  (24), In attitude and approaches to the project (25) . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 16, 2022. ; https://doi.org/10.1101/2022.11.16.22282319 doi: medRxiv preprint 3 0 7 Support and Learning 3 0 8 Solution: Offer and promote support and learning that builds confidence and skills for public 3 0 9 involvement in research. Seek to remove practical and social barriers that stop members of the public 3 1 0 and research professionals from making the most of public involvement in research.

1 1
The following seventeen reviews mentioned various aspects of support and learning (2, 16, 18, 21-22, 3 1 2 24-25, 27-29, 32-38). The results are shown in Table 6 below. The table is split into two sections to 3 1 3 reflect differences between support and learning methods, and explicitly addresses the problem area 3 1 4 of Knowledge. 3 1 5 Ensure support specific to topic area (29) and to their individual involvement (33).  (16), supporting participants to be informed and make informed decisions (25) and to understand specific parts of the research process and/or context (32) Research methods 22, 32, 37-38 Training in research components to give confidence in their involvement (32) and to explain 'rules' and constraints of . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint Public involvement in research governance can help research be more transparent and gain public 3 2 0

LEARNING -Attribute Study/Studies Examples of reasoning
trust. This section explicitly addresses the problem area of Conflict and Control. Only three of the 3 2 1 reviews mentioned governance (2, 24, 35). They discuss the need for shared decision making (at 3 2 2 every level), power and leadership, in order to lead to a culture of deeper involvement. As limited 3 2 3 suggestions were reported there is no table for this section. Solution: Use plain language for well-timed and relevant communications, as part of involvement 3 2 6 plans and activities. Communicate with a wider audience about public involvement and research, 3 2 7 using a broad range of approaches that are accessible and appealing. 3 2 8 Nine of the reviews discussed communication as being important to ensure PPI activity is successful 3 2 9 (2, 20, 24-25, 27, 32, 34-35, 37). Various attributes of good communication are discussed with the 3 3 0 main points as in Table 7 below, and explicitly addresses the problem area of Communications. 3 3 1  (24), ensures accountability (27) Ongoing/ regular updates 25, 32, 37 Contribute to motivation and engagement, and . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 16, 2022. ; https://doi.org/10.1101/2022.11.16.22282319 doi: medRxiv preprint to foster satisfying partnerships (32) Creating space to voice concern/ open communication climate 20,24,32 Contribute to motivation and engagement, and to foster satisfying partnerships (32) Avoid/ translate jargon 24-25, 32 Ensuring everyone understood and felt comfortable and confident to engage in meaningful dialogue (32) Use different materials (not just written reports etc) 32, 34, 37 Ensure people with different levels of literacy can participate (32) Sharing information, experiences and knowledge 2, 34 Across all groups involved (2) Clarifying and agree expectations upfront 24, 32 Could avoid conflicts, demotivation, dissolution of partnerships, or frustration in situations where stakeholders could perceive a lack of concrete actions (32), patients are 'partners' not 'are involved' (24) Have stakeholders lead groups 32 But be careful they include all groups in the discussion (32) Impact 3 3 3 Solution: Seek improvement by identifying and sharing the difference that public involvement makes 3 3 4 to research. Understand the changes, benefits and learning gained from the insights and experiences of 3 3 5 patients, carers and the public. Seven of the reviews discussed impact (2,20,24,32,(34)(35)38). The general theme was that impact 3 3 7 needs to be better evaluated throughout the whole research lifecycle. It was noted that this is an area 3 3 8 where the existing literature is scant and current working practices are perceived to be lacking in 3 3 9 terms of rigour.

4 0
Evaluating impact through continuous assessment and feedback was seen to be important in order to 3 4 1 ensure ongoing involvement, to identify best practice and areas for improvement, and to make sure 3 4 2 that the experience is working for everyone involved.

4 3
In addition to evaluating the process of PPI within health research, it was also noted that the impact of 3 4 4 findings that are translated to real world setting, and ideally the contribution of PPI activity to that 3 4 5 impact, should also be evaluated. 3 4 6 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 16, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 It is important to note that impact can be positive or negative and that impact may happen in a 3 4 7 complex way and to a range of areas, for example, impact on the research, on the research outcomes, 3 4 8 on the researchers, on the PPI members, on the wider community and stakeholders. 3 4 9 Other issues 3 5 0 Interestingly considering the topic of the reviews, the use of PPI members in the reviews was not 3 5 1 universal. 3 5 2 • 9 reviews described PPI throughout the review process 3 5 3 • 3 reviews took their findings to PPI members for discussion 3 5 4 • 3 reviews made use of external panels or organisations 3 5 5 • Single reviews reported utilising PPI at specific stages: • 2 reviews mentioned that there had not been any PPI in the review 3 6 2 • 9 reviews did not mention PPI in their own review process at all. There was no consistent terminology used for either types of PPI or stages of research. There has 3 6 8 been some attempt to categorise these at a national level. For example, in the UK, INVOLVE 3 6 9 distinguished between three PPI approaches: consultation, collaboration and user-led; while Health 3 7 0 Canada divides PPI into five stages: inform or educate, gather information, discuss, engage and 3 7 1 partner (Pii)(40). 3 7 2 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 16, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 Crocker et al (41) describes the types of involvement covered in the studies to range 'from one person 3 7 3 to many people or whole patient organisations, from one-off involvement in a particular aspect of the 3 7 4 trial (for example, reviewing draft information for patients or recruiting participants from their 3 7 5 communities) to involvement throughout the trial (for example, as members of a trial steering 3 7 6 committee), and from involvement with no decision making power (for example, as advisers) to 3 7 7 involvement in decision making as equal partners'.

7 8
Some examples of stages of research where PPI could be included were described as below in Table  3  7  9 8. 3 8 0 public involvement specific to population health research. The novelty in this review of reviews is 3 8 7 twofold: firstly, that the findings have been framed by the UK Standards and secondly, that the 3 8 8 challenges have been matched against potential solutions. Most reviews were about PPI activity in 3 8 9 specific thematic healthcare areas or in general health and social care research but the details of the 3 9 0 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

(which was not certified by peer review) preprint
The copyright holder for this this version posted November 16, 2022. ; https://doi.org/10.1101/2022.11.16.22282319 doi: medRxiv preprint studies included in the reviews makes it clear that many studies included were of direct relevance to 3 9 1 population health research. The findings are, therefore, both generic across health and social care 3 9 2 research as well as providing useful evidence-based suggestions as to what works in PPI in population 3 9 3 health research. 3 9 4 Comparing findings with recently published primary studies 3 9 5 Looking at recently published primary studies we found several of interest, mainly around data-driven 3 9 6 population health research. The principles that emerge from these studies fit well with the findings of 3 9 7 the review of reviews, but also suggest that there are a variety of approaches through which PPI can 3 9 8 be addressed and improved. We summarise recent primary studies in Table 9. 3 9 9 4 0 0 • There is little guidance on how to meaningfully involve the public in big data research. • Involvement in big data research is significantly limited in comparison with other study designs. • May be because common approaches to public involvement adopted in primary data research are not appropriate within big data analysis studies. • The highly data driven discussions that underline this type of research can present a barrier to public involvement. • There is now growing recognition that public involvement in big data research requires special considerations. • Involving patients in health economic research will require a serious investment of time and money for patients to get to a level at which they can contribute. • Patients need to be able to 'rise above' their conditionto find an interest in the material itself and have an objective view. • Proper selection procedures will have to be developed.

Representation & data-driven
Jewell et al (46) Report on the setting up of a service user and carer advisory group supporting data linkage in mental health research.
• The general public feel that the complexities of data linkage research may be difficult to explain in lay terms and that patients and the public have limited knowledge about data, anonymisation, aggregation, and the regulations surrounding these. • Training sessions were set up for all new group members. Training sought to provide members with information about data linkage, including the information governance procedures in place to protect the personal data of service users.

0 2
The specific aspect of longer-term duration that is often typical of population health studies is best . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 16, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 Considering involvement in longitudinal studies, one approach is that used by the ALSPAC study 4 0 8 (49). Based at the University of Bristol, the Avon Longitudinal Study of Parents and Children 4 0 9 (ALSPAC), also known as Children of the 90s, is a world-leading birth cohort study. One of the 4 1 0 governance aspects of the study is the original cohort advisory panel (OCAP) which is made up of 4 1 1 more than 30 study participants who meet bi-monthly to provide insights and advice on study design, 4 1 2 methodology and acceptability for participants. The group has been running since 2006. 4 1 3 The main aims of the OCAP group are: 4 1 4 • To represent the cohort of original study children 4 1 5 • To review study documentation and provide feedback to CO90s staff 4 1 6 • To represent and convey participants' opinions about planned research exercises. 4 1 7 These additional sources suggest that certain solutions identified in the reviews, such as good 4 1 8 communication and tailored training, are even more vital to PPI in population health research. One 4 1 9 thing that emerges strongly from these studies is the idea that PPI selection and recruitment for 4 2 0 population health research projects needs to be very carefully considered. 4 2 1

UK Standards
The UK Standards proved to be a coherent framework for capturing solutions and no solution was 4 2 3 offered that did not fit in to one of the six categories. It was, however, notable that two standards were 4 2 4 less discussed than others: Governance and Impact. Capturing, measuring and illustrating the impact 4 2 5 of PPI within the entire lifespan of a project is an issue that has not yet been resolved but is currently 4 2 6 being addressed by various organisations. The absence of Governance may be a result of language 4 2 7 use, as some attributes of Working Together were relevant in terms of this standard but were not 4 2 8 couched in terms of Governance specifically. It was also interesting to see that Communications is a 4 2 9 UK Standard separate from Working Together, as it was something that could be seen to be an 4 3 0 integral part of Working Together. 4 3 1 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 16, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 What is missing?
The solutions were matched against the problems identified as population health challenges. Not all 4 3 3 challenges were matched by solutions as shown in the table below (Table 10). Challenges not 4 3 4 matched to solutions are in bold and have been allocated to an appropriate UK Standard. By 4 3 5 combining the additional challenges to those matched against solutions we can anticipate producing a 4 3 6 more comprehensive list of things to address to help ensure that PPI within a project is both more 4 3 7 effective in itself, and also helps to achieve better outcomes for the research. 4 3 8 Canada, provides three questionnaires that examine involvement from the point of view of the 4 4 5 participant, the project or the organisation (50). Although there are some tools to assess impact in PPI 4 4 6 they tend to focus on certain aspects, for example the GRIPP2 tool assesses reporting of PPI (51).

7
This relative lack of scrutiny makes it difficult to assess actions that a project could or should 4 4 8 undertake to help ensure best practice in PPI.

9
The evaluation guidance provided on the National Institute for Health Research (NIHR) website 4 5 0 suggests the following categories of evaluation: 4 5 1 • Impact log -a simple method of recording outcomes 4 5 2 • 'Cube' framework -used to evaluate the process or quality of involvement 4 5 3 • Public Involvement Impact Assessment Framework (PiiAF) -more comprehensive method 4 5 4 consisting of two parts 1) planning involvement in a research project, 2) designing a plan to 4 5 5 evaluate the impact of involvement 4 5 6 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint Brett et al (53) notes particularly the lack of any evidence of any financial analysis and Jones et al (54) 4 6 0 suggests that the use of contemporaneous real time data concerning PPI within surgical trials, 4 6 1 currently lacking, could be made use of. Furthermore, '[T]he impact of involvement will always be 4 6 2 somewhat unpredictable, because at the start of any project researchers 'don't know what they don't 4 6 3 know'-they do not know precisely what problems they might anticipate, until the patients/public tell 4 6 4 them…. One of the most important contextual factors that influence the outcome of involvement is 4 6 5 the researchers themselves, in particular the skills, knowledge, values and assumptions they start with. 4 6 6 They are often the 'subjects' who experience the impact of involvement. For this reason, the answer 4 6 7 to the question 'Is involvement worth doing?' will always be 'It depends' (55).

6 8
One further point of consideration is that it could be considered that the aspirational end point of PPI 4 6 9 would be that any involvement would become so integral to the project that it would be difficult to 4 7 0 unpick whose contribution had led to an impact or outcome not originally anticipated.

7 1
Developing new tools for use in population health research 4 7 2 Based the synthesis of reviews, a guidance framework for use in health research projects including 4 7 3 PPI and based on the UK Standards was developed (Supporting Information 4). It combines the 4 7 4 solutions and outstanding challenges taken from the review of reviews and aims to provide a useful 4 7 5 tool for researchers to enhance and improve their PPI activity within population health and general 4 7 6 health research. 4 7 7 From this framework, and with additional information taken from the studies detailed in Table 9, the 4 7 8 following table (Table 11) has been developed which looks specifically at actions which could help 4 7 9 mitigate the key challenges of PPI in population health research.  CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 16, 2022. ;https://doi.org/10.1101https://doi.org/10. /2022 Representation: • Use a variety of methods and partners to recruit a range of PPI members with different knowledge, skills and experience at relevant points in the project lifecycle. • To fit better with wider community context, include relevant stakeholders and agencies also clinicians, charities, specialist support services plus patient and advocacy groups, be proactive and go out and get involved, don't expect people to come to you, build more meaningful relationships with the target population.
Complexity & Data-Driven: • Allow for the formal development of knowledge and skills, supporting PPI team members to be informed and make informed decisions and to understand specific parts of the research process and/or context. • Make learning relevant to the specific context of the research and at the appropriate level for the PPI team member to allow full participation and to build participant capacity. • Provide training in research components to give PPI team members confidence in their involvement and to explain 'rules' and constraints of research. • Use a variety of methods such as supervision, mentoring, formal training, workshops and team-based sessions, include everyone on the team if possible.

Duration:
All of the aspects above are important and need to be considered dynamically throughout the duration of the project.
Generally, researchers should plan for an evolving and varied PPI contribution over time with different PPIs being brought into the project to meet the specific requirements at the time. It may however be useful to maintain a continuous thread of a core group of PPI team members to ensure continuity.
Additional key components regarding the typically longer duration of population health projects are: • Communication -clear, accessible and timely communications using different methods as appropriate • Governance -clarity of roles and structures to support the project throughout its life • Relationship building -take time to build strong, meaningful relationships to improve PPI retention and understanding • Resources -ensure sufficient time, financial and practical resources are available for the duration of the project including lead in and post-project activity • Staff continuity -mitigating for changes in personnel, whether principal investigator, key researchers or PPI members • Flexibility -recognising that projects and external drivers do need to adapt to changes in a measured and managed way • Continuous evaluation and feedback -particularly important in longer projects to ensure that everything stays on track and to ensure that any issues are picked up sufficiently early to allow change.

8 3
What these tools (Table 11 and Supporting Information 4) do not provide is: 4 8 4

•
A comprehensive tick list style approach 4 8 5 . CC-BY 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) preprint The copyright holder for this this version posted November 16, 2022. ;