Challenges

Naturally, implementing ePROMs proved more difficult in some specialties than others. This, in some cases, was due to physicians being reluctant to use the ICHOM standard sets.5 Many departments used tools developed and endorsed by specialty societies which promoted hesitation when considering conversion to another data collection tool. In addition, bringing front desk staff on board prior to implementation also proved challenging due to reluctance to take on additional responsibility as well as perceived disruptions of the workflow. In both cases, mapping workflows, eliminating redundant steps as well as defining how the new standard sets could be either combined with or collected in place of previous sets provided evidence on how changes could prove beneficial. Once the net zero impact, and in some cases, time savings on work throughputs were demonstrated, the staff was typically receptive to the new process. This evidence demonstration and work process flow communication has been previously indicated as important to managing change, particularly with respect to ePROM implementation.5 9

Furthermore, some staff’s history with previous attempts to implement and collect ePROMs created implementation barriers. Initial trials with homegrown iPad solutions were unstable due to problems with WI-FI internet connection, complex logins designed to create secure connections, and the fact that computer adaptive testing and branching logic capabilities were not possible. The new platform and technology solved these issues; however, dedicated communication, demonstration and training were still required to help overcome these barriers. Again, this is consistent with previous research focused at change management and technology acceptance.10–12

Patient acceptance of remote capture of questionnaires also presented a challenge. It was found that when questionnaires were sent to the patient’s email or cell phone for completion rather than being presented on site, capture rates were lower. This may have been partially due to the email appearing as a generic email coming from the organisation and as such was disregarded as ‘junk’ mail. As a result, it was necessary to request clinicians interacting with the patient to provide context and set expectations with patients for the remote completion of questionnaires. This process of communication and expectation setting dramatically increased remote capture rates, which is consistent with findings of Atherton et al.13 In addition, customising the email to come from the provider versus the organisation was a key driver in acceptance and capture. The patient received an email from the provider to complete a questionnaire that was then used as part of their appointment. That personalisation created a connection between the patient and provider, as it was no longer presented like other organisational questionnaires that previously may have been deleted.

Finally, there were concerns revolving around data and security. Initial concerns about data storage and security revolved around using a third-party vendor rather than an internal technology solution. This was resolved through use of a HIPAA compliant data cloud owned by the organisation to store and maintain the data. Subsequently, there were concerns around ensuring the information collected in the ePROMs tool were included in the electronic health record (EHR). These concerns were resolved in two different manners. In the absence of an interface with the current EHR, physicians were required to login to an additional browser window, they were then able to import narrative reports into their notes in the EHR. Additionally, patient-completed questionnaires were routed through the organisation’s Health Information Management Systems so that appropriate documents could be scanned into the EHR. Each of these barriers is a reality of a complex organisational IT environment and is an important consideration for cost controls and securities. As such, it is necessary to allocate time and energies to ensure the framework of committees and organisational policies and procedures are followed.

Lessons Learnt

Minimising workflow disruption is essential in creating positive reception of the process changes by staff.5 14 In order to minimise workflow disruption, we observed and interviewed staff in order to better understand their current processes prior to implementation. In addition, we completed workflow mapping activities both preintervention and postintervention to show time and workflow savings.9 Ultimately, the key to gaining staff members’ participation was enthusiasm and support from leadership, face to face support, collaborative efforts in customising the tool as well as data that demonstrated tangible benefits to the practice and process of care.15

The turning point of the implementation was finding a physician champion to engage his colleagues by demonstrating the flexibility of the tool, highlighting efficiencies in the overall work process and convincing his colleagues of the value of the ePROM sets. Consistent with previous inquiry, engagement with physicians, nurses and administrators, that have an interest in using the tools and serve as champions within their own practices, has proven successful in each of the areas we have implemented the ePROMs.16 In particular, front-line staff perceptions of ePROMs usability are critical to implementation, as they are providing the iPad to the patient, explaining the process and are responsible for ensuring that patients complete questionnaires. Including front-line staff in the process flow discussions increased acceptance and willingness to support the ongoing project. In addition, having a coordinator or manager on the implementation project is extremely beneficial. Peer-to-peer communication has proven influential in increasing user acceptance as one’s colleagues are best able to express practice and time benefits.

Flexibility with the ePROM platform was also crucial in the successful adaptation of the tool by staff members. The platform had to be customisable in a number of ways to meet each practice’s specific needs, provider preferences and workflows; a team’s ability to interact in the process and make changes to better suit their needs also creates a collaborative environment that allows physicians and staff to become invested in the project.17 The patient questionnaires were built with a toolkit that allowed multiple-choice questions, multiselect questions, sliding scales, drawing and diagramming capabilities, free text as well as the ability to provide visuals and graphics to better engage patients. Also appealing to providers was the ability to send questionnaires remotely via SMS text messaging or email. This allows the patient to complete the questionnaires before their visit, and as a backup, patients were able to complete questionnaires on an iPad in the lobby on arriving for their appointment.18

In addition, providers could customise the ePROM system to meet their research or practice needs. For instance, they are able to define the ‘weight’ of individual questions, enabling them to create custom scoring systems tailored to particular conditions or the needs of specific patient populations. Prompts for additional follow-up questionnaires or provider interventions can be set based on specified scoring flags and thresholds. By applying scoring algorithms, patient responses can then be displayed to the provider with colour coding, allowing the provider to quickly identify the patient’s top concerns. For example, a red, orange, yellow and green system could be used to illustrate symptom severity to flag patients’ priority concerns in individual symptoms such as sleep, appetite, energy and concentration. Furthermore, the ePROM platform has the ability to prompt follow-up questionnaires should the patient’s score reach a predefined risk threshold for related disease or psychosocial condition. Results were immediately available and populated the provider’s inbox, accessible on the platform through any web browser. As a result, the provider could easily review patient responses in advance of the care visit to use that information with the patient to better focus on their specific needs. Moreover, triggers set to specific question responses or scores can result in an email to clinical staff prompting immediate contact with the patient when needed. The ability to easily add new and modify existing questionnaires allows for a smoother implementation process as specific practice needs can be addressed in real time. This also facilitates the provision of a scalable solution which can expand to additional condition specific areas of care.

Finally, encouraging individual physicians to customise the output summary using natural language processing was also important in the adoption of the new ePROM system. The natural language output examines the way a provider normally dictates their clinical note and uses that information to build a summary from the patient’s questionnaire that could be used as the basis for the clinical note for the patient’s visit. In turn, this decreases the physician work effort, and it is worth noting that there is substantial evidence showing that patient-reported symptoms more accurately reflect their health status than physician elicited responses.19 Incorporating ePROM results into the clinical encounter helps to engage the patient and provider in more relevant discussion that is focused on the patient’s needs.4 5 This allows for a more fruitful and focused conversation concerning patient needs, desires and thoughts about their visit or about the status of their recovery. It also increases the depth of the conversations allowing for more shared decisions regarding treatment paths. This opportunity, plus the ability to track scores for a specific patient, for physicians or for a department in general, allowed for greater buy-in from patients, physicians and administrators. Ultimately, these data may also prove beneficial when negotiating insurance contracts, physician recertification and managing referring providers relationships. A summary of the challenges and lessons learnt is available in table 1.