Intended for healthcare professionals

Feature Outcomes Data

Show us the data: why clinical outcomes matter

BMJ 2012; 344 doi: https://doi.org/10.1136/bmj.e66 (Published 10 January 2012) Cite this as: BMJ 2012;344:e66
  1. Michael Cross, freelance journalist
  1. michaelcross{at}fastmail.fm

The NHS’s use of clinical outcomes data is in its infancy, but accurate and usable measures are crucial to the success of new commissioning groups. Michael Cross maps out the road ahead

Data from the NHS will have a key role in the British government’s plan for national economic recovery. In a well trailed speech last month, Prime Minister David Cameron announced a plan to amend the NHS constitution to make data extracted from health records available for research unless patients actively opt out. Acknowledging “a little bit of controversy” over the issue of confidentiality, he said, “it doesn’t mean anyone can look at your health records, but it does mean using anonymous data to make new medical breakthroughs . . . it is simply a waste to have a health system like the NHS and not to do this kind of thing.”

Mr Cameron’s speech came hard on the heels of a series of announcements by Chancellor George Osborne about the opening up of NHS data for re-use by business. In his autumn statement on the economy, the chancellor set a timetable for publishing several new sets of data, including prescribing data at general practice level, under the government’s “open data” programme. The idea is to nurture a market in websites and online services that make use of “public sector information” ranging from geographical and meteorological information to anonymised data drawn from health and social care records.

The chancellor’s statement set out a new role for the NHS Health and Social Care Information Centre (set up by the last government to split the management of NHS information from the management of NHS information technology) as a hub for publicly available data. All its data sets would be available free under an “open government licence” apart from some prescribing data, which is “subject to a review of charging regimes.”

Need for better outcomes data

However, one other potential use of open NHS data is receiving less attention despite a more urgent need—in England, at least. This is the creation of accurate and timely, yet also easily usable, data on measures of clinical outcomes for the purpose of commissioning services under the government’s reforms.

There is little point in giving clinical commissioning groups the power to shop in an open market if they cannot do so intelligently. Today, the NHS’s use of clinical outcomes data is in its infancy. David Colin-Thomé, chair of the Intelligent Board reference group set up by the healthcare information firm Dr Foster Intelligence says: “NHS commissioning has achieved less than it should have because of a lack of clinical involvement and a dearth of high quality information.”

The problem with current crude outcome measures was summed up by Ailsa Claire, director of commissioning development at NHS Yorkshire and the Humber. She said that her previous trust, Barnsley PCT, had scored highly on official outcome indicators yet the population it served—in an area notorious for poor public health— “kept dying on us.”

Yet “clinical commissioning has to operate in an information-rich environment,” she said. “If we don’t radically change the way we operate . . . clinical commissioning will fail.”

The problem of comparing the effectiveness of clinical services between organisations serving populations with widely different lifestyles and health expectations also surfaces in the reluctance of some specialties to follow the lead of cardiothoracic surgery and publish individual surgeons’ mortality statistics. Michael Parker, president of the Association of Coloproctology and a council member of the Royal College of Surgeons, told a King’s Fund seminar on Making Healthcare Smarter last September that the royal college is keen to “facilitate the interpretation of outcome data” but that publishing raw data would be harmful because of the risk of misinterpretation.

Some efforts to fill the information gap are under way. A new experimental quarterly data set, the summary hospital level mortality indicator, compares the number of patients who die after treatment (up to 30 days of discharge) with the number expected to die, given the characteristics of patients treated. However, Tim Straughan, chief executive of the NHS Information Centre, says the summary index is best suited to function as a “smoke alarm” for local action by care providers rather than as a commissioning tool.

Meanwhile, the National Institute for Health and Clinical Excellence (NICE) is working on 150 quality standards to compare the performance of clinical teams. But as Fergus Macbeth, director for clinical practice, says: “There are not many true outcome measures in this. It is much easier to measure process or structure.”

One problem facing commissioners is the sheer difficulty of analysing the wealth of data that will become available. “We have to invest in analysis and expertise at every level in the system,” says Mr Straughan. To avoid information overload, the latest Intelligent Board report from Dr Foster Intelligence advises commissioners: “Less is more. Defining a few critical outcomes and related process indicators is crucial.”

The NHS medical director, Bruce Keogh, is a long standing advocate of publishing outcome data—he was one of the driving forces behind the cardiothoracic surgeons’ initiative—but he says his focus is enabling clinicians to compare their performance with peers rather than on the higher level data needed by commissioners picking services. He describes the NHS’s outcomes framework programme, an initiative to set out outcomes and corresponding indicators that will be used to hold the NHS Commissioning Board to account, as developing “very high level measures . . . . Commissioning would be interested in measures that I’m less interested in,” he said. Neither would transparency, on its own, fill the gap, he said. However, he stresses that what performance measures are created should be released for all. “I don’t think any information that is used for measuring the quality of care should be hidden. It should be transparent.”

Confidentiality

The press furore surrounding the prime minister’s announcements gave a hint of another potential problem with the use of data derived from patients’ records for any purpose beyond immediate clinical care. In recent years the NHS in England has come under repeated attack from professional and civil liberties bodies over the model of consent for viewing easily shared electronic health records. The new emphasis on re-use of data extracted from individuals’ health records is likely to revive the controversy. This is likely to be stoked by lobbies opposed to two of the proposed users of derived data, the drug industry and private firms entering the new market in NHS services.

Officially, the government holds the view—as stressed by the prime minister—that the new data sets will be anonymised and that patients who strongly object to re-use will be able to opt out. A spokesperson for the Department of Health said: “One of the guiding principles of government strategy is that ‘transparency will not be extended at the cost of privacy.’ Health and care data will be made available in anonymised and de-identified form. We will continue to see how we can further strengthen the approach to patient confidentiality and will monitor any concerns raised under the Data Protection Act as we implement our plans for publication.”

However, recent academic studies have warned that anonymised data may need to be protected as rigorously as identifiable data to remain on the right side of the law. Earlier this year, a study commissioned by the Cabinet Office as part of the government’s transparency programme warned of a “potential clash” over the use of anonymised data in publicly available releases of data. The problem is that in small data sets, such as from individual surgeons’ outcomes, it might be possible to identify individuals despite anonymisation. The study, Transparent Government, Not Transparent Citizens, by Kieron O’Hara, senior research fellow in electronics and computer science at Southampton University, warns: “There are no complete legal or technical fixes to the de-anonymisation problem” and that “it is essential that policymakers, data managers, data controllers, privacy officers, and lawyers do not automatically assume that anonymised data cannot be used to re-identify people.”1

Last year, a study by three experts in privacy law concluded that “merely replacing name and address with postcode and date of birth achieves a level of de-identification that is trivially reversible: many, if not most, patients are easily identifiable by reference to these facts. Such limited measures mean that, in data protection law, [anonymised] data remain identifiable, and thus as ‘personal data’ are subject to UK and European data protection rules.”2 This means that all processing of an individual’s data (including the act of anonymisation) must be carried out with consent. Today, the authors warn, “patients are not adequately informed about possible secondary uses of their medical data for medical research, are not asked to give clear, specific, free and informed consent, are not even offered unambiguous and effective opt-outs, and are misled about the level of anonymisation of their data and the likelihood of re-identification.” They call for “a much wider debate as to the role and importance of confidentiality and consent for medical research using patient records. This debate needs to include medical researchers, health professionals and, to a much greater extent, the public.”

With the prime minister’s announcement of a consultation on the consent model, a debate now seems set to get under way. Sir Bruce is confident that the vast majority of ordinary patients will understand the merits of making data available for re-use. “The public are really smart. They worry less than some people seem to think.”

He is equally confident that clinical professions can be won round to the cause of transparency. “There’s a saying that the best way to look after the surgeons is to look after the patients. If you can demonstrate you’re doing a good job, it reflects well on you and it is quite protective of you and your organisation.”

The next indicator of the way forward should be the emergence of the final NHS information revolution strategy from the Department of Health. It was originally due early last summer; publication is now set for spring 2012, after the department has digested the findings of the reports by the Futures Forum. Yet the transparency agenda, and the reforms in England, now seem to have their own momentum. The overall impression—not for the first time—is that government policies for the NHS seem to be moving independently of the information policies needed to support them.

Notes

Cite this as: BMJ 2012;344:e66

Footnotes

  • Competing interests: The author has completed the ICJME unified disclosure form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declares no support from any organisation for the submitted work; no financial relationships with any organisation that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work.

  • Provenance and peer review: Commissioned; not externally peer reviewed.

References

View Abstract