Patient Reported Outcome Measures (PROMs) – the who, what, where and why now?

Alex H Matthews^a and Jonathan P Evans^b
aNIHR Academic Clinical Fellow, University of Exeter
^bNIHR Academic Clinical Lecturer, University of Exeter

Corresponding author e-mail: [email protected]

Acknowledgements:
EMPROVE Collaborative
National PROMs Network

Published 10 February 2022

The importance of PROMs

Patient Reported Outcome Measures (PROMs) have become an accepted way of measuring the outcome of an intervention as the patient sees it. Since 2009, the national PROMs programme¹ has been routinely collecting these measures from National Health Service (NHS) patients before and after hip and knee replacement surgery in England. So much so that the collection of PROMs after major joint arthroplasty has become ubiquitous. However the routine collection of PROMs outside of this initiative have struggled to progress and little is known what happens at a local provider level. Mason et al.² commented on the British Orthopaedic Foot and Ankle Registry, a national audit tool that society members’ can use to audit outcomes with an emphasis on PROMs collection. The authors cited reasons of inadequate resources and data protection 'red tape' as the top barriers to its use. The importance of PROMs has recently been renewed following the publication of the Cumberledge 'First do no harm' report in 2020³. Guidance from the report is explicit in recommending long term PROMs follow-up for all cases associated with the use of medical devices. In essence the report calls for an all specialities PROMs registry, mandating the routine collection of PROMs which is central to medical device safety. This is a huge undertaking and there are lessons to be learnt from what is being done already. 

The National PROMs Network 

The National PROMs Network is a network of individuals (consultant orthopaedic surgeons, trainees, AHPs, researchers, and administrators) who work together to share experiences and best practice in collecting and using PROMs. In 2019, eight members of the National PROMs Network across the UK agreed to share their organisations’ current PROMs practice. They found considerable variation in PROMs used, their collection methods and their administration intervals. But does this variation exist on a national scale, outside of bias organisations?

The EMPROVE Audit 

In 2021 the South West Orthopaedic Research Division (SWORD) performed a national collaborative audit of practice involving 38 hospitals in the United Kingdom (Figure 1). The study known as Evaluating the Measures in Patient Reported Outcomes, Values and Experiences (EMPROVE) aimed to develop a national overview of PROMs administration in orthopaedics by assessing the PROMs used in common procedures, their frequency of use and their administration technique. A review of national society recommendations was undertaken to formulate a gold standard. This was the first national review of PROMs in clinical practice within orthopaedics in the UK.

The UK and PROMs 

Unsurprisingly EMPROVE found that the reporting of PROMs was most frequent and standardised among the group of conditions incentivised by the national PROMs programme. This incentivisation stems from a 10% reduction in tariff that is attributed with a less than 50% completion rate or if providers are identified as outliers in comparison with the national average. PROMs collection was 83.3% and 80.6% for hip and knee arthroplasty respectively. Data from the national PROMs programme would suggest an even higher frequency of PROMs collection of over 90%. Outside of procedures that attract best practice tariffs, PROMs collection was much more variable. There was a good uptake of PROMs use for patients undergoing shoulder replacement surgery with 27/36 (75%) of respondents. This was consistent with clear societal PROMs recommendations from the British Elbow and Shoulder Society. PROMs collection for all hand and wrist conditions, foot and ankle conditions and spinal conditions suffered from poor reporting frequency and standardisation precipitated by a lack of societal advice particularly amongst hand and wrist procedures. 

Current society recommendations

On review of the national societal guidelines EMPROVE identified some poor-quality guidance for PROMs collection. Recommendations were based on guidelines generated off the back of expert consensus meetings and less on a rigorous synthesis of the evidence. Psychometric testing of PROMs or the methods of assessing the reliability, validity and sensitivity of clinical measures is a vast field of existing research which warrants acknowledgement from societies. There are also measurement tools such as Consensus based Standards for the Selection of health Measurement Instruments (COSMIN)⁴. International societies such as the International Consortium for Health Outcomes Measurement (ICHOM) aim to provide an international standardisation of outcome sets, including PROMs, at a condition specific level⁵. They have produced standard sets for Hip, Knee osteoarthritis, hand and wrist conditions and lower back pain. However, there is a disparity between these outcome sets and those recommended by UK societies.  

The NHS burden 

The EMPROVE study showed across all conditions the rate of PROMs completion reduced over time. As most departments thought the responsibility of data collection was the consultant surgeon, there is often a lack of available time available for inputting this data. Only eight hospitals employed a designated PROMs manager. Therefore, compliance with PROMs may be affected by funding for staff and resources to directly collect this data which is primarily delivered on paper format. The use of electronic patient questionnaires may somewhat reduce this burden on trusts in terms of cost and timeliness. There is the concern that this formatting may fail to capture all sections of the population and this was a problem encountered in the British Orthopaedic Foot and Ankle Registry with a quarter of patients not having an email address. Here, the semi-automated digital approach using barcoded/QR coded paper PROMs may help with admin time and accuracy and ease of scanning when a digital PROMs solution is available. This digitalisation will increase their incorporation with Hospital Episode Statistics (HES) and National Joint Registry (NJR) data providing greater transparency in outcomes which would help establish a UK wide Medical Device Information System (MDIS).

The patient burden

Another aspect of PROMs which is burdensome for the patient is the large variety of questionnaires being used in practice to measure the same latent trait. These questionnaires are time consuming for the patient to complete and this could explain the high attrition rate. There is an evolving area of PROMs research which aims to reduce the admin burden using computer adaptive testing. The United States National Institute of Health has developed a platform called Patient-Reported Outcomes Measurement Information System (PROMIS) to create a universal PROMs language based on item response theory⁶. Item Response Theory (IRT) is a mathematical algorithm that quantifies the relationship between a person’s response to a question and their individual position on the continuum of what is being measured. PROMIS can be administered through a computer adaptive test using this algorithm to ensure patients are asked the minimal yet most relevant questions. When applied to the National PROMs programme these methods have been shown to reduce the number of questions needed to achieve the same precision rate. This improves the efficiency and may improve patient compliance. However, there is no doubt that there will be a large technology cost with the implementation of this system. Another interesting area of research is the use of app-based ecological momentary assessment (EMA). EMA repeatedly samples PROMs in real time and in real world settings in an attempt to minimise retrospective recall bias. A combination of shorter questionnaires using smart phone driven EMA could represent the future of PROMs collection. 

Conclusion 

The PROMs landscape is likely to change over the next ten years as we see more political investment. As a specialty in the UK we are performing well in the reporting of some PROMs. We have established registries that are collecting useable data. However, we are by no means a perfect system. We have identified ways to incorporate formatting that reduces staff resources. We should also be using PROMs that are supported through psychometric analyses and the creation of core outcome sets will help to define what we should be collecting. As a specialty we are well placed to share our experiences and we welcome the renewed interest in such an important area of patient safety. 

References 

1. Patient Reported Outcome Measures (PROMs) – NHS Digital. Available at: https://digital.nhs.uk/data-and-information/data-tools-and-services/data-services/patient-reported-outcome-measures-proms. Accessed January 20, 2022.
2. Mason L, Wood E, Halliwell P. British Orthopaedic Foot and Ankle Registry – Where are we now and where are we going? Journal of Trauma & Orthopaedics. 2021;9(4):26-28. Available at: www.boa.ac.uk/resources/knowledge-hub/jto-december-2021.html. Accessed January 20, 2022 
3. Cumberlege (2020). First Do No Harm – The report of the Independent Medicines and Medical Devices Safety Review. Available at: www.immdsreview.org.uk/downloads/IMMDSReview_Web.pdf. Accessed January 20, 2022.
4. COnsensus-based Standards for the selection of health Measurement Instrument (COSMIN). Available at: www.cosmin.nl.  Accessed January 20, 2022. 
5. International Consortium for Health Outcomes Measurement (ICHOM). Available at: www.ichom.org. Accessed January 20, 2022.
6. PROMIS Health Organisation. Introduction to PROMIS. Available at: www.promishealth.org/57461-2/. Accessed January 20, 2022.