Skip to main content

Advertisement

SpringerLink
Log in

The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment

  • Original Paper
  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

The use of item banks and computerized adaptive testing (CAT) begins with clear definitions of important outcomes, and references those definitions to specific questions gathered into large and well-studied pools, or “banks” of items. Items can be selected from the bank to form customized short scales, or can be administered in a sequence and length determined by a computer programmed for precision and clinical relevance. Although far from perfect, such item banks can form a common definition and understanding of human symptoms and functional problems such as fatigue, pain, depression, mobility, social function, sensory function, and many other health concepts that we can only measure by asking people directly. The support of the National Institutes of Health (NIH), as witnessed by its cooperative agreement with measurement experts through the NIH Roadmap Initiative known as PROMIS (www.nihpromis.org), is a big step in that direction. Our approach to item banking and CAT is practical; as focused on application as it is on science or theory. From a practical perspective, we frequently must decide whether to re-write and retest an item, add more items to fill gaps (often at the ceiling of the measure), re-test a bank after some modifications, or split up a bank into units that are more unidimensional, yet less clinically relevant or complete. These decisions are not easy, and yet they are rarely unforgiving. We encourage people to build practical tools that are capable of producing multiple short form measures and CAT administrations from common banks, and to further our understanding of these banks with various clinical populations and ages, so that with time the scores that emerge from these many activities begin to have not only a common metric and range, but a shared meaning and understanding across users. In this paper, we provide an overview of item banking and CAT, discuss our approach to item banking and its byproducts, describe testing options, discuss an example of CAT for fatigue, and discuss models for long term sustainability of an entity such as PROMIS. Some barriers to success include limitations in the methods themselves, controversies and disagreements across approaches, and end-user reluctance to move away from the familiar.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Reference List

  1. Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, N.J.: Lawrence Erlbaum Associates.

    Google Scholar 

  2. Lai, J. S., Cella, D., Peterman, A. et al. (2005). Anorexia/cachexia related quality of life for children with cancer: Testing the psychometric properties of the Pediatric Functional Assessment of Anorexia/Cachexia Therapy (peds-FAACT). Cancer, 104(7), 1531–1539.

    Article  PubMed  Google Scholar 

  3. Gershon, R., Cella, D., Dineen, K. et al. (2003). Item response theory and health-related quality of life in cancer. Expert Review of Pharmacoeconomics and Outcomes Research, 3(6), 783–791.

    Article  Google Scholar 

  4. Gershon, R. (2005). Computerized adaptive testing. Journal of Applied Measurements, 6(1), 109–127.

    Google Scholar 

  5. Bunderson, V. C., Inouye, D. K., & Olsen, J. B. (1986). The four generations of computerized educational measurement. In R. L. Linn (Ed.), Educational measurement (pp. 367–407). New York: Macmillan Publishing.

    Google Scholar 

  6. Lautenschlager, G. J., & Flaherty, V. L. (1990). Computer administration of questions: More desirable or more social desirability? Journal of Applied Psychology, 75(3), 310–314.

    Article  Google Scholar 

  7. Reckase, M. D. (1989). Adaptive testing: The evolution of a good idea. Measurement Issues and Practice, 8(3), 11–15.

    Article  Google Scholar 

  8. McHorney, C. A., & Cohen, A. S. (2000). Equating health status measures with Item Response Theory: Illustrations with functional status items. Medical Care, 38(Suppl 9), II43–II59.

    PubMed  CAS  Google Scholar 

  9. Ware, J. E., Bjorner, J. B., & Kosinski, M. (2000). Practical implications of Item Response Theory and computerized adaptive testing: A brief summary of ongoing studies of widely used headache impact scales. Medical Care, 38(Suppl 9), 473–483.

    Google Scholar 

  10. Cella, D., & Chang, C. H. (2000). A discussion of Item Response Theory (IRT) and its applications in health status assessment. Medical Care, 38(Suppl 9), II66–II72.

    PubMed  CAS  Google Scholar 

  11. Wolfe, F., & Pincus, T. (1999). Listening to the patient: A practical guide to self-report questionnaires in clinical care. Arthritis Rheumatism, 42(9), 1797–1808.

    Article  PubMed  CAS  Google Scholar 

  12. Detmar, S. B., Muller, M. J., Schornagel, J. H. et al. (2002). Health-related quality-of-life assessments and patient-physician communication: A randomized controlled trial. JAMA, 288(23), 3027–3034.

    Article  PubMed  Google Scholar 

  13. Velikova, G., Brown, J., Booth, L., Smith, A., Bown, P., Lynch, P., & Selby, P. (2003). A randomized study of quality of life measurements in oncology practice - effects on patient well-being and process of care. Proceedings of the American society of clinical oncology, 22, 728.

    Google Scholar 

  14. Jacobsen, P. B., Davis, K., & Cella, D. (2002). Assessing quality of life in research and clinical practice. Oncology, 16(Suppl 9), 133–139.

    PubMed  Google Scholar 

  15. Bode, R. K., Cella, D., Lai, J. -S. et al. (2003). Developing an initial physical function item bank from existing sources. Journal of Applied Measurements, 4(2), 124–136.

    Google Scholar 

  16. Bode, R., Lai, J. -S., Heinemann, A. et al. (2006). Expansion of a physical function item bank and development of an abbreviated form for clinical research. Journal of Applied Measurements, 7(1), 1–15.

    Google Scholar 

  17. Lai, J. -S., Cella, D., Chang, C. -H. et al. (2003). Item banking to improve, shorten and computerize self-reported fatigue: An illustration of steps to create a core item bank from the FACIT-Fatigue Scale. Quality of Life Research, 12, 485–501.

    Article  PubMed  Google Scholar 

  18. Lai, J. -S., Cella, D., Dineen, K. et al. (2005). An item bank was created to improve the measurement of cancer-related fatigue. Journal of Clinical Epidemiology, 58(2), 190–197.

    Article  PubMed  Google Scholar 

  19. Lai, J. -S., Dineen, K., Reeve, B. et al. (2005). An item response theory based pain item bank can enhance measurement precision. Journal of Pain and Symptomology Management, 30(3), 278–288.

    Article  Google Scholar 

  20. MSP for Windows (Version 5) [computer program]. (2000).

  21. Bode, R. K., Lai, J. -S., Cella, D. et al. (2003). Issues in the development of an item bank. Archives of Physical Medicine and Rehabiitation, 84(4 Suppl 2), S52–S60.

    Article  Google Scholar 

  22. Carlson, L. E., Speca, M., Hagen, N. et al. (2001). Computerized quality-of-life screening in a cancer pain clinic. Journal of Palliative Care, 17(1), 46–52.

    PubMed  CAS  Google Scholar 

  23. Chang, C. H., Cella, D., Fernandez, O. et al. (2002). Quality of life in multiple sclerosis patients in Spain. Multiple Sclerosis, 8(6), 527–531.

    Article  PubMed  CAS  Google Scholar 

  24. Detmar, S. B., & Aaronson, N. K. (1998). Quality of life assessment in daily clinical oncology practice: A feasibility study. European Journal of Cancer, 34(8), 1181–1186.

    Article  PubMed  CAS  Google Scholar 

  25. Taenzer, P., Bultz, B. D., Carlson, L. E. et al. (2000). Impact of computerized quality of life screening on physician behavior and patient satisfaction in lung cancer outpatients. Psycho-Oncology, 9(3), 203–213.

    Article  PubMed  CAS  Google Scholar 

  26. Velikova, G., Wright, P., Smith, A. B. et al. (2001) Self-reported quality of life of individual cancer patients: Concordance of results with disease course and medical records. Journal of Clinical Oncology, 19(7), 2064–2073.

    PubMed  CAS  Google Scholar 

  27. Velikova, G., Brown, J. M., Smith, A. B. et al. (2002) Computer-based quality of life questionnaires may contribute to doctor-patient interactions in oncology. British Journal of Cancer, 86(1), 51–59.

    Article  PubMed  CAS  Google Scholar 

  28. Velikova G., Booth L., Smith A. B. et al. (2004). Measuring quality of life in routine oncology practice improves communication and patient well-being: A randomized controlled trial. Journal of Clinical Oncology, 22(4), 714–724.

    Article  PubMed  Google Scholar 

  29. Yount, S., Davis, K., Cella, D. et al. (2003). Brief patient symptom assessment: Research and clinical applications. To be presented at the 10th World Conference on Lung Cancer.

  30. Hahn, E. A., Cella, D., Dobrez, D. G. et al. (2003). Quality of life assessment for low literacy Latinos: A new multimedia program for self-administration. Journal of Oncology Management, 12(5), 9–12.

    PubMed  Google Scholar 

  31. Higginson, I. J., & Carr, A. J. (2001). Measuring quality of life: Using quality of life measures in the clinical setting. BMJ, 322(7297), 1297–1300.

    Article  PubMed  CAS  Google Scholar 

  32. Buxton, J., White, M., & Osoba, D. (1998). Patients’ experiences using a computerized program with a touch-sensitive video monitor for the assessment of health-related quality of life. Quality of Life Research, 7(6), 513–519.

    Article  PubMed  CAS  Google Scholar 

  33. Davis, K. M., Chang, C. -H., Lai, J. -S., & Cella, D. (2002). Feasibility and acceptability of computerized adaptive testing (CAT) for fatigue monitoring in clinical practice. Quality of Life Research, 11(7), 134.

    Google Scholar 

  34. Bergstrom, B. A., & Lunz, M. E. (1999). CAT for certification and licensure. In F. Drasgow & J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 67–92). Mahwah, NJ: Lawrence Erlbaum Associates Publishers.

    Google Scholar 

  35. Gershon, R. C. (2004) The ABCs of Computerized Adaptive Testing. In T. M. Wood & W. Zhi (Eds.), Measurement issues and practice in physical activity. Champaign, IL: Human kinetics.

  36. Ware, J. E. Jr., Kosinski, M., Bjorner, J. B. et al. (2003) Applications of computerized adaptive testing (CAT) to the assessment of headache impact. Quality of Life Research, 12(8), 935–952.

    Article  PubMed  Google Scholar 

  37. Bergstrom, B., & Cline, A. (2003). Beyond multiple choice: Innovations in professional testing. CLEAR Exam Review. Summer.

  38. van der Linden, W. J., & Pashley, P. J. (2000) Item selection and ability estimation in adaptive testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: theory and practice. Norwell, MA: Kluwer.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Psychiatry and Behavioral Sciences, Institute for Healthcare Studies, Feinberg School of Medicine, Northwestern University, Evanston, IL, 60201, USA

    David Cella

  2. Center on Outcomes, Research and Education (CORE), Evanston Northwestern Healthcare, Evanston, IL, USA

    David Cella, Jin-Shei Lai & Seung Choi

  3. Psychometrics and Informatics, Center on Outcomes, Research and Education (CORE), Evanston Northwestern Healthcare, Evanston, IL, USA

    Richard Gershon

  4. Department of Psychology, Northwestern University, Evanston, IL, USA

    Richard Gershon

  5. Department of Pediatrics, Institute for Healthcare Studies, Feinberg School of Medicine, Northwestern University, Evanston, IL, USA

    Jin-Shei Lai

Authors
  1. David Cella
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard Gershon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin-Shei Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Seung Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Cella.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cella, D., Gershon, R., Lai, JS. et al. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res 16 (Suppl 1), 133–141 (2007). https://doi.org/10.1007/s11136-007-9204-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-007-9204-6

Keywords

Search

Navigation