Skip to main content
SpringerLink
Log in

Methodological challenges in the evaluation of prognostic factors in breast cancer

  • Published:
Breast Cancer Research and Treatment Aims and scope Submit manuscript

Abstract

Many studies are carried out in an effort to find factors that help explain the large unexplained variation in prognosis of breast cancer patients. The principles of good study design and analysis are less well appreciated for prognostic factor studies than for therapeutic trials. The oncology literature is full of results from studies of varying quality, often with conflicting findings. As a consequence, despite the large number of studies, there is still uncertainty about the importance of most prognostic factors. Few recently proposed prognostic factors for breast cancer have become widely accepted. This paper reviews the important methodological issues underlying such research. These issues are illustrated with examples from published studies and recent reviews of papers published in cancer journals. Guidelines are proposed for conducting and evaluating prognostic factor studies which should improve the quality of research in this important area.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. McGuire WL: Breast cancer prognostic factors: evaluation guidelines. J Natl Cancer Inst 83:154-155, 1991

    Google Scholar 

  2. Gasparini G, Pozza F, Harris AL: Evaluating the potential usefulness of new prognostic and predictive indicators in node negative breast cancer patients. J Natl Cancer Inst 85:1206-1219, 1993

    Google Scholar 

  3. Clark GM, Wenger CR, Beardslee S, et al: How to integrate steroid hormone receptor, flow cytometric, and other prognostic information in regard to primary breast cancer. Cancer 71:2157-2162, 1993

    Google Scholar 

  4. Simon R, Altman DG: Statistical aspects of prognostic factor studies in oncology. Br J Cancer 6:979-985, 1994

    Google Scholar 

  5. Gray-Donald K, Kramer MS: Causality inference in observational vs. experimental studies. An empirical comparison. Am J Epidemiol 127:885-892, 1988

    Google Scholar 

  6. Tukey JW: Some thoughts on clinical trials, especially problems of multiplicity. Science 198:679-684, 1977

    Google Scholar 

  7. Fayers PM, Machin D: Sample size: how many patients are necessary? Br J Cancer 72:1-9, 1995

    Google Scholar 

  8. Machin D, Campbell MJ, Fayers PM, Pinol APY: Sample Size Tables for Clinical Studies, 2nd edition. Blackwell, Oxford, 1997

    Google Scholar 

  9. Harrell FE, Lee KL, Matchar DB, Reichert TA: Regression models for prognostic prediction: advantages, problems, and suggested solutions. Cancer Treat Rep 69:1071-1077, 1985

    Google Scholar 

  10. Peduzzi P, Concato J, Feinstein AR, Holford TR: The importance of events per independent variable (EPV) in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 48:1503-1510, 1995

    Google Scholar 

  11. Altman DG, De Stavola BL, Love SB, Stepniewska KA: Review of survival analyses published in cancer journals. Br J Cancer 72:511-518, 1995

    Google Scholar 

  12. Axelsson K, Ljung B-ME, Moore DH, et al: Tumor angiogenesis as a prognostic assay for invasive ductal breast carcinoma. J Natl Cancer Inst 87:997-1008, 1995

    Google Scholar 

  13. Press MF, Hung G, Godolphin W, Slamon DJ: Sensitivity of HER-2/neu antibodies in archival tissue samples: potential source of error in immunohistochemical studies of oncogene expression. Cancer Res 54:2771-2777, 1994

    Google Scholar 

  14. Romero H, Schneider J: Different detection rates of HER-2/NEU overexpression in ovarian carcinoma using two different commercially available detection kits. Eur J Cancer 31A:1020-1021, 1995

    Google Scholar 

  15. Molino A, Micciolo R, Turazza M, et al: Prognostic significance of estrogen receptors in 405 primary breast cancers: a comparison of immunohistochemical and biochemical methods. Breast Cancer Res Treat 345:241-249, 1997

    Google Scholar 

  16. Bland JM, Altman DG: Statistical methods for comparing two methods of measurement. Lancet i:307-310, 1986

    Google Scholar 

  17. Cox DR: Regression models and life-tables. J R Stat Soc B 34:187-220, 1972

    Google Scholar 

  18. Fielding LP, Fenoglio-Preiser CM, Freedman LS: The future of prognostic factors in outcome prediction for patients with cancer. Cancer 70:2367-2377, 1992

    Google Scholar 

  19. Chen C-H, George SL: The bootstrap and identification of prognostic factors via Cox's proportional hazards regression model. Stat Med 4:39-46, 1985

    Google Scholar 

  20. Altman DG, Andersen PK: Bootstrap investigation of the stability of a Cox regression model. Stat Med 8:771-783, 1989

    Google Scholar 

  21. Sauerbrei W, Schumacher M: A bootstrap resampling procedure for model building: application to the Cox regression model. Stat Med 11:2093-2109, 1992

    Google Scholar 

  22. Schumacher M, Höllander, N, Sauerbrei W: Resampling and cross-validation techniques: a tool to reduce bias caused by model building? Stat Med 16:2813-2827, 1997

    Google Scholar 

  23. Gamel JW, McCurdy JB, McLean IW: A comparison of prognostic covariates for uveal melanoma. Invest Ophthalmol Vis Sci 33:1919-1922, 1992

    Google Scholar 

  24. Jenks S, Volkers N: Razors and refrigerators and reindeer — oh my. J Natl Cancer Inst 84:1863, 1992

    Google Scholar 

  25. Peters JM, Preston-Martin S, London SJ, et al: Processed meats and risk of childhood leukemia (California, USA). Cancer Causes Control 5:195-202, 1994

    Google Scholar 

  26. Wyatt JC, Altman DG: Prognostic models: clinically useful or quickly forgotten? Br Med J 311:1539-1541, 1995

    Google Scholar 

  27. Altman DG, Royston P: What do we mean by validating a prognostic model? Stat Med, in press

  28. Vach W: Some issues in estimating the effect of prognostic factors from incomplete covariate information. Stat Med 16:57-72, 1997

    Google Scholar 

  29. Sagman U, Maki E, Evans WK, et al: Small-cell carcinoma of the lung: derivation of a prognostic staging system. J Clin Oncol 9:1639-1649, 1991

    Google Scholar 

  30. Thor A, Benz C, Moore D, et al: Stress response protein (srp-27) determination in primary human breast carcinomas: clinical, histologic, and prognostic correlations. J Natl Cancer Inst 83:170-178, 1991

    Google Scholar 

  31. Hart A, Wyatt J: Evaluating black boxes as medical decision-aids: issues arising from a study of neural networks. Med Informatics 15:229-236, 1990

    Google Scholar 

  32. Ohno-Machado L: A comparison of Cox proportional hazards and artificial network models for medical prognosis. Comput Biol Med 27:55-65, 1997

    Google Scholar 

  33. Schwarzer G, Vach W, Schumacher M: On the misuses of artificial neural networks for prognostic factor and diagnostic classification in oncology. University of Freiburg Technical Report No. 46, 1997

  34. Morgan TM, Elashoff RM: Effect of categorizing a continuous covariate on the comparison of survival time. J Am Stat Assoc 81:917-921, 1986

    Google Scholar 

  35. Hilsenbeck SG, Clark GM, McGuire WL: Why do so many prognostic factors fail to pan out? Breast Cancer Res Treat 22:197-206, 1992

    Google Scholar 

  36. Altman DG, Lausen B, Sauerbrei W, Schumacher M: Dangers of using ‘optimal’ cutpoints in the evaluation of prognostic factors. J Natl Cancer Inst 86:829-835, 1994

    Google Scholar 

  37. Altman DG: Suboptimal analysis using ‘optimal’ cutpoints. Br J Cancer 78:556-557, 1998

    Google Scholar 

  38. Jänicke F, Schmitt M, Ulm K, et al: Urokinase-type plasminogen activator antigen and early relapse in breast cancer. Lancet 2:1049, 1989

    Google Scholar 

  39. Jänicke F, Schmitt M, Pache L, et al: Urokinase (uPA) and its inhibitor PAI-1 are strong and independent prognostic factors in node negative breast cancer. Breast Cancer Res Treat 24:195-208, 1993

    Google Scholar 

  40. Knoop A, Andreasen PA, Andersen JA, et al: Prognostic significance of urokinase-type plasminogen activator and plasminogen activator inhibitor-1 in primary breast cancer. Br J Cancer 77:932-940, 1998

    Google Scholar 

  41. Buettner P, Garbe C, Guggenmoos-Holzmann I: Problems in defining cutoff points of continuous prognostic factors: example of tumor thickness in primary cutaneous melanoma. J Clin Epidemiol 50:1201-1210, 1997

    Google Scholar 

  42. Budinha M, Skrk J, Zakotnik B, et al: Prognostic value of total cathepsin B in invasive ductal carcinoma of the breast. Eur J Cancer 31A:661-664, 1995

    Google Scholar 

  43. Durrleman S, Simon R: Flexible regression models with cubic splines. Stat Med 8:551-561, 1989

    Google Scholar 

  44. Hastie T, Sleeper L, Tibshirani R: Flexible covariate effects in the proportional hazards model. Breast Cancer Res Treat 22:241-250, 1992

    Google Scholar 

  45. Royston P, Altman DG: Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling. Appl Stat 43:429-467, 1994

    Google Scholar 

  46. Knorr KL, Hilsenbeck SG, Wenger CR, et al: Making the most of your prognostic factors: presenting a more accurate survival model for breast cancer patients. Breast Cancer Res Treat 22:251-262, 1992

    Google Scholar 

  47. Buyse M: Analysis of clinical trial outcomes: some comments on subgroup analyses. Controlled Clin Trials 10:187S-194S, 1989

    Google Scholar 

  48. Henry JA, McCarthy AL, Angus B, et al: Prognostic significance of the estrogen-regulated protein, cathepsin D, in breast cancer. An immunohistochemical study. Cancer 65:265-271, 1990

    Google Scholar 

  49. Simon R: Confidence limits for reporting results of clinical trials. Ann Intern Med 105:429-435, 1986

    Google Scholar 

  50. Gardner MJ, Altman DG (eds) Statistics with Confidence. British Medical Journal, London, 1989

  51. Simon R: Patient subsets and variation in therapeutic efficacy. Br J Clin Pharmacol 14:473-482, 1982

    Google Scholar 

  52. Aubele N, Auer G, Falkmer U, et al: Improved prognostication in small (pT1) breast cancers by image cytometry. Breast Cancer Res Treat 36:83-91, 1995

    Google Scholar 

  53. Harrell FE, Lee KL, Mark DB: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:361-387, 1996

    Google Scholar 

  54. Haybittle JL, Blamey RW, Elston CW, et al: A prognostic index in primary breast cancer. Br J Cancer 45:361-366, 1982

    Google Scholar 

  55. Todd JH, Dowle C, Williams MR, et al: Confirmation of a prognostic index in primary breast cancer. Br J Cancer 56:489-492, 1987

    Google Scholar 

  56. Brown JM, Benson EA, Jones M: Confirmation of a long-term prognostic index in breast cancer. Breast 2:144-147, 1993

    Google Scholar 

  57. Stern JM, Simes RJ: Publication bias: evidence of delayed publication in a cohort study of clinical research projects. Br Med J 315:640-645, 1997

    Google Scholar 

  58. Ferrandina G, Scambia G, Bardelli F, et al: Relationship between cathepsin-D content and disease-free survival in node-negative breast cancer patients: a meta-analysis. Br J Cancer 76:661-666, 1997

    Google Scholar 

  59. Fox SB, Smith K, Hollyer J, et al: The epidermal growth factor receptor as a prognostic marker: results of 370 patients and review of 3009 patients. Breast Cancer Res Treat 29:41-49, 1994

    Google Scholar 

  60. Rawson NSB, Peto J: An overview of prognostic factors in small cell lung cancer. Br J Cancer 61:597-604, 1990

    Google Scholar 

  61. The International Non-Hodgkin's Lymphoma Prognostic Factors Project: A predictive model for aggressive lymphoma. N Engl J Med 329:987-994, 1993

    Google Scholar 

Download references

Authors
  1. Douglas G. Altman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gary H. Lyman
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Altman, D.G., Lyman, G.H. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast Cancer Res Treat 52, 289–303 (1998). https://doi.org/10.1023/A:1006193704132

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1006193704132

Search

Navigation