A united interpretation of different uncertainty intervals

Abstract

The frequentist and Bayesian philosophies of statistical inference require different approaches to the calculation of an interval of uncertainty for a measurand. A frequentist (or classical) interval will have an associated confidence level, p, that is the probability of generating an interval enclosing the value of the measurand. A Bayesian interval will have an associated credible level, p, that is a ‘degree-of-belief’ that the value of the measurand subsequently lies within the interval. Since potential users are not primarily concerned with the method of analysis, a shared interpretation of the information given to them seems desirable. We obtain such an interpretation by recognising that in either case p is the proportion of independent intervals calculated over time that contain their respective measurands. This interpretation is also useful in explaining an interval calculated according to the procedure of Guide to the Expression of Uncertainty in Measurement.

Introduction

Different understandings of probability can lead to different statistical analyses of measurement data, and hence to different interpretations of the measurement uncertainty. This paper is primarily concerned with two such understandings, as represented by the frequentist (or classical) and Bayesian frameworks of statistical inference.

In the frequentist framework the quantities of interest are regarded as fixed statistical parameters to be estimated using data, and not as quantities possessing probability density functions (pdfs). In uncertainty analysis, this approach asks the question “what range of (fixed) values for the measurand could have given rise to the data?” The result is a 100p% confidence interval for the measurand,¹ typically with p = 0.95. Several recent papers in the metrological literature have adopted this approach, e.g., [1], [2].

In Bayesian statistics pdfs are attached to all quantities of interest. These pdfs are first constructed from informed opinions, subsequently updated using the sampling model and data, and finally combined appropriately to obtain the distribution of the measurand. An interval containing 100p% of the mass of this distribution is often called a 100p% credible interval for the measurand [3], [4].² There are also several recent papers written within this framework, e.g., [5], [6].

The frequentist approach deals adequately with so-called ‘random’ errors. However, Bayesian methodology seems preferable for the incorporation of ‘systematic’ errors. In practice, a measurement involves both types of error. The procedure for uncertainty analysis advocated in the Guide to the Expression of Uncertainty in Measurement [7], which results in an interval of ‘expanded uncertainty’, seems neither strictly frequentist nor strictly Bayesian [4]. Partly because of the utility of this procedure, a shared interpretation is required for confidence intervals and credible intervals, both of which might be called intervals of uncertainty. This paper presents such an interpretation designed for clear communication to those who will make use of the interval.

The suggestion that there is no obvious existing shared interpretation may appear surprising. However, the paragraphs above indicate that the level of certainty p relates to different concepts in the two cases given. So what should be a client’s understanding of a figure, say p = 0.95, associated with the uncertainty interval on a calibration certificate? And if that figure needs no communicable meaning then why not prefer p = 0.90, because this would give intervals that were narrower and hence more informative to the client about the actual value of the measurand? So, to what meaning of p is the analysis accountable?

We do not discuss the superiority of either the frequentist or Bayesian framework over the other. Surely, both have advantages and disadvantages; else there would be a unique way of evaluating the uncertainty of measurement. Rather, our purpose is to point out a single interpretation of the corresponding intervals to assist the users of measurement results and to encourage their confidence in measurement processes. We suggest that both a 100p% confidence interval and a 100p% credible interval may be interpreted as an interval the like of which contains the value of the relevant measurand on 100p% of independent occasions. (Alternative wordings are given later.) In our experience this simple interpretation is not often made explicit.

We begin by considering some different definitions of probability, including a seemingly natural way of quantifying the concept of degree-of-belief.