Extension of Latin hypercube samples with correlated variables

Abstract

A procedure for extending the size of a Latin hypercube sample (LHS) with rank correlated variables is described and illustrated. The extension procedure starts with an LHS of size m and associated rank correlation matrix C and constructs a new LHS of size 2m that contains the elements of the original LHS and has a rank correlation matrix that is close to the original rank correlation matrix C. The procedure is intended for use in conjunction with uncertainty and sensitivity analysis of computationally demanding models in which it is important to make efficient use of a necessarily limited number of model evaluations.

Introduction

The evaluation of the uncertainty associated with analysis outcomes is now widely recognized as an important part of any modeling effort [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11]. A number of approaches to such evaluations are in use, including differential analysis [12], [13], [14], [15], [16], [17], response surface methodology [18], [19], [20], [21], [22], [23], [24], [25], [26], variance decomposition procedures [27], [28], [29], [30], [31], and Monte Carlo (i.e., sampling-based) procedures [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42]. Additional information is available in a number of reviews [43], [44], [45], [46], [47], [48], [49], [50], [51]. Monte Carlo analysis employing Latin hypercube sampling [52], [53] is one of the most popular and effective approaches for the evaluation of the uncertainty associated with analysis outcomes and is the focus of this presentation.

Conceptually, an analysis can be formally represented by a function of the form

$$\mathbf{y}=f(\mathbf{x})\text{,}$$

where

$$\mathbf{x}=[x_1,x_2,\dots ,x_n]$$

is a vector of analysis inputs and

$$\mathbf{y}=[y_1,y_2,\dots ,y_p]$$

is a vector of analysis results. In turn, uncertainty with respect to the appropriate values to use for the elements of x leads to uncertainty with respect to the values for the elements of y. Most analyses use probability to characterize the uncertainty associated with the elements of x and hence the uncertainty associated with the elements of y. In particular, a sequence of probability distributions

$$D_1,D_2,\dots ,D_n$$

is used to characterize the uncertainty associated with the elements of x, where the distribution D_j characterizes the uncertainty associated with the element x_j of x. The definition of the preceding distributions is often accomplished through an expert review process and can be accompanied by the specification of correlations and other restrictions involving the interplay of the possible values for the elements of x [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], [68], [69].

In a Monte Carlo (i.e., sampling-based) analysis, a sample

$${\mathbf{x}}_i=[x_{i1},x_{i2},\dots ,x_{\mathit{in}}],i=1,2,\dots ,m\text{,}$$

is generated from the possible values for x in consistency with the distributions indicated in Eq. (1.4) and any associated restrictions. In turn, the evaluations

$${\mathbf{y}}_i=f({\mathbf{x}}_i),i=1,2,\dots ,m\text{,}$$

create a mapping

$$[{\mathbf{x}}_i,{\mathbf{y}}_i],i=1,2,\dots ,m\text{,}$$

between analysis inputs and analysis outcomes that forms the basis for uncertainty analysis (i.e., the determination of the uncertainty in the elements of y that derives from uncertainty in the elements of x) and sensitivity analysis (i.e., the determination of how the uncertainty in individual elements of x contributes to the uncertainty in elements of y).

As previously indicated, Latin hypercube sampling is a very popular method for the generation of the sample indicated in Eq. (1.5). Further, this generation is often performed in conjunction with a procedure introduced by Iman and Conover to induce a desired rank correlation structure on the resultant sample [70], [71]. As a result of this popularity, the original paper introducing Latin hypercube sampling was recently declared a Technometrics classic in experimental design [72]. The effectiveness of Latin hypercube sampling, and hence the cause of its popularity, derives from the fact that it provides a dense stratification over the range of each uncertain variable with a relatively small sample size while preserving the desirable probabilistic features of simple random sampling. More specifically, Latin hypercube sampling combines the desirable features of simple random sampling with the desirable features of a multilevel, highly fractionated fractional factorial design. Latin hypercube sampling accomplishes this by using a highly structured, randomized procedure to generate the sample indicated in Eq. (1.5) in consistency with the distributions indicated in Eq. (1.4).

A drawback to Latin hypercube sampling is that its highly structured form makes it difficult to increase the size of an already generated sample while simultaneously preserving the stratification properties that make Latin hypercube sampling so effective. Unlike simple random sampling, the size of a Latin hypercube sample (LHS) cannot be increased simply by generating additional sample elements as the new sample containing the original LHS and the additional sample elements will no longer have the structure of an LHS. For the new sample to also be an LHS, the additional sample elements must be generated with a procedure that takes into account the existing LHS that is being increased in size and the definition of Latin hypercube sampling.

The purpose of this presentation is to describe a procedure for the extension of the size of an LHS that results in a new LHS with a correlation structure close to that of the original LHS. The basic idea is to start with an LHS

$${\mathbf{x}}_i=[x_{i1},x_{i2},\dots ,x_{\mathit{in}}],i=1,2,\dots ,m\text{,}$$

of size m and then to generate a second sample

$${\tilde{\mathbf{x}}}_i=[{\tilde{x}}_{i1},{\tilde{x}}_{i2},\dots ,{\tilde{x}}_{\mathit{in}}],i=1,2,\dots ,m\text{,}$$

of size m such that

$${\mathbf{x}}_i=\{\begin{array}{cc}{\mathbf{x}}_i\hfill & \mathrm{for}i=1,2,\dots ,m\hfill \\ {\tilde{\mathbf{x}}}_{i-m}\hfill & \mathrm{for}i=m+1,m+2,\dots ,2m\hfill \end{array}$$

is an LHS of size 2m and also such that the correlation structures associated with the original LHS in Eq. (1.8) and the extended LHS in Eq. (1.10) are similar. A related extension technique for LHSs has been developed by Tong [73] but does not consider correlated variables. Extensions to other integer multiples of the original sample size are also possible.

There are at least three reasons why such extensions of the size of an LHS might be desirable. First, an analysis could have been performed with a sample size that was subsequently determined to be too small. The extension would permit the use of a larger LHS without the loss of any of the already performed, and possibly quite expensive, calculations. Second, the implementation of the Iman and Conover procedure to induce a desired rank correlation structure on an LHS of size m requires the inversion of an m×m matrix. This inversion can be computationally demanding when a large sample is to be generated. The presented extension procedure provides a way to generate an LHS of size 2m with a specified correlation structure at a computational expense that is approximately equal to that of generating two LHSs of size m with the desired correlation structure. Third, the extension procedure provides a way to perform replicated Latin hypercube sampling [74], [75] to test the stability of results that enhances the quality of results obtained when the replicates are pooled.