The study of variability in engineering design—An appreciation and a retrospective

Abstract

We explore the concept of parameter design applied to the production of glass beads in the manufacture of metal-encapsulated transistors. The main motivation is to complete the analysis hinted at in the original publication by Jim Morrison in 1957, which was an early example of discussing the idea of transmitted variation in engineering design, and an influential paper in the development of analytic parameter design as a data-centric engineering activity. Parameter design is a secondary design activity focused on selecting the nominals of the design variables to achieve the required target performance and to simultaneously reduce the variance around the target. Although the 1957 paper is not recent, its approach to engineering design is modern.

1. Introduction

The concept of engineering design robustness has been well established since the work of Japanese engineer Genichi Taguchi became known in the western economies in the 1980s, for example, Taguchi and Wu (1985) and subsequently Taguchi (1986).

Design robustness is formulating a design with a functional response that is as immune as possible to so-called noise factors.¹

The challenge of design robustness has been around for a long time and was uppermost in the minds of 18th century engineers, for example, John Harrison in his design of a clock that could keep time despite humidity, and the rolling motion of a ship, to solve the longitude problem, and Josiah Wedgewood in making good quality China pieces, despite variability of temperature inside his kilns, although the word robustness was not common parlance at that time.

Many decades prior to the awareness of Taguchi’s work a seminal paper was published by Jim Morrison² who addressed a particular type of robustness problem, that of transmitted variation (Morrison, 1957). At the time Morrison was working as a glass engineer for the British Thomson–Houston company in Rugby UK. This early paper addressed a key problem of robustness associated with the manufacture of encapsulated transistors.

2. The Glass Beads Problem

This problem arose in the early days of semiconductors, whereby development work was proceeding on a new range of metal-encapsulated transistors. The envelope comprised a metal disk to which the germanium or silicon wafer would be attached and a flanged metal “top hat” which would be welded to the disk to complete the enclosure. The electrical connections to the transistor consisted of two fine wires which were sealed into flanged holes on the disk by fusing glass beads to form a hermetic seal. (see Figure 1). This problem provided the motivation for the 1957 paper.

Figure 1. A diagram of a single bore bead together with a metal-encapsulated transistor. The glass bead (left figure) ends up inside the “top hat” enclosing the transistor (right figure).

Although this case study is not recent, it is modern. Its main teaching point as we will see, is that the “obvious” (to the uninitiated) solution to this simple problem is wrong, which motivates application of Morrison’s method to more complicated problems, whereby the solution will not in any sense be obvious.

It is worth noting that the small glass beads are not just parts but are part of something (a bigger system—e.g., the complete circuit hardware). Often the performance of an entire system relies heavily on the performance of the smallest parts, for example, in the case of the 1986 Challenger disaster—see Feynman (1986).

After the 1957 paper surfaced (due in large part to the work of George Box and his colleagues at the Center for Quality and Productivity Improvement at the University of Wisconsin³), Morrison published a coda to the 1957 paper (Morrison, 1998), where he introduced the term variance synthesis and described his general approach as statistical engineering. ⁴ To quote directly from the 1998 paper “the concept of transmitted error was implicit in the 1957 article, yet it remained neglected and undeveloped until the early 1980s when Taguchi and Wu (1985) introduced off-line quality control and Box and Fung (1986) showed that in suitable circumstances, the error transmission formula provided a better approach…(than those proposed by Taguchi and Wu)” Additionally, the 1998 paper gave more detail on how the glass beads were made, which we repeat here verbatim to provide context for the study.

“Because of the low volume of this special glass, the glass was melted in a small pot and the longer glass tubes from which the beads were cut, were then “drawn” by hand. A skilled glass blower would gather a quantity of molten glass on the end of a blow pipe and would “marver” it to a cylindrical shape on a flat metal plate after putting a puff of air into the gather to keep it hollow, A co-worker would then stick a “punty” on the opposite end of the gather and the two would then walk backwards away from each other along a tube-drawing alley.

A third colleague armed with a board would fan the glass to cool it until the tubing was thought to be the right size.⁵ The tubing was then cut up into lengths of about 1 m prior to shipping to the customer.”

There were several types of glass beads mentioned in the 1957 paper: A standard wall bead, a thick wall bead, and a triple-bore bead, but following Morrison (1957), the focus of our discussion is on the standard-wall bead—see Figure 1.

The volume of a single bore bead is given by the following objective function. ⁶

(1)$$ V=\frac{\pi \left({D}^2-{B}^2\right)L}{4} $$

There are three relevant dimensions (the outer diameter $ \left[D\right] $, the inner bore $ \left[B\right] $, and the length $ \left[L\right] $) and these are, collectively the design variables. Table 1 shows the measurements of these dimensions taken on a sample of 30 beads, carefully measured with a measurement microscope.

Table 1. The mean dimensions and variances of a sample of 30 beads, as reported in Morrison (1957).

The volumes in Table 1 were estimated from the volume equation (1) and were calibrated by estimating the volumes by weighing (ignoring any variation in glass density). The volume estimates derived from the glass density, were on average lower than those calculated geometrically, primarily due to the beads not being perfectly circular, and some of the beads losing some material by being chipped (a failure mode which we ignore in this paper).

The volumes in Table 1 were estimated from the volume equation (1) and were calibrated by estimating the volumes by weighing (ignoring any variation in glass density). The volume estimates derived from the glass density, were on average lower than those calculated geometrically, primarily due to the beads not being perfectly circular, and some of the beads losing some material by being chipped (a failure mode which we ignore in this paper).

There were two distinct failure modes which hampered high-quality production of the beads. Firstly, the glass beads would crack under temperature gradients due to the differences in thermal properties of the glass and the metal of the mounting surface that was being sealed. A so-called one-sided failure mode (Clausing and Frey, 2005). Secondly, at the preproduction stage, a high failure rate was being experienced in the glass/metal sealing process because the volume of the glass in production was too variable. For example, (a) if the volume of the glass bead was too great, the fused glass would spread across the disk and impede the mounting of the wafer at the next production stage and (b) if the volume was too small, surface tension would pull the glass to one side leaving a gap in the seal meaning the seal was not airtight. This is a two-sided failure mode illustrated in Figure 2 and is the main focus of this paper.

Figure 2. Figure reproduced with permission from the Chartered Quality Institute. This figure first appeared in the January 1998 edition of Quality World, in the article “Looking Back” by Jim Morrison.

Both failure modes are caused by noise factors. In the glass cracking case, the noise factor is an environmental variable (temperature), and in the excessive glass volume variability case, the noise factor is variation in the design variables which is transmitted to the volume.

The countermeasure for the glass cracking was to make the glass from a special borosilicate composition whose expansion characteristics matched those of the metal it was sealing, so that the seal would be stress free (in other words robust to temperature).

With regard to the variability that is transmitted from an input variable $ x $ say, to an output variable $ y $ say, is given by the expression $ {\sigma}_y^2\hskip1.5pt \approx \hskip1.5pt \frac{\partial y}{\partial x}{\sigma}_x^2 $, where the derivative of the objective function is evaluated at the point of interest in the $ x $ space (usually the nominal or target value, such that when $ x $ takes this value, the required output value $ y $ is achieved. The expression is of course exact if the relationship between $ y $ and $ x $ is linear, and will provide an adequate approximation if $ y=f(x) $ is roughly linear in a near-neighborhood of $ x. $⁷

When there is more than one design variable, there usually is, say $ {\left\{\left.{x}_i\right\}\right.}_{i=1}^k $. In the glass bead example, there are three design variables, $ {\left\{\left.{x}_i\right\}\right.}_{i=1}^3=\left\{D,B,L\right\} $, and so the variance transmission formula is (assuming the $ {x}_i $s are independent⁸)

(2)$$ {\sigma}_y^2={\sum}_{i=1}^3{\left(\frac{\partial y}{\partial {x}_i}\right)}^2.{\sigma}_{x_i}^2, $$

where the derivatives are calculated at the nominals for the $ {x}_i $s. we take the nominal values as the mean values in Table 1. The partial derivatives are easily derived and so equation (2) (with $ V $ taking the role of $ y $) becomes

(3)$$ {\sigma}_V^2={\left(\frac{\partial V}{\partial D}\right)}^2{\sigma}_D^2+{\left(\frac{\partial V}{\partial B}\right)}^2{\sigma}_B^2+{\left(\frac{\partial V}{\partial L}\right)}^2{\sigma}_L^2, $$

(3a)$$ {\sigma}_V^2={\left(\frac{\pi DL}{2}\right)}^2{\sigma}_D^2+{\left(\frac{\pi BL}{2}\right)}^2{\sigma}_B^2+{\left(\frac{\pi \left({D}^2-{B}^2\right)}{4}\right)}^2{\sigma}_L^2, $$

so

(3b)$$ {\sigma}_V^2=\left(25.98\times 0.00125\right)+\left(3.55\times 0.00254\right)+\left(3.75\times 0.00536\right), $$

and finally,

(3c)$$ {\sigma}_V^2=0.0325+0.0090+0.0201=0.0617\ {\mathrm{mm}}^6. $$

Note that each derivative has units mm⁴ So that these equations are dimensionally consistent.

Observe that it is the diameter $ D $ that accounts for most of the variability transmitted to $ V $, and not $ L $ even though $ {\sigma}_L^2>{\sigma}_D^2 $. This is because the gradient in the direction of $ D $ (25.98) is much greater than in the direction $ L $ (3.75). This example shows that even in such a simple case, the “obvious” solution of attacking the design variable with the largest variance is wrong. It can only be speculated as to how intuition might let us down in more complicated examples.

Before this analysis was done by Jim Morrison, The British Thomson-Houston Company, Ltd. (which was part of GE) was planning to spend a lot of money investing in a new cutting machine to improve the accuracy of the cut length (L) of the beads.

It turned out that the reduction in the variance of the diameter was easier to achieve than reduction in the variance of the length because it was observed that D only varied gradually along a cut length of glass tubing of ~1 m in length.

By cutting these longer 1-m lengths into shorter pieces of ~10 cm and producing batches of beads from these shorter lengths, the variability of D within a batch of these 10 cm lengths was reduced which in turn resulted in a large reduction in the variance of the glass bead volume. This clever control plan based on sorting the shorter batches bypassed the need for new cutting equipment to control cut length L, since the glass beads were easily batched according to their origin from the shorter pieces.⁹

The glass bead example nicely illustrates the importance of analyzing transmitted variation in making the right engineering and manufacturing decisions to improve quality and avoid unnecessary cost.

The analysis of transmitted variation was called tolerance design by Taguchi (1986). Jim Morrison came to call this variance synthesis (Morrison, 1998).

Tolerance design is a tertiary design activity—the primary design activity is establishing the basic design concept to deliver the functionality as required by the customer or end-user.

A secondary design activity is choosing the correct nominals for the design variables, so that the objective target is achieved. If the secondary design activity also has the objective of improving design robustness (minimizing transmitted variation), then Taguchi called this parameter design.

In his 1957 paper, Jim Morrison did not pursue a parameter design solution for the glass beads, although a careful reading of the paper did show that he had thought about it: “In situations in which bead volume is a critical factor, the designer can use this analysis as a guide in determining the most suitable proportions of bead for a given application.”

Bisgaard and Ankemann (2005) claim that this is the first reference to parameter design in the literature. So, we look at parameter design in some detail now.

3. A Parameter Design Study of the Glass Beads

The parameter design problem for the glass beads can be stated as minimize the transmitted variation given by (3) subject to $ V=3.72\ \mathrm{mm} $ (equation (1)). Here, we use the Solver function in MS Excel® to determine the solution.

We make some realistic assumptions before proceeding; firstly, we assume a constant coefficient of variation $ \frac{\sigma_{x_i}}{\mu_{x_i}} $ for each of the three design variables $ {\left\{{x}_i\right\}}_{i=1}^3=\left\{D,B,L\right\} $ since in engineering and manufacturing applications, it is often true that the variability and the mean are linked. A general link function is $ {\sigma}_{x_i} \propto \hskip1.5pt {\mu}_{x_i}^p $, where a constant coefficient of variation corresponds to the case $ p=1 $. But other values of $ p $ may be appropriate (see Box, 1988). We begin by assuming that $ p=1 $. The coefficients of variation can be calculated directly from Table 1. Secondly, we add the constraint that the bore dimension B, must be in the interval (0.6−0.65 mm), so that the thin wires can pass through the bead easily. Also, the diameter D clearly needs to be greater than the bore, so D>, and it cannot be too large, otherwise the bead will not locate into the “top hat,” so we arbitrarily set the constraint D ≤ 2 mm.

We take as the starting values or initial design specification the measurements given in Morrison (1957). These are summarized in Table 2.

Table 2. The initial design specification prior to conducting parameter design, with the resulting transmitted variation.

After running the Excel solver with the constraints as specified, the results are as given in Table 3. Note the coefficients of variation are the same in each case per our assumption.

Table 3. The parameter design solution.

Note that the coefficients of variation are as in Table 2 per our assumption, that is, $ {\sigma}_{x_i} \propto \hskip1.5pt {\mu}_{x_i} $.

Note that comparing the postparameter design in Table 3 with the initial specification in Table 2, the transmitted variation has reduced from $ 0.0616 $ to $ 0.0529 $ or about $ 14\% $. This gain is equivalent to reducing the variance of $ D $ in the initial specification by about 20%. Note in Table 3 the changes in the nominal values for $ D $, $ B $, $ \mathrm{and}\ L $ compared to Table 2. Figure 3 illustrates the parameter design solution graphically.

Figure 3. Contours of transmitted variation from bead dimensions $ D $ and $ L $ to volume $ V $ under the assumption that $ {\sigma}_{x_i} \propto \hskip2pt {\mu}_{x_i} $. The red line corresponds to $ V=3.72\ \mathrm{mm} $. In this plot, the bore diameter $ B $ is set to its optimal value of 0.6 mm. The open dot is the initial specification (slightly off the red line because initially $ B\ne 0.6 $), and the solid dot is the parameter design solution.

Note from Figure 3 that the optimal value of the outer diameter D is on the edge of its constraint, so if this constraint could be relaxed, for example, by using a larger “top hat,” a better solution (i.e., a further reduction in $ {\sigma}_V^2 $) could be realized.¹⁰

It has long been recognized that the importance of the assumption regarding the relationship between $ {\mu}_{x_i} $ and $ {\sigma}_{x_i} $ is crucial in parameter design. For example, if we repeat the previous analysis, but now assume that the variances for the bead dimensions are fixed at the values in Table 1 (i.e., $ p=0 $), the parameter design solution is as in Table 4.

Table 4. Parameter design solution assuming that the variances of the bead dimensions are fixed at the values given in Table 2.

Note from Table 4 that the proposed nominals for $ D $ and $ L $ are quite different to the previous solution. This sensitivity of the parameter design method was acknowledged by Morrison (1957) and was discussed more extensively by Box and Fung (1994). The contour plot under the assumption of fixed variances for $ D $, $ B $, $ \mathrm{and}\ L $ is shown in Figure 4.

Figure 4. Contours of the transmitted variation now assuming fixed variances for the bead dimensions, showing the new parameter design solution (solid dot), compared to the previous solution under the assumption of constant coefficient of variation (the solid square, now with $ {\sigma}_V^2=0.068470 $). The initial design specification is shown by the open dot. Note that the contours in Figure 4 are oriented differently compared to those in Figure 3, illustrating the sensitivity of parameter design to underlying assumptions, to the extent that optimal value of the outer diameter $ D $ is now well within the imposed constraint. Clearly in this case, and in general, the functional relationship between $ {\sigma}_{x_i} $ and $ {\mu}_{x_i} $ needs to be carefully established.

Additionally, to the functional relationship between $ {\sigma}_{x_i} $ and $ {\mu}_{x_i} $, we should also check for the effect of a lack of independence between the design variables. Morrison (1957) hints that variables $ B $ and $ D $ were slightly correlated (larger bores tended to result in larger outer diameters). If we denote the correlation between $ B $ and $ D $ by $ {\rho}_{DB} $ an additional term must be added to the variance transmission formula, (3), namely

$$ 2\frac{\partial V}{\partial B}.\frac{\partial V}{\partial D}{\sigma}_B{\sigma}_D{\rho}_{DB}=\frac{\pi^2 BD{L}^2}{2}.{\sigma}_B{\sigma}_D{\rho}_{DB}. $$

Note that with this extra term equation (3) is still dimensionally consistent.

The results in Table 5 illustrate the effects of a weak correlation between $ B $ and $ D $ (say $ {\rho}_{BD}\hskip1.5pt \le \hskip1.5pt 0.3 $). Note that the optimal value for the design variables only change when $ p=0 $, and the transmitted variation to $ V $ increases with $ {\rho}_{BD} $. The orientation of the contours of transmitted variation do not change with $ {\rho}_{DB} $, so we do not show them here.

Table 5. The consequences of a correlation between design variables (in this case $ D $ and $ B $).

Note that when $ p=0 $, the optimal values for D and L depend on $ {\rho}_{DB} $, due to the orientation of the contours, so this highlights that the independence of the design variables needs to be considered in parameter design studies in addition to the considerations emphasized in Box and Fung (1994).

One other approach to deal with correlated variables in a parameter design study is to express the objective function in dimensionless variables using an application of Buckingham’s “Pi theorem” (e.g., see Shen et al., 2014). For the objective function (1), if we take as our dimensionless variables, $ {\pi}_0=\frac{V}{B^3} $, $ {\pi}_1=\frac{D}{B} $, and $ {\pi}_2=\frac{L}{B} $, then $ {\pi}_0\hskip1.5pt \propto \hskip1.5pt \left({\pi}_1^2-1\right).\hskip1.5pt {\pi}_2. $ The dimension of the problem is reduced by one. Note that the two variables which exhibit a correlation now appear as a ratio. Grove and Davis (1992) show how dimensionless variables help with parameter design by exploring this idea while analyzing Taguchi’s celebrated and widely taught Wheatstone Bridge problem (see Box, 2000, Chapter E.3), albeit in that case to deal with an interaction between design variables rather than a correlation.

Determining a value for $ p $ from the summary data for $ \left\{D,B,L\right\} $ regarding the three types of beads discussed in the 1957 paper is inconclusive, but there is some evidence that for $ D $ and $ B $, $ p=0 $, while for $ L, $ $ p=1 $. To conclude the parameter design discussion, we investigate the effect that this hybrid assumption has on the parameter design solution.

Firstly assuming $ {\rho}_{DB}=0 $ to allow for direct comparison to the solutions illustrated in Figures 3 and 4, the resulting contour plot is shown in Figure 5. The optimal value for $ {\sigma}_V^2 $ is now $ 0.44830. $

Figure 5. The parameter design solution with $ {\rho}_{DB}=0 $ and $ p=0 $ for $ D $ and $ B $, and $ p=1 $ for $ L $. The optimal values for $ \left\{D,B,L\right\} $ are $ \left\{\mathrm{2.0,0.6,1.3}\right\} $, with $ {\sigma}_V^2=0.044830 $. As in the previous figures the open dot is the initial bead specification and the parameter design solution by the solid dot.

Note that in Figure 5, the orientation of the contours is similar to Figure 3 when it was assumed that $ =1 $ $ \forall \left\{D,B,L\right\} $, so it is the variance assumption on $ L $ which dictates the re-orientation of the contours observed in Figure 4.

Exploring this hybrid assumption now for the range of values for $ {\rho}_{DB} $ that were considered previously, we find the results in Table 6. Note that although the optimal values of $ \left\{D,B,L\right\} $ do not alter from the assumption of $ p=1 $ for each design variable, the optimal value for $ {\sigma}_V^2 $ is improved from the previous results.

Table 6. Parameter design solution under the assumption $ p=\left\{0,0,1\right\} $ for $ \left\{D,B,L\right\} $ for various values of $ {\rho}_{DB} $.

In summary, we see that for the glass bead study it is the assumption regarding the link between $ {\sigma}_L $ and $ {\mu}_L $ that is crucial for this analysis on three counts—the orientation of the contours and the optimal settings for $ \left\{D,B,L\right\} $ and magnitude of the transmitted variance.

4. In Retrospect

Morrison’s (1957) paper was it seems the first paper to formulate the idea of parameter design although he did not call it that and he did not explore that idea in his paper. Together with the fact that he published the paper in a journal that engineers do not usually read, may explain why Morrison’s early ideas were not picked up in the engineering mainstream until the work of Taguchi became better known. Subsequently George Box and his co-workers at the Center for Quality and productivity improvement at the University of Wisconsin-Madison regularly referenced the 1957 paper in their many publications.

The method was used in the Ford Motor Company (Parry-Jones, 1999; Davis, 2006) as part of a coordinated strategy to embed robust design into engineering practice and was taught in company training programs.

It is hoped that as more data-centric engineering methods (Girolami, 2020) are developed as part of the statistical engineering framework, Morrison’s work with suitable modern adaptions will find continue to find its place in current engineering design challenges. Although the glass bead example is simple to understand, the solution to reducing the transmitted variation is not obvious, which has implications for applications with many more design variables. Recent developments in applying parameter design for situations that rely on a computer model (a “digital-twin”) to evaluate design alternatives, because an explicit objective function is not available due to complexity and high dimension, is given in Shen (2017).

In his 1998 coda to the 1957 paper, Morrison laid out a 11-step process for reducing variability in engineering design. The final step was (my italics).

“The possibility of experimenting with the nominal values of the design parameters … can be explored especially if the variances are not constant but alter with changes in the nominal values.”

The analysis of the glass beads problem presented here illustrates step 11 for the glass beads problem and emphasizes the importance of checking the assumptions regarding the way in which the standard deviation and mean of the design variables may be linked.

The purpose of this paper is to recognize and honor the importance of Jim Morrison’s paper as a foundational paper for statistical engineering, and to investigate the parameter design solution to the glass bead problem that Morrison envisaged but did not actually do. My one regret is that I did not do this sooner, so that I could have shared this work with him.¹¹