Uncertain Risk: The Role and Limits of Quantitative Assessment

doi:10.1017/CBO9780511814662.006

6 - Uncertain Risk: The Role and Limits of Quantitative Assessment

Published online by Cambridge University Press: 05 June 2012

Alison C. Cullen and

Mitchell J. Small

Edited by

Timothy McDaniels and

Mitchell Small

Show author details

Alison C. Cullen: Affiliation:
The Daniel J. Evans School of Public Affairs, University of Washington, Seattle, WA
Mitchell J. Small: Affiliation:
Departments of Civil Engineering & Environmental Engineering and Public Policy, Carnegie Mellon University, Pittsburgh, PA
Timothy McDaniels: Affiliation:
University of British Columbia, Vancouver
Mitchell Small: Affiliation:
Carnegie Mellon University, Pennsylvania

Book contents

Get access

Summary

INTRODUCTION

The assessment and management of uncertain risks is an essential component of science, engineering, medicine, business, law, and other modern disciplines. I. Good argues that risk science based on the evaluation of probabilistic events has been useful in saving and improving lives throughout history, “since the assessment of uncertainty incorporates the idea of learning from experience which most creatures do” (Good, 1959). Learning from experience is only part of the challenge, since many decisions involve new or evolving risk scenarios in which humans have little or no experience. Risk management capabilities must include the ability to respond to risks that are known and well understood, and the ability to detect and anticipate those that are newly emerging (e.g., U.S. EPA SAB, 1995; NRC, 1999; U.S. EPA ORD, 2000). Uncertainty analysis provides a framework for characterizing the position of a problem along this continuum and the implications for appropriate response and management.

Uncertainty about risk arises as a result of a fundamental lack of understanding about how the world works, randomness in the outcome of natural and human processes, and an inability to measure, characterize, or even at times define key quantities of interest that affect, or are affected by, risk. Risks can vary across time, space, and from individual to individual in a population. Characterization of these variations is an essential part of a risk analysis; however, the basic knowledge or data to fully and effectively characterize risk, and its variability, are often lacking.

Type: Chapter
Information: Risk Analysis and Society
An Interdisciplinary Characterization of the Field
, pp. 163 - 212

DOI: https://doi.org/10.1017/CBO9780511814662.006 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2003

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Adlassnig, K.-P. 1986. Fuzzy set theory in medical diagnosis. IEEE Transactions on Systems, Man and Cybernetics, SMC-16: 260–5CrossRef Google Scholar

Andrews, R. 1997. Risk-based decision making. In Environmental Policy in the 1990s, N. Vig and M. Kraft, eds., Congressional Quarterly, 208–230, Washington, DC

Apostolakis, G. E. and Pickett, S. E.. 1998. Deliberation: Integrating analytical results into environmental decision involving multiple stakeholders. Risk Analysis, 18 (5): 621–47CrossRef Google Scholar

Ashby, D., and Hutton, J. L. 1996. Bayesian epidemiology. In D. K. Berry, ed., Bayesian Biostatistics, pp. 109–38. Marcel Dekker, New York

Bankes, S. C. 1993. Exploratory modeling for policy analysis. Operations Research, 41: 435–49CrossRef Google Scholar

Bankes, S. C. 1994. Computational experiments and exploratory modeling. Chance, 7 (1): 50–1, 57Google Scholar

Bedford, T., and R. Cooke. 2001. Probabilistic Risk Analysis: Foundations and Methods, Cambridge University Press, Cambridge

Berry, D. A. and Stangl, D. K., eds. 1996. Bayesian Biostatistics. Marcel Dekker, New York

Bier, V. M., Haimes, Y. Y., Lambert, J. H., Ferson, S., and Small, M. J. 2003. Quantifying risk of extreme or rare events: Lessons from a selection of approaches. This volume.

Bier, V. M., Haimes, Y. Y., Lambert, J. H., Matalas, N. C., and Zimmerman, R. 1999. A survey of approaches for assessing and managing the risk of extremes. Risk Analysis, 19 (1): 83–94CrossRef Google Scholar

Bogen, K. T. 1990. Uncertainty in Environmental Health Risk Assessment, Garland Publishing, New York

Bogen, K. T. 1995. Methods to approximate joint uncertainty and variability in risk. Risk Analysis, 15 (3): 411–19CrossRef Google Scholar

Bostrom, A., Fischhoff, B., and Morgan, M. G. 1992. Characterizing mental models of hazardous processes: A methodology with an application to radon. J. Social Issues, 48 (4): 85–100CrossRef Google Scholar

Boudet, C., Zmirou, D., Laffond, M., Balducci, F., and Benoit-Guyod, J-L. 1999. Health risk assessment of a modern municipal waste incinerator. Risk Analysis, 19: 1215–22CrossRef Google Scholar PubMed

Brasquet, C., Bourges, B., and Cloirec, P. 1999. Quantitative structure-property relationship (QSPR) for the adsorption of organic compounds onto activated carbon cloth: Comparison between multiple linear regression and neural network. Environmental Science and Technology, 33 (23): 4226–31CrossRef Google Scholar

Brattin, W. J., Barry, T. M., and Chiu, N. 1996. Monte Carlo modeling with uncertain probability density functions. Human and Ecological Risk Assessment, 2 (4): 820–40CrossRef Google Scholar

Brieman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. 1984. Classification and regression trees. Wadsworth&Brooks/Cole Advanced Books & Software, Pacific Grove, CA

Browner, C. 1995. Policy for Risk Characterization at the US Environmental Protection Agency, Memorandum to assistant and regional administrators

Bumb, R. R., Crummet, W. B., Cutie, S. S., Gledhill, J. R., Hummel, R. H., Kagel, R. O., Lamparski, L. L., Luoma, E. V., Miller, D. L., Nestrick, T. J., Shadoff, L. A., Stehl, R. H., and Woods, J. S. 1980. Trace chemistries of fire: A source of chlorinated dioxins. Science 210 (4468): 385–90CrossRef Google Scholar PubMed

Burmaster, D. E., and Thompson, K. M. 1997. Estimating exposure point concentrations for surface soils for use in deterministic and probabilistic risk assessments. Human and Ecological Risk Assessment, 3 (3): 363–84CrossRef Google Scholar

Burmaster, D. E., and Thompson, K. M. 1998. Fitting second-order parametric distributions to data using maximum likelihood estimation. Human and Ecological Risk Assessment, 4 (2): 319–39CrossRef Google Scholar

Burmaster, D. E., and Wilson, A. M. 1996. An introduction to second-order random variables in human health risk assessment. Human and Ecological Risk Assessment, 2 (4): 892–919CrossRef Google Scholar

Cao, H., and Chen, G. 1983. Some applications of fuzzy sets to meteorological forecasting. Fuzzy Sets and Systems, 9: 1–12CrossRef Google Scholar

Carrington, C. D. 1996. Logical probability and risk assessment. Human and Ecological Risk Assessment, 2: 62

Carrington, C. D. 1997. An administrative view of model uncertainty in public health. Risk, 8: 273ff. See also: http://www.fplc.edu/risk/vol8/summer/Carringt.htm

Cash, D. W. 2000. Distributed assessment systems: An emerging paradigm of research, assessment, and decision-making for environmental change. Global Environmental Change, 10 (4): 241–4CrossRef Google Scholar

Casman, E. A., Morgan, M. G., and Dowlatabadi, H. 1999. Mixed levels of uncertainty in complex policy models. Risk Analysis, 19 (1): 33–42CrossRef Google Scholar

Chaloner, K. 1996. Elicitation of prior distributions. In D. K. Berry Stangl, ed., Bayesian Biostatistics, pp. 141–56. Marcel Dekker, New York

Charniak, E. 1991. Bayesian networks without tears. AI Magazine, 12 (4): 50–63Google Scholar

Clark, W. C., and Dickson, N. M. 1999. The global environmental assessment project: Learning from efforts to link science and policy in an interdependent world. Acclimations, 8: 6–7. http://www.nacc.usgcrp.gov/ewsletter/Google Scholar

Clarke, L. 1988. Politics and bias in risk assessment. The Social Science Journal, 15: 155–65CrossRef Google Scholar

Cooke, R. M. 1991. Experts in Uncertainty: Opinion and Subjective Probability in Science. Oxford Press, New York

Costello, C. J., Adams, R. M., and Polasky, S. (1998). The value of El Niño forecasts in the management of salmon: A stochastic dynamic assessment. American Journal of Agricultural Economics, 80: 765–77CrossRef Google Scholar

Covello, V. T., and Mumpower, J. 1985. Risk analysis and risk management: A historical perspective. Risk Analysis, 5: 103–20CrossRef Google Scholar

Cox, D. C., and Baybutt, P. 1981. Methods for uncertainty analysis: A comparative study. Risk Analysis, 1: 251–8CrossRef Google Scholar

CRAM Presidential/Congressional Commission on Risk Assessment and Risk Management. 1997. Risk Assessment and Risk Management in Regulatory Decision Making. Final Report. Washington, DC

Cullen, A. C. 1994. Measures of conservatism in probabilistic risk assessment. Risk Analysis, 14: 389–93CrossRef Google Scholar PubMed

Cullen, A. C. 1995. The sensitivity of Monte Carlo simulation results to model assumptions: The case of municipal solid waste combustor risk assessment. Journal of Air Waste Management Association, 45: 538–46CrossRef Google Scholar

Cullen, A. C., and Eschenroeder, A. Q. 1997. Coping with muncipal waste. In J. Graham and J. Hartwell eds., The Greening of Industry. Harvard University Press, Cambridge, MA

Cullen, A. C., and Frey, H. C. 1999. Probabilistic Techniques in Exposure Assessment: A Handbook for Dealing with Variability and Uncertainty in Models and Inputs. Plenum Press, New York

Dakins, M. E., Toll, J. E., Small, M. J., and Brand, K. P. 1996. Risk-based environmental remediation: Bayesian Monte Carlo analysis and the expected value of sample information. Risk Analysis, 16 (1): 67–79CrossRef Google Scholar PubMed

De Finetti, B. 1937. La Prevision: Ses Lois Logiques, Ses Sources Subjectives. Annales de L'Institut Henri Poincaré, 7 (1). pp. 1–68. English translation in H. Kyburg and H. Smokler, eds. 1964. Studies in Subjective Probability. Wiley, New York

DeGroot, M. H. 1969. Optimal Statistical Decisions, McGraw-Hill Book Company, New York

Dourson, M. L. and Stara, J. F. 1983. Regulatory history and experimental support of uncertainty (safety factors). Regulatory Toxicology and Pharmacology, 3: 224–38CrossRef Google Scholar

Draper, D. 1995. Assessment and propagation of model uncertainty. Journal Royal Statistical Society, Series B, 57 (1): 45–97Google Scholar

Eddy, D. M., Hasselblad, V., and Shachter, R. 1990. An introduction to a Bayesian method for meta-analysis: The confidence profile method. Medical Decision Making, 10 (1): 15–23CrossRef Google Scholar PubMed

Eisenberg, J. N. S., and McKone, T. E. 1998. Decision tree method for the classification of chemical pollutants: Incorporation of across-chemical variability and within-chemical uncertainty. Environmental Science and Technology, 32 (21): 3396–404CrossRef Google Scholar

Evans, J. S. 1985. The value of improved exposure estimates: A decision analytic approach, Proceedings of the 78th Annual Meeting of the Air Pollution Control Association (June 16–21, 1985), Detroit, Michigan

Evans, J. S., Graham, J. D., Gray, G. M., and Sielken, R. L. 1994a. A distributional approach to characterizing low-dose cancer risk. Risk Analysis, 14 (1): 25–34CrossRef Google Scholar

Evans, J. S., Gray, G. M., Sielken, R. L., Smith, A. E., Valdez-Flores, C., and Graham, J. D. 1994b. Use of probabilistic expert judgment in distributional analysis of carcinogenic potency, Reg Tox and Pharm, 20: 15–36CrossRef Google Scholar

Evans, J., Hawkins, N., and Graham, J. 1988. The value of monitoring for radon in the home: A decision analysis, Journal of the Air Pollution Control Association, 38 (11): 1380–5Google Scholar PubMed

Evans, J. S. 1985. The value of improved exposure estimates: A decision analytic approach, Proceedings of the 78thAnnual Meeting of the Air Pollution Control Association, (June 16–21, 1985), Detroit, Michigan

Ferson, S. 1996. What Monte Carlo methods can not do. Human and Ecological Risk Assessment, 2: 990–1007CrossRef Google Scholar

Ferson, S., Ginzburg, L., and Akcakaya, R. Whereof one cannot speak: When input distributions are unknown. Unpublished (apparently) Risk Analysis

Finkel, A. 1990. Confronting Uncertainty in Risk Management: A Guide for Decision Makers. Center for Risk Management, Resources for the Future, Washington, DC

Finkel, A. M., and Evans, J. S. 1987. Evaluating the benefits of uncertainty reduction in environmental health risk management. J Air Pollution Control Association, 37: 1164–71Google Scholar PubMed

Fischhoff, B., Riley, D., Kovacs, D. C., and Small, M. 1998. What information belongs in a warning?Psychology and Marketing, 15 (7): 663–863.0.CO;2-D>CrossRef Google Scholar

Fischhoff, B., Slovic, P., and Lichtenstein, S. 1982. Lay foibles and expert fables in judgments about risk. American Statistician, 36: 240–55Google Scholar

Freudenburg, W. 1988. Perceived risk, real risk: Social science and the art of probabilistic risk assessment. Science, 241: 44–9CrossRef Google Scholar

Frey, H. C. 1992. Quantitative Analysis of Uncertainty and Variability in Environmental Policy Making. American Association for the Advancement of Science, Washington, DC

Frey, H. C., and Burmaster, D. F. 1999. Methods for characterizing variability and uncertainty: Comparison of bootstrap simulation and likelihood-based approaches. Risk Analysis, 19 (1): 109–30CrossRef Google Scholar

Frey, H. C., and Rhodes, D. S. 1996. Characterization and simulation of uncertain frequency distributions: Effects of distribution choice, variability, uncertainty, and parameter dependence. Human and Ecological Risk Assessment, 4 (2): 423–68CrossRef Google Scholar

Gee. D. 1997. Criteria for Managing Uncertainty and Regulation in Public Policy. Draft manuscript

Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. 1995. Bayesian Data Analysis. Chapman & Hall, London

Gentle, J. 1998. Random Number Generation and Monte Carlo Methods. Springer-Verlag, Heidelberg

Giering, E. W., and Kandel, A. 1983. The application of fuzzy set theory to the modeling of competition in ecological systems. Fuzzy Sets and Systems, 9: 103–27CrossRef Google Scholar

Goldfarb, T. D. 2000. Environmental Studies. Dushkin McGraw-Hill, Guilford, CT

Good, I. J. 1959. Kinds of probability. Science, 129: 443–7CrossRef Google Scholar

Gurian, P. L., Small, M. J., Lockwood III, J. R., and Schervish, M. J. 2001a. Benefit-cost estimation for alternative drinking water maximum contaminant levels. Water Resources Research, 37 (9): 2213–26CrossRef Google Scholar

Gurian, P. L., Small, M. J., Lockwood III, J. R., and Schervish, M. J. 2001b. Addressing uncertainty and conflicting cost estimates in revising the arsenic MCL. Environmental Science & Technology, 35 (22): 4414–20CrossRef Google Scholar

Haas, C. N. 1999. On modeling correlated random variables in risk assessment. Risk Analysis, 19 (6): 1205–14CrossRef Google Scholar PubMed

Habicht, H. 1992. Guidance on Risk Characterization for Risk Managers and Risk Assessors. USEPA memorandum to Assistant and Regional Administrators

Haimes, Y. Y., and Lambert, J. 1994. When and how can you specify a probability distribution when you don't know much, Risk Analysis, 14 (5): 661–706CrossRef Google Scholar

Hammitt, J. K., and Shlyakhter, A. I. 1999. The expected value of information and the probability of surprise. Risk Analysis, 19 (1): 135–52CrossRef Google Scholar

Henrion, M. 1982. The value of knowing how little you know: The advantages of probabilistic treatment in policy analysis, Ph.D. dissertation, Carnegie-Mellon University, Pittsburgh

Henrion, M., and Fischhoff, B. 1986. Assessing uncertainty in physical constants. American Journal of Physics, 54 (9): 791–8CrossRef Google Scholar

Hertwich, E. G., McKone, T. E., and Pease, W. S. 1999. Parameter uncertainty and variability in evaluative fate and exposure models. Risk Analysis, 19: 1193–1204CrossRef Google Scholar PubMed

Hilton, R. W. 1981. The determinants of information value: Synthesizing some general results. Management Science, 27 (1): 57–64CrossRef Google Scholar

Hoffman, F. O., and Hammonds, J. S. 1994. Propagation of uncertainty in risk assessments: The need to distinguish between uncertainty due to lack of knowledge and uncertainty due to variability. Risk Analysis, 14 (5): 707–12CrossRef Google Scholar PubMed

Hunt, S., Frewer, L. J., and Shepard, R. 1999. Public trust in sources of information about radiation risks in the UK. Journal of Risk Research, 2: 167–80CrossRef Google Scholar

Iman, R. L., and Conover, W. J. 1982. A distribution-free approach to inducing rank correlation among input variables. Communications in Statistics, B11 (3): 311–34CrossRef Google Scholar

Iman, R. L., and Hora, S. C. 1989. Bayesian methods for modeling recovery times with an application to the loss of off-site power at nuclear power plants. Risk Analysis, 9 (1): 25–36CrossRef Google Scholar

IPCC (Intergovernmental Panel on Climate Change). 1995. Impacts, Adaptations and Mitigation of Climate Change: Scientific-Technical Analyses Contribution of Working Group II to the Second Assessment of the Intergovernmental Panel on Climate Change, R. T. Watson, M. C. Zinyowera, and R. H. Moss, eds. Cambridge University Press, Cambridge

IPCC (Intergovernmental Panel on Climate Change). 2001a. Climate Change 2001: The Scientific Basis. See http://www.ipcc.ch/pub/tar/

IPCC (Intergovernmental Panel on Climate Change). 2001b. Climate Change 2001: Impacts, Adaptation and Vulnerability. See http://www.ipcc.ch/pub/tar/

IPCC (Intergovernmental Panel on Climate Change). 2001c. Climate Change 2001: Mitigation. See http://www.ipcc.ch/pub/tar/

Isukapalli, S. S., and Georgopoulos, P. G. 2001. Computational Methods for Sensitivity and Uncertainty Analysis for Environmental and Biological Models. Project Report to U.S. EPA Office of Research and Development (ORD) National Exposure Research Laboratory. EPA/600/R-01-068. Research Triangle Park, NC

Jasanoff, S. 1987. Cultural aspects of risk assessment in Britain and the United States. In B. B. Johnson and V. T. Covello, eds., The Social and Cultural Construction of Risk, pp. 359–97. Reidel, Dordrecht, Netherlands

Jasanoff, S. 1990. The Fifth Branch, Science Advisers as Policymakers. Harvard University Press, Cambridge, MA

Jasanoff, S. 1993. Bridging the two cultures of risk analysis. Risk Analysis, 13: 123–9CrossRef Google Scholar

Johnson, B. B., and Slovic, P. 1995. Presenting uncertainty in health risk assessment: Initial studies of its effects on risk perception and trust. Risk Analysis, 15: 485–94CrossRef Google Scholar PubMed

Johnson, R., Pankow, J., Bender, D., Price, C., and Zogorski, J. 2000. MTBE: To what extent will past releases contaminate community water supply wells?ES&T, 34 (9): 210A–217AGoogle Scholar PubMed

Julien, B., Fenves, S. J., and Small, M. J. 1992. Knowledge acquisition methods for environmental evaluation. AI Applications, 6: 1–20Google Scholar

Kadane, J. B., and Wolfson, L. J. 1998. Experiences in elicitation. Journal of the Royal Statistical Society, Series D, 47: 3–19 (with discussion)CrossRef Google Scholar

Kahneman, D., Slovic, P., and Tversky, A. 1982. Judgment under Uncertainty: Heuristics and Biases. Cambridge University Press, New York

Kalagnanam, J., and Diwekar, U. 1997. An efficient sampling technique for off-line quality control, Technometrics, 38: 308

Kammen, D. M., and Hassenzahl, D. M. 1999. Should We Risk It? Exploring Environmental, Health, and Technological Problem Solving, Princeton University Press, Princeton, NJ

Kasperson, R. E., and Kasperson, J. X. 1996. The social amplification and attenuation of risk. Annals of the American Academy of Political and Social Sciences, 545: 95–105CrossRef Google Scholar

Kasperson, R., Renn, O., Slovic, P., Brown, H., Emel, J., Goble, R., Kasperson, J., and Ratick, S. 1988. The social amplification of risk: A conceptual framework. Risk Analysis, 8: 177–87CrossRef Google Scholar

Keeney, R. L. 1982. Decision analysis: An overview. Operations Research, 39: 803–38CrossRef Google Scholar

Klir, G. J., and Folger, T. A. 1988. Fuzzy Sets, Uncertainty, and Information, Prentice Hall, Englewood Cliffs, NJ

Larkin, L. I. 1985. A fuzzy logic controller for aircraft flight control. In M. Sugeno, ed., Industrial Applications of Fuzzy Control. North-Holland, New York

Lave, L. 1978. “Eight Frameworks of Regulation: The Strategy of Social Regulation,” Brookings Institute, pp. 8–28

Lave, L. B., Resendiz-Carrillo, D., and McMichael, F. C. 1990. Safety goals for high-hazard dams: Are dams too safe?Water Resources Research, 26 (7): 1383–91Google Scholar

Lee, P. M. 1989. Bayesian Statistics: An Introduction, Oxford University Press, Oxford

Lempert, R., Schlesinger, M., and Bankes, S. 1996. When we don't know the costs or the benefits: Adaptive strategies for abating climate change. Climactic Change, 33: 235–74CrossRef Google Scholar

Levin, A., Fratt, D. B., Leonard, A., et al.1991. Comparative analysis of health risk assessments for muncipal waste combustors. Journal of the Air and Waste Management Association, 41 (1): 20–31CrossRef Google Scholar

Lewis, H. W., Budnitz, R. J., Kouts, H. J. C., Lowenstein, W. B., Rowe, W. D., Von Hippel, F., and Zachariasen, F. 1979. Risk Assessment Review Group Report to theU.S.Nuclear Regulatory Commission. NUREG/CR-0400

Lichtenstein, S., and Fischhoff, B. 1977. Do those who know more also know more about how much they know?Organizational Behavior and Human Performance, 20: 159–83CrossRef Google Scholar

Lockwood III, J. R., Schervish, M. J., Gurian, P., and Small, M. H. 2001. Characterization of arsenic occurrence in US drinking water treatment facility source waters. Journal of American Statistical Association, 96: 456, pp. 1184–1193CrossRef Google Scholar

MacKenzie, D. 1990. Inventing Accuracy: A Historical Sociology of Nuclear Missile Guidance. MIT Press, Cambridge, MA

March, J. G., and Simon, H. A. 1958. Organizations. John Wiley and Sons, New York

McElhany, P., Ruckelshaus, M., Ford, M. J., Wainwright, T., and Bjorkstedt, E. 2000. Viable Salmonid Populations and the Recovery of Evolutionarily Significant Units. National Marine Fisheries Service

McKay, M. D., Conover, W. J., and Beckman, R. J. 1979. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics, 21 (2): 239–45Google Scholar

McKone, T. E., and Bogen, K. T. 1992. Uncertainties in health-risk assessment: An integrated case study based on tetrachloroethylene in California groundwater. Regulatory Toxicology and Pharmacology, 15: 86–103CrossRef Google Scholar PubMed

Merkhofer, M. W. 1987. Quantifying judgmental uncertainty: Methodology, experiences, and insights. IEEE Transactions on Systems, Man, and Cybernetics. 17 (5): 741–52CrossRef Google Scholar

Merz, J., Small, M. J., and Fischbeck, P. 1992. Measuring decision sensitivity: A combined Monte Carlo-logistic regression approach. Medical Decision Making, 12: 189–96CrossRef Google Scholar PubMed

Morgan, M. G., and Dowlatabadi, H. 1996. Learning from integrated assessment of climate change. Climatic Change, 34 (3–4): 337–68CrossRef Google Scholar

Morgan, M. G., and Henrion, M. 1990. Uncertainty: A Guide for Dealing with Uncertainty in Quantitative Risk and Policy Analysis. Cambridge University Press, Cambridge

Morgan, M. G., Fischhoff, B., Bostrom, A., Lave, L., and Atman, C. 1992. Communicating risk to the public. Environmental Science and Technology, 26 (11): 2048–56Google Scholar

Morgan, M. G., Henrion, M., and Morris, S. C. 1980, Expert Judgment for Policy Analysis. BNL 51358. Brookhaven National Laboratory. Brookhaven, NY

Morokoff, W. J., and Caflisch, R. E. 1994. Quasi-random sequences and their discrepancies. SIAMJournal on Scientific Computing, 15 (6): 1251–79CrossRef Google Scholar

Nakatsuyama, M., Nagahashi, H., and Nishizuka, N. 1984. Fuzzy logic phase controller for traffic functions in the one-way arterial road. ProceedingIFAC9thTriennial World Congress, pp. 2865–70, Pergamon Press, Oxford

NAS. 1987. Report on the Delaney Paradox. National Academics Press. Washington, DC

NCRP. 1996. A Guide for Uncertainty Analysis in Dose and Risk Assessments Related to Environmental Contamination. National Council on Radiation Protection and Measurements. NCRP Commentary No. 14. Bethesda, MD

Norsys. 1998. Netica™ Application for Belief Networks and Influence Diagrams: User's Guide, Versions 1.03 for Macintosh and 1.05 for Windows. Norsys Software Corporation, Vancouver, BC, Canada

NRC. 1975. Reactor Safety Study: An Assessment of Accident Risks inU.S.Commercial Nuclear Power Plants. WASH-1400/NUREG 751014. U.S. Nuclear Regulatory Commission, Washington, DC

NRC. 1983. Risk Assessment in the Federal Government: Managing the Process (also known as “The Red Book”). National Academy Press, Washington, DC

NRC. 1994. Science and Judgement in Risk Assessment. National Academy Press, Washington, DC

NRC. 1996. Understanding Risk: Informing Decisions in a Democratic Society. National Academy Press, Washington, DC

NRC. 1999. Our Common Journey: A Transition Toward Sustainability, Board on Sustainable Development, National Research Council, National Academy Press, Washington, DC

NRC. 2000. Incorporating Science, Economics, and Sociology in Developing Sanitary and Phytosanitary Standards in International Trade, Proceedings of a Conference. National Research Council, National Academy Press, Washington, DC

Pate, E. 1983. Acceptable decision processes and acceptable risks in public sector regulations. IEEE Transactions on Systems, Man, and Cybernetics, SMC-13: 113–24CrossRef Google Scholar

Pate-Cornell, E. M. 1996. Uncertainties in risk analysis: Six levels of treatment. Reliability Engineering and System Safety, 54: 95–111CrossRef Google Scholar

Patt, A. 1999. Extreme outcomes: The strategic treatment of low probability events in scientific assessments. Risk, Decision, and Policy, 4 (1): 1–15CrossRef Google Scholar

Pearl, J. 1988. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufman, San Mateo, CA

Peters, R. G., Covello, V. T., and McCallum, D. B. 1997. The determinants of trust and credibility in environmental risk communication: An empirical study. Risk Analysis, 17: 43–54CrossRef Google Scholar

Petts, J. 1998. Trust and waste management information expectation versus observation. Journal of Risk Research, 1: 307–20CrossRef Google Scholar

Press, S. J. 1989. Bayesian Statistics: Principles, Models, and Applications. Wiley, New York

Raffensperger, C., and Tickner, J. A., eds. 1999. Protecting Public Health and the Environment – Implementing the Precautionary Principle. Island Press, Washington, DC

Raiffa, H. 1968. Decision Analysis: Introductory Lectures on Choices Under Uncertainty. Addison-Wesley Publishing, Reading, MA

Raiffa, H., and Schlaifer, R.O. 1961. Applied Statistical Decision Theory, Harvard University Press, Cambridge, MA

Ramsey, F. P. 1931. The Foundation of Mathematics and Other Logical Essays. Kegan Paul, London

Renn, O., Burns, W. J., Kasperson, J. X., Kasperson, R. E., and Slovic, P. 1992. The social amplification of risk: Theroetical foundations and empirical applications. Journal of Social Issues, 48: 127–60CrossRef Google Scholar

Shlyakhter, A. I. 1994. “Improved Framework for Uncertainty Analysis: Accounting for Unsuspected Errors,” Risk Analysis, 14: 441–7CrossRef Google Scholar

Simon, T. W. 1999. Two dimensional Monte Carlo simulation and beyond: A comparison of several probabilistic risk assessment methods applied to a Superfund site. Human and Ecological Risk Assessment, 5 (4): 823–43CrossRef Google Scholar

Slovic, P. 1993. Perceived risk, trust, and democracy. Risk Analysis, 13: 675–82CrossRef Google Scholar

Slovic, P., Fischoff, B., and Lichtenstein, S. 1979. Rating the risks. Environment, 21: 14–20, 36–9Google Scholar

Small, M. J., and Fischbeck, P. F. 1999. False precision in Bayesian updating with incomplete models. Human and Ecological Risk Assessment, 5 (2): 291–304CrossRef Google Scholar

Smith, A. E., Ryan, P. B., and Evans, J. S. 1992. The effect of neglecting correlations when propagating uncertainty and estimating the population distribution of risk. Risk Analysis, 12: 467–74CrossRef Google Scholar

Solow, A. R., Adams, R. F., Bryant, K. J., Legler, D. M., O'Brien, J. J., McCarl, B. A., Nayda, W., and Weiher, R. 1998. The value of improved ENSO prediction to U.S. agriculture. Climatic Change, 39: 47–60CrossRef Google Scholar

Song, X.-H., and Hopke, P. K. 1996. Solving the chemical mass balance problem using an artificial neural network. Environmental Science & Technology, 30 (2): 531–5CrossRef Google Scholar

Spensley, J. W. 1995. National Environmental Policy Act. In T. F. P. Sullivan, ed., Environmental Law Handbook, 13th ed., Chapter 10. Government Institutes, Rockville, MD

Spetzler, C. S., and Holstein, S. 1975. Probability encoding in decision analysis. Management Science, 22 (3): 340–58CrossRef Google Scholar

Squillace, P. J., Zogorski, J. S., Wilber, W. G., and Price, C. V. 1996. Preliminary assessment of the occurrence and possible sources of MTBE in groundwater in the United States, 1993–1994. Environmental Science and Technology, 30 (5): 1721–30CrossRef Google Scholar

Stiber, N. A., Pantazidou, M., and Small, M. J. 1999. Expert system methodology for evaluating reductive dechlorination at TCE sites. ES&T, 33 (17): 3012–20Google Scholar

Taylor, A. C. 1993. Using objective and subjective information to generate distributions for probabilistic exposure assessment. Journal of Exposure Analysis and Environmental Epidemiology, 3: 285–98Google Scholar

Taylor, A., Evans, J., and McKone, T. 1993. The value of animal test information in environmental control decisions. Risk Analysis, 13: 403–12CrossRef Google Scholar PubMed

Tezuka, S. 1995. Uniform Random Numbers: Theory and Practice. Kluwer Academic Publishers, Dordrecht

Thompson, K. M. 1999. Developing univariate distributions from data for risk analysis. Human and Ecological Risk Assessment, 5 (4): 755–83CrossRef Google Scholar

Thompson, K. M., and Evans, J. S. 1997. The value of improved exposure information for perchlorethylene (Perc): A case study for dry cleaners. Risk Analysis, 17 (2): 253–71CrossRef Google Scholar

Thompson, K. M., and Graham, J. D. 1996. Going beyond the single number: Using probabilistic risk assessment to improve risk management. Human and Ecological Risk Assessment, 2 (4): 1008–34CrossRef Google Scholar

Thompson, P. B., and Dean, W. 1996. Competing conceptions of risk. Risk: Health, Safety & Environment, 7: 361–84Google Scholar

Tickner, J. A. 1999. A map toward precautionary decision making. In C. Raffensperger and J. A. Tickner, eds., Protecting Public Health and the Environment – Implementing the Precautionary Principle. Island Press, Washington, DC

UCC Commission for Racial Justice. 1987. Toxic Wastes and Race in the United States. United Church of Christ and Public Data Access, New York

UK-ILGRA. 1996. Use of Risk Assessment within Government Departments. UK Health and Safety Executive, HSE Books, Sudbury, Suffolk, UK. See also http://www.hse.gov.uk/dst/ilgra/minrpt1.htm

UK-ILGRA. 1999. Risk Assessment and Risk Management – Improving Policy and Practice within Government Departments. UK Health and Safety Executive, HSE Books, Sudbury, Suffolk, UK

U.S. CEQ. 1971. Environmental Quality. GPO, Washington, DC

U.S. EPA Science Advisory Board. 1995. Beyond the Horizon: Using Foresight to Protect the Environmental Future. USEPA Science Advisory Board, Washington, DC

U.S. EPA National Center for Environmental Assessment. 1997a. Exposure Factors Handbook, NTIS: PB98-124217, See also http://www.epa.gov/ncea/exposfac.htm, Washington, DC

U.S. EPA Science Policy Council. 1997b. Proposed Policy for Use of Monte Carlo Analysis in Agency Risk Assessment. Memorandum of William P. Wood, Executive Director, Risk Assessment Forum, to Dorothy E. Patton, Executive Director of Science Policy Council (8104), January 29, 1997

U.S. EPA Risk Assessment Forum. 1998. Report of the Workshop on Selecting Input Distributions for Probabilistic Assessments. Prepared by Eastern Research Group, Inc., for EPA under EPA Contract No. 68-D5-0028, Washington, DC

U.S. EPA Risk Assessment Forum. 1999. Report of the Workshop on Selecting Input Distributions for Probabilistic Assessments USEPA 630/R-98/004. January 1, 1999. U.S. EPA Risk Assessment Forum, Washington, DC

U.S. EPA Office of Research and Development. 2000. Exploratory Research to Anticipate Future Environmental Issues. FY 2000 STAR Program Request for Applications. U.S. EPA ORD National Center for Environmental Research, Washington, DC. http://es.epa.gov/ncerqa/rfa/explfuturefnl.html

Winkler, R. L. 1967. The assessment of prior distributions in Bayesian analysis. Journal of the American Statistical Association, 62: 776–800CrossRef Google Scholar

Wiser, G. 2002. Analysis and perspective: Kyoto Protocol packs powerful implementation punch. International Reporter Current Report, 25 (2): 86. See http://www.ciel.org/Publications/INER_Compliance.pdf

Wolfson, L. J. 1995. Elicitation of Priors and Utilities for Bayesian Analysis. PhD Thesis, Department of Statistics, Carnegie Mellon University, Pittsburgh

Wolfson, L. J., Kadane, J. B., and Small, M. J. 1996. Bayesian environmental policy decisions: Two case studies. Ecological Applications, 6 (4): 1056–66CrossRef Google Scholar

Wolpert, R. L. 1989. Eliciting and combining subjective judgments about uncertainty. International Journal of Technology Assessment in Health Care, 5 (4): 537–57CrossRef Google Scholar PubMed

Wood, E. F., and Rodriguez-Iturbe, I. 1975. Bayesian inference and decision making for extreme hydrologic events. Water Resources Research, 11 (4): 533–42CrossRef Google Scholar

Word, C. J., Harding, A. K., Bilyard, G. R., and Weber, J. R. 1999. Basic science and risk communication: A dialogue-based study. Risk: Health Safety and Environment, 10 (2), 231–42Google Scholar

World Watch Institute. 2000. State of the World. Norton, New York

Wynne, B. 1980. Technology, risk and participation: The social treatment of uncertainty. In J. Conrad, ed., Society, Technology and Risk, pp. 83–107. Academic Press, New York

Wynne, B. 1987. Risk Management and Hazardous Wastes: Implementation and the Dialectics of Credibility. Springer, Berlin

Bedford, T., and Cooke, R. 2001. Probabilistic Risk Analysis: Foundations and Methods, Cambridge University Press, Cambridge

Covello, V. T., and Merkhofer, M. W. 1993. Risk Assessment Methods: Approaches for Assessing Health and Environmental Risks. Plenum Press, New York

Crawford-Brown, D. J. 1999. Risk-Based Environmental Decisions: Methods and Culture. Kluwer Academic Publishers, Boston

Cullen, A. C., and Frey, H. C. 1999. Probabilistic Techniques in Exposure Assessment: A Handbook for Dealing With Variability and Uncertainty in Models and Inputs. Plenum Press, New York

Jaeger, C. C., Renn, O., Rosa, E. A., and Webler, T. 2001. Rational Action, Risk, and Uncertainty. Earthscan, London

Kadane, J. B., and Wolfson, L. J. 1998. Experiences in elicitation. Journal of the Royal Statistical Society D, 47: 3–19 (with discussion)CrossRef Google Scholar

Kammen, D. M., and Hassenzahl, D. M. 1999. Should We Risk It? Exploring Environmental, Health, and Technological Problem Solving, Princeton University Press, Princeton, NJ

Klapp, M. G. 1992. Bargaining With Uncertainty: Decision-Making in Public Health, Technologial Safety, and Environmental Quality. Auburn House, Dover, MA

Morgan, M. G., and Henrion, M. 1990. Uncertainty: A Guide for Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press, Cambridge

Porter, T. M. 1995. Trust in Numbers: The Pursuit of Objectivity in Science and Public Life, Princeton University Press, Princeton, NJ

Slovic, P. 2000. The Perception of Risk. Earthscan, London

Stirling, A. 1999. On Science and Precaution in the Management of Technological Risk: Volume I. A Synthesis Report of Case Studies. Report EUR 19056/EN. European Commission for Prospective Technological Studies, Seville, Spain

Stirling, A. 2002. On Science and Precaution in the Management of Technological Risk: Volume II. Case Studies. Report EUR 19056/EN/2. European Commission for Prospective Technological Studies, Seville, Spain

Warren-Hicks, W. H., and Moore, D. R. J., eds. 1998. Uncertainty Analysis in Ecological Risk Assessment. Society of Environmental Toxicology and Chemistry (SETAC), Pensacola, FL