10 - Assessing and Reporting Uncertainty: Some Alternative Methods

Summary

INTRODUCTION

We have now considered the major sources of uncertainty related to evidence and have shown how uncertainty arises in chains of reasoning linking evidence to hypotheses we entertain. A major credential of evidence we have only mentioned briefly is its inferential force or weight. As we noted, this credential is always expressed in probabilistic terms but is a source of great controversy. In Figure 4.6 (p. 73), we showed how the force or weight of evidence concerns the strength of all the believability and relevance links in our chains of reasoning. But this illustration concerns the chains of reasoning from just few items of evidence. In any intelligence analysis, there will be many items of evidence to consider and very many sources of uncertainty or doubt that will be associated with complex arguments linking this mass of evidence to hypotheses at issue in the analysis. It would not be uncommon to be able to identify hundreds or even thousands of sources of doubt arising from masses of evidence being considered.

There are other matters apart from assessing the force or weight of evidence in which uncertainty arises. One way of describing evidence-based reasoning is to say that it involves the revision of probabilistic beliefs about hypotheses based on the evidence we have obtained. To say that we are revising these beliefs suggests that they must have had some initial state in order for them to be revised. The term prior probability is used to indicate the initial conditions of our uncertainty before we consider evidence that begins to emerge. In truth, there has been considerable controversy about prior probabilities and how they can be assessed. When we consider our evidence and its force or weight, we can begin the process of revising these prior beliefs based upon evidence. In the process, we revise our prior beliefs to form what are usually termed posterior beliefs, those assessed after we receive and incorporate the evidence we have. But we must take a bit of care concerning the process of belief revision just described.