We propose a new method to facilitate comparison of aggregated forecasts based on different aggregation, elicitation and calibration methods. Aggregates are evaluated by their relative position on the cumulative distribution of the corresponding individual scores. This allows one to compare methods using different measures of quality that use different scales. We illustrate the use of the method by re-analyzing various estimates from Budescu and Du (Management Science, 2007).

The Agricultural Act of 2014 introduced new crop insurance policies to manage agricultural risk, especially to cotton farmers. A representative farm panel was used to elicit the yield distribution of the farm, county, and correlation. Results suggest that the optimal underlying insurance policy is Revenue Protection at a 75% coverage level for both high- and low-productivity farms even with a Yield Exclusion provision. The Stacked Income Protection Plan benefit is mostly attributable to a higher insurance premium subsidy. For any crop, efficient agricultural risk management can be achieved through understanding the guaranteed yield and its relation to the farm and county yield.

Expert judgement is frequently used within general insurance. It tends to be a method of last resort and used where data is sparse, non-existent or non-applicable to the problem under consideration. Whilst such judgements can significantly influence the end results, their quality is highly variable. The use of the term “expert judgement” itself can lend a generous impression of credibility to what may be a little more than a guess. Despite the increased emphasis placed on the importance of robust expert judgements in regulation, actuarial research to date has focussed on the more technical or data-driven methods, with less emphasis on how to use and incorporate softer information or how best to elicit judgements from others in a way that reduces cognitive biases. This paper highlights the research that the Getting Better Judgement Working Party has conducted in this area. Specifically, it covers the variable quality of expert judgement, both within and outside the regulatory context, and presents methods that may be applied to improve its formation. The aim of this paper is to arm the insurance practitioner with tools to distinguish between low-quality and high-quality judgements and improve the robustness of judgements accordingly, particularly for highly material circumstances.

Objectives: A treatment pathway model was developed to examine the costs and benefits of the current bowel cancer service in England and to evaluate potential alternatives in service provision. To use the pathway model, various parameters and probability distributions had to be specified. They could not all be determined from empirical evidence and, instead, expert opinion was elicited in the form of statistical quantities that gave the required information. The purpose of this study is to describe the procedures used to quantify expert opinion and note examples of good practice contained in the case study.

Methods: The required information was identified and preparatory discussion with four experts refined the questions they would be asked. In individual elicitation sessions they quantified their opinions, mainly in the form of point and interval estimates for specified variables. New methods have been developed for quantifying expert opinion and these were implemented in specialized software that uses interactive graphics. This software was used to elicit opinion about quantities related to measurable covariates.

Results: Assessments for thirty-four quantities were elicited and available checks supported their validity. Eight points of good practice in eliciting and using expert judgment were evident. Parameters and probability distributions needed for the pathway model were determined from the elicited assessments. Simulation results from the pathway model were used to inform policy on bowel cancer service provision.

Conclusions: The study illustrates that quantifying and using expert judgment can be acceptable in real problems of practical importance. For full benefit to be gained from expert knowledge, elicitation must be conducted carefully and should be reported in detail.