Hostname: page-component-586b7cd67f-dsjbd Total loading time: 0 Render date: 2024-11-27T18:32:51.871Z Has data issue: false hasContentIssue false
  • English
  • Français

Disentangling Wine Judges’ Consensus, Idiosyncratic, and Random Expressions of Quality or Preference*

Published online by Cambridge University Press:  10 November 2017

Jeffrey Bodington
Jeffrey Bodington*
Affiliation:
Bodington & Company, 50 California Street, San Francisco, California, 94111; e-mail: [email protected].
Get access Rights & Permissions [Opens in a new window]

Abstract

Judges confer various awards on wines entered in dozens of wine competitions each year. This article employs data on blind replicates to show that those awards are based on one instance of stochastic ratings assigned by wine judges; awards based on the expected values of those stochastic ratings would be different. This article recognizes the stochastic nature of ratings and builds on the work of many others to propose and test a conditional-probability model that yields maximum-likelihood estimates of judges’ latent consensus, idiosyncratic, and random assignments of scores to wines. The exact p-value for a likelihood test of the null hypothesis that the model's results are random is less than 0.001. Applying the notion of conditional probability may lead to better methods of assigning awards to entries in wine competitions and of assessing the capabilities of wine judges. (JEL Classifications: A10, C10, C00, C12, D12)

Keywords

consensusidiosyncraticrandomstatisticswine tasting
Type
Articles
Information
Journal of Wine Economics , Volume 12 , Issue 3 , August 2017 , pp. 267 - 281
Copyright
Copyright © American Association of Wine Economists 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

The author thanks an anonymous reviewer for insightful and constructive comments. All remaining errors are the responsibility of the author alone.

References

Alvo, M., and Yu, P. L. H. (2014). Statistical Methods for Ranking Data. New York: Springer.Google Scholar
Ashton, R. H. (2012). Reliability and consensus of experienced wine judges: Expertise within and between? Journal of Wine Economics, 7(1), 70–87.CrossRefGoogle Scholar
Ashton, R. H. (2014). Nothing good ever came from New Jersey: Expectations and the sensory perception of wine. Journal of Wine Economics, 9(3), 304–319.CrossRefGoogle Scholar
Ashton, R. H. (2016). The value of expert opinion in the pricing of Bordeaux wine futures. Journal of Wine Economics, 11(2), 261–288.CrossRefGoogle Scholar
Bayer, P., Ferreira, F., and McMillan, R. (2003). A Unified Framework for Measuring Preferences for Schools and Neighborhoods. Yale University, Economic Growth Center, Center Discussion Paper No. 872.Google Scholar
Bockenholt, U. (1992). Thurstonian representation for partial ranking data. British Journal of Mathematical and Statistical Psychology, 45, 31–49.Google Scholar
Bodington, J. C. (2015a). Evaluating wine-tasting results and randomness with a mixture of rank preference models. Journal of Wine Economics, (10)1, 31–46.Google Scholar
Bodington, J. C. (2015b). Testing a mixture of rank preference models on judges’ scores in Paris and Princeton. Journal of Wine Economics, 10(2), 173–189.Google Scholar
Bodington, J. C. (2017a). Wine, women, men and type II error. Journal of Wine Economics, 12(2), 161–172.Google Scholar
Bodington, J. C. (2017b). The distribution of ratings assigned to blind replicates. Journal of Wine Economics, forthcoming.Google Scholar
Cao, J., and Stokes, L (2017). Comparison of different ranking methods in wine tasting. Journal of Wine Economics, 12(2), 203–210.CrossRefGoogle Scholar
Chen, W. (2014). How to order sushi. PhD diss., Harvard University.Google Scholar
Cicchetti, D. (2014). Blind Tasting of South African Wines: A Tale of Two Methodologies. American Association of Wine Economists, Working Paper No. 164.Google Scholar
Cicchetti, D. (2017). Evaluating the value of triplicate tastings of a given wine: Biostatistical considerations. Journal of Wine Research, 28(2), 135–143.CrossRefGoogle Scholar
Cleaver, G., and Wedel, M. (2001). Identifying random-scoring respondent in sensory research using finite mixture regression results. Food Quality and Preference, 12, 373–384.Google Scholar
Cortez, P., Cerdeira, A., Almeida, F., Matos, T., and Reis, J. (2009). Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems, 47(4), 547–553.Google Scholar
Critchlow, D. E. (1980). Metric methods for analyzing partially ranked data. Lecture notes in Statistics 34. New York: Springer-Verlag.Google Scholar
Frost, M. B., and Nobel, A. (2002). Preliminary study of the effect of knowledge and sensory expertise on liking for red wines. American Journal of Enology and Viticulture, 53(4), 275–284. See also related UC Davis Viticulture and Enology, Summary 115.CrossRefGoogle Scholar
Ginsburgh, V., and Zang, I. (2012). Shapley ranking of wines. Journal of Wine Economics, 7(2), 169–180.CrossRefGoogle Scholar
Greene, W. H., and Hensher, D. A. (2010). Modeling Ordered Choices. Cambridge: Cambridge University Press.Google Scholar
Green, P. E., and Rao, V. (1972). Applied Multidimensional Scaling: A Comparison of Approaches and Algorithms. Holt, Rienhart and Winston, Austen.Google Scholar
Hastings, J. S., Kane, T. J., and Staiger, D. O. (2006). Preferences and heterogeneous treatment effects in a public school choice lottery. National Bureau of Economic Research, Working Paper 12145.Google Scholar
Hodgson, R. T. (2008). An examination of judge reliability at a major U.S. wine competition. Journal of Wine Economics, 3(2), 105–113.Google Scholar
Hodgson, R., and Cao, J. (2014). Criteria for accrediting expert wine judges. Journal of Wine Economics, 9(1), 62–74.Google Scholar
Keane, M. P., and Wasi, N. (2013). The structure of consumer taste heterogeneity in revealed vs. stated preference data. University of Oxford, Nuffield College, Economics Papers from Economics Group, No. 2013-W10.Google Scholar
Mallows, C. L. (1957). Non-null ranking models. Biometrika, 44, 114–130.Google Scholar
Mantonakis, A., Rodero, P., Lesschaeve, I., and Hastie, R. (2009). Order in choice: Effects of serial position on preferences. Psychological Science, 20(11), 1309–1312.Google Scholar
Marden, J. I. (1995). Analyzing and Modeling Rank Data. London: Chapman and Hall.Google Scholar
Mc Breen, J., Goffette-Nagot, F., and Jensen, P. (2009). An agent-based simulation of rental housing markets. Groupe d'Analyse et de Théorie Economique: Working Paper GATER 2009-08.Google Scholar
Nachev, A., and Hogan, M. (2013). Using data mining techniques to predict product quality from physiochemical data. Business Information Systems, Cairnes Business School, NUI, Galway, Ireland.Google Scholar
Olkin, I., Lou, Y., Stokes, L., and Cao, J. (2015). Analyses of wine-tasting data: A tutorial. Journal of Wine Economics, 10(1), 4–30.Google Scholar
Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine, 50(302), 157–175.Google Scholar
Plackett, R. L. (1975). The analysis of permutations. Applied Statistics, 24, 193–202.Google Scholar
Rajan, U., and Sinha, A. (2008). Equilibria in a hotelling model: First-mover advantage? Ross School of Business, Working Paper No. 1114.Google Scholar
Rhee, D., de Palma, A., and Thisse, J. (1998). First-mover disadvantage with consumers’ idiosyncratic preferences along unobservable characteristics. Regional Science and Economics, 36(1), 99–117.CrossRefGoogle Scholar
Theusen, K. F. (2007). Analysis of ranked preference data. Informatics and mathematical modeling. Master's thesis, Technical University of Denmark, Kongens Lyngby, Denmark.Google Scholar
Train, K. (2002). Discrete Choice Methods with Simulation. Cambridge: Cambridge University Press.Google Scholar
Vargo, M. D. (1989). Microbiological spoilage of a moderate acid food system using a dairy-based salad dressing model. Master's thesis, Department of Food Science and Nutrition, The Ohio State University. See also discussion in Fligner, M. A., and Verducci, J. S. (1993), Probability Models and Statistical Analyses for Ranking Data. Springer-Verlag, 11–14.Google Scholar
Vigneau, E., Courcoux, P., and Semenou, M. (1999). Analysis of ranked preference data using latent class models. Food Quality and Preference, 10(1999), 201–207.Google Scholar
Wine and Spirit Education Trust. (2014). Wines and spirits, looking behind the label. London: Wine and Spirit Education Trust.Google Scholar
Yue, C., Zhao, S., and Kuzma, J. (2015). Heterogeneous consumer preferences for nanotechnology and genetic-modification technology in food products. Journal of Agricultural Economics, 66, 308–328.CrossRefGoogle Scholar