Hostname: page-component-cd9895bd7-jn8rn Total loading time: 0 Render date: 2024-12-26T19:24:33.085Z Has data issue: false hasContentIssue false
  • English
  • Français

Sampling Plans for Fitting the Psychometric Function

Published online by Cambridge University Press:  10 April 2014

Miguel A. García-Pérez  and
Rocío Alcalá-Quintana
Miguel A. García-Pérez*
Affiliation:
Universidad Complutense de Madrid
Rocío Alcalá-Quintana
Affiliation:
Universidad Complutense de Madrid
*
Address correspondence to: Miguel A. García-Pérez, Departamento de Metodología, Facultad de Psicología, Universidad Complutense, Campus de Somosaguas, 28223 Madrid (Spain). Phone: +34 913 943 061. Fax: +34 913 943 189. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Research on estimation of a psychometric function Ψ has usually focused on comparing alternative algorithms to apply to the data, rarely addressing how best to gather the data themselves (i.e., what sampling plan best deploys the affordable number of trials). Simulation methods were used here to assess the performance of several sampling plans in yes–no and forced-choice tasks, including the QUEST method and several variants of up–down staircases and of the method of constant stimuli (MOCS). We also assessed the efficacy of four parameter estimation methods. Performance comparisons were based on analyses of usability (i.e., the percentage of times that a plan yields usable data for the estimation of all the parameters of Ψ) and of the resultant distributions of parameter estimates. Maximum likelihood turned out to be the best parameter estimation method. As for sampling plans, QUEST never exceeded 80% usability even when 1000 trials were administered and rendered accurate estimates of threshold but misestimated the remaining parameters. MOCS and up–down staircases yielded similar and acceptable usability (above 95% with 400–500 trials) and, although neither type of plan allowed estimating all parameters with optimal precision, each type appeared well suited to estimating a distinct subset of parameters. An analysis of the causes of this differential suitability allowed designing alternative sampling plans (all based on up–down staircases) for yes–no and forced-choice tasks. These alternative plans rendered near optimal distributions of estimates for all parameters. The results just described apply when the fitted Ψ has the same mathematical form as the actual Ψ generating the data; in case of form mismatch, all parameters except threshold were generally misestimated but the relative performance of all the sampling plans remained identical. Detailed practical recommendations are given.

Los estudios sobre estimación de la función psicométrica Ψ se han centrado tradicionalmente en comparar los algoritmos que se pueden aplicar a los datos, dejando al margen el problema de cómo recoger los propios datos (es decir, qué esquema de muestreo despliega de mejor forma los ensayos disponibles). Aquí se utilizan técnicas de simulación para evaluar el rendimiento de varios esquemas de muestreo en tareas de sí–no y de elección forzada, incluyendo QUEST y distintas variantes de escaleras de paso fijo y del método de los estímulos constantes. También se evalúa la eficacia de cuatro métodos de estimación de parámetros. Las comparaciones se basan en análisis de usabilidad (es decir, del porcentaje de veces que un esquema proporciona datos válidos para estimar todos los parámetros de Ψ) y de las distribuciones de las estimaciones. El mejor método de estimación resultó ser el de máxima verosimilitud. En cuanto a esquemas de muestreo, QUEST no llegó a rendir una usabilidad del 80% ni siquiera cuando se administraron 1000 ensayos y, aunque proporcionó buenas estimaciones del umbral, estimó erróneamente el resto de los parámetros. El método de los estímulos constantes y las escaleras de paso fijo rindieron una usabilidad similar (superior al 95% con 400–500 ensayos) y, aunque ninguno de estos esquemas permitió estimar con precisión óptima todos los parámetros, cada tipo de esquema se mostró adecuado para estimar un subconjunto distinto de parámetros. El análisis de las causas de estas diferencias permitió diseñar esquemas alternativos (todos ellos basados en escaleras de paso fijo) para tareas de sí–no y de elección forzada. Estos esquemas alternativos proporcionaron estimaciones con distribuciones casi óptimas. Los resultados descritos son válidos cuando la función cuyos parámetros se estiman tiene la misma forma analítica que la función psicométrica que ha generado los datos; cuando esas funciones difieren en forma, todos los parámetros excepto el umbral resultan estimados erróneamente, aunque la eficacia relativa de los distintos esquemas de muestreo no varía. Se ofrecen recomendaciones prácticas basadas en estos resultados.

Keywords

psychometric functionpsychophysical methodsleast squaresmaximum likelihoodsimulationfunción psicométricamétodos psicofísicosmínimos cuadradosmáxima verosimilitudsimulación
Type
Articles
Information
The Spanish Journal of Psychology , Volume 8 , Issue 2 , November 2005 , pp. 256 - 289
Copyright
Copyright © Cambridge University Press 2005

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Alcalá-Quintana, R., & García-Pérez, M.A. (2002, August). Bias and standard errors in forced-choice Bayesian staircases. Paper presented at the 33rd European Mathematical Psychology Group Meeting, Bremen, Germany.Google Scholar
Alcalá-Quintana, R., & García-Pérez, M.A. (2004a). The role of parametric assumptions in adaptive Bayesian estimation. Psychological Methods, 9, 250–271.CrossRefGoogle ScholarPubMed
Alcalá-Quintana, R., & García-Pérez, M.A. (2004b). Empirical performance of optimal Bayesian adaptive psychophysical methods. Perception (Suppl.), 33, 178.Google Scholar
Balakrishnan, N. (1992). Handbook of the logistic distribution. New York: Marcel Dekker.Google Scholar
Berkson, J. (1955). Maximum likelihood and minimum χ 2 estimates of the logistic function. Journal of the American Statistical Association, 50, 130–162.Google Scholar
Brand, T., & Kollmeier, B. (2002). Efficient adaptive procedures for threshold and concurrent slope estimates for psychophysics and speech intelligibility tests. Journal of the Acoustical Society of America, 111, 2801–2810.CrossRefGoogle ScholarPubMed
Brown, L.G. (1996). Additional rules for the transformed up-down method in psychophysics. Perception & Psychophysics, 58, 959–962.CrossRefGoogle ScholarPubMed
Dixon, W.J., & Mood, A.M. (1948). A method for obtaining and analyzing sensitivity data. Journal of the American Statistical Association, 43, 109–126.CrossRefGoogle Scholar
Evans, M., Hastings, N., & Peacock, B. (1993). Statistical distributions (2nd ed.). New York: Wiley.Google Scholar
Foster, D.H., & Bischof, W.F. (1991). Thresholds from psychometric functions: Superiority of bootstrap to incremental and probit variance estimators. Psychological Bulletin, 109, 152–159.CrossRefGoogle Scholar
Freeman, P.R. (1970). Optimal Bayesian sequential estimation of the median effective dose. Biometrika, 57, 79–89.CrossRefGoogle Scholar
García-Pérez, M.A. (1998). Forced-choice staircases with fixed step sizes: Asymptotic and small-sample properties. Vision Research, 38, 1861–1881.CrossRefGoogle ScholarPubMed
García-Pérez, M.A. (2001). Yes-no staircases with fixed step sizes: Psychometric properties and optimal setup. Optometry & Vision Science, 78, 56–64.CrossRefGoogle ScholarPubMed
García-Pérez, M.A., Giorgi, R., Woods, R.L., & Peli, E. (2005). Thresholds vary between spatial and temporal forced-choice paradigms: The case of lateral interactions in peripheral vision. Spatial Vision, 18, 99–127.Google ScholarPubMed
Hall, J.L. (1981). Hybrid adaptive procedure for estimation of psychometric functions. Journal of the Acoustical Society of America, 69, 1763–1769.CrossRefGoogle ScholarPubMed
Harvey, L.O. Jr., (1997). Efficient estimation of sensory thresholds with ML-PEST. Spatial Vision, 11, 121–128.CrossRefGoogle ScholarPubMed
Kaernbach, C. (1991). Simple adaptive testing with the weighted up—down method. Perception & Psychophysics, 49, 227–229.CrossRefGoogle ScholarPubMed
Kaernbach, C. (2001). Slope bias of psychometric functions derived from adaptive data. Perception & Psychophysics, 63, 1389–1398.CrossRefGoogle ScholarPubMed
Kershaw, C.D. (1985). Statistical properties of staircase estimates from two interval forced choice experiments. British Journal of Mathematical & Statistical Psychology, 38, 35–43.CrossRefGoogle Scholar
King-Smith, P.E., & Rose, D. (1997). Principles of an adaptive method for measuring the slope of a psychometric function. Vision Research, 37, 1595–1604.CrossRefGoogle ScholarPubMed
King-Smith, P.E., Grigsby, S.S., Vingrys, A.J., Benes, S.C., & Supowit, A. (1994). Efficient and unbiased modifications of the QUEST threshold method: Theory, simulations, experimental evaluation and practical implementation. Vision Research, 34, 885–912.CrossRefGoogle ScholarPubMed
Klein, S.A. (2001). Measuring, estimating, and understanding the psychometric function: A commentary. Perception & Psychophysics, 63, 1421–1455.CrossRefGoogle ScholarPubMed
Kontsevich, L.L., & Tyler, C.W. (1999). Bayesian adaptive estimation of psychometric slope and threshold. Vision Research, 39, 2729–2737.CrossRefGoogle ScholarPubMed
Lam, C.F., Mills, J.H., & Dubno, J.R. (1996). Placement of observations for the efficient estimation of a psychometric function. Journal of the Acoustical Society of America, 99, 3689–3693.CrossRefGoogle ScholarPubMed
Leek, M.R., Hanna, T.E., & Marshall, L. (1992). Estimation of psychometric functions from adaptive tracking procedures. Perception & Psychophysics, 51, 247–256.CrossRefGoogle ScholarPubMed
Maloney, L.T. (1990). Confidence intervals for the parameters of psychometric functions. Perception & Psychophysics, 47, 127–134.CrossRefGoogle ScholarPubMed
Marks, B.L. (1962). Some optimal sequential schemes for estimating the mean of a cumulative normal quantal response curve. Journal of the Royal Statistical Society, Series B, 24, 393–400.Google Scholar
McKee, S.P., Klein, S.A., & Teller, D.Y. (1985). Statistical properties of forced-choice psychometric functions: Implications of probit analysis. Perception & Psychophysics, 37, 286–298.CrossRefGoogle ScholarPubMed
Miller, J., & Ulrich, R. (2001). On the analysis of psychometric functions: The Spearman—Kärber method. Perception & Psychophysics, 63, 1399–1420.CrossRefGoogle ScholarPubMed
Müller, H.G., & Schmitt, T. (1990). Choice of number of doses for maximum likelihood estimation of the ED50 for quantal dose-response data. Biometrics, 46, 117–129.CrossRefGoogle ScholarPubMed
Myers, R.H. (1990). Classical and modern regression with applications. Boston, MA: PWS-KENT.Google Scholar
Numerical Algorithms Group (1999). NAG Fortran library manual, Mark 19. Oxford: Numerical Algorithms Group.Google Scholar
O'Regan, J.K., & Humbert, R. (1989). Estimating psychometric functions in forced-choice situations: Significant biases found in threshold and slope estimations when small samples are used. Perception & Psychophysics, 46, 434–442.CrossRefGoogle ScholarPubMed
Owen, R.J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351–356.CrossRefGoogle Scholar
Ramsey, F.L. (1972). A Bayesian approach to bioassay. Biometrics, 28, 841–858.CrossRefGoogle ScholarPubMed
Robbins, H., & Monro, S. (1951). A stochastic approximation method. Annals of Mathematical Statistics, 22, 400–407.CrossRefGoogle Scholar
Santoro, L., Burr, D., & Morrone, M. C. (2002). Saccadic compression can improve detection of Glass patterns. Vision Research, 42, 1361–1366.CrossRefGoogle ScholarPubMed
Serrano-Pedraza, I., & Sierra-Vázquez, V. (2003, August). Comparison between two methods for estimation of the parameters of a psychometric function: Effect of initial guess. Poster presented at the 34th European Mathematical Psychology Group Meeting, Madrid, Spain.Google Scholar
Simmers, A.J., Bex, P.J., Smith, F.K.H., & Wilkins, A.J. (2001). Spatiotemporal visual function in tinted lens wearers. Investigative Ophthalmology & Visual Science, 42, 879–884.Google ScholarPubMed
Snoeren, P.R., & Puts, M.J.H. (1997). Multiple parameter estimation in an adaptive psychometric method: MUEST, an extension of the QUEST method. Journal of Mathematical Psychology, 41, 431–439.CrossRefGoogle Scholar
Snowden, R.J., & Hammett, S.T. (1998). The effects of surround contrast on contrast thresholds, perceived contrast and contrast discrimination. Vision Research, 38, 1935–1945.CrossRefGoogle ScholarPubMed
Solomon, J.A., & Morgan, M.J. (2000). Facilitation of collinear flanks is cancelled by non-collinear flanks. Vision Research, 40, 279–286.CrossRefGoogle ScholarPubMed
Strasburger, H. (2001). Invariance of the psychometric function for character recognition across the visual field. Perception & Psychophysics, 63, 1356–1376.CrossRefGoogle ScholarPubMed
Swanson, W.H., & Birch, E.E. (1992). Extracting thresholds from noisy psychophysical data. Perception & Psychophysics, 51, 409–422.CrossRefGoogle ScholarPubMed
Treutwein, B. (1997). YAAP: Yet another adaptive procedure. Spatial Vision, 11, 129–134.Google ScholarPubMed
Treutwein, B., & Strasburger, H. (1999). Fitting the psychometric function. Perception & Psychophysics, 61, 87–106.CrossRefGoogle ScholarPubMed
Watson, A.B., & Pelli, D.G. (1983). QUEST: A Bayesian adaptive psychometric method. Perception & Psychophysics, 33, 113–120.CrossRefGoogle Scholar
Watson, A.B., & Turano, K. (1995). The optimal motion stimulus. Vision Research, 35, 325–336.CrossRefGoogle ScholarPubMed
Werkhoven, P., & Snippe, H.P. (1996). An efficient adaptive procedure for psychophysical discrimination experiments. Behavior Research Methods, Instruments, & Computers, 28, 556–562.CrossRefGoogle Scholar
Wetherill, G.B., & Levitt, H. (1965). Sequential estimation of points on a psychometric function. British Journal of Mathematical & Statistical Psychology, 18, 1–10.CrossRefGoogle ScholarPubMed
Wichmann, F.A., & Hill, N.J. (2001a). The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics, 63, 1293–1313.CrossRefGoogle ScholarPubMed
Wichmann, F.A., & Hill, N.J. (2001b). The psychometric function: II. Bootstrap-based confidence intervals and sampling. Perception & Psychophysics, 63, 1314–1329.CrossRefGoogle ScholarPubMed