Confidence intervals for the mean function of the true proportion score (μζ|x), where ζ and x respectively denote the true proportion and observed test scores, can be approximated by the Efron, Bayesian, and parametric empirical Bayes (PEB) bootstrap procedures. The similarity of results yielded by all the bootstrap methods suggests the following: the unidentifiability problem of the prior distribution g(ζ) can be bypassed with respect to the construction of confidence intervals for the mean function, and a beta distribution for g(ζ) is a reasonable assumption for the test scores in compliance with a negative hypergeometric distribution. The PEB bootstrap, which reflects the construction of Morris intervals, is introduced for computing predictive confidence bands for ζ|x. It is noted that the effect of test reliability on the precision of interval estimates varies with the two types of confidence statements concerned.

Reporting effect size index estimates with their confidence intervals (CIs) can be an excellent way to simultaneously communicate the strength and precision of the observed evidence. We recently proposed a robust effect size index (RESI) that is advantageous over common indices because it’s widely applicable to different types of data. Here, we use statistical theory and simulations to develop and evaluate RESI estimators and confidence/credible intervals that rely on different covariance estimators. Our results show (1) counter to intuition, the randomness of covariates reduces coverage for Chi-squared and F CIs; (2) when the variance of the estimators is estimated, the non-central Chi-squared and F CIs using the parametric and robust RESI estimators fail to cover the true effect size at the nominal level. Using the robust estimator along with the proposed nonparametric bootstrap or Bayesian (credible) intervals provides valid inference for the RESI, even when model assumptions may be violated. This work forms a unified effect size reporting procedure, such that effect sizes with confidence/credible intervals can be easily reported in an analysis of variance (ANOVA) table format.