Model selection procedures have become an integral part of almost any statistical or econometric analysis of data. Prominent examples include the selection of explanatory variables in (non)linear regression or lag-length selection in time series models. It is safe to say that these methods have become an integral part of the toolbox of modern data analysis. Moreover, research on model selection (or related procedures such as model averaging and shrinkage procedures) and its implications for statistical analysis has intensified in recent years. This is witnessed by the fact that it is difficult to pick up any recent issue of an econometrics or statistics journal and avoid encountering a paper where some form of model selection, or model averaging, or general shrinkage estimation is considered.

Shrinkage estimation procedures such as ridge regression and the lasso have been proposed for stabilizing estimation in linear models when high collinearity exists in the design. In this paper, we consider asymptotic properties of shrinkage estimators in the case of “nearly singular” designs.I thank Hannes Leeb and Benedikt Pötscher and also the referees for their valuable comments. This research was supported by a grant from the Natural Sciences and Engineering Research Council of Canada.

We consider the problem of estimating the unconditional distribution of a post-model-selection estimator. The notion of a post-model-selection estimator here refers to the combined procedure resulting from first selecting a model (e.g., by a model-selection criterion such as the Akaike information criterion [AIC] or by a hypothesis testing procedure) and then estimating the parameters in the selected model (e.g., by least squares or maximum likelihood), all based on the same data set. We show that it is impossible to estimate the unconditional distribution with reasonable accuracy even asymptotically. In particular, we show that no estimator for this distribution can be uniformly consistent (not even locally). This follows as a corollary to (local) minimax lower bounds on the performance of estimators for the distribution; performance is here measured by the probability that the estimation error exceeds a given threshold. These lower bounds are shown to approach ½ or even 1 in large samples, depending on the situation considered. Similar impossibility results are also obtained for the distribution of linear functions (e.g., predictors) of the post-model-selection estimator.The research of the first author was supported by the Max Kade Foundation and by the Austrian National Science Foundation (FWF), Grant no. P13868-MAT. A preliminary draft of the material in this paper was written in 1999.

We consider the properties of the highest posterior probability model in a linear regression setting. Under a spike and slab hierarchy we find that although highest posterior model selection is total risk consistent, it possesses hidden undesirable properties. One such property is a marked underfitting in finite samples, a phenomenon well noted for Bayesian information criterion (BIC) related procedures but not often associated with highest posterior model selection. Another concern is the substantial effect the prior has on model selection. We employ a rescaling of the hierarchy and show that the resulting rescaled spike and slab models mitigate the effects of underfitting because of a perfect cancellation of a BIC-like penalty term. Furthermore, by drawing upon an equivalence between the highest posterior model and the median model, we find that the effect of the prior is less influential on model selection, as long as the underlying true model is sparse. Nonsparse settings are, however, problematic. Using the posterior mean for variable selection instead of posterior inclusion probabilities avoids these issues.

It is common in econometric applications that several hypothesis tests are carried out simultaneously. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. The classical approach is to control the familywise error rate (FWE), which is the probability of one or more false rejections. But when the number of hypotheses under consideration is large, control of the FWE can become too demanding. As a result, the number of false hypotheses rejected may be small or even zero. This suggests replacing control of the FWE by a more liberal measure. To this end, we review a number of recent proposals from the statistical literature. We briefly discuss how these procedures apply to the general problem of model selection. A simulation study and two empirical applications illustrate the methods.We thank three anonymous referees for helpful comments that have led to an improved presentation of the paper. The research of the third author has been partially supported by the Spanish Ministry of Science and Technology and FEDER, Grant BMF2003-03324.

An unknown constant matrix M is observed with additive random error. The basic problem considered is to devise an estimator of M that trades off bias against variance so as to achieve relatively low quadratic risk. This paper develops an adaptive total least squares estimator and an adaptive total shrinkage estimator of M that minimize estimated risk over certain large classes of linear estimators. It is shown that the asymptotic risk of the adaptive total least squares estimator is the smallest attainable among reduced rank total least squares fits to the data matrix. The asymptotic risk of the adaptive total shrinkage estimator is shown to be smaller still. A close link is established between total shrinkage and the Efron–Morris estimator of M. In the asymptotics, the row dimension of M tends to infinity, and the column dimension stays fixed. The risks converge uniformly when the signal-to-noise ratio and the measurement error variance are both bounded. A second problem treated is estimation of M under the assumption that a linear relation holds among its columns. In this formulation of the errors-in-variables linear regression model, rank constrained adaptive total least squares asymptotically dominates the usual total least squares estimator of M, and rank constrained adaptive total shrinkage is better still.This research was supported in part by National Science Foundation Grant DMS 0404547.

Research on model/procedure selection has focused on selecting a single model globally. In many applications, especially for high-dimensional or complex data, however, the relative performance of the candidate procedures typically depends on the location, and the globally best procedure can often be improved when selection of a model is allowed to depend on location. We consider localized model selection methods and derive their theoretical properties.This research was supported by U.S. National Science Foundation CAREER Grant DMS0094323. We thank three referees and the editors for helpful comments on improving the paper.

Most model selection mechanisms work in an “overall” modus, providing models without specific concern for how the selected model is going to be used afterward. The focused information criterion (FIC), on the other hand, is geared toward optimum model selection when inference is required for a given estimand. In this paper the FIC method is extended to weighted versions. This allows one to rank and select candidate models for the purpose of handling a range of similar tasks well, as opposed to being forced to focus on each task separately. Applications include selecting regression models that perform well for specified regions of covariate values. We derive these weighted focused information criteria (wFIC), give asymptotic results, and apply the methods to real data. Formulas for easy implementation are provided for the class of generalized linear models.We express our sincere thanks to all reviewers of this paper, including the special issue guest editors and editor Professor Phillips, whose comments and questions have contributed to significant improvements. We also thank Dr. Ronald Klein for kindly giving permission to use the WESDR data. The work of Claeskens has been supported in part by the Fund for Scientific Research Flanders (G.0542.06).

Suppose we observe X ∼ Nm(Aβ,σ2I) and would like to estimate the predictive density p(y|β) of a future Y ∼ Nn(Bβ,σ2I). Evaluating predictive estimates by Kullback–Leibler loss, we develop and evaluate Bayes procedures for this problem. We obtain general sufficient conditions for minimaxity and dominance of the “noninformative” uniform prior Bayes procedure. We extend these results to situations where only a subset of the predictors in A is thought to be potentially irrelevant. We then consider the more realistic situation where there is model uncertainty and this subset is unknown. For this situation we develop multiple shrinkage predictive estimators and obtain general minimaxity and dominance conditions. Finally, we provide an explicit example of a minimax multiple shrinkage predictive estimator based on scaled harmonic priors.We acknowledge Larry Brown, Feng Liang, Linda Zhao, and three referees for their helpful suggestions. This work was supported by various NSF grants, DMS-0605102 the most recent.

We consider complexity penalization methods for model selection. These methods aim to choose a model to optimally trade off estimation and approximation errors by minimizing the sum of an empirical risk term and a complexity penalty. It is well known that if we use a bound on the maximal deviation between empirical and true risks as a complexity penalty, then the risk of our choice is no more than the approximation error plus twice the complexity penalty. There are many cases, however, where complexity penalties like this give loose upper bounds on the estimation error. In particular, if we choose a function from a suitably simple convex function class with a strictly convex loss function, then the estimation error (the difference between the risk of the empirical risk minimizer and the minimal risk in the class) approaches zero at a faster rate than the maximal deviation between empirical and true risks. In this paper, we address the question of whether it is possible to design a complexity penalized model selection method for these situations. We show that, provided the sequence of models is ordered by inclusion, in these cases we can use tight upper bounds on estimation error as a complexity penalty. Surprisingly, this is the case even in situations when the difference between the empirical risk and true risk (and indeed the error of any estimate of the approximation error) decreases much more slowly than the complexity penalty. We give an oracle inequality showing that the resulting model selection method chooses a function with risk no more than the approximation error plus a constant times the complexity penalty.We gratefully acknowledge the support of the NSF under award DMS-0434383. Thanks also to three anonymous reviewers for useful comments that improved the presentation.

We consider the problem of combining forecasts from two different levels (called “macro” and “micro”), where we have access to the forecasts and their precisions but not to the full data set. We develop a theoretical framework and provide Monte Carlo evidence in the cases of both perfect and imperfect aggregation. Our proposed procedure is simple and robust. We also extend the procedure to time series and propose a forecast model for the European zero rates, combining quarterly and monthly observations. We show that forecast accuracy is improved at both levels.We are grateful to the editors of this special issue and to two referees for their constructive comments, which greatly helped improve the paper. Earlier versions of this paper were presented at the NAKE workshop in Amsterdam, at Tilburg University, and at ESEM 2006 in Vienna. We thank the participants for their useful comments. In addition, we thank Steffan Berridge, John Einmahl, and Hans Schumacher for their constructive comments.

In our paper published in vol. 22, no. 1, of Econometric Theory (2006, pp. 69–97) the proof of (22) in Lemma 3.1 is incorrect as given. The problem in the proof occurs after the first display on p. 93 where we attempted to deduce (22) from the already established relation (21). The argument given there is incorrect, because an existence-quantor was mistaken for an all-quantor. We show how the problem can be repaired. In the course of this, we also note a useful variant of another result in Lemma 3.1.