Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-12T13:09:36.091Z Has data issue: false hasContentIssue false
  • English
  • Français

INFERENCE AFTER MODEL AVERAGING IN LINEAR REGRESSION MODELS

Published online by Cambridge University Press:  04 September 2018

Xinyu Zhang  and
Chu-An Liu
Xinyu Zhang
Affiliation:
Chinese Academy of Sciences Qingdao University
Chu-An Liu*
Affiliation:
Academia Sinica
*
*Address correspondence to Chu-An Liu, Institute of Economics, Academia Sinica, Taipei 115, Taiwan; e-mail: [email protected].
Get access Rights & Permissions [Opens in a new window]

Abstract

This article considers the problem of inference for nested least squares averaging estimators. We study the asymptotic behavior of the Mallows model averaging estimator (MMA; Hansen, 2007) and the jackknife model averaging estimator (JMA; Hansen and Racine, 2012) under the standard asymptotics with fixed parameters setup. We find that both MMA and JMA estimators asymptotically assign zero weight to the under-fitted models, and MMA and JMA weights of just-fitted and over-fitted models are asymptotically random. Building on the asymptotic behavior of model weights, we derive the asymptotic distributions of MMA and JMA estimators and propose a simulation-based confidence interval for the least squares averaging estimator. Monte Carlo simulations show that the coverage probabilities of proposed confidence intervals achieve the nominal level.

Type
ARTICLES
Information
Econometric Theory , Volume 35 , Issue 4 , August 2019 , pp. 816 - 841
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We thank three anonymous referees, the co-editor Liangjun Su, and the editor Peter C.B. Phillips for many constructive comments and suggestions. We also thank conference participants of SETA 2016, AMES 2016, and CFE 2017 for their discussions and suggestions. Xinyu Zhang gratefully acknowledges the research support from National Natural Science Foundation of China (Grant numbers 71522004, 11471324 and 71631008). Chu-An Liu gratefully acknowledges the research support from the Ministry of Science and Technology of Taiwan (MOST 104-2410-H-001-092-MY2). All errors remain the authors’.

References

REFERENCES

Andrews, D.W.K. (1991) Asymptotic optimality of generalized CL, cross-validation, and generalized cross-validation in regression with heteroskedastic errors. Journal of Econometrics 47, 359–377.10.1016/0304-4076(91)90107-OCrossRefGoogle Scholar
Breiman, L. (1996) Bagging predictors. Machine Learning 24, 123–140.10.1007/BF00058655CrossRefGoogle Scholar
Buckland, S.T., Burnham, K.P., & Augustin, N.H. (1997) Model selection: An integral part of inference. Biometrics 53, 603–618.10.2307/2533961CrossRefGoogle Scholar
Camponovo, L. (2015) On the validity of the pairs bootstrap for lasso estimators. Biometrika 102(4), 981–987.10.1093/biomet/asv039CrossRefGoogle Scholar
Chatterjee, A. & Lahiri, S.N. (2011) Bootstrapping lasso estimators. Journal of the American Statistical Association 106(494), 608–625.10.1198/jasa.2011.tm10159CrossRefGoogle Scholar
Chatterjee, A. & Lahiri, S.N. (2013) Rates of convergence of the adaptive lasso estimators to the oracle distribution and higher order refinements by the bootstrap. The Annals of Statistics 41(3), 1232–1259.10.1214/13-AOS1106CrossRefGoogle Scholar
Claeskens, G. & Hjort, N.L. (2008) Model Selection and Model Averaging. Cambridge University Press.10.1017/CBO9780511790485CrossRefGoogle Scholar
DiTraglia, F. (2016) Using invalid instruments on purpose: Focused moment selection and averaging for GMM. Journal of Econometrics 195, 187–208.10.1016/j.jeconom.2016.07.006CrossRefGoogle Scholar
Hansen, B.E. (2007) Least squares model averaging. Econometrica 75, 1175–1189.10.1111/j.1468-0262.2007.00785.xCrossRefGoogle Scholar
Hansen, B.E. (2008) Least-squares forecast averaging. Journal of Econometrics 146(2), 342–350.10.1016/j.jeconom.2008.08.022CrossRefGoogle Scholar
Hansen, B.E. (2014) Model averaging, asymptotic risk, and regressor groups. Quantitative Economics 5(3), 495–530.10.3982/QE332CrossRefGoogle Scholar
Hansen, B.E. (2018) Econometrics. Unpublished manuscript, University of Wisconsin.Google Scholar
Hansen, B.E. & Racine, J. (2012) Jacknife model averaging. Journal of Econometrics 167, 38–46.10.1016/j.jeconom.2011.06.019CrossRefGoogle Scholar
Hansen, P., Lunde, A., & Nason, J. (2011) The model confidence set. Econometrica 79, 453–497.Google Scholar
Hjort, N.L. & Claeskens, G. (2003a) Frequentist model average estimators. Journal of the American Statistical Association 98, 879–899.10.1198/016214503000000828CrossRefGoogle Scholar
Hjort, N.L. & Claeskens, G. (2003b) Rejoinder to the focused information criterion and frequentist model average estimators. Journal of the American Statistical Association 98(464), 938–945.10.1198/016214503000000882CrossRefGoogle Scholar
Hoeting, J.A., Madigan, D., Raftery, A.E., & Volinsky, C.T. (1999) Bayesian model averaging: A tutorial. Statistical Science 14, 382–417.Google Scholar
Inoue, A. & Kilian, L. (2008) How useful is bagging in forecasting economic time series? A case study of US consumer price inflation. Journal of the American Statistical Association 103, 511–522.10.1198/016214507000000473CrossRefGoogle Scholar
Kabaila, P. (1995) The effect of model selection on confidence regions and prediction regions. Econometric Theory 11(3), 537–549.10.1017/S0266466600009403CrossRefGoogle Scholar
Kabaila, P. (1998) Valid confidence intervals in regression after variable selection. Econometric Theory 14(4), 463–482.10.1017/S0266466698144031CrossRefGoogle Scholar
Kim, J. & Pollard, D. (1990) Cube root asymptotics. The Annals of Statistics 18, 191–219.10.1214/aos/1176347498CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2003) The finite-sample distribution of post-model-selection estimators and uniform versus non-uniform approximations. Econometric Theory 19(1), 100–142.10.1017/S0266466603191050CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2005) Model selection and inference: Facts and fiction. Econometric Theory 21(1), 21–59.10.1017/S0266466605050036CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2006) Can one estimate the conditional distribution of post-model-selection estimators? The Annals of Statistics 34(5), 2554–2591.10.1214/009053606000000821CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2008) Can one estimate the unconditional distribution of post-model-selection estimators? Econometric Theory 24(02), 338–376.10.1017/S0266466608080158CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2017) Testing in the presence of nuisance parameters: Some comments on tests post-model-selection and random critical values. In Ahmed, S.E. (ed.), Big and Complex Data Analysis: Methodologies and Applications, pp. 69–82. Springer International Publishing.10.1007/978-3-319-41573-4_4CrossRefGoogle Scholar
Li, K.-C. (1987) Asymptotic optimality for Cp, CL, cross-validation and generalized cross-validation: Discrete index set. The Annals of Statistics 15, 958–975.10.1214/aos/1176350486CrossRefGoogle Scholar
Liang, H., Zou, G., Wan, A.T.K., & Zhang, X. (2011) Optimal weight choice for frequentist model average estimators. Journal of the American Statistical Association 106, 1053–1066.10.1198/jasa.2011.tm09478CrossRefGoogle Scholar
Liu, C.-A. (2015) Distribution theory of the least squares averaging estimator. Journal of Econometrics 186, 142–159.10.1016/j.jeconom.2014.07.002CrossRefGoogle Scholar
Liu, Q. & Okui, R. (2013) Heteroscedasticity-robust Cp model averaging. Econometrics Journal 16, 462–473.10.1111/ectj.12009CrossRefGoogle Scholar
Lu, X. (2015) A covariate selection criterion for estimation of treatment effects. Journal of Business and Economic Statistics 33, 506–522.10.1080/07350015.2014.982755CrossRefGoogle Scholar
Lu, X. & Su, L. (2015) Jackknife model averaging for quantile regressions. Journal of Econometrics 188(1), 40–58.10.1016/j.jeconom.2014.11.005CrossRefGoogle Scholar
Magnus, J., Powell, O., & Prüfer, P. (2010) A comparison of two model averaging techniques with an application to growth empirics. Journal of Econometrics 154(2), 139–153.10.1016/j.jeconom.2009.07.004CrossRefGoogle Scholar
Moral-Benito, E. (2015) Model averaging in economics: An overview. Journal of Economic Surveys 29(1), 46–75.10.1111/joes.12044CrossRefGoogle Scholar
Pötscher, B. (1991) Effects of model selection on inference. Econometric Theory 7(2), 163–185.10.1017/S0266466600004382CrossRefGoogle Scholar
Pötscher, B. (2006) The distribution of model averaging estimators and an impossibility result regarding its estimation. Lecture Notes-Monograph Series 52, 113–129.10.1214/074921706000000987CrossRefGoogle Scholar
Pötscher, B. & Leeb, H. (2009) On the distribution of penalized maximum likelihood estimators: The lasso, scad, and thresholding. Journal of Multivariate Analysis 100(9), 2065–2082.10.1016/j.jmva.2009.06.010CrossRefGoogle Scholar
Raftery, A.E. & Zheng, Y. (2003) Discussion: Performance of Bayesian model averaging. Journal of the American Statistical Association 98(464), 931–938.10.1198/016214503000000891CrossRefGoogle Scholar
Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 58, 267–288.Google Scholar
Van der Vaart, A. & Wellner, J. (1996) Weak Convergence and Empirical Processes. Springer Verlag.10.1007/978-1-4757-2545-2CrossRefGoogle Scholar
Wan, A.T.K., Zhang, X., & Zou, G. (2010) Least squares model averaging by Mallows criterion. Journal of Econometrics 156, 277–283.10.1016/j.jeconom.2009.10.030CrossRefGoogle Scholar
Yang, Y. (2000) Combining different procedures for adaptive regression. Journal of Multivariate Analysis 74(1), 135–161.10.1006/jmva.1999.1884CrossRefGoogle Scholar
Yang, Y. (2001) Adaptive regression by mixing. Journal of the American Statistical Association 96, 574–588.10.1198/016214501753168262CrossRefGoogle Scholar
Yuan, Z. & Yang, Y. (2005) Combining linear regression models: When and how? Journal of the American Statistical Association 100, 1202–1214.10.1198/016214505000000088CrossRefGoogle Scholar
Zhang, X. & Liang, H. (2011) Focused information criterion and model averaging for generalized additive partial linear models. The Annals of Statistics 39, 174–200.10.1214/10-AOS832CrossRefGoogle Scholar
Zhang, X., Wan, A.T., & Zhou, S.Z. (2012) Focused information criteria, model selection, and model averaging in a Tobit model with a nonzero threshold. Journal of Business and Economic Statistics 30, 132–142.10.1198/jbes.2011.10075CrossRefGoogle Scholar
Zhang, X., Wan, A.T.K., & Zou, G. (2013) Model averaging by jackknife criterion in models with dependent data. Journal of Econometrics 174, 82–94.10.1016/j.jeconom.2013.01.004CrossRefGoogle Scholar
Zhang, X., Zou, G., & Liang, H. (2014) Model averaging and weight choice in linear mixed-effects models. Biometrika 101, 205–218.10.1093/biomet/ast052CrossRefGoogle Scholar
Zou, H. (2006) The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101, 1418–1429.10.1198/016214506000000735CrossRefGoogle Scholar