Hostname: page-component-586b7cd67f-r5fsc Total loading time: 0 Render date: 2024-11-30T15:16:43.018Z Has data issue: false hasContentIssue false
  • English
  • Français

REGULARIZING PRIORS FOR LINEAR INVERSE PROBLEMS

Published online by Cambridge University Press:  06 November 2014

Jean-Pierre Florens  and
Anna Simoni
Jean-Pierre Florens
Affiliation:
Toulouse School of Economics
Anna Simoni*
Affiliation:
CNRS – GRECSTA and CREST
*
*Address correspondence to Anna Simoni, CREST, 15 Boulevard Gabriel Péri, 92240 Malakoff, France; e-mail: [email protected].
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper proposes a new Bayesian approach for estimating, nonparametrically, functional parameters in econometric models that are characterized as the solution of a linear inverse problem. By using a Gaussian process prior we propose the posterior mean as an estimator and prove frequentist consistency of the posterior distribution. The latter provides the frequentist validation of our Bayesian procedure. We show that the minimax rate of contraction of the posterior distribution can be obtained provided that either the regularity of the prior matches the regularity of the true parameter or the prior is scaled at an appropriate rate. The scaling parameter of the prior distribution plays the role of a regularization parameter. We propose a new data-driven method for optimally selecting in practice this regularization parameter. We also provide sufficient conditions such that the posterior mean, in a conjugate-Gaussian setting, is equal to a Tikhonov-type estimator in a frequentist setting. Under these conditions our data-driven method is valid for selecting the regularization parameter of the Tikhonov estimator as well. Finally, we apply our general methodology to two leading examples in econometrics: instrumental regression and functional regression estimation.

Type
ARTICLES
Information
Econometric Theory , Volume 32 , Issue 1 , February 2016 , pp. 71 - 121
Copyright
Copyright © Cambridge University Press 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Agapiou, S., Larsson, S., & Stuart, A.M. (2013) Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems. Stochastic Processes and Applications 123, 3828–3860.CrossRefGoogle Scholar
Ai, C. & Chen, X. (2003) Efficient estimation of models with conditional moment restrictions containing unknown functions. Econometrica 71, 1795–1843.Google Scholar
Belitser, E. & Ghosal, S. (2003) Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution. Annals of Statistics 31, 536–559.Google Scholar
Bissantz, N., Hohage, T., Munk, A., & Ruymgaart, F. (2007) Convergence rates of general regularization methods for statistical inverse problems and applications. SIAM Journal on Numerical Analysis 45, 2610–2636.Google Scholar
Carrasco, M. & Florens, J.P. (2000) Generalization of GMM to a continuum of moment conditions. Econometric Theory 16, 797–834.CrossRefGoogle Scholar
Carrasco, M., Florens, J.P., & Renault, E. (2005) Estimation based on spectral decomposition and regularization. In Heckman, J.J. & Leamer, E. (eds.), Handbook of Econometrics, vol. 6, pp. 5633–5751. Elsevier.Google Scholar
Cavalier, L. (2011) Inverse problems in statistics. In Alquier, P., Gautier, E., & Stoltz, G. (eds.), Inverse Problems and High-dimensional Estimation, pp. 3–96. Springer-Verlag.Google Scholar
Cavalier, L. & Tsybakov, A.B. (2002) Sharp adaptation for inverse problems with random noise. Probability Theory and Related Fields 123, 323–354.Google Scholar
Chen, X. & Pouzo, D. (2012) Estimation of nonparametric conditional moment models with possibly nonsmooth generalized residuals. Econometrica 80, 277–321.Google Scholar
Chen, X. & Reiss, M. (2011) On rate optimality for ill-posed inverse problems in econometrics. Econometric Theory 27, 497–521.Google Scholar
Daouia, A., Florens, J.-P., & Simar, L. (2009) Regularization of nonparametric frontier estimators. Journal of Econometrics 168, 285–299.Google Scholar
Darolles, S., Fan, Y., Florens, J.P., & Renault, E. (2011) Nonparametric instrumental regression. Econometrica 79, 1541–1565.Google Scholar
Diaconis, P.W. & Freedman, D. (1986) On the consistency of Bayes estimates. Annals of Statistics 14, 1–26.Google Scholar
Engl, H.W., Hanke, M., & Neubauer, A. (2000) Regularization of Inverse Problems. Kluwer Academic.Google Scholar
Florens, J.-P. (2003) Inverse problems and structural econometrics: The example of instrumental variables. Invited Lectures to the World Congress of the Econometric Society, Seattle 2000. In Dewatripont, M., Hansen, L.-P., & Turnovsky, S.J. (eds.), Advances in Economics end Econometrics: Theory and Applications, vol. II, pp. 284–311. Cambridge University Press.CrossRefGoogle Scholar
Florens, J.-P., Johannes, J., & Van Bellegem, S. (2011) Identification and estimation by penalization in nonparametric instrumental regression. Econometric Theory 27, 472–496.Google Scholar
Florens, J.P. & Simoni, A. (2010) Regularizing Priors for Linear Inverse Problems. IDEI Working paper, n. 621.Google Scholar
Florens, J.P. & Simoni, A. (2012a) Nonparametric estimation of an instrumental regression: A quasi-Bayesian approach based on regularized posterior. Journal of Econometrics 170, 458–475.CrossRefGoogle Scholar
Florens, J.P. & Simoni, A. (2012b) Regularized posteriors in linear ill-posed inverse problems. Scandinavian Journal of Statistics 39, 214–235.Google Scholar
Ghosh, J.K. & Ramamoorthi, R.V. (2003) Bayesian Nonparametrics. Springer Series in Statistics. Springer.Google Scholar
Hall, P. & Horowitz, J. (2005) Nonparametric methods for inference in the presence of instrumental variables. Annals of Statistics 33, 2904–2929.Google Scholar
Hall, P. & Horowitz, J.L. (2007) Methodology and convergence rates for functional linear regression. Annals of Statistics 35, 70–91.Google Scholar
Helin, T. (2009) On infinite-dimensional hierarchical probability models in statistical inverse problems. Inverse Problems and Imaging 3, 567–597.CrossRefGoogle Scholar
Hoderlein, S., Nesheim, L., & Simoni, A. (2013) Semiparametric Estimation of Random Coefficients in Structural Economic Models. CEMMAP Working paper, CWP09/12.Google Scholar
Hofinger, A. & Pikkarainen, H.K. (2007) Convergence rate for the Bayesian approach to linear inverse problems. Inverse Problems 23, 2469–2484.Google Scholar
Hofinger, A. & Pikkarainen, H.K. (2009) Convergence rate for linear inverse problems in the presence of an additive normal noise. Stochastic Analysis and Applications 27, 240–257.Google Scholar
Horowitz, J. (2014) Adaptive nonparametric instrumental variables estimation: Empirical choice of the regularisation parameter. Journal of Econometrics 180, 158–173.Google Scholar
Johannes, J., Schenk, R., & Simoni, A. (2014) Adaptive Bayesian estimation in Gaussian sequence space models. In Bongiorno, E.G., Goia, A., Salinelli, E., & Vieu, P. (eds.), Contributions in Infinite-Dimensional Statistics and Related Topics, pp. 167–172. Società Editrice Esculapio.Google Scholar
Kaipio, J. & Somersalo, E. (2004) Statistical and Computational Inverse Problems. Applied Mathematical Series, 160. Springer.Google Scholar
Kato, T. (1995) Perturbation Theory for Linear Operators. Springer.Google Scholar
Knapik, B.T., Van der Vaart, A.W., & Van Zanten, J.H. (2011) Bayesian inverse problems. Annals of Statistics 39, 2626–2657.Google Scholar
Krein, S.G. & Petunin, J.I. (1966) Scales of Banach spaces. Russian Mathematical Surveys 21, 85–160.Google Scholar
Kress, R. (1999) Linear Integral Equation. Springer-Verlag.Google Scholar
Kuo, H.H. (1975) Gaussian Measures in Banach Spaces. Springer-Verlag.Google Scholar
Le Cam, L. (1986) Asymptotic Methods in Statistical Decision Theory. Springer-Verlag.Google Scholar
Liao, Y. & Jiang, W. (2011) Posterior consistency of nonparametric conditional moment restricted models. Annals of Statistics 39, 3003–3031.Google Scholar
Linton, O. & Mammen, E. (2005) Estimating semiparametric ARCH(∞) models by kernel smoothing methods. Econometrica 73, 771–836.Google Scholar
Luschgy, H. (1995) Linear estimators and radonifying operators. Theory of Probability and Its Applications 40, 167–175.CrossRefGoogle Scholar
Mandelbaum, A. (1984) Linear estimators and measurable linear transformations on a Hilbert space. Z. Wahrcheinlichkeitstheorie 3, 385–398.Google Scholar
Natterer, F. (1984) Error bounds for Tikhonov regularization in Hilbert scale. Applicable Analysis 18, 29–37.Google Scholar
Newey, W.K. & Powell, J.L. (2003) Instrumental variable estimation of nonparametric models. Econometrica 71, 1565–1578.Google Scholar
Ruymgaart, F.H. (1998) A note on weak convergence of density in Hilbert spaces. Statistics 30, 331–343.Google Scholar
Van der Vaart, A.W. & Van Zanten, J.H. (2009) Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth. Annals of Statistics 37, 2655–2675.CrossRefGoogle Scholar
Van Rooij, A.C.M. & Ruymgaart, F.H. (1999) On inverse estimation. In Ghosh, S. (ed.), Asymptotic, Nonparametric and Time Series, pp. 579–613. Dekker.Google Scholar
Zellner, A. (1986) On assessing prior distributions and Bayesian regression analysis with g-prior distribution. In Goel, P.K. & Zellner, A. (eds.), Bayesian Inference and Decision Techniques: Essays in Honour of Bruno de Finetti, vol. 6, pp. 233–243. North Holland.Google Scholar