Hostname: page-component-586b7cd67f-tf8b9 Total loading time: 0 Render date: 2024-11-28T15:26:44.288Z Has data issue: false hasContentIssue false
  • English
  • Français

Logarithmic Sobolev inequalities in discrete product spaces

Part of: Markov processes General convexity Time-dependent statistical mechanics (dynamic and nonequilibrium)

Published online by Cambridge University Press:  13 June 2019

Katalin Marton
Katalin Marton*
Affiliation:
H-1364 POB 127, Budapest, Hungary
*
Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

The aim of this paper is to prove an inequality between relative entropy and the sum of average conditional relative entropies of the following form: for a fixed probability measure q on , ( is a finite set), and any probability measure on , (*)

$$D(p||q){\rm{\le}}C \cdot \sum\limits_{i = 1}^n {{\rm{\mathbb{E}}}_p D(p_i ( \cdot |Y_1 ,{\rm{ }}...,{\rm{ }}Y_{i - 1} ,{\rm{ }}Y_{i + 1} ,...,{\rm{ }}Y_n )||q_i ( \cdot |Y_1 ,{\rm{ }}...,{\rm{ }}Y_{i - 1} ,{\rm{ }}Y_{i + 1} ,{\rm{ }}...,{\rm{ }}Y_n )),} $$
where pi(· |y1, …, yi−1, yi+1, …, yn) and qi(· |x1, …, xi−1, xi+1, …, xn) denote the local specifications for p resp. q, that is, the conditional distributions of the ith coordinate, given the other coordinates. The constant C depends on (the local specifications of) q.

The inequality (*) ismeaningful in product spaces, in both the discrete and the continuous case, and can be used to prove a logarithmic Sobolev inequality for q, provided uniform logarithmic Sobolev inequalities are available for qi(· |x1, …, xi−1, xi+1, …, xn), for all fixed i and fixed (x1, …, xi−1, xi+1, …, xn). Inequality (*) directly implies that the Gibbs sampler associated with q is a contraction for relative entropy.

In this paper we derive inequality (*), and thereby a logarithmic Sobolev inequality, in discrete product spaces, by proving inequalities for an appropriate Wasserstein-like distance.

MSC classification

Primary: 60J10: Markov chains (discrete-time Markov processes on discrete state spaces)
Secondary: 52A40: Inequalities and extremum problems 82C22: Interacting particle systems
Type
Paper
Information
Combinatorics, Probability and Computing , Volume 28 , Issue 6 , November 2019 , pp. 919 - 935
Copyright
© Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

This work was supported by grant OTKA K 105840 of the Hungarian Academy of Sciences and by National Research, Development and Innovation Office NKFIH K 120706.

References

Boucheron, S., Lugosi, G. and Massart, P. (2013) Concentration Inequalities, Oxford University Press.CrossRefGoogle Scholar
Caputo, P., Menz, G. and Tetali, P. (2015) Approximate tensorization of entropy at high temperature. Ann Fac Sci Toulouse Math Sér 6 24 691–716.CrossRefGoogle Scholar
Cesi, F. (2001) Quasi-factorization of the entropy and logarithmic Sobolev inequalities for Gibbs random fields. Probab. Theory Rel. Fields 120 569–584.CrossRefGoogle Scholar
Dobrushin, R. L. (1968) The description of a random field by means of conditional probabilities and condition of its regularity (in Russian). Theory Probab. Appl. 13 197–224.CrossRefGoogle Scholar
Dobrushin, R. L. (1970) Prescribing a system of random variables by conditional distributions. Theory Probab. Appl. 15 458–486.CrossRefGoogle Scholar
Dobrushin, R. L. and Shlosman, S. B. (1985) Constructive criterion for the uniqueness of Gibbs field. In Statistical Physics and Dynamical Systems (Fritz, J., Jaffe, A. and Szász, D., eds), Springer, pp. 371–403.CrossRefGoogle Scholar
Dobrushin, R. L. and Shlosman, S. B. (1987) Completely analytical interactions: Constructive description. J. Statist. Phys. 46 983–1014.CrossRefGoogle Scholar
Diaconis, P. and Saloff-Coste, L. (1996) Logarithmic Sobolev inequalities for finite Markov chains. Ann. Appl. Probab. 6 695–750.Google Scholar
Gibbs, A. L. and Su, F. E. (2002) On choosing and bounding probability metrics. Internat. Statist. Rev. 70 419–435.CrossRefGoogle Scholar
Goldstein, S. (1979) Maximal coupling. Z. Wahrscheinlichkeitstheor. verw. Geb. 46, 193–204.CrossRefGoogle Scholar
Gross, L. (1975) Logarithmic Sobolev inequalities. Amer. J. Math. 97 1061–1083.CrossRefGoogle Scholar
Ledoux, M. (1999) Concentration of measure and logarithmic Sobolev inequalities. In Séminaire de Probabilités XXXIII, Vol. 1709 of Lecture Notes in Mathematics, Springer, pp. 120–216.CrossRefGoogle Scholar
Martinelli, F. and Olivieri, E. (1994) Approach to equilibrium of Glauber dynamics in the one phase region, I: The attractive case. Commun. Math. Phys. 161 447–486.CrossRefGoogle Scholar
Martinelli, F. and Olivieri, E. (1994) Approach to equilibrium of Glauber dynamics in the one phase region, II: The general case. Commun. Math. Phys. 161 487–514.CrossRefGoogle Scholar
Marton, K. (2013) An inequality for relative entropy and logarithmic Sobolev inequalities in Euclidean spaces. J. Funct. Anal. 264 34–61.CrossRefGoogle Scholar
Olivieri, E. (1988) On a cluster expansion for lattice spin systems: A finite size condition for the convergence. J. Statist. Phys. 50 1179–1200.CrossRefGoogle Scholar
Olivieri, E. and Picco, P. (1990) Clustering for D-dimensional lattice systems and finite volume factorization properties. J. Statist. Phys. 59 221–256.CrossRefGoogle Scholar
Otto, F. and Reznikoff, M. (2011) A new criterion for the logarithmic Sobolev inequality and two applications. J. Funct. Anal. 243 121–157.CrossRefGoogle Scholar
Royer, G. (1999) Une Initiation aux Inegalités de Sobolev Logarithmiques, Société Mathématique de France.Google Scholar
Sason, I. (2015) Tight bounds for symmetric divergence measures and a refined bound for lossless source coding. IEEE Trans. Inform. Theory 61 701–707.CrossRefGoogle Scholar
Sason, I. (2015) On reverse Pinsker inequalities. arXiv:1503.07118v4Google Scholar
Stroock, D. W. and Zegarlinski, B. (1992) The equivalence of the logarithmic Sobolev inequality and the Dobrushin– Shlosman mixing condition. Commun. Math. Phys. 144 303–323.CrossRefGoogle Scholar
Stroock, D. W. and Zegarlinski, B. (1992) The logarithmic Sobolev inequality for discrete spin systems on the lattice. Comm. Math. Phys. 149 175–193.CrossRefGoogle Scholar
Zegarlinski, B. (1992) Dobrushin uniqueness theorem and logarithmic Sobolev inequalities. J. Funct. Anal. 105 77–111.CrossRefGoogle Scholar