Groundstates of the planar Schrödinger–Poisson system with potential well and lack of symmetry

Abstract

The Schrödinger–Poisson system describes standing waves for the nonlinear Schrödinger equation interacting with the electrostatic field. In this paper, we are concerned with the existence of positive ground states to the planar Schrödinger–Poisson system with a nonlinearity having either a subcritical or a critical exponential growth in the sense of Trudinger–Moser. A feature of this paper is that neither the finite steep potential nor the reaction satisfies any symmetry or periodicity hypotheses. The analysis developed in this paper seems to be the first attempt in the study of planar Schrödinger–Poisson systems with lack of symmetry.

1. Introduction

This paper deals with the qualitative analysis of solutions to Schrödinger–Poisson systems of the type

(1.1)\begin{equation} \left\{\begin{array}{@{}ll} i\psi_t -\Delta\psi+E(x)\psi +\nu\phi \psi=0, & (x,t)\in\mathbb{R}^2\times\mathbb{R}, \\ \Delta \phi=|\psi|^2, & (x,t)\in\mathbb{R}^2\times\mathbb{R}, \end{array}\right. \end{equation}

where $\psi :\mathbb {R}^2\times \mathbb {R}\rightarrow {\mathbb {C}}$ is the wave function, $E:\mathbb {R}^2\rightarrow \mathbb {R}$ is an external potential and $\nu$ is a real parameter. The function $\phi$ represents an internal potential for a nonlocal self-interaction of the wave function $\psi$.

By the standing wave ansatz $\psi (x,t)={\rm e}^{-i\lambda t}u(x)$ (with $\lambda \in \mathbb {R}$), problem (1.1) reduces to the following stationary planar Schrödinger–Poisson system:

(1.2)\begin{equation} \left\{\begin{array}{@{}ll} -\Delta u+V(x)u +\nu\phi u=0, & x\in\mathbb{R}^2, \\ \Delta \phi=u^2, & x\in\mathbb{R}^2, \end{array}\right. \end{equation}

where $V(x)=E(x)+\lambda$. In some recent works, local nonlinear terms of the form $f(u)$ have been added to the right-hand side of the first equation in (1.2). In this case, the nonlinear term $f(u)$ describes the interaction effect among particles; see Benci and Fortunato [9]. We shall be concerned in this paper with the case where $f$ is a continuous function with exponential critical or subcritical growth in the Trudinger–Moser sense.

The analysis developed in this paper is performed in the case where $V(x)$ is a finite steep potential well. This is a variation on the infinite potential well, in which a particle is trapped in a ‘box’ with limited potential ‘walls’. Unlike the infinite potential well, there is a probability that the particle will be detected outside the box. The quantum mechanical interpretation contrasts from the classical interpretation in that the particle cannot be detected outside the box if its total energy is less than the potential energy barrier of the walls. Even when the particle's energy is less than the potential energy barrier of the walls, there is a non-zero probability of the particle surviving outside the box according to the quantum interpretation.

During the last few decades, quantum modelling of semiconductors has become a very active area of research. The (local or nonlocal) Schrödinger–Poisson system explains the thermodynamical and electrostatic equilibrium of electrons trapped in tiny quantum wells. In reality, the interaction of a charge particle with an electromagnetic field can be characterized by coupling the nonlinear Schrödinger's and Poisson's equations, according to a classical model. Physicists proposed the Schrödinger–Poisson system to quantify the precise energy levels of electrons in semiconductor heterostructures; see Nier [39].

The first equation in system (1.2) is referred as the Schrödinger equation while the second equation in (1.2) is known as the Poisson equation. The Schrödinger equation plays the role of Newton's laws and conservation of energy in classical mechanics, that is, it predicts the future behaviour of a dynamic system. The linear Schrödinger equation is a central tool of quantum mechanics, which provides a thorough description of a particle in a non-relativistic setting.

In recent decades, a considerable amount of research has been conducted on the nature and behaviour of solutions to the Schrödinger–Poisson system. This is due in part to the fact that this class of nonlinear problems contains a large number of fundamental physical models including, for instance the interaction of a charge particle with the electrostatic field in quantum mechanics. In this case, the unknowns $u$ and $\phi$ represent the wave functions associated with the particle and the electric potential, respectively. Related applications include the study of obstacle problem, the seepage surface problem or Elenbaas's equation. In astrophysics, the Schrödinger–Poisson system has been suggested to model certain theoretical concepts; see Schunck and Mielke [45]. Self-gravitating boson stars, for example, may be a source of exotic laser interferometer gravitational-wave observatory detections in addition to the predicted gravitational wave merger signals of black hole and neutron star binary systems; see Sennett et al. [46].

The structure of the nonlinear Schrödinger equation is much more complicated. This equation is a prototypical dispersive nonlinear partial differential equation that has been central for almost four decades now to a variety of areas in mathematical physics. The relevant fields of application vary from Bose–Einstein condensates and nonlinear optics (see Byeon and Wang [10]), propagation of the electric field in optical fibers (see Malomed [36]) to the self-focusing and collapse of Langmuir waves in plasma physics (see Zakharov [56]) and the behaviour of deep water waves and freak waves (the so-called rogue waves) in the ocean (see Onorato et al. [41]). The nonlinear Schrödinger equation also describes various phenomena arising in the theory of Heisenberg ferromagnets and magnons, self-channelling of a high-power ultra-short laser in matter, condensed matter theory, dissipative quantum mechanics, electromagnetic fields (see Avron et al. [6]), plasma physics (e.g., the Kurihara superfluid film equation). We refer to Sulem and Sulem [49] for a modern overview, including applications.

Schrödinger also established the classical derivation of his equation, based upon the analogy between mechanics and optics, and closer to de Broglie's ideas. Schrödinger developed a perturbation method, inspired by the work of Lord Rayleigh in acoustics, and he proved the equivalence between his wave mechanics and Heisenberg's matrix. The importance of Schrödinger's perturbation method was pointed out by Einstein [27], who wrote: ‘The Schrödinger method, which has in a certain sense the character of a field theory, does indeed deduce the existence of only discrete states, in surprising agreement with empirical facts. It does so on the basis of differential equations applying a kind of resonance argument.’

In the literature, in order to overcome the lack of compactness, such problems have been investigated under periodicity assumptions or various symmetry hypotheses, namely, either in radially symmetric spaces or in axially symmetry spaces. For instance, Chen and Tang [22] proposed a new approach to recover the compactness for Cerami sequences, while Tang [51] developed a direct method (the non-Nehari manifold method) to find a minimizing Cerami sequence for the energy functional outside the Nehari–Pankov manifold by using the diagonal method. We also refer to Albuquerque et al. [3], Alves et al. [4], Chen and Tang [20, 21], Sun and Ma [50], Wen et al. [55], etc. In a nonlocal setting, Tang and Cheng [52] introduced an original method to recover the compactness of Palais–Smale sequences. A key of the present paper is that in our framework, the compactness is recovered by virtue of the Trudinger–Moser inequality, see Trudinger [53] and Moser [38].

The main novelty in the present paper is that we establish the existence of groundstates under general hypotheses and without assuming any symmetry or periodicity restrictions neither for the steep potential nor for the reaction.

1.1 Overview and historical comments

In the present paper, we are concerned with the following planar Schrödinger–Poisson system

(1.3)\begin{equation} \left\{\begin{array}{@{}ll} -\Delta u+V(x)u+\nu\phi u=f(u), & x\in\mathbb{R}^2, \\ \Delta \phi=u^2, & x\in\mathbb{R}^2, \end{array}\right. \end{equation}

where $\nu \in \mathbb {R}$ and $f\in C(\mathbb {R},\mathbb {R})$. We assume that $V\in C(\mathbb {R}^2,\mathbb {R})$ satisfies the following hypotheses:

1. (V1) $V$ is weakly differentiable and satisfies $(\nabla V(x), x)\in L^{\infty }(\mathbb {R}^3)\cup L^{\kappa }(\mathbb {R}^3)$ for $\kappa >1$ and

   \[ V(x)+\frac{1}{2}(\nabla V(x), x)\geq0,\quad\text{for all}\ x\in\mathbb{R}^2, \]
   where $(\cdot,\cdot )$ is the usual inner product in $\mathbb {R}^2$;

2. (V2) for all $x\in \mathbb {R}^2$, $V(x)\leq \lim \limits _{|y|\rightarrow +\infty }V(y)=V_\infty <+\infty$ and the inequality is strict in a subset of positive Lebesgue measure;

3. (V3) $\inf \sigma (-\Delta +V(\cdot ))>0$, where $\sigma (-\Delta +V(\cdot ))$ denotes the spectrum of the self-adjoint operator $-\Delta +V(\cdot ):H^1(\mathbb {R}^2)\rightarrow L^2(\mathbb {R}^2)$, that is,

   \[ \inf\sigma(-\Delta+V({\cdot}))=\inf\limits_{u\in H^1(\mathbb{R}^2){\setminus}\{0\} }\frac{\int_{\mathbb{R}^2}(|\nabla u|^2+V(x)u^2)\,{\rm d}\kern 0.06em x}{\int_{\mathbb{R}^2}u^2\,{\rm d}\kern 0.06em x}>0. \]

Condition (V2) expresses that $V$ is a finite steep potential well. Assumptions of this type have been used in many recent papers for various types of elliptic problems; we refer only to Rabinowitz [43] for the study of a nonlinear Schrödinger equation with the nonlinear subcritical growth. The existence of a potential well rather than simply a local minimum has advantages in several situations. It is, for example, a crucial requirement when using a Lyapunov function to determine the stability of a stationary solution of an infinite dimensional dynamical system; see Ball and Marsden [8, § 4] and Marsden and Hughes [37, § 6.6] for a discussion of this issue in the context of nonlinear elasticity.

As one of the typical examples of nonlinear Schrödinger equations with nonlocal nonlinearities, there has been a large amount of literature to Schrödinger–Poisson systems in dimension three, see [18, 28, 29, 33, 44, 50, 54] and the references therein. This kind of systems arises in the physical literature as an approximation of the Hartree–Fock model of a quantum many-body system of electrons under the presence of the external potential $V(x)$; see [9, 30, 32] and the references therein.

In the last decade, planar Schrödinger–Poisson systems have attracted a lot of attention after Stubbe [48] introduced an analytic framework to a system of this type. Indeed, the second equation in system (1.3) is called the Poisson equation, which can be solved by

\[ \phi(x) = \Gamma(x) \ast u^2(x)=\int_{\mathbb{R}^2}\Gamma(x-y)u^2(y)\,{\rm d}y, \]

where $\Gamma$ is the Newtonian kernel in dimension $2$ and is expressed by

\[ \Gamma(x)=\frac{1}{2\pi}\ln|x|,\quad x\in\mathbb{R}^2{\setminus}\{0\}. \]

And so, formally, problem (1.3) has a variational structure with the associated energy functional

\begin{align*} I(u)& =\frac{1}{2} \int_{\mathbb{R}^{2}}\left(|\nabla u|^{2}+V(x) u^{2}\right) {\rm d}\kern 0.06em x\\ & \quad+\frac{\nu}{4} \int_{\mathbb{R}^{4}} \Gamma(|x-y|) u^{2}(x) u^{2}(y)\,{\rm d}\kern 0.06em x\,{\rm d} y-\int_{\mathbb{R}^{2}}F(u)\,{\rm d}\kern 0.06em x, \end{align*}

where $F(t)=\int _{0}^{t}f(\tau )\,{\rm d}\tau$. Note that the approaches dealing with higher-dimensional cases seem difficult to be adapted to the planar case, since $\Gamma (x)=\frac {1}{2\pi }\ln |x|$ is sign-changing and presents singularities both at zero and infinity, and the corresponding energy functional is not well-defined on $H^1(\mathbb {R}^2)$. Precisely, the energy functional $I$ involves a convolution term

\[ \int_{\mathbb{R}^{2}} \int_{\mathbb{R}^{2}} \ln(|x-y|) u^{2}(x) u^{2}(y)\,{\rm d}\kern 0.06em x\,{\rm d}y \]

which is not well defined for all $u\in H^1(\mathbb {R}^2)$. So the rigorous study of planar Schrödinger–Newton systems had remained open for a long time. This is why much less is known in the planar case.

In [48], Stubbe introduced a variational framework for (1.3) with $V(x)\equiv 1$ by setting a weighted Sobolev space

\[ X:= \left\{u\in H^1(\mathbb{R}^2):\int_{\mathbb{R}^2}\ln(1+|x|)|u(x)|^{2}\,{\rm d}\kern 0.06em x<{+}\infty\right\}, \]

endowed with the norm

\[ \|u\|_{X}^{2}=\int_{\mathbb{R}^{2}}\left(|\nabla u|^{2}+V(x)|u|^{2}\right) {\rm d}\kern 0.06em x+\int_{\mathbb{R}^{2}} \ln (1+|x|)|u(x)|^{2}\,{\rm d}\kern 0.06em x, \]

which yields that the associated energy functional is well-defined and continuously differentiable on the space $X$. Cingolani and Weth [23] further developed the above variational framework to (1.3) with $f(x,u)=|u|^{p-2}u,\ p\geq 4$ and gave a variational characterization of ground state solutions when $V(x)$ is positive and $1$-periodic. Later, Du and Weth [26] extended the above results to the case $p\in (2,4)$. Chen and Tang [20] proved that there exists at least a ground state solution to (1.3) in an axially symmetric functions space, when $f$ satisfies some subcritical polynomial growth conditions, see also [7, 14, 21, 55] and so on. Some results on the existence and multiplicity of nontrivial solutions are obtained in [12, 21, 35] for the subcritical exponential growth case. In particular, the authors in [35] developed an asymptotic approximation procedure to set the problem (1.3) in the standard Sobolev space $H^1(\mathbb {R}^2)$. It is worthy in [11, 17] that a different approach has been also developed by establishing new weighted versions of the Trudinger–Moser inequality, for which the problems are well defined in a log-weighted Sobolev space where variational methods can be applied up to cover the maximal possible nonlinear growth. For the critical exponential growth case, we also refer to [5, 19, 22], which is introduced later.

In all of these works, we should point out that, what has been considered on system (1.3) is the potential $V$ is either autonomous or periodic, see [5, 14, 48], or axially symmetric, see [19, 20, 22, 55]. So it is quite natural to ask if there exist nontrivial solutions for planar Schrödinger–Poisson systems without any symmetry or periodicity assumption on $V$. The main focus of the present paper is at the existence of positive solutions to system (1.3) with $V$ satisfying some suitable finite potential well condition.

2. Main results

As is well known, the classical Sobolev embedding theorem asserts that

(2.1)\begin{equation} W^{1,p}_0(\Omega)\subset L^q(\Omega)\ \text{for}\ 1\leq q\leq p^* \ \text{and}\ p< N, \end{equation}

where $\Omega \subset \mathbb {R}^N$ is a bounded domain and $p^* = Np/(N-p)$ is the critical Sobolev exponent. In the limiting case $p = N$, the critical Sobolev exponent becomes infinite and $W^{1,N}(\Omega )\subset L^q(\Omega )$ for $1 \leq q < \infty$. However, one cannot take the limit as $q\nearrow N$ in (2.1), that is, the embedding $W^{1,N}_0(\Omega )\subset L^\infty (\Omega )$ is no longer valid. To fill this gap, Trudinger [53] discovered a borderline embedding result; see also Pohozaev [42]. Roughly speaking, this is an exponential-type inequality which asserts that

\[ u\in W^{1,N}_0(\Omega) \Rightarrow \int_\Omega\,{\rm e}^{u^2}\,{\rm d}\kern 0.06em x<\infty. \]

This inequality was subsequently sharpened by Moser [38] as follows:

\[ \sup_{\|\nabla u\|_{L^N(\Omega)}\leq 1}\int_\Omega\,{\rm e}^{\mu |u|^{N'}}\,{\rm d}\kern 0.06em x\ \left\{ \begin{array}{@{}ll} \leq C\,|\Omega| & \text{if}\ \mu\leq \mu_N:=N\omega_N^{1/(N-1)}\\ ={+}\infty & \text{if}\ \mu>\mu_N, \end{array}\right. \]

where $N':=N/(N-1)$ and $\mu _{N-1}$ is the measure of the unit sphere in $\mathbb {R}^N$.

Since we consider planar Schrödinger–Poisson systems involving nonlinearities of exponential growth in the sense of Trudinger–Moser, we first recall the Trudinger–Moser inequality, in the sense established in [13]; see also [1, 15]. This inequality plays a crucial role in estimating subcritical or critical nonlinearities of Trudinger–Moser type.

Lemma 2.1 [13] If $\alpha >0$ and $u\in H^1(\mathbb {R}^2)$, then

\[ \int_{\mathbb{R}^2}\left({\rm e}^{\alpha u^2}-1\right){\rm d}\kern 0.06em x<\infty. \]

Moreover, if $u\in H^1(\mathbb {R}^2),\|\nabla u\|_2^2\leq 1, \|u\|_2^2<\theta <\infty$ and $\alpha <4\pi$, then there exists a constant $C_{\theta,\alpha }$ which depends only on $\theta,\alpha$ such that

\[ \int_{\mathbb{R}^2}\left({\rm e}^{\alpha u^2}-1\right){\rm d}\kern 0.06em x\leq C_{\theta,\alpha}. \]

We also recall a notion of criticality which is totally different from the Sobolev type.

1. (f ₀) There exists $\alpha _0>0$ such that

   \[ \lim\limits_{|t|\rightarrow\infty}\frac{f(t)}{{\rm e}^{\alpha t^2}}=0,\ \forall \alpha>\alpha_0,\quad \lim\limits_{|t|\rightarrow\infty}\frac{f(t)}{{\rm e}^{\alpha t^2}}={+}\infty,\ \forall \alpha<\alpha_0, \]

which was introduced by Adimurthi and Yadava [2], see also de Figueiredo et al. [24] to the planar nonlinear elliptic problems. At present, there have been a large number of works in the literature to nonlinear elliptic problems involving critical growth of Trudinger–Moser type. We refer the readers to [4, 16, 40] and the references therein.

2.1 The subcritical case

For the subcritical exponential growth case, we make the following assumptions on the nonlinearity $f$.

1. (f ₁) For every $\theta >0$, there exists $C_\theta >0$ such that $|f(s)|\leq C_\theta \min \{1,|s|\}\,{\rm e}^{\theta |s|^2}, \forall s>0$.

2. (f ₂) $f\in C(\mathbb {R},\mathbb {R})$ and $f(s)=o(s)$ as $s\rightarrow 0$, $f(s)\equiv 0$ for $s\leq 0$ and $f(s)>0$ for $s>0$.

3. (f ₃) The function $\frac {f(t)t-F(t)}{t^3}: t\mapsto \mathbb {R}$ is nondecreasing in $(0,+\infty )$.

4. (f ₄) There exist $M_0>0$ and $t_0>0$ such that

   \[ F(t)\leq M_0|f(t)|,\quad \forall |t|\geq t_0. \]

Now we state our result about the existence of positive ground state solutions to problem (1.3).

Theorem 2.2 Assume (V1)–(V3) and $(f_1)$–$(f_4)$ hold. Then, for any $\nu >0$, problem (1.3) has at least a positive ground state solution $u\in X$ satisfying

(2.2)\begin{equation} \left|\int_{\mathbb{R}^2}\int_{\mathbb{R}^2}\ln|x-y|u^2(x)u^2(y)\,{\rm d}\kern 0.06em x\,{\rm d}y\right|<{+}\infty. \end{equation}

Remark 2.3 In the present paper, we can also endow $X$ with the norm (see [19, 20])

\[ \|u\|_{X}^{2}=\int_{\mathbb{R}^{2}}\left(|\nabla u|^{2}+V(x)|u|^{2}\right) {\rm d}\kern 0.06em x+\int_{\mathbb{R}^{2}} \ln (2+|x|)|u(x)|^{2}\,{\rm d}\kern 0.06em x, \]

from which we can easily see that

\[ \int_{\mathbb{R}^{2}} \ln (2+|x|)|u(x)|^{2}\,{\rm d}\kern 0.06em x>\int_{\mathbb{R}^{2}} \ln 2|u(x)|^{2}\,{\rm d}\kern 0.06em x. \]

As a clear estimate on bound from below for the term $\int _{\mathbb {R}^{2}} \ln (2+|x|)|u(x)|^{2}\,{\rm d}\kern 0.06em x$, it is good for us to develop an energy estimate inequality in analysis. Thus, it seems possible to improve condition (V3) by relaxing the lower bound of $V$.

Remark 2.4 In order to obtain the compactness directly, the authors in [19–22, 55] studied system (1.3) in an axially symmetric space $E:=X\cap H_{as}^1$ with

\[ H_{as}^1=\left\{u\in H^1(\mathbb{R}^2):\, u(x):=u(x_1,x_2)=u(|x_1|,|x_2|),\ \forall x\in\mathbb{R}^2\right\}. \]

This is a natural constraint set, since critical points of the functional $I$ restricted to $E$ are also critical points of the functional $I$ in $X$. More importantly, for any axially symmetric function $u\in E$, by decomposing the convolution term in functional $I$, they can obtain an estimate

\[ \int_{\mathbb{R}^4}\ln(1+|x-y|)u^2(x)u^2(y)\,{\rm d}\kern 0.06em x\,{\rm d}y\geq \frac{1}{4}\|u\|_2^2\int_{\mathbb{R}^2}\ln(1+|x|)u^2\,{\rm d}\kern 0.06em x,\quad u\in E, \]

which is crucial in proving that $I$ satisfies the Cerami condition at arbitrary energy level in $E$. Moreover, compared with [21] where the authors studied the case of subcritical polynomial growth, and used a monotonicity condition on $V$:

\begin{align*} & V\in C^1(\mathbb{R}^2,\mathbb{R}),\, t\mapsto t^2[2V(tx)\\ & \quad - \nabla V(tx)\cdot(tx)]\ \text{is nondecreasing in}\ (0,\infty)\ \text{for all}\ x\in\mathbb{R}^2 \end{align*}

to prove the existence of ground state solutions of (1.3), this condition is removed in theorem 2.2. However, we do not provide a minimax characterization of this ground state energy.

In addition to the difficulties that the quadratic part of $I$ is not coercive on $X$ and the norm of $X$ is not translation invariant, the boundedness and compactness of any Cerami sequence $\{u_n\}$ associated with functional $I$ are also the main obstacles that we need to overcome in our arguments. In the autonomous or periodic case, see [5], some strong compactness lemma can be established with the help of the translation invariance of $I$. However, their approaches do not work any more for the non-autonomous equation (1.3), since it seems difficult to construct a Cerami sequence satisfying asymptotically some Pohozaev identity due to the lack of the translation invariance of $I$.

In contrast to the symmetry or axial symmetry case (see [21]), where one can establish a new inequality on $V_1(u)$ to prove the boundedness of any Cerami sequence $\{u_n\}$ for functional $I$, as mentioned in remark 2.4, it becomes tougher to prove the boundedness of $\{u_n\}$ in $X$ for our case without any symmetry assumption on $V$. To bypass this obstacle, a new nonlocal perturbation approach is introduced to prove firstly that any Cerami sequence $\{u_n\}$ is bounded in $H^1(\mathbb {R}^2)$. And then by virtue of Lions’ vanishing lemma combined with estimates on the convolution term, we can rule out the vanishing case for sequence $\{u_n\}$. Finally, a delicate analysis on the mountain-pass values corresponding to the functional $I$ and the associated limit functional $I_\infty$ is given to exclude energy bubbling of sequence $\{u_n\}$ at infinity.

2.2 The critical case

We now turn our attention to the critical case. In the literature, there have been a few results on planar Schrödinger–Poisson systems with critical growth of Trudinger–Moser type. Recently, Alves and Figueiredo [5] investigated the existence of positive ground state solutions for (1.3) when $V(x)\equiv 1$ and $f$ satisfies ($f_0$),($f_2$) and the following conditions:

1. (f'₃) $\frac {f(t)}{t^3}$ is increasing in $(0,\infty )$;

2. (f ₅) there exists $\theta >2$ such that $0<\theta F(t)\leq f(t)t$ for all $t>0$;

3. (f ₆) there exist constants $p>4$ and

   \[ \lambda_0>\max\left\{1,\left[\frac{2(p-2)\alpha_0c_p}{\pi(p-4)}\right]^{(p-2)/2}\right\} \]
   such that $f(t)\geq \lambda _0t^{p-1}$ for $t\geq 0$, where $c_p=\inf _{\mathcal {N}_p} I_p$ with
   \[ \mathcal{N}_p:=\{u\in X{\setminus}\{0\}:\,I'_p(u)u=0\} \]
   and
   \begin{align*} I_p(u)& =\frac{1}{2}\int_{\mathbb{R}^2}(|\nabla u|^2+u^2)\,{\rm d}\kern 0.06em x\\ & \quad +\frac{\nu}{8\pi}\int_{\mathbb{R}^4}\ln(|x-y|)u^2(x)u^2(y)\,{\rm d}\kern 0.06em x{\rm d}y-\frac{1}{p}\int_{\mathbb{R}^2}|u|^p\,{\rm d}\kern 0.06em x. \end{align*}

Observe from [5] that the monotonicity condition ($f'_3$) is often used to guarantee the boundedness of the Palais–Smale sequence $\{u_n\}$ associated with $I$. With the aid of ($f_6$), the authors established directly an estimate on the norm of sequence $\{u_n\}$ in $H^1(\mathbb {R}^2)$. Then thanks to the Trudinger–Moser inequality, the compactness was obtained. However, as a global condition, ($f_6$) requires $f(t)$ to be super-cubic for all $t\geq 0$, which seems a little bit strict especially for $t>0$ small.

Later, under weaker assumptions than ($f'_3$) and ($f_6$), Chen and Tang [22] studied the existence of nontrivial solutions to (1.3) when $f(u)$ is replaced by $f(x,u)$. Motivated by [24], $f(x,u)$ is required to satisfy the following conditions:

1. (F1) $f(x,t)t>0$ for all $(x,t)\in \mathbb {R}^2\times (\mathbb {R}{\setminus} \{0\})$ and there exist $M_0>0$ and $t_0>0$ such that

   \[ F(x,t)\leq M_0|f(x,t)|,\quad \forall x\in\mathbb{R}^2,\ |t|\geq t_0; \]

2. (F2) $\liminf _{t\rightarrow \infty }\frac {t^2F(x,t)}{{\rm e}^{\alpha _0 t^2}}\geq \kappa >\frac {2}{\alpha _0^2\rho ^2}$ where $\rho \in (0,1/2)$ such that $\rho ^2\max _{|x|\leq \rho } V(x) \leq 1$.

Conditions (F1) and (F2) are devoted to giving a sharp estimate on the minimax level to guarantee that any Cerami sequence or any minimizing sequence $\{u_n\}$ of the associated functional does not vanish. Moreover, (F1) together with a weaker monotonicity condition

1. (f ₇) for all $x\in \mathbb {R}^2$, the mapping $(0,\infty )\ni t\mapsto \displaystyle \frac {f(t)-V(x)t}{t^3}$ is non-decreasing,

than ($f'_3$) can be used to verify that the weak limit of any Cerami sequence $\{u_n\}$ is a nontrivial solution of (1.3) in [22]. It is worth pointing out that the authors in [22] need to introduce some sort of axially symmetric assumptions on $V$ and $f$.

Another feature of the present paper is that we study the existence of nontrivial solutions to system (1.3) without hypotheses (F1) and (F2). Let $\mathcal {S}_2$ be the best constant of Sobolev embedding $H_0^1(B_{1/4}(0))\hookrightarrow L^2(B_{1/4}(0))$, that is,

(2.3)\begin{equation} \mathcal{S}_2\left(\int_{B_{1/4}(0)}u^2\,{\rm d}\kern 0.06em x\right)^{1/2}\leq\left(\int_{B_{1/4}(0)}|\nabla u|^2+V(x)u^2\,{\rm d}\kern 0.06em x\right)^{1/2}, \end{equation}

which has been one well-known fact. Moreover, the compactness of the embedding guarantees the achievement of $\mathcal {S}_2$, since $B_{1/4}(0)$ is a specific bounded domain. Recently, one fine bound of constant $\mathcal {S}_2$ has been also obtained in Du [25], which makes possible to give one specific estimate of $\nu$ from below in the following result.

Theorem 2.5 Assume that conditions (V2)–(V3) and (f$_0$), (f$_2$), (f$_7$) hold. Then, for

\[ \nu>\frac{\alpha_0 \mathcal{S}_2^4}{4\pi\ln 2}, \]

equation (1.3) has at least a positive ground state solution $u\in X$ satisfying

(2.4)\begin{equation} \left|\int_{\mathbb{R}^2}\int_{\mathbb{R}^2}\ln|x-y|u^2(x)u^2(y)\,{\rm d}\kern 0.06em x\,{\rm d}y\right|<{+}\infty. \end{equation}

Essentially, we adopt some ideas used in the subcritical exponential growth case to prove theorem 2.5. However, instead of the nonlocal perturbation approach used in theorem 2.2, the boundedness of any Cerami sequence $\{u_n\}$ in $H^1(\mathbb {R}^2)$ is obtained by a contradiction argument combining with Lions’ vanishing lemma.

In order to obtain the existence of weak solutions to the limiting problem and further to exclude energy bubbling of sequence $\{u_n\}$ at infinity, a refined estimate on the mountain pass value is given under an additional restriction on $\nu$.

Remark 2.6 Different from [19, 22], the potential $V$ and the nonlinearity $f$ are only required to satisfy some weaker assumptions as in theorem 2.5, and are neither axially symmetric nor satisfy condition (F1) or (F2). Moreover, the conditions of theorem 2.5 do not involve ($f'_3$), ($f_5$) and ($f_6$) in [5] where the authors studied system (1.3) in the radially symmetric setting. Indeed, ($f_7$) is weaker than ($f'_3$). Since nonlinearity $f$ in theorem 2.5 is very general, the restriction on $\nu$ has to be stated to ensure that the associated mountain pass value is less than $\frac {\pi }{\alpha _0}$, so that the compactness is recovered by virtue of the Trudinger–Moser inequality.

Throughout the paper, we need the following notations. Denote by $L^s(\mathbb {R}^2)$, $s\in [1,\infty ]$ the usual Lebesgue space with the norm $\|\cdot \|_s$. For any $r>0$ and any $z\in \mathbb {R}^2$, $B_r(z)$ stands for the ball of radius $r$ centred at $z$. $X^*$ denotes the dual space of $X$. At last, $C,C_1,C_2,\ldots$ denote various positive generic constants.

3. Preliminary results

Consider the Hilbert space $H^1(\mathbb {R}^2)$ with the norm

\[ \|u\|:=\left(\int_{\mathbb{R}^2}|\nabla u|^2+V(x)u^2\,{\rm d}\kern 0.06em x\right)^{\frac{1}{2}}, \]

which is equivalent to the standard norm in $H^1(\mathbb {R}^2)$ with (V2)–(V3). For any $u\in X$, we also define

\[ \|u\|_\star:=\left(\int_{\mathbb{R}^2}\ln(1+|x|)u^2\,{\rm d}\kern 0.06em x\right)^{\frac{1}{2}} \]

and

\[ \|u\|_X=\left(\|u\|^2+\|u\|_\star^2\right)^{\frac{1}{2}}. \]

In what follows, we recall a few basic properties about the Newton kernel to problem (1.3). Define the symmetric bilinear forms

\begin{align*} & (u,v)\mapsto B_1(u,v)=\frac{1}{2\pi}\int_{\mathbb{R}^4}\ln(1+|x-y|)u(x)u(y)\,{\rm d}\kern 0.06em x\,{\rm d}y,\\ & (u,v)\mapsto B_2(u,v)=\frac{1}{2\pi}\int_{\mathbb{R}^4}\ln\left(1+\frac{1}{|x-y|}\right)u(x)u(y)\,{\rm d}\kern 0.06em x\,{\rm d}y,\\ & (u,v)\mapsto B_0(u,v)=B_1(u,v)-B_2(u,v)=\frac{1}{2\pi}\int_{\mathbb{R}^4}\ln(|x-y|)u(x)u(y)\,{\rm d}\kern 0.06em x\,{\rm d}y, \end{align*}

where $u,v:\mathbb {R}^2\rightarrow \mathbb {R}$ are measurable functions in the Lebesgue sense. We define on $X$ the associated functionals

\begin{align*} & V_1(u)=B_1(u^2,u^2)=\frac{1}{2\pi}\int_{\mathbb{R}^4}\ln(1+|x-y|)u^2(x)u^2(y)\,{\rm d}\kern 0.06em x\,{\rm d}y,\\ & V_2(u)= B_2(u^2,u^2)=\frac{1}{2\pi}\int_{\mathbb{R}^4}\ln\left(1+\frac{1}{|x-y|}\right)u^2(x)u^2(y)\,{\rm d}\kern 0.06em x\,{\rm d}y,\\ & u\mapsto V_0(u)=V_1(u)-V_2(u)=\frac{1}{2\pi}\int_{\mathbb{R}^4}\ln(|x-y|)u^2(x)u^2(y)\,{\rm d}\kern 0.06em x\,{\rm d}y. \end{align*}

Observe that

\[ \ln(1+|x-y|)\leq \ln(1+|x|+|y|)\leq \ln(1+|x|)+\ln(1+|y|)\quad \text{for}\ x,y\in\mathbb{R}^2, \]

then we have the estimate for $u,v,w,z\in X$

(3.1)\begin{align} B_1(uv,wz)& \leq \frac{1}{2\pi}\int_{\mathbb{R}^4}[\ln(1+|x|)+\ln(1+|y|)]|u(x)v(x)||w(y)z(y)|\,{\rm d}\kern 0.06em x\,{\rm d}y\nonumber\\ & \leq\frac{1}{2\pi}(\|u\|_\star\|v\|_\star\|w\|_2\|z\|_2+\|u\|_2\|v\|_2\|w\|_\star\|z\|_\star). \end{align}

Due to the Hardy–Littlewood–Sobolev inequality, we deduce that

(3.2)\begin{equation} |B_2(u,v)|\leq \frac{1}{2\pi}\int_{\mathbb{R}^4}\frac{|u(x)u(y)|}{|x-y|}\,{\rm d}\kern 0.06em x\,{\rm d}y\leq C\|u\|_{4/3}\|v\|_{4/3},\quad u,v\in X, \end{equation}

which implies that

(3.3)\begin{equation} |V_2(u)|\leq C\|u\|^4_{8/3},\quad u\in X. \end{equation}

Lemma 3.1 [23] Let $\{u_n\}$ be a sequence in $L^2(\mathbb {R}^2)$ such that $u_n\rightarrow u\in L^2(\mathbb {R}^2){\setminus} \{ 0\}$ pointwise almost everywhere on $\mathbb {R}^2$. Moreover, let $\{v_n\}$ be a bounded sequence in $L^2(\mathbb {R}^2)$ such that $\sup _{n\in \mathbb {N}}B_1(u_n^2,v_n^2)<\infty$. Then there exist $n_0\in \mathbb {N}$ and $C>0$ such that $\|v_n\|_\star < C$ for $n\geq n_0$. If, moreover,

\[ B_1(u_n^2,v_n^2)\rightarrow0\quad \text{and}\quad \|v_n\|\rightarrow0\quad \text{as}\ n\rightarrow\infty, \]

then $\|v_n\|_\star \rightarrow 0$ as $n\rightarrow \infty$.

Lemma 3.2 [23] Let $\{u_n\},\{v_n\},\{w_n\}$ be bounded sequences in $X$ such that $u_n\rightharpoonup u$ weakly in $X$. Then, for every $z \in X$, we have $B_1(v_n w_n,z(u_n-u))\rightarrow 0$ as $n\rightarrow \infty$.

Lemma 3.3 [23] The following conclusions hold true.

1. (i) The space $X$ is compactly embedded into $L^{s}(\mathbb {R}^2)$ for all $s\in [2,+\infty )$.

2. (ii) The functionals $V_0,V_1,V_2$ are of $C^1$ class on $X$. Moreover, $V'_i(u)v=4B_i(u^2,uv)$ for $u,v\in X$ and $i=0,1,2$.

3. (iii) $V_2$ is continuously differentiable on $L^{\frac {8}{3}}(\mathbb {R}^2)$.

4. (iv) $V_1$ is weakly lower semicontinuous on $H^1(\mathbb {R}^2)$.

We now recall a version of the Mountain Pass Theorem which plays a crucial role in proving the existence of nontrivial solutions.

Theorem 3.4 [47] Let $X$ be a real Banach space and let $I\in C^1(X,\mathbb {R})$. Let $S$ be a closed subset of $X$ which disconnects (arcwise) $X$ in distinct connected components $X_1$ and $X_2$. Suppose further that $I(0) = 0$ and

1. (i) $0\in X_1$ and there is $\alpha >0$ such that $I_S\geq \alpha$,

2. (ii) there is $e\in X_2$ such that $I(e)\leq 0$.

Then $I$ possesses a sequence $\{u_n\}\subset X$(called as $(Ce)_c$ sequence) satisfying

\[ I(u_n)\rightarrow c,\ \|I'(u_n)\|_{X^*}(1+\|u_n\|_X)\rightarrow0 \]

with $c\geq \alpha > 0$ given by $c=\inf \limits _{\gamma \in \Gamma }\max \limits _{t\in [0,1]}I(\gamma (t))$, where

\[ \Gamma=\{\gamma\in C([0,1],X):\,\gamma(0)=0,\,\gamma(1)=e\}. \]

4. The subcritical case

Without loss of generality, we assume $\nu =1$ in this section. We now give more details to describe the nonlocal perturbation approach mentioned in §2. Since we do not impose the well-known Ambrosetti–Rabinowitz condition, the boundedness of the Palais–Smale sequence cannot be obtained easily. In order to overcome this difficulty, we introduce a perturbation technique developed in [33, 34] to equation (1.3). Set

\[ \lambda\in(0,1],\quad r\in\left(\max\{p,4\},+\infty\right). \]

Consider the following modified problem:

(4.1)\begin{equation} \left\{\begin{array}{@{}ll} -\Delta u+V(x)u+\phi u+\lambda\left(\int_{\mathbb{R}^2}u^2\,{\rm d}\kern 0.06em x\right)^{\dfrac14}u =f(u)+\lambda|u|^{r-2}u, & x\in\mathbb{R}^2, \\ \Delta \phi=u^2, & x\in\mathbb{R}^2, \end{array}\right. \end{equation}

whose associated functional is given by

\begin{align*} I_{\lambda}(u)& =\frac{1}{2}\|u\|^2+\frac{1}{8\pi}\int_{\mathbb{R}^4}\ln(|x-y|)u^2(x)u^2(y)\,{\rm d}\kern 0.06em x\,{\rm d}y+\frac{2\lambda}{5} \|u\|_2^{\frac52}\\ & \quad -\int_{\mathbb{R}^2}F(u)\,{\rm d}\kern 0.06em x-\frac{\lambda}{r}\|u\|_r^r. \end{align*}

Now we provide a Pohozaev type identity for the modified equation (4.1).

Lemma 4.1 Suppose that $u\in X$ is a weak solution of (4.1). Then we have the following identity:

\begin{align*} P_\lambda(u)& :=\int_{\mathbb{R}^2}V(x)u^2\,{\rm d}\kern 0.06em x+\frac{1}{2}\int_{\mathbb{R}^2}(\nabla V(x),x)u^2\,{\rm d}\kern 0.06em x+V_0(u)+\lambda\|u\|_2^{5/2} +\frac{1}{4}\|u\|_2^4\nonumber\\ & \quad-2\int_{\mathbb{R}^2}F(u)\,{\rm d}\kern 0.06em x-\frac{2\lambda}{r}\|u\|_r^r=0. \end{align*}

In the following, we verify the geometry assumption of theorem 3.4 so that we can get the associated $(Ce)_{c_\lambda }$ sequence $\{u_{n,\lambda }\}$ (still denoted by $\{u_{n}\}$) with $c_\lambda \geq \alpha > 0$ and

(4.2)\begin{equation} c_\lambda:=\inf\limits_{\gamma\in\Gamma}\max\limits_{t\in[0,1]}I_\lambda(\gamma(t)). \end{equation}

From the following lemma, we can observe that $c_\lambda$ has an uniform bound independently of $\lambda$. That is, there exist $a,b>0$ such that $c_\lambda \in [a,b]$, where $a,b$ do not depend on $\lambda$.

Lemma 4.2 Assume (V1)–(V3) and $(f_1)$–$(f_3)$ hold, then assumption (i) and (ii) of theorem 3.4 hold true.

Proof. Choosing $\theta \in (0,4\pi )$ and $p>2$, it follows from ($f_1$)–($f_2$) that, for any $\varepsilon >0$, there exists $C_{\varepsilon } > 0$ such that

(4.3)\begin{equation} |F(s)|\leq \varepsilon ({\rm e}^{\theta|s|^2}-1)+C_{\varepsilon}|s|^p,\quad s\in\mathbb{R}. \end{equation}

By Moser–Trudinger's inequality (lemma 2.1), we claim that there exists $C>0$ (independent of $u$ and $\varepsilon$) such that, for any $u\in H^1(\mathbb {R}^2)$ with $\|u\|^2<1$, there holds that

(4.4)\begin{equation} \int_{\mathbb{R}^2}F(u)\,{\rm d}\kern 0.06em x\leq \varepsilon C\|u\|^2+C_{\varepsilon}\|u\|^{p}. \end{equation}

In fact, due to $e^x-1-x\leq x(e^x-1)$ for any $x\ge 0$,

\begin{align*} \int_{\mathbb{R}^2}({\rm e}^{\theta|u|^2}-1){\rm d}\kern 0.06em x& =\theta\int_{\mathbb{R}^2}u^2\,{\rm d}\kern 0.06em x+\int_{\mathbb{R}^2}({\rm e}^{\theta|u|^2}-1-\theta|u|^2)\,{\rm d}\kern 0.06em x\\ & \leq\theta\int_{\mathbb{R}^2}u^2\,{\rm d}\kern 0.06em x+\theta\int_{\mathbb{R}^2}u^2({\rm e}^{\theta|u|^2}-1)\,{\rm d}\kern 0.06em x. \end{align*}

By lemma 2.1 and Hölder's inequality, for some $c>0$ (independent of $u$), one has

\[ \int_{\mathbb{R}^2}u^2({\rm e}^{\theta|u|^2}-1)\,{\rm d}\kern 0.06em x\leq c\|u\|^2. \]

So (4.4) follows from (4.3) and Sobolev's embedding.

Then, we have by (3.3)

(4.5)\begin{align} I_\lambda(u)& =\frac{1}{2}\|u\|^2+\frac{1}{4}[V_1(u)-V_2(u)]+\frac{2\lambda}{5} \|u\|_2^{\frac52}-\int_{\mathbb{R}^2}F(u)\,{\rm d}\kern 0.06em x-\frac{\lambda}{r}\|u\|_r^r\nonumber\\ & \geq\frac{1-2\varepsilon C}{2}\|u\|^2-C_1\|u\|^4-C_\varepsilon\|u\|^p-C_3\|u\|^r, \end{align}

which implies that there exist $\alpha >0$ and $\rho >0$ small such that

(4.6)\begin{equation} I_\lambda(u)\geq \alpha,\quad \forall u\in S=\{u\in X: \|u\|=\rho\}. \end{equation}

On the other hand, take $e\in C_0^\infty (\mathbb {R}^2)$ such that $e(x)\equiv 0$ for $x\in \mathbb {R}^2{\setminus} {B_{\frac {1}{4}}(0)}$, $e(x)\equiv 1$ for $x\in B_{\frac {1}{8}}(0)$, and $|\nabla e(x)|\leq C$. Then we have the following estimate:

(4.7)\begin{align} I_\lambda(se) & =\frac{s^2}{2}\|e\|^2+\frac{s^4}{4}[V_1(e)-V_2(e)]+\frac{2\lambda s^{5/2}}{5} \|e\|_2^{\frac52}-\int_{\mathbb{R}^2}F(se)\,{\rm d}\kern 0.06em x-\frac{\lambda s^r}{r}\|e\|_r^r\nonumber\\ & \leq\frac{s^2}{2}\|e\|^2 -\frac{s^4}{8\pi}\int_{|x|\leq\frac{1}{4}}\int_{|y|\leq\frac{1}{4}}\ln\frac{1}{|x-y|}\,{\rm e}^2(y)\,{\rm e}^2(x)\,{\rm d}y\,{\rm d}\kern 0.06em x+\frac{2 s^{5/2}}{5} \|e\|_2^{\frac52}\nonumber\\ & \leq\frac{s^2}{2}\|e\|^2 -\frac{s^4\ln2}{8\pi}\left(\int_{\mathbb{R}^2}\,{\rm e}^2(x)\,{\rm d}\kern 0.06em x\right)^2+\frac{2 s^{5/2}}{5} \|e\|_2^{\frac52}. \end{align}

Hence, we can choose $t_0>0$ large enough such that $I_{\lambda }(t_0e)<0$.

Lemma 4.3 Assume (V1)–(V3) and $(f_1)$–$(f_4)$ hold, then any $(Ce)_{c_\lambda }$ sequence $\{u_n\}$ is bounded in $H^1(\mathbb {R}^2)$.

Proof. Assume $\{u_n\}$ is a $(Ce)_{c_\lambda }$ sequence. Then the following holds$:$

(4.8)\begin{align} C& \geq I_{\lambda}(u_n)-\frac{1}{4}I'_{\lambda}(u_n)u_n\nonumber\\ & = \frac{1}{4}\|u_n\|^2 +\frac{3\lambda}{20}\|u_n\|_2^{\frac{5}{2}}+\int_{A_n\cup\{\mathbb{R}^2{\setminus}{A_n}\}}\left(\frac{1}{4}f(u_n)u_n-F(u_n)\right){\rm d}\kern 0.06em x\nonumber\\ & \quad +\frac{r-4}{4r}\lambda\int_{\mathbb{R}^2}|u_n|^{r}\,{\rm d}\kern 0.06em x\nonumber\\ & \geq \frac{1}{4}\|u_n\|^2 +\frac{3\lambda}{20}\|u_n\|_2^{\frac{5}{2}}-\int_{A_n}(F(u_n)-\frac{1}{4}f(u_n)u_n)\,{\rm d}\kern 0.06em x +\frac{r-4}{4r}\lambda\int_{\mathbb{R}^2}|u_n|^{r}\,{\rm d}\kern 0.06em x, \end{align}

where $A_n:=\{x|\,\frac {1}{4}f(u_n)u_n-F(u_n)\leq 0\}$. Using ($f_4$), the definition of $A_n$ implies that there exists $T>0$ such that $|u_n|\leq T$ for $x\in A_n$. So, by ($f_2$) there exists $C_T>0$ such that

(4.9)\begin{equation} \int_{A_n}(F(u_n)-\frac{1}{4}f(u_n)u_n)\,{\rm d}\kern 0.06em x\leq C_{T}\int_{A_n}u_n^2\,{\rm d}\kern 0.06em x\leq C_{T}\|u_n\|_2^2. \end{equation}

Observe that for any large $B_1>0$, there exists $B_2>0$ such that $\frac {3}{20}\|u_n\|_2^{\frac {5}{2}}\geq B_1\|u_n\|_2^2-B_2$. So combining (4.8) and (4.9) we have

(4.10)\begin{equation} C+\lambda B_2\geq \frac{1}{8}\|u_n\|^2+\int_{\mathbb{R}^2}\left[(\lambda B_1-C_{T})|u_n|^2 +\frac{r-4}{4r}\lambda|u_n|^{r}\right]{\rm d}\kern 0.06em x. \end{equation}

Let $B_1$ be large enough, then for fixed $\lambda$, the following holds:

\[ (\lambda B_1-C_{T})t^2 +\frac{r-4}{4r}\lambda t^{r}\ge0 \]

for $t\geq 0$. Thus, it follows from (4.10) that $\|u_n\|\leq C$ for some $C$ (independent of $n$).

Lemma 4.4 Assume (V1)–(V3) and $(f_1)$, $(f_2)$ and $(f_4)$ hold true, if $I'_{\lambda,\infty }(u)=0$ and $u\in X{\setminus} \{0\}$, then $I_{\lambda,\infty }(u)=\max \limits _{t\in (0,+\infty )}I_{\lambda,\infty }(u_t)$, where $u_t:=t^2u(tx)$. Here,

\begin{align*} I_{\lambda,\infty}(u)& {:=}\frac{1}{2}\int_{\mathbb{R}^2}|\nabla u|^2\,{\rm d}\kern 0.06em x+\frac{1}{2}\int_{\mathbb{R}^2}V_\infty u^2\,{\rm d}\kern 0.06em x+\frac{1}{4}V_0(u) +\frac{2\lambda}{5}\|u\|_2^{5/2}\nonumber\\ & \quad -\int_{\mathbb{R}^2}F(u)\,{\rm d}\kern 0.06em x-\frac{\lambda}{r}\|u\|_r^r. \end{align*}

Proof. By $I'_{\lambda,\infty }(u)=0$, we have

(4.11)\begin{align} 2\|\nabla u\|_2^2& +\int_{\mathbb{R}^2} V_\infty u^2\,{\rm d}\kern 0.06em x+\lambda\|u\|_2^{5/2} +V_0(u)\nonumber\\ & \quad-\frac{1}{4}\|u\|_2^4-2\int_{\mathbb{R}^2}(f(u)u-F(u))\,{\rm d}\kern 0.06em x-\frac{2(r-1)\lambda}{r}\|u\|_r^r=0, \end{align}

which comes from $2I'_{\lambda,\infty }(u)u-\mathcal {P}_{\lambda,\infty }(u)=0$. Here,

\[ \mathcal{P}_{\lambda,\infty}(u)=\int_{\mathbb{R}^2}V_\infty u^2\,{\rm d}\kern 0.06em x+\lambda\|u\|_2^{5/2}+V_0(u)+\frac{1}{4}\|u\|_2^4 -2\int_{\mathbb{R}^2} F(u)\,{\rm d}\kern 0.06em x-\frac{2\lambda}{r}\|u\|_r^r. \]

Let us define a function $\chi (t):=I_{\lambda,\infty }(u_t)$ on $[0,+\infty )$. Obviously, $\chi (0)=0$ and $\chi (t)>0$ for $t>0$ small and $\chi (t)<0$ for $t$ sufficiently large. Thus, $\max _{t\in (0,+\infty )}I_{\lambda,\infty }(u_t)$ is achieved at some $t_u>0$. So $\chi '(t_u)=0$ and $u_{t_u}$ satisfies (4.11). Observe that

\begin{align*} \chi'(t)& =2t^3\|\nabla u\|_2^2+t\int_{\mathbb{R}^2} V_\infty u^2\,{\rm d}\kern 0.06em x+\lambda t^{3/2}\|u\|_2^{5/2}+ t^3V_0(u)-\int_{\mathbb{R}^2}\left(\frac{F(t^2u)}{t^2}\right)'_t\,{\rm d}\kern 0.06em x\\ & \quad-{t}^{3}\left(\ln t +\frac{1}{4}\right)\|u\|_{2}^{4} -\frac{2(r-1)\lambda t^{2r-3}}{r}\|u\|_r^r\\ & \quad = t^3\left[2\|\nabla u\|_2^2+\frac{1}{t}\int_{\mathbb{R}^2} V_\infty u^2\,{\rm d}\kern 0.06em x+\frac{\lambda}{t^{3/2}}\|u\|_2^{5/2}+V_0(u)-\left(\ln t +\frac{1}{4}\right)\|u\|_{2}^{4}\right.\\ & \quad\left.-2\int_{\mathbb{R}^2}\frac{(f(t^2u)t^2u-F(t^2u))}{t^6}\,{\rm d}\kern 0.06em x-\frac{2(r-1)\lambda t^{2r-6}}{r}\|u\|_r^r\right]. \end{align*}

From (4.11) we infer that $\chi '(1)=0$ and $\chi '(t)>0$ for $t<1$ and $\chi '(t)<0$ for $t>1$. Thus, $t_u=1$. The proof is complete.

Motivated by the strategy of proposition 3.1 in [23], we have

Lemma 4.5 Assume (V1)–(V3) and $(f_1)$–$(f_4)$ hold, for $\lambda \in (0,1]$ fixed, there exists $u_0\in X{\setminus} \{0\}$ such that $I'_\lambda (u_0)=0$ with $I_\lambda (u_0)=c_\lambda$.

Proof. By using lemmas 4.2 and 4.3, we observe that there exists a $(Ce)_{c_\lambda }$ sequence $\{u_n\}\subset X$ with $\|u_n\|\leq C$ uniformly for $n$. The remaining proof will be divided into three steps.

Step 1. We claim that

(4.12)\begin{equation} \liminf\limits_{n\rightarrow\infty}\sup\limits_{y\in\mathbb{R}^2}\int_{B_2(y)}u_n^2(x)\,{\rm d}\kern 0.06em x>0. \end{equation}

If (4.12) does not occur, then by Lions’ vanishing lemma (see [31]), we have $u_n\rightarrow 0$ in $L^s(\mathbb {R}^2)$ for all $s>2$. From ($f_1$)–($f_2$), take $\theta$ small enough, we deduce that, for any $\varepsilon >0$, there exists $C_{\varepsilon } > 0$ such that for $p>2$

(4.13)\begin{equation} |f(u_n)|\leq \varepsilon ({\rm e}^{\theta|u_n|^2}+1)|u_n|+C_{\varepsilon}|u_n|^p, \end{equation}

from which we have $\int _{\mathbb {R}^2}f(u_n)u_n\,{\rm d}\kern 0.06em x=o(1)$ for large $n$. Thus, it follows from (4.4) and (3.3) that

\begin{align*} & \|u_n\|^2+V_1(u_n)+\lambda \|u_n\|_2^{5/2}=I_\lambda'(u_n)u_n+V_2(u_n)\\ & \quad +\int_{\mathbb{R}^2}f(u_n)u_n\,{\rm d}\kern 0.06em x+\lambda\|u_n\|_r^r+o(1)=o(1), \end{align*}

which yields that $u_n\rightarrow 0$ in $H^1(\mathbb {R}^2)$ and $V_1(u_n)\rightarrow 0$ as $n\rightarrow \infty$. Thus, by (4.4) one has

\[ I_\lambda(u_n)\!=\!\frac{1}{2}\|u_n\|^2\!+\!\frac{2\lambda}{5} \|u_n\|_2^{5/2}\!+\!\frac{1}{4}(V_1(u_n)\!-V_2(u_n))-\int_{\mathbb{R}^2}F(u_n)\,{\rm d}\kern 0.06em x-\frac{\lambda}{r}\|u_n\|_r^r\rightarrow0 \]

as $n\rightarrow \infty$, which contradicts $c_\lambda >a$. So the claim is true. Going if necessary to a subsequence, there exists a sequence $\{y_n\}\subset \mathbb {R}^2$ such that $v_n=u_n(\cdot +y_n)$ is still bounded in $H^1(\mathbb {R}^2)$ and $v_n\rightharpoonup v_0$ for some non-zero function $v_0\in H^1(\mathbb {R}^2)$, and $v_n\rightarrow v_0$ a.e. in $\mathbb {R}^2$.

Step 2. We claim that $\{y_n\}$ is bounded. Assume by contradiction that $|y_n|\rightarrow +\infty$. Since $\{u_n\}$ is a $(Ce)_{c_\lambda }$ sequence for $I_\lambda$, the following holds:

\[ V_1(v_n)=V_1(u_n)=o(1)+V_2(u_n)+\int_{\mathbb{R}^2}f(u_n)u_n\,{\rm d}\kern 0.06em x+\lambda\|u_n\|_r^r-\|u_n\|^2-\lambda\|u_n\|_2^{\frac{5}{2}}, \]

which yields that $V_1(v_n)$ is bounded uniformly for $n$ due to the boundedness of $\{u_n\}$ in $H^1(\mathbb {R}^2)$. Recalling lemma 3.1, we obtain that $\|v_n\|_{\star }$ are also bounded uniformly for $n$, and thus $\{v_n\}$ is bounded in $X$. Up to subsequence, we may assume that $v_n\rightharpoonup v_0$ in $X$. So, $v_0\in X$. It then follows by lemma 3.3 (i) that $v_n\rightarrow v_0$ in $L^s(\mathbb {R}^2)$ for $s\geq 2$ as $n\rightarrow \infty$. Observe that for small $r>0$, due to $y_n\rightarrow +\infty$, one has

\begin{align*} \|u_n\|^2_{{\star}}& =\int_{\mathbb{R}^2}\ln(1+|x+y_n|)v_n^2(x)\,{\rm d}\kern 0.06em x\geq \int_{B_r(0)}\ln(1+|x+y_n|)v_n^2(x)\,{\rm d}\kern 0.06em x\\ & \geq \frac{1}{2}\int_{B_r(0)}\ln(1+|y_n|)v_n^2(x)\,{\rm d}\kern 0.06em x\geq C_2\ln(1+|y_n|) \end{align*}

with some $C_2>0$, and

\begin{align*} \|v_0({\cdot}{-}y_n)\|^2_{{\star}}& =\int_{\mathbb{R}^2}\ln(1+|x+y_n|)v_0^2(x)\,{\rm d}\kern 0.06em x\\ & \leq \int_{B_r(0)}[\ln(1+|x|)+\ln(1+|y_n|)]v_0^2(x)\,{\rm d}\kern 0.06em x\\ & \leq C_3\ln(1+|y_n|) \end{align*}

for some $C_3>0$. In view of the above inequalities, we deduce from the Fatou lemma that there exists $C>0$ such that

(4.14)\begin{align} \|v_0({\cdot}{-}y_n)\|_X^2& \leq \|v_0\|^2+\|v_0({\cdot}{-}y_n)\|_\star^2\nonumber\\ & \leq C(\|u_n\|^2+\|v_0({\cdot}{-}y_n)\|_\star^2)\nonumber\\ & \leq C\|u_n\|_X^2. \end{align}

Define

\begin{align*} & \tilde{I}_{\lambda,n}(v_n):=\frac{1}{2}\int_{\mathbb{R}^2}|\nabla v_n|^2\,{\rm d}\kern 0.06em x+\frac{1}{2}\int_{\mathbb{R}^2}V(x+y_n)v_n^2\,{\rm d}\kern 0.06em x\\ & \quad +\frac{1}{4}V_0(v_n) +\frac{2\lambda}{5}\|v_n\|_2^{5/2}-\int_{\mathbb{R}^2}F(v_n)\,{\rm d}\kern 0.06em x-\frac{\lambda}{r}\|v_n\|_r^r. \end{align*}

Therefore, we have for every $n$,

(4.15)\begin{align} \left|\tilde{I}'_{\lambda,n}(v_n)(v_n-v_0)\right|& =\left|I'_\lambda(u_n)(u_n-v_0({\cdot}{-}y_n))\right|\nonumber\\ & \quad \leq \|I'_\lambda(u_n)\|_{X^*}(\|u_n\|_X+\|v_0({\cdot}{-}y_n))\|_X), \end{align}

which, together with (4.14), implies that

(4.16)\begin{equation} |\tilde{I}'_{\lambda,n}(v_n)(v_n-v_0)|\rightarrow0,\quad\text{as}\ n\rightarrow\infty. \end{equation}

Based on the fact that $v_n\rightarrow v_0$ in $L^s(\mathbb {R}^2)$ for $s\geq 2$ as $n\rightarrow \infty$, by assumption (V2) one has

(4.17)\begin{equation} \int_{\mathbb{R}^2}V(x+y_n)v_n(v_n-v_0)\,{\rm d}\kern 0.06em x\rightarrow0, \end{equation}

as $n\rightarrow \infty$, and by (4.13) and (3.2) one has

(4.18)\begin{equation} \begin{aligned} & \int_{\mathbb{R}^2}f(v_n)(v_n-v_0)\,{\rm d}\kern 0.06em x\rightarrow0,\quad \|v_n\|_2^{1/2}\int_{\mathbb{R}^2}v_n(v_n-v_0)\,{\rm d}\kern 0.06em x\rightarrow0,\\ & |\frac{1}{4}V'_2(v_n)(v_n-v_0)|\leq |B_2(v_n^2,v_n(v_n-v_0))|\leq \|v_n\|_{8/3}^3\|v_n-v_0\|_{8/3}\rightarrow0, \end{aligned} \end{equation}

as $n\rightarrow \infty$. By the definition of $\tilde {I}_{\lambda,n}(v_n)$, we have

(4.19)\begin{align} & B_1(v_n^2,v_n(v_n-v_0))\nonumber\\ & = \tilde{I}'_{\lambda,n}(v_n)(v_n-v_0)-\int_{\mathbb{R}^2}V(x+y_n)v_n(v_n-v_0)\,{\rm d}\kern 0.06em x-\|\nabla (v_n-v_0)\|_2^2\nonumber\\ & \quad+B_2(v_n^2,v_n(v_n-v_0))-\lambda\|v_n\|_2^{1/2}\int_{\mathbb{R}^2}v_n(v_n-v_0)\,{\rm d}\kern 0.06em x+\int_{\mathbb{R}^2}f(v_n)(v_n-v_0)\,{\rm d}\kern 0.06em x\nonumber\\ & \quad+\lambda\int_{\mathbb{R}^2}|v_n|^{r-2}v_n(v_n-v_0)\,{\rm d}\kern 0.06em x+o(1). \end{align}

Combining (4.16)–(4.19), we infer that $\|\nabla (v_n-v_0)\|_2^2\rightarrow 0$ and $B_1(v_n^2, v_n(v_n-v_0))\rightarrow 0$ as $n\rightarrow \infty$, and then $v_n\rightarrow v_0$ in $H^1(\mathbb {R}^2)$. Recalling lemmas 3.1 and 3.2, we have $\|v_n-v_0\|_\star \rightarrow 0$ as $n\rightarrow \infty$. We therefore deduce that $v_n\rightarrow v_0$ in $X$.

Note that, for any $\varphi \in C_0^\infty (\mathbb {R}^2)$, we have, after passing to a subsequence,

\[ \int_{\mathbb{R}^2}V(x+y_n)v_n\varphi\,{\rm d}\kern 0.06em x\rightarrow\int_{\mathbb{R}^2}V_\infty v_0\varphi\,{\rm d}\kern 0.06em x\quad \text{as}\ n\rightarrow\infty. \]

Thus, from (4.14) and (4.15) we deduce that for any $\varphi \in C_0^\infty (\mathbb {R}^2)$

\begin{align*} |I'_{\lambda,\infty}(v_0)\varphi|& =|\tilde{I}'_{\lambda,n}(v_n)\varphi|+o(1)=|I'_\lambda(u_n)\varphi({\cdot}{-}y_n)|+o(1)\\ & \leq \|I'_\lambda(u_n)\|_{X^*}\|\varphi({\cdot}{-}y_n)\|_{X}+o(1)\\ & \leq C\|I'_\lambda(u_n)\|_{X^*}\|\varphi\|_{X}+o(1), \end{align*}

which implies that $I'_{\lambda,\infty }(v_0)=0$. That is, $v_0$ is a nontrivial critical point of functional $I_{\lambda,\infty }$ with $I_{\lambda,\infty }(v_0)=c_\lambda$. Recalling the definition of $c_\lambda$ and lemma 4.4, we have

\[ c_\lambda\leq \max_{t\in(0,+\infty)}I_\lambda(v_{0t})<\max_{t\in(0,+\infty)}I_{\lambda,\infty}(v_{0t})=I_{\lambda,\infty}(v_{0})=c_\lambda, \quad v_{0t}=t^2v_0(tx), \]

which is a contradiction. Therefore, $\{y_n\}$ is a bounded sequence.

Step 3. We show that $u_n\rightarrow u$ in $X$. Since we have known from step 2 that $\{y_n\}$ is a bounded sequence, there exists $u_0\in H^1(\mathbb {R}^2){\setminus} \{0\}$ such that $u_n\rightharpoonup u_0$ in $H^1(\mathbb {R}^2)$ and $u_n\rightarrow u_0$ a.e. in $\mathbb {R}^2$. Arguing as in step 2, we deduce that $u_n\rightarrow u_0$ in $X$. We conclude that $u_0$ is critical point of $I_\lambda$ with $I_\lambda (u_\lambda )=c_\lambda$.

4.1 Proof of theorem 2.2

In view of lemma 4.5, for fixed $\lambda \in (0,1]$, we have $I'_{\lambda }(u_\lambda )=0$ for some $u_\lambda \in X{\setminus} \{0\}$. Choosing a sequence $\{\lambda _n\}\subset (0,1]$ satisfying $\lambda _n\rightarrow 0^+$, there exists a sequence of nontrivial critical points $\{u_{\lambda _n}\}$(still denoted by $\{u_n\}$) of $I_{\lambda _n}$ with $I_{\lambda _n}(u_n)=c_{\lambda _n}$. We now show that $\{u_n\}$ is bounded in $H^1(\mathbb {R}^2)$. In fact, according to the definition of $I'_{\lambda _n}(u_n)u_n=0$, we have

(4.20)\begin{equation} V_0(u_n)={-}(\|\nabla u_n\|_2^2+\int_{\mathbb{R}^2}V(x)u_n^2\,{\rm d}\kern 0.06em x) -\lambda_n\|u_n\|_2^{5/2} +\lambda_n\|u_n\|_r^r+\int_{\mathbb{R}^2}f(u_n)u_n\,{\rm d}\kern 0.06em x. \end{equation}

Substituting $V_0(u_n)$ into $P_\lambda (u_n)=0$ gives

(4.21)\begin{align} & -\|\nabla u_n\|_2^2 +\frac{1}{2}\int_{\mathbb{R}^2}(\nabla V(x),x)u_n^2+\frac{1}{4}\|u_n\|_2^4\nonumber\\ & \quad +\int_{\mathbb{R}^2}f(u_n)u_n-2F(u_n)\,{\rm d}\kern 0.06em x +\frac{r-2}{r}\lambda_n\|u_n\|_r^r=0. \end{align}

We use the same fashion to get

(4.22)\begin{align} I_{\lambda_n}(u_n)& =\frac{1}{4}\|\nabla u_n\|_2^2+\frac{1}{4}\int_{\mathbb{R}^2}V(x)u_n^2\,{\rm d}\kern 0.06em x\nonumber\\ & \quad+\int_{\mathbb{R}^2}\frac{1}{4}f(u_n)u_n-F(u_n)\,{\rm d}\kern 0.06em x+\frac{3\lambda_n}{20}\|u_n\|_2^{5/2} +\frac{\lambda_n(r-4)}{4r}\|u_n\|_r^r. \end{align}

Putting (4.22) into (4.21), we get

(4.23)\begin{equation} \begin{aligned} & -[4I_{\lambda_n}(u_n)-\int_{\mathbb{R}^2}V(x)u_n^2\,{\rm d}\kern 0.06em x+\left(\frac{4}{r}-1\right)\lambda_n\|u_n\|_r^r\\ & \quad +\int_{\mathbb{R}^2}4F(u_n)-f(u_n)u_n\,{\rm d}\kern 0.06em x-\frac{3\lambda_n}{5}\|u_n\|_2^{5/2}] \\ & +\frac{1}{2}\int_{\mathbb{R}^2}(\nabla V(x),x)u_n^2 +\frac{1}{4}\|u_n\|_2^4 +\int_{\mathbb{R}^2}f(u_n)u_n-2F(u_n)\,{\rm d}\kern 0.06em x +\frac{r-2}{r}\lambda_n\|u_n\|_r^r=0, \end{aligned} \end{equation}

which can be rewritten as

(4.24)\begin{align} & \frac{1}{4}\|u_n\|_2^4+\int_{\mathbb{R}^2}V(x)u_n^2\,{\rm d}\kern 0.06em x+\left(2-\frac{6}{r}\right)\lambda_n\|u_n\|_r^r \nonumber\\ & \quad + \frac{1}{2}\int_{\mathbb{R}^2}(\nabla V(x),x)u_n^2\,{\rm d}\kern 0.06em x+\frac{3\lambda_n}{5}\|u_n\|_2^{5/2}\nonumber\\ & \quad +\int_{\mathbb{R}^2}2f(u_n)u_n-6F(u_n){\rm d}\kern 0.06em x=4I_{\lambda_n}(u_n). \end{align}

From condition ($f_3$), we can conclude that $f(u_n)u_n\geq 3F(u_n)$, see lemma 4.2 in [21]. Thus, (4.24) implies that $\{u_n\}$ is bounded in $L^2(\mathbb {R}^2)$ uniformly for $n$. Moreover, observe from (4.24) that

(4.25)\begin{equation} \begin{aligned} & \int_{\mathbb{R}^2}V(x)u_n^2\,{\rm d}\kern 0.06em x+\frac{1}{2}\int_{\mathbb{R}^2}\nabla V(x)\cdot xu_n^2\,{\rm d}\kern 0.06em x\leq C,\\ & \left(2-\frac{6}{r}\right)\lambda_n\|u_n\|_r^r\leq C,\quad \frac{3\lambda_n}{5}\|u_n\|_2^{5/2}\leq C,\\ & \int_{\mathbb{R}^2}2f(u_n)u_n-6F(u_n)\,{\rm d}\kern 0.06em x\leq C. \end{aligned} \end{equation}

In view of lemma 4.1, we have

(4.26)\begin{align} & 2I'_{\lambda_n}(u_n)u_n-P_{\lambda_n}(u_n)= 2\|\nabla u_n\|_2^2\nonumber\\ & \qquad +\int_{\mathbb{R}^2} (V(x)-\frac{1}{2}\nabla V(x)\cdot x) u_n^2\,{\rm d}\kern 0.06em x+\lambda_n\|u_n\|_2^{5/2} +V_0(u)\nonumber\\ & \qquad-\frac{1}{4}\|u_n\|_2^4-2\int_{\mathbb{R}^2}(f(u_n)u_n-F(u_n))\,{\rm d}\kern 0.06em x-\frac{2(r-1)\lambda_n}{r}\|u_n\|_r^r=0. \end{align}

For $t>0$, from (4.26) we deduce that

(4.27)\begin{align} & I_{\lambda_n}(u_n)-I_{\lambda_n}(u_{nt})= \frac{1-t^4}{2}\|\nabla u_n\|_2^2 +\frac{1}{2}\int_{\mathbb{R}^2} [V(x)-t^2V(t^{{-}1}x)] u_n^2\,{\rm d}\kern 0.06em x+\lambda_n\frac{2(1-t^{5/2})}{5}\|u_n\|_2^{5/2}\nonumber\\ & \quad+\frac{1-t^4}{4}V_0(u_n) +\frac{t^4\ln t}{4}\|u_n\|_2^4+\int_{\mathbb{R}^2}\left[\frac{F(t^2u_n)}{t^2}-F(u_n)\right]{\rm d}\kern 0.06em x-\frac{(1-t^{2(r-1)})\lambda_n}{r}\|u_n\|_r^r\nonumber\\ & \quad= \frac{1-t^4}{4}[2I'_{\lambda_n}(u_n)u_n-P_{\lambda_n}(u_n)]+\frac{1}{t^2}F(t^2u_n)\bigg]{\rm d}\kern 0.06em x\nonumber\\ & \quad+\frac{1}{4}\int_{\mathbb{R}^2}\left[(1+t^4)V(x)-2t^2V(t^{{-}1}x) +\frac{1-t^4}{2}\nabla V(x)\cdot x\right]u_n^2\,{\rm d}\kern 0.06em x\nonumber\\ & \quad+\frac{1-t^4+4t^4\ln t}{16}\|u_n\|_2^4+\int_{\mathbb{R}^2}\left[\frac{1-t^4}{2}f(u_n)u_n +\frac{t^4-3}{2}F(u_n)\right.\nonumber\\ & \quad+\left(\frac{3}{20}+\frac{t^4}{4}-\frac{2t^{5/2}}{5}\right)\lambda_n\|u_n\|_2^{5/2}+\left[\frac{(r-1)(1-t^4)}{2r}-\frac{(1-t^{2(r-1)})} {r}\right]\lambda_n\|u_n\|_r^r. \end{align}

where $u_{nt}(x)=t^2u_{n}(tx)$. Now we show that $\{\|\nabla u_n\|_2\}$ is bounded uniformly for $n$. Suppose by contradiction that $\|\nabla u_n\|_2\rightarrow \infty$. Take $t_n=(\sqrt {M}/\|\nabla u_n\|_2)^{1/2}$ for some $M>0$ large, then $t_n\rightarrow 0$. Obviously, $t_n^4\ln t_n\rightarrow 0$. Letting $t=t_n$ in (4.27), since $\{u_n\}$ is bounded in $L^2(\mathbb {R}^2)$ uniformly for $n$, by (V2) and ($f_1$) and (4.4), we have for large $t_n$

(4.28)\begin{align} & I_{\lambda_n}(u_n)-I_{\lambda_n}(t_n^2u_{nt_{n}})\nonumber\\ & = \frac{1}{4}\int_{\mathbb{R}^2}\left[V(x) +\frac{1}{2}\nabla V(x)\cdot x\right]u_n^2\,{\rm d}\kern 0.06em x\nonumber\\ & \quad+\frac{1}{16}\|u_n\|_2^4+\int_{\mathbb{R}^2}\left[\frac{1}{2}f(u_n)u_n-\frac{3}{2}F(u_n)\right]{\rm d}\kern 0.06em x,\nonumber\\ & \quad+ \frac{3}{20}\lambda_n\|u_n\|_2^{5/2}+\frac{r-3}{2r}\lambda_n\|u_n\|_r^r+o(1). \end{align}

Therefore, it follows from (3.3), (4.4), (4.25), (4.28) and (V1), ($f_1$), the Gagliardo–Nirenberg inequality that

(4.29)\begin{align} c_{\lambda_n}& =I_{\lambda_n}(u_n)\geq I_{\lambda_n}(t_n^2u_{nt_{n}})+o(1)\nonumber\\ & = \frac{t_n^4}{2}\|\nabla u_n\|_2^2+\frac{t_n^4}{4}V_0(u_n)-\frac{t_n^4\ln t_n}{4}\|u_n\|_2^4 -\frac{1}{t_n^2}\int_{\mathbb{R}^2}F(t_n^2u_n)\,{\rm d}\kern 0.06em x\nonumber\\ & \quad+\frac{1}{2}\int_{\mathbb{R}^2} t_n^2V(t_n^{{-}1}x) u_n^2\,{\rm d}\kern 0.06em x+\lambda_n\frac{2t_n^{5/2}}{5}\|u_n\|_2^{5/2}-\frac{t_n^{2r-2}}{r}\lambda_n\|u_n\|_r^{r}+o(1)\nonumber\\ & \geq\frac{t_n^4}{2}\|\nabla u_n\|_2^2-\frac{t_n^4}{4}V_2(u_n)-t_n^{2(p-1)}C\|u_n\|_p^p+o(1)\nonumber\\ & \geq\frac{M}{2}-\frac{t_n^4C}{4}\|u_n\|_2^3\|\nabla u_n\|_2-t_n^{2(p-1)}C\|u_n\|_2^2\|\nabla u_n\|_2^{p-2}+o(1)\nonumber\\ & \geq\frac{M}{2}-\frac{CM}{4\|\nabla u_n\|_2}\|u_n\|_2^3-\frac{CM^{(p-1)/2}}{\|\nabla u_n\|_2}\|u_n\|_2^2+o(1), \end{align}

which, together with the fact that $\|\nabla u_n\|_2\to +\infty$, implies a contradiction by letting $M>0$ large enough. Hence, $\{u_n\}$ is bounded in $H^1(\mathbb {R}^2)$. Arguing similarly as in lemma 4.5, we obtain that $u_n\rightarrow u_0$ in $X$. Moreover, by letting $n\rightarrow +\infty$, one has $c_{\lambda _n}\rightarrow c^*>a>0$ and $I_{\lambda _n}(u_{n})=c^*+o(1)$. Here, $a$ is positive number given in (4.2) below. For any $\varphi \in C_0^\infty (\mathbb {R}^2)$, we have

\[ I'_{\lambda_n}(u_{n})\varphi= I'(u_{n})\varphi+\lambda_n\|u_n\|_2^\frac{1}{2}\int_{\mathbb{R}^2}u_{n}\varphi\,{\rm d}\kern 0.06em x +\lambda_n\int_{\mathbb{R}^2}|u_n|^{r-2}u_n\varphi\,{\rm d}\kern 0.06em x=o(1)\|\varphi\|_X. \]

Thus, $\{u_n\}$ is a Palais–Smale sequence of $I$ with level $c^*$. Therefore, arguing as that in lemma 4.5, there exists a nontrivial $u_0\in X$ such that $I'(u_0)=0$ and $I(u_0)=c^*$. Now let us define the set of solutions

\[ \mathcal{S}:=\{u\in X{\setminus}\{0\}:\,I'(u)=0\}. \]

It is clear that $\mathcal {S}\not =\emptyset$ and $\mathcal {S}$ is bounded away from zero. To be precise, a short estimate yields that for any $u\in \mathcal {S}$, the following holds:

(4.30)\begin{equation} \|u\|\geq C\quad\text{for}\; \text{some}\ C>0. \end{equation}

We claim that

\[ c_*:=\inf\limits_{u\in\mathcal{S}}I(u)>0. \]

For a contradiction, we assume $c_*=0$. Then there exists $\{u_n\}\subset \mathcal {S}$ such that $I(u_n)\rightarrow 0$ as $n\rightarrow \infty$. In view of (4.24), $I(u)\geq \frac {1}{16}\|u\|_2^4$ for all $u\in \mathcal {S}$. So, $\|u_n\|_2^4\rightarrow 0$. Arguing as above, we have $u_n$ is bounded in $H^1(\mathbb {R}^2)$ uniformly for $n$. Using the Gagliardo–Nirenberg inequality:

\[ \|u_n\|_p^p\leq C_p\|u_n\|_2^2\|\nabla u_n\|_2^{p-2} \]

we infer that $u_n\rightarrow 0$ in $L^p(\mathbb {R}^2)$ for $p\in [2,+\infty )$. By (4.13), we have ${\int _{\mathbb {R}^2}f(u_n)u_n{\rm d}\kern 0.06em x=o(1)}$ for large $n$, and furthermore $u_n\rightarrow 0$ in $H^1(\mathbb {R}^2)$ which contradicts (4.30). The claim is true. Take finally a minimizing sequence $\{u_n\}\subset \mathcal {S}$ so that $I(u_n)\rightarrow c_*$. Similarly to lemma 4.5, there exists $u_{*}\in X$ so that $u_n\rightarrow u_{*}$ in $X$ and $I'(u_{*})=0$. It follows that $u_{*}$ is a positive ground state solution of problem (1.3).

5. The critical case

In this section, we are devoted to the proof of theorem 2.5. Differently from the subcritical case, we prove the existence of critical point of functional $I$ directly by using the Mountain-Pass Theorem (see theorem 3.4). It is not hard to check the mountain pass geometry of $I$ as similar arguments to the subcritical case. Therefore, recalling theorem 3.4, we can also get the associated $(Ce)_{c_{mp}}$ sequence $\{u_{n}\}$ with $c_{mp}\geq \alpha > 0$ and $c_{mp}:=\inf \limits _{\gamma \in \Gamma }\max \limits _{t\in [0,1]}I(\gamma (t)),$ where $\Gamma =\{\gamma \in C([0,1],X):\,\gamma (0)=0,\,\gamma (1)=e\}$. Now let us verify the $(Ce)_{c_{mp}}$ sequence $\{u_{n}\}$ contains a bounded subsequence in $H^1(\mathbb {R}^2)$.

Lemma 5.1 Assume the conditions of theorem 2.5 hold, then if $\nu >\frac {\alpha _0 \mathcal {S}_2^4}{4\pi \ln 2}$, we have

(5.1)\begin{equation} c_{mp}<\frac{\pi}{\alpha_0}, \end{equation}

where $\alpha _0$ has been given by assumption ($f_0$).

Proof. Recalling (2.3), there exists $\tilde {e}\in H_0^1(B_{\frac {1}{4}}(0))$ such that

(5.2)\begin{equation} \|\tilde{e}\|=1\quad\text{and}\quad \|\tilde{e}\|_2=\mathcal{S}_2^{{-}1}. \end{equation}

Then, similarly to lemma 4.2, for any $s>0$ we have

(5.3)\begin{align} I(s\tilde{e}) & =\frac{s^2}{2}\|\tilde{e}\|^2+\frac{s^4\nu}{4}[V_1(\tilde{e})-V_2(\tilde{e})]-\int_{\mathbb{R}^2}F(s\tilde{e}){\rm d}\kern 0.06em x\nonumber\\ & \leq\frac{s^2}{2}\|\tilde{e}\|^2 -\frac{s^4\nu}{8\pi}\int_{|x|\leq\frac{1}{4}}\int_{|y|\leq\frac{1}{4}}\ln\frac{1}{|x-y|}\tilde{e}^2(y)\tilde{e}^2(x)\,{\rm d}y\,{\rm d}\kern 0.06em x\nonumber\\ & \leq\frac{s^2}{2} -\frac{s^4\nu\ln2}{8\pi}\mathcal{S}_2^{{-}4}. \end{align}

And so,

\[ c_{mp}<\max_{s\in(0,+\infty)}\left\{\frac{s^2}{2} -\frac{s^4\nu\ln2}{8\pi}\mathcal{S}_2^{{-}4}\right\}. \]

A direct computation shows that

\[ \max_{s\in(0,+\infty)}\left\{\frac{s^2}{2} -\frac{s^4\nu\ln2}{8\pi}\mathcal{S}_2^{{-}4}\right\}=\frac{\pi\mathcal{S}_2^4}{2\ln 2 \cdot\nu}. \]

Thus, the conclusion follows from $\nu >\frac {\alpha _0 \mathcal {S}_2^4}{2\ln 2}$.

Lemma 5.2 Assume the conditions of theorem 2.5 hold, then any $(Ce)_{c_{mp}}$ sequence $\{u_n\}$ is bounded in $H^1(\mathbb {R}^2)$.

Proof. From ($f_{7}$) we can deduce that (see also lemma 2.3 in [20])

(5.4)\begin{align} & \frac{1-s^4}{4}f(t)t+F(st)-F(t)+\frac{(1-s^2)^2}{4}V(x)t^2\nonumber\\ & = \int_{1}^{s}\left[\frac{f(\tau t)-V(x)\tau t}{(\tau t)^3}-\frac{f(t)-V(x)t}{t^3}\right]\tau^3t^4\,{\rm d}\tau\geq 0,\quad \forall t\not=0,s\geq0. \end{align}

Then by the definition of $I$, we have

(5.5)\begin{align} C& \geq I(u_n)-\frac{1}{4}I'(u_n)u_n\nonumber\\ & = \frac{1}{4}\|\nabla u_n\|_2^2 +\int_{\mathbb{R}^2}\left(\frac{1}{4}f(u_n)u_n-F(u_n)+\frac{1}{4}V(x)u_n^{2}\right){\rm d}\kern 0.06em x, \end{align}

which implies that $\{\|\nabla u_n\|_2\}$ is bounded uniformly for $n$. In view of lemma 5.1, there exists $\varepsilon _0>0$ small such that

(5.6)\begin{equation} c_{mp}<\frac{\pi}{\alpha_0}(1-5\varepsilon_0)=:\tilde{c}<\frac{\pi}{\alpha_0}. \end{equation}

We now prove the boundedness of $\{u_n\}$ in $H^1(\mathbb {R}^2)$. Suppose by contradiction that $\|u_n\|\rightarrow \infty$. Set $v_n=\sqrt {4\tilde {c}} u_n/\|u_n\|$, then $\|v_n\|^2=4\tilde {c}$ and $\|\nabla v_n\|=o(1)$. So, we have $v_n\rightharpoonup v$ in $H^1(\mathbb {R}^2)$ and $v_n\rightarrow v$ a.e. in $\mathbb {R}^2$ after passing to a subsequence. Furthermore, we have either $\{v_n\}$ is vanishing, i.e.,

(5.7)\begin{equation} \lim\limits_{n\rightarrow\infty}\sup\limits_{y\in\mathbb{R}^2}\int_{B_2(y)}v_n^2(x)\,{\rm d}\kern 0.06em x=0 \end{equation}

or non-vanishing, i.e., there exist $\delta >0$ and a sequence $\{y_n\}\subset \mathbb {R}^2$ such that

(5.8)\begin{equation} \lim\limits_{n\rightarrow\infty}\int_{B_2(y_n)}v_n^2(x)\,{\rm d}\kern 0.06em x>\delta. \end{equation}

If (5.7) occurs, then it follows from Lions’ vanishing lemma (see [31]) that $v_n\rightarrow 0$ in $L^s(\mathbb {R}^2)$ for all $s>2$.

Moreover, a straightforward computation shows by assumption ($f_{7}$) and (5.4) that

(5.9)\begin{equation} I(u)\geq I(tu)+\frac{1-t^4}{4}I'(u)u,\quad \forall u\in X,t\geq 0. \end{equation}

It then follows from (3.3) and ($f_{7}$) that

(5.10)\begin{align} I(u_n)& \geq I(v_n)+\frac{1-16\tilde{c}^{2}\|u_n\|^{{-}4}}{4}I'(u_n)u_n\nonumber\\ & \geq \frac{1}{2}\|v_n\|^2+\frac{\nu}{4}V_0(v_n)-\int_{\mathbb{R}^2}F(v_n)\,{\rm d}\kern 0.06em x+o(1)\nonumber\\ & \geq \frac{1}{2}\|v_n\|^2-\int_{\mathbb{R}^2}\frac{1}{4}\left(f(v_n)v_n+V(x)v^2_n\right){\rm d}\kern 0.06em x+o(1). \end{align}

Since $\|\nabla v_n\|=o(1)$, by $(f_0)$ and Trudinger–Moser's inequality, for any $\varepsilon >0$ there exists $C_\varepsilon >0$ such that

\begin{align*} \int_{\mathbb{R}^2}f(v_n)v_n\,{\rm d}\kern 0.06em x& \leq \varepsilon\int_{\mathbb{R}^2}u_n^2\,{\rm d}\kern 0.06em x+C_\varepsilon\int_{\mathbb{R}^2}({\rm e}^{\alpha_0v_n^2}-1)v_n\,{\rm d}\kern 0.06em x\\ & \leq \varepsilon C\|v_n\|^2+C_\varepsilon\left[\int_{\mathbb{R}^2}({\rm e}^{\frac{4}{3}\alpha_0v_n^2}-1)\,{\rm d}\kern 0.06em x\right]^{\frac{3}{4}}\|v_n\|_4\\ & =\varepsilon C\|v_n\|^2+C_\varepsilon o(1), \end{align*}

which implies by the arbitrariness of $\varepsilon$ that

\[ \lim\limits_{n\rightarrow\infty}\int_{\mathbb{R}^2}f(v_n)v_n\,{\rm d}\kern 0.06em x=0, \]

which, together with (5.10), implies a contradiction due to (5.6) because if $\|\nabla v_n\|_2^2\to 0$, then $\int _{\mathbb {R}^2} V(x)v_n^2\,{\rm d}\kern 0.06em x\to 4\tilde c$. Thus non-vanishing must hold, namely relation (5.8) holds true. Since $\|\nabla v_n\|=o(1)$ and $\|v_n\|^2=4\tilde {c}$, in view of (5.10), the Trudinger–Moser inequality implies that there exists $C>0$ such that

(5.11)\begin{equation} V_1(v_n)\leq C. \end{equation}

Therefore, from ($f_1$), ($f_7$), (3.3), (5.8), (5.10) and (5.11) we infer that for $n$ large

(5.12)\begin{align} o(1)& =\frac{I'(u_n)u_n}{\|u_n\|^4}=\nu V_1(v_n)-\nu V_2(v_n)-\int_{\mathbb{R}^2}\frac{f(u_n)u_n}{\|u_n\|^4}\,{\rm d}\kern 0.06em x+o(1)\nonumber\\ & \leq C-\int_{\mathbb{R}^2}\frac{f(\sqrt{4\tilde{c}}\|u_n\|v_n)v^4_n}{\sqrt{4\tilde{c}}\|u_n\|^3v^3_n}\,{\rm d}\kern 0.06em x+o(1)\nonumber\\ & \leq C-\int_{B_2(y_n)}\frac{f(\sqrt{4\tilde{c}}\|u_n\|v_n)v^4_n}{\sqrt{4\tilde{c}}\|u_n\|^3v^3_n}\,{\rm d}\kern 0.06em x+o(1)={-}\infty. \end{align}

This is a contradiction and the conclusion follows.

Lemma 5.3 Assume the conditions of theorem 2.5 hold, there exists $u_0\in X{\setminus} \{0\}$ such that $I'(u_0)=0$ with $I(u_0)=c_{mp}$.

Proof. Assume that $\{u_{n}\}$ is a $(Ce)_{c_{mp}}$ sequence of functional $I$, then we have from lemma 5.2 that there exists $M>0$ such that $\|u_n\|\leq M$ uniformly for $n$. Similarly to lemma 4.5, the proof of this lemma will be also divided into three steps.

Step 1. We show that

(5.13)\begin{equation} \liminf\limits_{n\rightarrow\infty}\sup\limits_{y\in\mathbb{R}^2}\int_{B_2(y)}u_n^2(x)\,{\rm d}\kern 0.06em x>0. \end{equation}

Otherwise, it follows from Lions’ vanishing lemma (see [31]) that $u_n\rightarrow 0$ in $L^s(\mathbb {R}^2)$ for all $s>2$. By recalling (5.5) and lemma 5.1, there exists $\varepsilon _0>0$ small such that

(5.14)\begin{equation} \|\nabla u_n\|_2^2\leq 4c_{mp}<\frac{4\pi}{\alpha_0}(1-5\varepsilon_0). \end{equation}

In view of ($f_0$), for $s\in (1,2)$ and some $M_1>0$, we have

(5.15)\begin{equation} |f(u)|^s\leq C[{\rm e}^{\alpha_0(1+\varepsilon_0)su^2}-1],\quad |u|\geq M_1. \end{equation}

By choosing $s\in (1,2)$ such that

\[ (1+\varepsilon_0)(1-5\varepsilon_0)s<1, \]

we infer from lemma 2.1, (5.14) and (5.15), Hölder's inequality that

(5.16)\begin{align} \int_{|u_n|\geq M_1}f(u_n)u_n\,{\rm d}\kern 0.06em x & \leq \left(\int_{|u_n|\geq M_1}|f(u_n)|^s\,{\rm d}\kern 0.06em x\right)^{1/s}\|u_n\|_{s'}\nonumber\\ & \leq C\left(\int_{\mathbb{R}^2}[{\rm e}^{\alpha_0(1+\varepsilon_0)su_n^2}-1]\,{\rm d}\kern 0.06em x\right)^{1/s}\|u_n\|_{s'}\nonumber\\ & \leq C\|u_n\|_{s'}=o(1). \end{align}

Here, $s'=\frac {s}{s-1}\in (2,+\infty )$. So, for any $\varepsilon >0$, by ($f_2$), there exist $\bar {M}_\varepsilon >0$ small and $C_\varepsilon >0$ such that

(5.17)\begin{align} \int_{\mathbb{R}^2}f(u_n)u_n\,{\rm d}\kern 0.06em x& \leq\int_{|u_n|\leq\bar{M}_\varepsilon}\varepsilon u_n^2\,{\rm d}\kern 0.06em x +\int_{ \bar{M}_\varepsilon\leq|u_n|\leq M_1}f(u_n)u_n\,{\rm d}\kern 0.06em x \nonumber\\ & +\int_{|u_n|\geq M_1}f(u_n)u_n\,{\rm d}\kern 0.06em x \leq\varepsilon C+C_\varepsilon\int_{\tilde{M}_\varepsilon\leq|u_n|\leq M_\varepsilon}|u_n|^p\,{\rm d}\kern 0.06em x+\varepsilon\nonumber\\ & \leq\varepsilon C+C_\varepsilon o(1)+\varepsilon,\quad p\in(2,\infty). \end{align}

Thus,

(5.18)\begin{equation} \lim\limits_{n\rightarrow\infty}\int_{\mathbb{R}^2}f(u_n)u_n\,{\rm d}\kern 0.06em x=0. \end{equation}

Thus, it follows from (5.18) and (3.3) that

(5.19)\begin{equation} \|u_n\|^2+\nu V_1(u_n)=I_\lambda'(u_n)u_n+\nu V_2(u_n)+\int_{\mathbb{R}^2}f(u_n)u_n\,{\rm d}\kern 0.06em x+o(1)=o(1), \end{equation}

which implies that $u_n\rightarrow 0$ in $H^1(\mathbb {R}^2)$, $V_1(u_n)\rightarrow 0$. Moreover,

(5.20)\begin{equation} \int_{\mathbb{R}^2}F(u_n)\,{\rm d}\kern 0.06em x\leq \frac{1}{4}\int_{\mathbb{R}^2}\left(f(u_n)u_n+V(x)u^2_n\right){\rm d}\kern 0.06em x=o(1), \end{equation}

and then

\[ \lim_{n\rightarrow\infty}I(u_n)=0=c_{mp} \]

which contradicts $c_{mp}>0$. So (5.13) holds true. After passing to a subsequence, there exists a sequence $\{y_n\}\subset \mathbb {R}^2$ such that $v_n=u_n(\cdot +y_n)$ is still bounded in $H^1(\mathbb {R}^2)$ and $v_n\rightharpoonup v_0\in H^1(\mathbb {R}^2){\setminus} \{0\}$ and $v_n\rightarrow v_0$ a.e. in $\mathbb {R}^2$.

Step 2. We claim that $\{y_n\}$ is bounded. Assume by contradiction that $y_n\rightarrow +\infty$. In view of (5.17), we can easily see that $\int _{\mathbb {R}^2}f(u_n)u_n\,{\rm d}\kern 0.06em x\leq C$ uniformly for $n$. So, by the definition of $I$, we have

\begin{align*} \frac{\nu}{4}V_1(v_n)& =\frac{\nu}{4}V_1(u_n)\\ & =I(u_n)+\frac{\nu}{4}V_2(u_n) +\int_{\mathbb{R}^2}F(u_n)\,{\rm d}\kern 0.06em x-\frac{1}{2}\|u_n\|^2\\ & \leq I(u_n)+\frac{\nu}{4}V_2(u_n) +\int_{\mathbb{R}^2}\frac{1}{4}\left(f(u_n)u_n+V(x)u_n^2\right){\rm d}\kern 0.06em x-\frac{1}{2}\|u_n\|^2, \end{align*}

which implies that $V_1(v_n)$ is bounded uniformly for $n$, due to the boundedness of $\{u_n\}$ in $H^1(\mathbb {R}^2)$. It follows from lemma 3.1 that $\|v_n\|_{\star }$ is also bounded in $n$, and so $\{v_n\}$ is bounded in $X$. Up to subsequence, there exists $v_0\in X$ such that $v_n\rightharpoonup v_0$ in $X$ and $v_n\rightarrow v_0$ in $L^s(\mathbb {R}^2)$ for $s\geq 2$ as $n\rightarrow \infty$. Arguing as in lemma 4.5, we obtain that

(5.21)\begin{equation} |\tilde{I}'(v_n)(v_n-v_0)|\rightarrow0,\quad\text{as}\ n\rightarrow\infty. \end{equation}

where

\[ \tilde{I}(v_n):=\frac{1}{2}\int_{\mathbb{R}^2}|\nabla v_n|^2\,{\rm d}\kern 0.06em x+\frac{1}{2}\int_{\mathbb{R}^2}V(x+y_n)v_n^2\,{\rm d}\kern 0.06em x+\frac{\nu}{4}V_0(v_n) -\int_{\mathbb{R}^2}F(v_n)\,{\rm d}\kern 0.06em x. \]

Based on the fact that $v_n\rightarrow v_0$ in $L^s(\mathbb {R}^2)$ for $s\geq 2$ as $n\rightarrow \infty$, by assumption (V2) and (3.2) one has

(5.22)\begin{equation} \int_{\mathbb{R}^2}V(x+y_n)v_n(v_n-v_0)\,{\rm d}\kern 0.06em x\rightarrow0,\quad \left|\frac{1}{4}V'_2(v_n)(v_n-v_0)\right|\rightarrow0, \quad\text{as}\ n\rightarrow\infty . \end{equation}

Arguing similarly as (5.16) and (5.17), we have

(5.23)\begin{equation} \int_{\mathbb{R}^2}f(v_n)(v_n-v_0)\,{\rm d}\kern 0.06em x\rightarrow0,\quad \text{as}\ n\rightarrow\infty. \end{equation}

By the definition of $\tilde {I}(v_n)$, we have

(5.24)\begin{align} & B_1(v_n^2,v_n(v_n-v_0))+\|\nabla (v_n-v_0)\|_2^2\nonumber\\ & = \tilde{I}'(v_n)(v_n-v_0)-\int_{\mathbb{R}^2}V(x+y_n)v_n(v_n-v_0)\,{\rm d}\kern 0.06em x\nonumber\\ & \quad+B_2(v_n^2,v_n(v_n-v_0))+\int_{\mathbb{R}^2}f(v_n)(v_n-v_0)\,{\rm d}\kern 0.06em x +o(1). \end{align}

Combining (5.21)–(5.24), we infer that $\|\nabla (v_n-v_0)\|_2^2\rightarrow 0$ and $B_1(v_n^2,v_n(v_n-v_0))\rightarrow 0$ as $n\rightarrow \infty$, and then $v_n\rightarrow v_0$ in $H^1(\mathbb {R}^2)$. Recalling lemma 3.1, we have $\|v_n-v_0\|_\star \rightarrow 0$ as $n\rightarrow \infty$. We therefore deduce that $v_n\rightarrow v_0$ in $X$.

Furthermore, similarly to the proof of lemma 4.5, we obtain that $v_0$ is a nontrivial critical point of functional $I_\infty$ and

\[ I_\infty(v_0)=\lim\limits_{n\rightarrow\infty}I(v_n)=c_{mp}, \]

where

\[ I_\infty(v_0):=\frac{1}{2}\int_{\mathbb{R}^2}|\nabla v_0|^2\,{\rm d}\kern 0.06em x+\frac{1}{2}\int_{\mathbb{R}^2}V_\infty v_0^2\,{\rm d}\kern 0.06em x+\frac{\nu}{4}V_0(v_0) -\int_{\mathbb{R}^2}F(v_0)\,{\rm d}\kern 0.06em x. \]

Using the conditions of theorem 2.5, like (5.9) we can also estimate

\[ I_\infty(u)\geq I_\infty(tu)+\frac{1-t^4}{4}I'_\infty(u)u, \quad \forall u\in X, t\geq0, \]

which yields

(5.25)\begin{equation} I_\infty(v_0)\geq \max_{t\geq0}I_\infty(tv_0). \end{equation}

Recalling the definition of $c_{mp}$, we have

\[ c_{mp}\leq \max_{t\in(0,+\infty)}I(t v_{0})<\max_{t\in(0,+\infty)}I_{\infty}(tv_{0})\leq I_{\infty}(v_{0})=c_{mp}, \]

which is a contradiction. Therefore, $\{y_n\}$ is a bounded sequence.

Step 3. We show that $u_n\rightarrow u$ in $X$ and then $u$ is critical point of $I$ with $I(u)=c_{mp}$. The proof is the same as that in lemma 4.5, and we omit it.

5.1 Proof of theorem 2.5

Similarly to theorem 2.2, let us define the set of solutions

\[ \mathcal{\tilde{S}}:=\{u\in X{\setminus}\{0\}:\,I'(u)=0\}. \]

By lemma 5.3, $\mathcal {\tilde {S}}\not =\emptyset$. For $u\in \mathcal {\tilde {S}}$ by ($f_0$), for any $\varepsilon >0$ there exists $C_\varepsilon >0$ such that

\begin{align*} \|u\|^2& \leq \|u\|^2+\nu V_1(u)\\ & \leq\varepsilon\int_{\mathbb{R}^2}u^2\,{\rm d}\kern 0.06em x+C_\varepsilon\int_{\mathbb{R}^2}({\rm e}^{\alpha_0 u^2}-1)u^3\,{\rm d}\kern 0.06em x+\nu V_2(u)\\ & \leq \varepsilon\|u\|^2+C_\varepsilon\left(\int_{\mathbb{R}^2}\,{\rm e}^{2\alpha_0 u^2}-1)\right)^{\frac{1}{2}}\|u\|_6^3+\nu C\|u\|^4\\ & \leq\varepsilon\|u\|^2+C_\varepsilon C\|u\|^3+\nu C\|u\|^4, \end{align*}

which implies that there exists $C>0$ such that $\|u\|\geq C$ for any $u\in \mathcal {\tilde {S}}$.

We claim that

(5.26)\begin{equation} \tilde{c}_*:=\inf\limits_{u\in\mathcal{\tilde{S}}}I(u)>0. \end{equation}

Assume by contradiction that $\tilde {c}_*=0$ and $\{u_n\}\subset \mathcal {\tilde {S}}$ satisfies $I(u_n)\rightarrow 0$ as $n\rightarrow \infty$. Recalling lemma 5.2, we deduce that $\{u_n\}$ is bounded in $H^1(\mathbb {R}^2)$ uniformly for $n$, and then by (5.5), $I(u_n)\geq \frac {1}{4}\|\nabla u_n\|_2^2$. Obviously, $\|\nabla u_n\|_2^2\rightarrow 0$. By the Gagliardo–Nirenberg inequality, we have $u_n\rightarrow 0$ in $L^p(\mathbb {R}^2)$ for $p>2$. The Trudinger–Moser inequality implies $\int _{\mathbb {R}^2}f(u_n)u_n{\rm d}\kern 0.06em x=o(1)$ as $n\rightarrow \infty$. And so using $I'(u_n)=0$ and (3.3), we have $V_1(u_n)\rightarrow 0$ and $u_n\rightarrow 0$ in $H^1(\mathbb {R}^2)$ which is impossible. (5.26) holds true. It is easy to obtain from lemma 5.1 that $\tilde {c}_*<\frac {\pi }{\alpha _0}$.

Finally, let $\{u_n\}\subset \mathcal {\tilde {S}}$ be a minimizing sequence, hence $I(u_n)\rightarrow \tilde {c}_*\in \left (0,\frac {\pi }{\alpha _0}\right )$. By lemma 5.2, we know that $\{u_n\}$ is bounded in $H^1(\mathbb {R}^2)$. Similar to the proof of lemma 5.3, there exists $\tilde {u}_{*}\in X$ such that $u_n\rightarrow \tilde {u}_{*}$ in $X$ and $I'(\tilde {u}_{*})=0$ and $I(\tilde {u}_{*})=\tilde {c}_*$. Thus, $\tilde {u}_{*}$ is a positive ground state solution of problem (1.3). The proof is now complete.