Concentration and multiplicity of solutions for fractional double phase problems

Abstract

In the present paper, we consider the following fractional double phase problem with nonlocal reaction:

\begin{align*}\qquad \quad \left\{\begin{array}{lll}(- \Delta)_p^s u + (- \Delta)_q ^s u + V(\varepsilon x) (|u|^{p - 2} u + |u|^{q - 2} u) = \left(\frac{1}{|x|^\mu} * F(u)\right) f(u) & \ \text{in}\ \mathbb{R}^N, \\u \in W^{s,p}(\mathbb{R}^N) \cap W^{s,q}(\mathbb{R}^N) ,\ u \gt 0 & \ \text{in}\ \mathbb{R}^N,\end{array}\right\}\end{align*}

where ɛ is apositive parameter, $0 \lt s \lt 1$, $2 \leqslant p \lt q \lt \min\{2p, N / s\}$, $0 \lt \mu \lt sp$, $(- \Delta)_t^s$ $(t \in \left\{p,q\right\})$ is the fractional t-Laplace operator, the reaction term $f : \mathbb{R} \mapsto \mathbb{R}$ is continuous, and the potential $V \in C (\mathbb{R}^N , \mathbb{R})$ satisfying a local condition. Using a variational approach and topological tools (the non-standard C¹-Nehari manifold analysis and the abstract category theory), multiplicity of positive solutions and concentration properties for the above problem are established. Our results extend and complement some previous contributions related to double phase variational integrals.

1. Introduction

1.1. Features of the paper and historical comments

In this paper, we are concerned with the study of concentration and multiplicity properties of solutions for a class of fractional double phase problems with Choquard nonlinearity. The features of this paper are the following:

(i) the presence of several nonlocal operators with different growth, which generates a double phase associated energy;

(ii) the analysis developed in this paper is concerned with the combined effects of a nonstandard fractional operator with unbalanced growth and a nonlocal Choquard term;

(iii) the potential describing the absorption term satisfies a local condition and no information on the behaviour of the potential at infinity is imposed;

(iv) the main concentration property creates a bridge between the global maximum point of the solution and the global minimum of the potential;

(v) since the nonlinearity is only assumed to be continuous, one cannot apply the standard C ¹-Nehari manifold arguments due to the lack of differentiability of the associated Nehari manifold;

(vi) our analysis combines the nonlocal nature of the fractional (p, q)-operator and of the Choquard nonlinearity with the local perturbation in the absorption term.

The problem studied in this paper combines both the above-mentioned features. More exactly, we are interested in concentration phenomena associated with a nonlinear Choquard problem driven by fractional Laplace operators with different power. This integro-differential operator appears in several relevant models of applied sciences. We only recall that the fractional power of the Laplace operator is the infinitesimal generator of the Lévy stable diffusion process and arises in anomalous diffusion in plasma, population dynamics, geophysical fluid dynamics, flame propagation, chemical reactions in liquids, and American options in finance. Moreover, fractional Sobolev spaces have been well known since the beginning of the last century, especially within the framework of harmonic analysis. The starting point in the study of fractional problems is attributed to the pioneering papers of Caffarelli et al. [20–22]. For a comprehensive introduction to the study of fractional equations and the use of variational methods in the treatment of these problems, we refer to the monographs by Di Nezza, Palatucci, and Valdinoci [27] and Molica Bisci, Rădulescu, and Servadei [45]. Therefore, the nonlocal operators are becoming increasingly popular in applied sciences, theoretical research, and real-world applications.

Since the content of this paper is at interplay between ‘double phase problems’ and ‘Choquard problems’, we now recall some pioneering achievements in these fields.

We start with a short description on the development of double phase problems. To the best of our knowledge, the first contributions to this field are due to the study by Ball [13, 14], in relationship with problems in nonlinear elasticity and composite materials. Let $\Omega\subset\mathbb{R}^N$ be a bounded domain with a smooth boundary. If $u:\Omega\to\mathbb{R}^N$ is the displacement and if Du is the N × N matrix of the deformation gradient, then the total energy can be represented by an integral of the type

(1.1)\begin{equation}I(u)=\int_\varOmega f(x,\text{D}u(x))dx,\end{equation}

where the energy function $f=f(x,\xi):\Omega\times\mathbb{R}^{N\times N}\to\mathbb{R}$ is quasiconvex with respect to ξ, see the study by Morrey [47]. A simple example considered by Ball is given by functions f of the type

\begin{align*}f(\xi)=g(\xi)+h({\mathrm{det}}\,\xi),\end{align*}

where ${\mathrm{det}}\,\xi$ is the determinant of the N × N matrix ξ and g, h are non-negative convex functions, which satisfy the growth conditions

\begin{align*}g(\xi)\geqslant c_1\,|\xi|^p;\quad\lim_{t\to+\infty}h(t)=+\infty,\end{align*}

where c ₁ is a positive constant and $1 \lt p \lt N$. The condition $p\leqslant N$ is necessary to study the existence of equilibrium solutions with cavities, that is, minima of the variational integral (1.1) that are discontinuous at one point where a cavity forms; in fact, every u with finite energy belongs to the Sobolev space $W^{1,p}(\Omega,\mathbb{R}^N)$, and thus it is a continuous function if p > N. In accordance with these problems arising in nonlinear elasticity, Marcellini [41, 42] considered continuous functions $f=f(x,u)$ with unbalanced growth that satisfy

\begin{align*}c_1\,|u|^p\leqslant |f(x,u)|\leqslant c_2\,(1+|u|^q)\quad\text{for all}\ (x,u)\in\Omega\times\mathbb{R},\end{align*}

where c ₁, c ₂ are positive constants and $1\leqslant p\leqslant q$. Recently, a great deal of works have enriched the mathematical analysis of nonlinear models with unbalanced growth; we refer to the works of Bahrouni, Rădulescu, and Repovš [12]; Crespo-Blanco, Gasínski, Harjulehto; and Winkert [23]; Liu and Papageorgiou [40]; and Papageorgiou, Pudełko, Rădulescu [49]. In addition, an overview of recent developments on the regularity of nonlocal double problems can be found in Byun, Ok, and Kyeong [19] and De Filippis and Palatucci [25].

In the pioneering works of Fröhlich [30] and Pekar [50], the following Choquard equation was introduced,

(1.2)\begin{align} -\Delta u +u = \left(\frac{1}{|x|}* |u|^2\right) u \ \text{in}\ \mathbb{R}^3, \end{align}

which models quantum polaron and corresponds to the study of the free electrons in ionic lattices interacting with phonons associated with deformations of the lattices or with the polarization created on the medium. Choquard also used it to study steady states of the one component plasma approximation in the Hartree–Fock theory describing an electron trapped in its own hole [37]. Equation (1.2) is also called the Schrödinger–Newton equation which combines the Schrödinger equation of quantum physics with nonrelativistic Newtonian gravity. Such a model can also be deduced from the Einstein–Klein–Gordon and Einstein–Dirac systems related to boson stars and the collapse of galaxy fluctuations of scalar field dark matter; we bring the reader’s attention to Elgart and Schlein [29], Jones [36], Lions [39], Penrose [51, 56], and Schunck and Mielke [54].

In conclusion, based on the historical research background of the fractional double problem and the Choquard problem and the related theoretical foundation, focusing on the characteristics of such problems and the combined effects produced by their combination, we will apply analytical, topological, and variational methods and develop some new techniques to devote ourselves to the study of new phenomena of nonlocal double problems with nonlocal reaction terms, when the potential V satisfies a general local condition and the nonlinearity f possesses only the continuity property.

1.2. Statement of the problem and main result

In this paper, we focus on the existence, multiplicity, and concentration behaviour of positive solutions for the nonlinear fractional (p, q)-Choquard problem:

(1.3)\begin{align} \left\{ \begin{array}{lll} (- \Delta)_p^s u + (- \Delta)_q ^s u + V(\varepsilon x) (|u|^{p - 2} u + |u|^{q - 2} u) = \left(\frac{1}{|x|^\mu} * F(u)\right) f(u) & \ \text{in}\ \mathbb{R}^N, \\ u \in W^{s,p}(\mathbb{R}^N) \cap W^{s,q}(\mathbb{R}^N) ,\ u \gt 0 & \ \text{in}\ \mathbb{R}^N, \end{array} \right\} \end{align}

where ɛ is a positive parameter, $0 \lt s \lt 1$, $2 \leqslant p \lt q \lt \min\{2p, N / s\}$, $0 \lt \mu \lt sp$, $(- \Delta)_t^s$ $(t \in \left\{p,q\right\})$ is the fractional t-Laplace operator, $V : \mathbb{R}^N \mapsto \mathbb{R}$ and $f : \mathbb{R} \mapsto \mathbb{R}$ are continuous functions, F is the primitive function of f, and $*$ denotes the convolution product.

In the local setting corresponding to $s \rightarrow 1^-$ (up to normalization), the double phase problem (1.3) is motivated by numerous models arising in mathematical physics. For instance, we can refer to the following Born–Infeld equation [16] that appears in electromagnetism:

\begin{equation*}-\mathrm{div}\,\left(\frac{\text{D}u}{(1-2\vert\text{D}u\vert^2)^{1/2}}\right)=h(u)\;\text{in}\;\varOmega.\end{equation*}

Indeed, by the Taylor formula, we have

\begin{align*}(1-x)^{-1/2}=1+\frac x2+\frac{3}{2\cdot 2^2}x^2+\frac{5!!}{3!\cdot 2^3}x^3+\cdots +\frac{(2n-3)!!}{(n-1)!2^{n-1}}x^{n-1}+\cdots\ \text{for}\ |x| \lt 1.\end{align*}

Taking $x=2\vert\text{D}u\vert^2$ and adopting the first-order approximation, we get a particular case of the problem (1.3) for p = 2 and q = 4. Furthermore, the n-th order approximation problem is driven by the multi-phase differential operator

\begin{align*}-\Delta u-\Delta_4u-\frac 32\Delta_6u-\cdots -\frac{(2n-3)!!}{(n-1)!}\Delta_{2n}u.\end{align*}

Finally, we also refer to the following fourth-order relativistic operator:

\begin{align*}u \mapsto {\mathrm{div}\, }\left(\frac{|\nabla u|^2}{(1-|\nabla u|^4)^{3/4}} \nabla u\right),\end{align*}

which describes large classes of phenomena arising in relativistic quantum mechanics. Clearly, we can use the Taylor formula to conclude that the fourth-order relativistic operator can be approximated by the following double phase operator:

\begin{align*}u \mapsto \Delta_4 u + \frac{3}{4}\Delta_8 u.\end{align*}

The purpose of this paper is to study a class of fractional unbalanced double phase problems in which the nonlocal term appears also in the nonlinear part. Problem (1.3) is closely related to the analysis of nonlinear problems and stationary waves for models arising in mathematical physics, such as phase transitions, anomalous diffusion, composite materials, image processing, fractional quantum mechanics in the study of particles on stochastic fields, fractional superdiffusion, and fractional white-noise limit, see the study by Pucci and Saldi [52]. For more details, we refer interested readers to the preliminary introduction of this topic in [27, 45]. From the point of view of physics, in the semi-classical sense (namely, as $ \varepsilon \rightarrow 0$), it is of great significance to study the existence and shape of the standing wave solutions of problem (1.3), which can be used to describe the transition relationship between quantum mechanics and classical mechanics. From a mathematical point of view, in the framework of this semi-classical state, we can obtain more dynamic behaviour information, such as concentration, convergence, decay, bifurcation, and other properties.

Note that, if $p=q=2$, after rescaling, problem (1.3) reduces to the following nonlinear fractional Choquard equation:

(1.4)\begin{align} \varepsilon^{2s}(- \Delta)^s u + V(x) u =\varepsilon^{\mu -N} \left(\frac{1}{|x|^\mu} * F(u)\right) f(u) & \ \text{in }\mathbb{R}^N. \end{align}

When ɛ = 1, $F(u)=|u|^p$, and $f(u)=|u|^{p-2}u$, d’Avenia, Siciliano and Squassina [24] dealt with the regularity, existence and non-existence, symmetry, and decay properties of solutions to problem (1.4). Under assumption that the potential V has a local minimum, Ambrosio [9] established the multiplicity and concentration of positive solutions to problem (1.4) via the penalization technique and Ljusternik-Schnirelmann theory.

When $p=q \neq 2$, after rescaling, problem (1.3) boils down to the following nonlinear fractional Choquard problem:

(1.5)\begin{align} \left\{ \begin{array}{lll} \varepsilon^{sp}(- \Delta)_p^s u + V(x)|u|^{p-2}u = \varepsilon^{\mu-N}\left(\frac{1}{|x|^\mu} * F(u)\right) f(u) & \ \text{in }\mathbb{R}^N, \\ u \in W^{s,p}(\mathbb{R}^N),\ u \gt 0 & \ \text{in } \mathbb{R}^N. \end{array} \right\} \end{align}

In [7], Ambrosio investigated the existence, multiplicity, and concentration of positive solutions for the fractional Choquard problem (1.5) by assuming that the potential $V :\mathbb{R}^N \mapsto \mathbb{R}$ fulfills the following condition due to the study by Rabinowitz [53]:

1. (V) $V_\infty=\liminf_{|x| \rightarrow +\infty} V(x) \gt \inf_{x\in\mathbb{R}^N}V(x)$.

These solutions concentrate at global minimum points of V under the global hypothesis (V). In the local sense s = 1, problem (1.5) becomes the following quasi-linear Choquard problem with the p-Laplace operator

(1.6)\begin{align} \left\{ \begin{array}{lll} - \varepsilon^{p}\Delta_p u + V(x)|u|^{p-2}u = \varepsilon^{\mu-N}\left(\frac{1}{|x|^\mu} * F(u)\right) f(u) & \ \text{in }\mathbb{R}^N, \\ u \in W^{1,p}(\mathbb{R}^N),\ u \gt 0 & \ \text{in } \mathbb{R}^N, \end{array} \right\} \end{align}

for which several existence, multiplicity, and concentration results of positive solutions have been studied by different authors, under suitable assumptions on the potential function V and the reaction f; see, for instance, Alves and Yang [3–5]. On the other hand, if s = 1 and $1 \lt p \lt q \lt N$, after rescaling, problem (1.3) reduces to the following (p, q)-Laplace problem:

(1.7)\begin{align} \left\{ \begin{array}{lll} - \varepsilon^{p}\Delta_p u - \varepsilon^q \Delta_q u + V(x) (|u|^{p - 2} u + |u|^{q - 2} u) =\varepsilon^{\mu -N} \left(\frac{1}{|x|^\mu} * F(u)\right) f(u) & \ \text{in }\mathbb{R}^N, \\ u \in W^{1,p}(\mathbb{R}^N) \cap W^{1,p}(\mathbb{R}^N) ,\ u \gt 0 & \ \text{in }\mathbb{R}^N. \end{array} \right\} \end{align}

Very recently, by assuming that the potential V and the nonlinear reaction f satisfy the following conditions:

• $(\overline{V})$ $V \in C(\mathbb{R}^N,\mathbb{R})$ and $+ \infty \gt V_\infty=\liminf_{|x| \rightarrow +\infty} V(x) \gt \inf_{x\in\mathbb{R}^N}V(x) \gt 0$;

• (f)$~~f \in C^1(\mathbb{R},\mathbb{R})$,

Zhang, Zhang, and Rădulescu studied the multiplicity and concentration behavior of positive solutions to problem (1.7) in [60].

For related concentration and multiplicity properties of solutions, we refer to the recent paper by Alves and de Morais Filho [2]; Alves, Ambrosio, and Isernia [1]; Ambrosio [8]; Ambrosio, Isernia, and Rădulescu [10]; Ambrosio and Rădulescu [11]; Del Pino and Felmer [26]; Gao, Tang, and Chen [31]; Gu and Tang [32]; Ji and Rădulescu [34, 35]; Moroz and Van Schaftingen [46]; Zhang and Zhang [61]; Zhang, Zhang, and Rădulescu [59]; and Zhang, Tang, and Rădulescu [62]. We also cite the study by Mingione and Rădulescu [44] for an overview of recent results concerning elliptic variational problems with nonstandard growth and nonuniform ellipticity.

Inspired by the above works, in this paper, we are interested in studying the multiplicity and concentration behaviour as $\varepsilon \rightarrow 0$ of positive solutions for problem (1.3), when we suppose a local condition on the potential V, and the nonlinear term f is only assumed to be continuous (without C ¹ differentiability). Precisely, we first impose that V is a continuous map satisfying the following assumptions:

• there exists $V_0 \gt 0$ such that $V_0 := \inf_{x \in \mathbb{R}^N} V(x)$;

• there exists an open bounded domain $\Lambda \subset \mathbb{R}^N$ such that

  \begin{align*}V_0 \lt \min_{\partial \Lambda}V\quad \ \text{and }\quad M :=\{x \in \Lambda : V(x) = V_0\} \neq \emptyset.\end{align*}

Next, we assume that f is a merely continuous function that verifies the following conditions:

1. (f₁) $\lim_{t \rightarrow 0} \frac{|f(t)|}{|t|^{p - 1}} = 0$;

2. (f₂) there exists $\nu \in (q, q_s^*(2 - \mu / N) / 2)$ such that

   \begin{align*}\lim_{|t| \rightarrow + \infty} \frac{|f(t)|}{|t|^{\nu - 1}} = 0,\end{align*}

   where $q_s^* = Nq / (N - sq)$;

3. (f₃) $0 \lt p F(t) : = p \int^t_0 f(\tau) d\tau \leqslant f(t)t$ for all t > 0;

4. (f₄) the map $t \mapsto \frac{f(t)}{t^{q - 1}}$ is increasing for all t > 0.

Remark 1.1. We would like to point out that, since the hypotheses on V and f are different from [7, 53, 59–61], our arguments are totally distinct with them and improve the previous results for the singularly perturbed fractional problem with nonlocal Choquard reaction, in the sense that we establish multiplicity results and concentration behaviour for the fractional (p, q)-problems involving continuous nonlinearity and by imposing a local condition on the potential V. Compared with the local case s =1, we point out that our result improves theorem 1.2 in [60]. In addition, we believe that the ideas contained here can be used in other cases to study problems driven by more general operators, under local conditions on the potential V and without the differentiability of the nonlinearity f.

In order to look for positive solutions of problem (1.3), we may assume that $f(t) = 0$ for all $t \leqslant 0$.

Recall that if A is a closed subset of a topological space Y, then we use ${\mathrm{cat}}_Y(A)$ to denote the Ljusternik–Schnirelmann category of A in Y, that is, the smallest number of closed and contractible sets in Y which cover A. For more details, we refer to the study by Willem [57].

Our main result in this paper establishes the following concentration and multiplicity properties.

Theorem 1.2 Let $0 \lt \mu \lt sp.$ Assume that the nonlinearity f fulfils $(f_1)$–$(f_4)$ with $\nu \lt (N - \mu) q / (N - sq)$ and the potential V verifies hypotheses $(V_1)$–$(V_2).$ Then for all δ > 0 such that $M_\delta :=\left\{x \in \mathbb{R}^N : {\mathrm{dist}\,}(x,M) \leqslant \delta\right\} \subset \Lambda,$ there exists $\varepsilon_\delta \gt 0$ such that, for any $\varepsilon \in (0,\varepsilon_\delta)$ problem (1.3) has at least $cat_{M_\delta}(M)$ positive solutions. Moreover, if u_ɛ denotes one of these solutions and $x_\varepsilon \in \mathbb{R}^N$ is the global maximum point of $u_\varepsilon,$ then $\lim_{\varepsilon \rightarrow 0} V(\varepsilon x_\varepsilon) = V_0.$

The proof of theorem 1.2 is based on topological and variational methods and refined analytic techniques. Let us now take the next steps to outline our strategies and methods for proving theorem 1.2:

(1) concerning our variational approach: because there is no information on the behaviour of V at infinity, as in [26], we first modify in a convenient way the nonlinear reaction outside the set Λ, and then we will study an auxiliary problem. The main characteristic of the corresponding modified energy functional is that it verifies all the conditions of the mountain pass theorem [6]. Finally, we prove that, for ɛ > 0 sufficiently small, the solutions of the auxiliary problem are indeed solutions of the original one;

(2) topological techniques: in order to get multiple solutions of the auxiliary problem, we will use the generalized Neahri manifold method and some well-known topological techniques proposed by Benci and Cerami in [15] based on accurate comparisons between the category of some sub-level sets of the modified functional and the category of the set M. Note that the nonlinearity is only continuous, and we stress that standard C ¹-Nehari manifold arguments as in [60] cannot be employed in our setting due to the lack of differentiability of the associated Nehari manifold. To this end, we take inspiration by some ideas developed in [55] and make use of some abstract critical point results obtained in [55] to overcome this obstacle;

(3) the combination effects of nonlocality and nonhomogeneous: the lack of homogeneity caused by fractional (p, q)-Laplacian operator, together with the appearance of nonlocal Choquard reaction term, makes our analysis more refined and interesting than the above works and also brings some new difficulties to our analysis. In particular, we need to develop a suitable Moser iteration scheme to obtain $L^\infty$-estimates and absorb some ideas from [11, 62] to build on the Hölder regularity results under this work.

Throughout this paper, C, C ₀, C ₁, C ₂, ... denote positive constants whose exact values are inessential and can change from line to line, and the same C, C ₀, C ₁, C ₂, ... may represent different constants; $B_\rho(y)$ denotes the open ball centred at $y \in \mathbb{R}^N$ with radius ρ > 0, and $B_\rho^c(y)$ denotes the complement of $B_\rho(y)$ in $\mathbb{R}^N$. In particular, B_ρ and $B_\rho^c$ denote $B_\rho(0)$ and $B_\rho^c(0)$, respectively.

2. Auxiliary results

2.1. Notations

Let $u : \mathbb{R}^N \mapsto \mathbb{R}$. For $0 \lt s \lt 1$ and p > 1, we define the fractional Sobolev space

\begin{align*}W^{s,p}(\mathbb{R}^N) & := \left\{u : |u|_p^p := \int_{\mathbb{R}^N} |u|^p dx \lt + \infty,\ [u]_{s,p}^p \right.\\ & \quad \left. := \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u(x) - u(y)|^p}{|x - y|^{N + sp}} dxdy \lt + \infty\right\}\end{align*}

equipped with the natural norm

\begin{align*}\|u\|_{W^{s,p}(\mathbb{R}^N)} := \left([u]_{s,p}^p + |u|_p^p\right)^{\frac{1}{p}}.\end{align*}

For all u, $v \in W^{s,p}(\mathbb{R}^N)$, we define

\begin{align*}\langle u, v \rangle_{s,p} := \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u(x) - u(y)|^{p - 2} (u(x) - u(y)) (v(x) - v(y))}{|x - y|^{N + sp}} dxdy.\end{align*}

We now recall some fundamental embeddings. Let $s \in (0,1)$ and $p \in (1, + \infty)$ such that N > sp. Then for any $r \in [p,p_s^*]$ there exists a constant such that $C_r \gt 0$ such that

(2.1)\begin{align} |u|_r \leqslant C_r \|u\|_{W^{s,p}(\mathbb{R}^N)} \end{align}

for all $u \in W^{s,p}(\mathbb{R}^N)$. Moreover, $W^{s,p}(\mathbb{R}^N)$ is compactly embedded in $L^r_{\mathrm{loc}}(\mathbb{R}^N)$ for any $r \in [1,p_s^*)$.

In order to treat the nonlinear fractional (p, q)-Choquard problem, we use the following space:

\begin{align*}X = W^{s,p}(\mathbb{R}^N) \cap W^{s,q}(\mathbb{R}^N)\end{align*}

endowed with the norm

\begin{align*}\|u\|_X := \|u\|_{W^{s,p}(\mathbb{R}^N)} + \|u\|_{W^{s,q}(\mathbb{R}^N)}.\end{align*}

Additionally, since $W^{s,r}(\mathbb{R}^N)$ ($1 \lt r \lt + \infty$) is a separable reflexive Banach space, then X is also a separable reflexive Banach space.

2.2. The penalized problem

In order to overcome the lack of compactness of problem (1.3), we shall adapt the penalization method introduced by del Pino and Felmer [26] to deal with the nonlinear fractional (p, q)-Choquard problem. Furthermore, without loss of generality, we can assume that

\begin{align*}0 \in \Lambda\quad \ \text{and }\quad V(0) = V_0.\end{align*}

So, we need to find a constant K > 0 (which is determined later, see lemma 2.6) and take a unique number a > 0 such that $f(a) = \frac{V_0}{K} \left(a^{p - 1} + a^{q - 1}\right)$.

Now, we define

\begin{align*} \widetilde{f}(t) = \begin{cases} \ f(t), & \ \text{if } t \leqslant a,\\ \ \frac{V_0}{K} \left( t^{p - 1} + t^{q - 1} \right), & \ \text{if } t \gt a \end{cases} \end{align*}

and

\begin{align*} g(x,t) = \begin{cases} \ \chi_\Lambda (x) f(t) + (1 - \chi_\Lambda(x)) \widetilde{f}(t), & \ \text{if } t \gt 0,\\ \ 0, & \ \text{if } t \leqslant 0, \end{cases} \end{align*}

where $\chi_\Omega$ is the characteristic function on $\Omega \subset \mathbb{R}^N$.

We can easily observe that g is a Carathéodory function and fulfills the following properties:

1. (g₁) $\lim_{t \rightarrow 0^+} \frac{g(x,t)}{t^{p - 1}} = 0$ uniformly for all $x \in \mathbb{R}^N$;

2. (g₂) $g(x,t) \leqslant f(t)$ for all $x \in \mathbb{R}^N$ and t > 0;

3. (g₃)_i $0 \lt p G(x,t) := p \int^t_0 g(x,\tau) d\tau \leqslant g(x,t) t$ for all $x \in \Lambda$ and t > 0;

4. (g₃)_ii $0 \lt p G(x,t) \leqslant g(x,t) t \leqslant \frac{V_0}{K} \left(t^p + t^q\right)$ for all $x \in \Lambda^c$ and t > 0;

5. (g₄) the maps $t \mapsto \frac{g(x,t)}{t^{\frac{q}{2} - 1}}$ and $t \mapsto \frac{G(x,t)}{t^\frac{q}{2}}$ are both increasing for all $x \in \mathbb{R}^N$ and t > 0.

Remark 2.1. We shall consider the following penalized problem:

(2.2)\begin{align} \left\{ \begin{array}{lll} (- \Delta)_p^s u + (- \Delta)_q ^s u + V(\varepsilon x) (|u|^{p - 2} u + |u|^{q - 2} u) = \left(\frac{1}{|x|^\mu} * G(\varepsilon x, u)\right) g(\varepsilon x, u),\\ u \in X,\ u \gt 0 \end{array} \right\} \end{align}

in $\mathbb{R}^N.$ If u_ɛ is a solution of problem (2.2) such that $u_\varepsilon(x) \leqslant a$ for all $x \in \Lambda_\varepsilon^c,$ where $\Lambda_\varepsilon:=\left\{x \in \mathbb{R}^N : \varepsilon x \in \Lambda \right\},$ then $g(\varepsilon x, u_\varepsilon) = f(u_\varepsilon),$ $G(\varepsilon x, u_\varepsilon) = F(\varepsilon x,u_\varepsilon).$ So, u_ɛ is also a solution of problem (1.3).

For any $\varepsilon \geqslant 0$, we define the space

\begin{align*}X_\varepsilon := \left\{u \in X : \int_{\mathbb{R}^N} V(\varepsilon x)\left(|u|^p + |u|^q\right) dx \lt + \infty\right\}\end{align*}

equipped with the norm

\begin{align*}\|u\|_{X_\varepsilon} := \|u\|_{V_\varepsilon,p} + \|u\|_{V_\varepsilon,q},\end{align*}

where $\|u\|_{V_\varepsilon,t}^t := [u]_{s,t}^t + \int_{\mathbb{R}^N} V(\varepsilon x) |u|^t dx$ for $t \in \left\{p,q\right\}$.

From now on, we focus on the critical points of the Euler–Lagrange functional $J_\varepsilon : {X_\varepsilon} \mapsto \mathbb{R}$ defined by

\begin{align*}J_\varepsilon(u) := \frac{1}{p} \|u\|_{V_\varepsilon,p}^p + \frac{1}{q} \|u\|_{V_\varepsilon,q}^q - \frac{1}{2} \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, u)\right)G(\varepsilon x, u) dx\end{align*}

for all $u \in {X_\varepsilon}$. By a standard argument, we can infer that $J_\varepsilon \in C^1({X_\varepsilon},\mathbb{R})$ and its derivative is given by

\begin{align*} \langle J_\varepsilon'(u),v \rangle & = \langle u,v \rangle_{s,p} + \langle u,v \rangle_{s,q} + \int_{\mathbb{R}^N} V(\varepsilon x)\left(|u|^{p -2} u + |u|^{q -2} u \right) vdx\\ & - \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, u)\right)g(\varepsilon x, u) v dx \ \text{for all } u,\ v \in {X_\varepsilon}. \end{align*}

Let us define the Nehari manifold associated with problem (2.2), that is,

\begin{align*}\mathcal{N}_\varepsilon :=\left\{u \in {X_\varepsilon} \setminus \{0\} : \langle J_\varepsilon'(u),u \rangle = 0\right\}.\end{align*}

We use ${X_\varepsilon^+}$ to denote the open set defined by

\begin{align*}{X_\varepsilon^+} := \left\{u \in {X_\varepsilon} : |{\mathrm{supp}\,} (u^+)| \gt 0\right\},\end{align*}

and we introduce the set $S_\varepsilon^+ := S_\varepsilon \cap {X_\varepsilon^+}$, where $S_\varepsilon : = \left\{u \in {X_\varepsilon} : \|u\|_{X_\varepsilon} = 1\right\}$. We first observe that $S_\varepsilon^+$ is an incomplete $C^{1,1}$-manifold of codimension one. So, for all $u \in S_\varepsilon^+$ we have $X_\varepsilon = T_u (S_\varepsilon^+) \bigoplus \mathbb{R} u$, where

\begin{align*} T_u (S_\varepsilon^+) := \left\{v \in {X_\varepsilon} : \langle u, v \rangle_{s,p} + \langle u, v \rangle_{s,q} + \int_{\mathbb{R}^N} V(\varepsilon x) \left(|u|^{p - 2} u + |u|^{q - 2} u\right) v dx= 0 \right\}. \end{align*}

Due to the fact that f is only continuous, the following result is crucial to bypass the non-differentiability of $\mathcal{N}_\varepsilon$.

Lemma 2.2. Assume that $(f_1)$–$(f_4)$ and $(V_1)$–$(V_2)$ are fulfilled. Then the following properties hold true:

1. (a) for all $u \in {X_\varepsilon^+},$ there exists a unique $t_u \gt 0$ such that $t_u u \in \mathcal{N}_\varepsilon.$ Furthermore, $\hat{m}_\varepsilon(u) = t_u u$ is the unique maximum of $\ell_u(t) :=J_\varepsilon(t u);$

2. (b) there exists τ > 0 independent of u such that $t_u \geqslant \tau$ for all $u \in S_\varepsilon^+.$ Moreover, for each compact set $\mathcal{W} \subset S_\varepsilon^+,$ there exists a constant $C_\mathcal{W} \gt 0$ such that $t_u \leqslant C_\mathcal{W}$ for all $u \in \mathcal{W};$

3. (c) the mapping $\hat{m}_\varepsilon : {X_\varepsilon^+} \mapsto \mathcal{N}_\varepsilon$ is continuous and $m_\varepsilon :=\hat{m}_\varepsilon |_{S_\varepsilon^+}$ is a homeomorphism between $S_\varepsilon^+$ and $\mathcal{N}_\varepsilon,$ and the inverse of m_ɛ is given by $m_\varepsilon^{-1}(u) := u / \|u\|_{X_\varepsilon};$

4. (d) $c_\varepsilon : = \inf_{u \in \mathcal{N}_\varepsilon} J_\varepsilon(u) \geqslant \varrho_0 \gt 0$ and J_ɛ is bounded below $\mathcal{N}_\varepsilon,$ where ϱ ₀ is independent of $\varepsilon,$ K, and $a;$

5. (e) let $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ If there exists a sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset S_\varepsilon^+$ such that ${\mathrm{dist}\,}(u_n,\partial S_\varepsilon^+) \rightarrow 0$ as $n \rightarrow \infty,$ then $\|m_\varepsilon(u_n)\|_{X_\varepsilon} \rightarrow + \infty$ and $J_\varepsilon(m_\varepsilon(u_n)) \rightarrow + \infty$ as $n \rightarrow \infty.$

Proof. (a) For each $u \in {X_\varepsilon^+}$ and t > 0, $\ell_u(0) = 0$. From theorem 4.3 of Lieb and Loss [38], $(g_2)$, $(f_1)$–$(f_2)$, (2.1), and hypothesis $(V_1)$ we can deduce that there exists some constant C > 0 such that

\begin{align*}\ell_u(t) \geqslant \frac{1}{2^{q - 1}q} \|u\|_{X_\varepsilon}^q t^q - C \|u\|_{X_\varepsilon}^{2 p }t^{2 p}\ \text{for}\ 0 \lt t \lt \frac{1}{\|u\|_{X_\varepsilon}}.\end{align*}

Due to $2p \gt q$, we see that $\ell_u(t) \gt 0$ for t > 0 sufficiently small. Using $(g_3)_i$ and $(g_3)_{ii}$, we can find a constant

\begin{align*}C_u = \frac{1}{2} \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, u)\right)G(\varepsilon x, u) dx \gt 0\end{align*}

such that

\begin{align*}\ell_u(t) \leqslant \frac{1}{p} \left(\|u\|_{X_\varepsilon}^p + \|u\|_{X_\varepsilon}^q\right)t^q - C_u t^{2p}\end{align*}

for all t > 1. Applying $2p \gt q$ again, we know that $\ell_u(t) \lt 0$ for t > 1 large enough. Hence, $\max_{t \geqslant 0} \ell_u(t)$ is attained at some $t_u \gt 0$ verifying $\ell_u'(t_u)= 0$ and $t_u u \in \mathcal{N}_\varepsilon$. We point out that

(2.3)\begin{align} t u \in \mathcal{N}_\varepsilon \Longleftrightarrow \|u\|^q_{V_\varepsilon,q} = \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{G(\varepsilon y, t u(y))}{t^{\frac{q}{2}} |x - y|^\mu} \frac{g(\varepsilon x, t u (x))}{t^{\frac{q}{2} - 1}} u(x) dxdy - t^{p - q} \|u\|^p_{V_\varepsilon,p}. \end{align}

According to q > p and $(g_4)$, we conclude that the right-hand side in (2.3) is an increasing function with respect to t > 0. Therefore, the uniqueness of t_u is now obvious.

(b) For any $u \in S_\varepsilon^+$, in view of lemma 2.2-(a), there exists $t_u \gt 0$ such that

\begin{align*}t_u^p \|u\|_{V_\varepsilon,p}^p + t_u^q \|u\|_{V_\varepsilon,q}^q = \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, t_u u) \right) g(\varepsilon x, t_u u) t_u u dx.\end{align*}

By theorem 4.3 of Lieb and Loss [38], together with $(g_2)$, $(f_1)$, $(f_2)$, (2.1), and hypothesis $(V_1)$, we can infer that for any σ > 0, there exists some constant $C_\sigma \gt 0$ such that for all $u \in S_\varepsilon^+$,

\begin{align*}t_u^p \|u\|_{V_\varepsilon,p}^p + t_u^q \|u\|_{V_\varepsilon,q}^q \leqslant \sigma t_u^{2 p} \|u\|_{V_\varepsilon,p}^{2 p} + C_\sigma t_u^{2 \nu} \|u\|_{V_\varepsilon,q}^{2 \nu}.\end{align*}

Assume that $t_u \leqslant 1$. Choosing $\sigma = \frac{1}{2}$, we have

\begin{align*} Ct_u^q & \leqslant \frac{1}{2} t_u^q \left(\|u\|_{V_\varepsilon,p}^q + \|u\|_{V_\varepsilon,q}^q\right) \ \text{for some constant } C \gt 0\\ & \leqslant \frac{1}{2} t_u^p \|u\|_{V_\varepsilon,p}^p + t_u^q \|u\|_{V_\varepsilon,q}^q\ (\text{since } q \gt p,\, t_u \leqslant 1 \ \text{and } 1 = \|u\|_{X_\varepsilon} \geqslant \|u\|_{V_\varepsilon,p})\\ & \leqslant C_{1 / 2} t_u^{2 \nu}\ (\text{since } 1 = \|u\|_{X_\varepsilon} \geqslant \|u\|_{V_\varepsilon,q}),\\ \Rightarrow\ \ & t_u \geqslant \tau \ \text{for some constant } \tau \gt 0\ (\text{since } \nu \gt q), \ \text{where } \tau \ \text{is independent of } u. \end{align*}

Assume that $t_u \gt 1$. Taking σ = 1 and applying $1 = \|u\|_{X_\varepsilon} \geqslant \|u\|_{V_\varepsilon,p}$, we get

\begin{align*} C t_u^p & \leqslant t_u^p \left(\|u\|_{V_\varepsilon, p}^q + \|u\|_{V_\varepsilon, q}^q\right) \ \text{for some constant } C \gt 0\\ & \leqslant t_u^p \|u\|_{V_\varepsilon,p}^p + t_u^q \|u\|_{V_\varepsilon,q}^q \ (\text{since } q \gt p \ \text{and } 1 = \|u\|_{X_\varepsilon} \geqslant \|u\|_{V_\varepsilon,p})\\ & \leqslant (1 + C_1) t_u^{2 \nu}\ (\text{since } q \gt p,\, t_u \gt 1 \ \text{and } 1 = \|u\|_{X_\varepsilon} \geqslant \|u\|_{V_\varepsilon,p},\ \|u\|_{V_\varepsilon,q}),\\ \Rightarrow\ \ & t_u \geqslant \tau \ \text{for some constant } \tau \gt 0 \ (\text{since } \nu \gt q), \ \text{where } \tau \ \text{is independent of } u. \end{align*}

So, there exists τ > 0 independent of u such that $t_u \geqslant \tau$ for all $u \in S_\varepsilon^+.$

Let $\mathcal{W} \subset S_\varepsilon^+$ be a compact set. Arguing by contradiction, we may assume that there exists a sequence $\{u_n\}_{n \in \mathbb{N}} \subset \mathcal{W}$ such that $1 \leqslant t_n :=t_{u_n} \rightarrow + \infty$ as $n \rightarrow \infty$. Since $\mathcal{W}$ is a compact set, there is $u \in \mathcal{W}$ such that $u_n \rightarrow u$ in X_ɛ as $n \rightarrow \infty$. As in the proof of lemma 2.2-(a), together with Fatou’s Lemma, we can see that

(2.4)\begin{align} J_\varepsilon(t_n u_n) \rightarrow - \infty \ \text{as } n \rightarrow \infty. \end{align}

In addition, for any fixed $\varphi \in \mathcal{N}_\varepsilon$, we have $\langle J_\varepsilon'(\varphi), \varphi \rangle = 0$. Combining $(g_3)_i$ with $(g_3)_{ii}$, we have

\begin{align*} J_\varepsilon(\varphi) & = J_\varepsilon(\varphi) - \frac{1}{2 p} \langle J_\varepsilon'(\varphi), \varphi \rangle \\ & \geqslant \left(\frac{1}{q} - \frac{1}{2p}\right) \left(\|\varphi\|_{V_\varepsilon,p}^p + \|\varphi\|_{V_\varepsilon,q}^q\right). \end{align*}

Taking $\varphi_n = t_n u_n \in \mathcal{N}_\varepsilon$ in the above inequality, we get

\begin{align*} J_\varepsilon(t_n u_n) & \geqslant \left(\frac{1}{q} - \frac{1}{2p}\right) \left(t_n^p \|u_n\|_{V_\varepsilon,p}^p + t_n^q \|u_n\|_{V_\varepsilon,q}^q\right)\\ & \geqslant \left(\frac{1}{q} - \frac{1}{2p}\right) \left(t_n^p \|u_n\|_{V_\varepsilon,p}^q + t_n^q \|u_n\|_{V_\varepsilon,q}^q\right)\\ & (\text{since } q \gt p,\ t_n \geqslant 1 \ \text{and } 1 = \|u_n\|_{X_\varepsilon} \geqslant \|u_n\|_{V_\varepsilon,p})\\ & \geqslant C t_n^p \ \text{for some constant } C \gt 0,\\ \Rightarrow\ \ & - \infty \geqslant + \infty \ (\text{see (2.4) and use the assumption } t_n \rightarrow + \infty \ \text{as } n \rightarrow \infty). \end{align*}

This is a contradiction.

(c) As in lemma 2.2-(a), we can define the maps $\hat{m}_\varepsilon : {X_\varepsilon^+} \mapsto \mathcal{N}_\varepsilon$ and $m_\varepsilon : S_\varepsilon^+ \mapsto \mathcal{N}_\varepsilon $ by

(2.5)\begin{align} \hat{m}_\varepsilon(u) = t_u u \quad \ \text{and } \quad m_\varepsilon = \hat{m}_\varepsilon |_{S_\varepsilon^+}. \end{align}

Firstly, we note that $\hat{m}_\varepsilon$, m_ɛ, and $m_\varepsilon^{- 1}$ are well-defined. Indeed, using lemma 2.2 (a), for any fixed $u \in {X_\varepsilon^+}$ it follows that there exists a unique $\hat{m}_\varepsilon(u) \in \mathcal{N}_\varepsilon$. In addition, if $u \in \mathcal{N}_\varepsilon$, and so $u \in {X_\varepsilon^+}$. Otherwise, we obtain $|{\mathrm{supp}\,}(u^+)| = 0$. The above equality, hypothesis $(V_1)$, the definition of g yield that u = 0. This is impossible since u ≠ 0. Thus, the inverse of m_ɛ is given by $m_\varepsilon^{- 1}(u) = \frac{u}{\|u\|_{X_\varepsilon}} \in S_\varepsilon^+$ for all $u \in \mathcal{N}_\varepsilon$. Consequently, $m_\varepsilon^{- 1}$ is well-defined and continuous. On the other hand, for all $u \in S_\varepsilon^+$, we can deduce that

\begin{align*}m_\varepsilon^{- 1}(m_\varepsilon(u)) = m_\varepsilon^{- 1}(t_u u) = \frac{t_u u}{\|t_u u\|_{X_\varepsilon}} = \frac{u}{\|u\|_{X_\varepsilon}} = u.\end{align*}

This yields that m_ɛ is bijection.

Next, we show that $\hat{m}_\varepsilon$ is a continuous map. To this end, let $\{u_n,u\}_{n \in \mathbb{N}} \subset {X_\varepsilon^+}$ such that $u_n \rightarrow u$ in X_ɛ as $n \rightarrow \infty$. On account of the fact that $\hat{m}_\varepsilon (t u) = \hat{m}_\varepsilon(u)$ for all t > 0, we can assume that $\|u_n\|_{X_\varepsilon} = \|u\|_{X_\varepsilon} = 1$ for all $n \in \mathbb{N}$. According to lemma 2.2 (b), we know that there exists $t_n :=t_{u_n} \rightarrow t_0 \gt 0$ as $n \rightarrow \infty$ such that $t_n u_n \in \mathcal{N}_\varepsilon$, then we have

\begin{align*}t_n^p \|u_n\|_{V_\varepsilon,p}^p + t_n^q \|u_n\|_{V_\varepsilon,q}^q = \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, t_n u_n)\right) g(\varepsilon x, t_nu_n) t_n u_n dx.\end{align*}

In the above relation, we pass to the limit as $n \rightarrow \infty$. Then,

\begin{align*}t_0^p \|u\|_{V_\varepsilon,p}^p + t_0^q \|u\|_{V_\varepsilon,q}^q = \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, t_0 u)\right) g(\varepsilon x, t_0u) t_0 u dx.\end{align*}

This implies that $t_0 u \in \mathcal{N}_\varepsilon$. From lemma 2.2 (a), we know that $t_u = t_0$. Hence, it follows that $\hat{m}_\varepsilon(u_n) \rightarrow \hat{m}_\varepsilon(u)$ in ${X_\varepsilon^+}$ as $n \rightarrow \infty$. Thus, $\hat{m}_\varepsilon$ and m_ɛ are continuous mappings.

(d) For ɛ > 0, $0 \lt t \lt 1$, and $u \in S_\varepsilon^+$, using theorem 4.3 of Lieb and Loss [38], $(g_2)$, $(f_1)$–$(f_2)$, (2.1), and hypothesis $(V_1)$ again, we can conclude that there exists C > 0 such that

\begin{align*}J_\varepsilon(t u) \geqslant \frac{1}{2^{q - 1}q} t^{q} - C t^{2 p}.\end{align*}

Thus, we can find $\varrho_0 \gt 0$ such that $J_\varepsilon(t u) \geqslant \varrho_0$ for $0 \lt t \lt 1$ sufficiently small (since $2 p \gt q \gt 0$), where ϱ ₀ is independent of $\varepsilon,$ K, and a. In addition, by lemma 2.2 (a), (b), and (c), we observe (see the study by Szulkin and Weth [55]) that

\begin{align*}c_\varepsilon = \inf_{u \in \mathcal{N}_\varepsilon} J_\varepsilon(u) = \inf_{u \in {X_\varepsilon^+}} \max_{t \gt 0} J_\varepsilon(t u) = \inf_{u \in S_\varepsilon^+} \max_{t \gt 0} J_\varepsilon(t u).\end{align*}

So, $J_\varepsilon(u) |_{\mathcal{N}_\varepsilon} \geqslant\varrho_0$.

(e) Let $\left\{u_n\right\}_{n \in \mathbb{N}} \subset S_\varepsilon^+$ be a sequence such that ${\mathrm{dist}\,}(u_n,\partial S_\varepsilon^+) \rightarrow 0$ as $n \rightarrow \infty$. For any $\varphi \in \partial S_\varepsilon^+$ and $n \in \mathbb{N}$, then we obtain $|u_n^+| \leqslant |u_n - \varphi|$ a.e. in $\Lambda_\varepsilon$. Hence, from $(V_1)$ and Sobolev embedding, for any $r \in [p,q_s^*]$ and $n \in \mathbb{N}$, it follows that

\begin{align*} |u_n^+|_{L^r(\Lambda_\varepsilon)} & \leqslant \inf_{\varphi \in \partial S_\varepsilon^+} |u_n - \varphi|_{L^r(\Lambda_\varepsilon)} \leqslant C_r \inf_{\varphi \in \partial S_\varepsilon^+} \|u_n - \varphi\|_{X_\varepsilon}. \end{align*}

Taking into account $\|u_n\|_{X_\varepsilon} = 1$ and using hypothesis $(V_1)$, we have

\begin{align*} \|u_n\|_{W^{s,p}(\mathbb{R}^N)}^p + \|u_n\|_{W^{s,q}(\mathbb{R}^N)}^q & \leqslant \frac{1}{\min\left\{V_0,1\right\}}\left(\|u_n\|_{V_\varepsilon,p}^p + \|u_n\|_{V_\varepsilon,q}^q\right) \\ &\leqslant \frac{1}{\min\left\{V_0,1\right\}}\left(\|u_n\|_{X_\varepsilon}^p + \|u_n\|_{X_\varepsilon}^q\right) \\ &\leqslant \frac{2}{V_0} + 2. \end{align*}

Note that $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ Then, for all t > 0, we can deduce from lemma 2.6, $(V_1)$, $(g_3)_{ii}$, $(g_2)$, and $(f_1)$–$(f_2)$ that

\begin{align*} & \quad \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, tu_n) \right) G(\varepsilon x, tu_n) dx \\ & = \frac{K}{2}\int_{\Lambda_\varepsilon^c} G(\varepsilon x, tu_n) dx + \frac{K}{2} \int_{\Lambda_\varepsilon} G(\varepsilon x, tu_n) dx\\ & \leqslant \frac{V_0}{2p} \int_{\Lambda_\varepsilon^c} \left(t^p|u_n|^p + t^q|u_n|^q \right) dx+ \frac{K}{2} \int_{\Lambda_\varepsilon} F(t u_n^+) dx\\ & \leqslant \frac{t^p}{2p} \int_{\mathbb{R}^N} V(\varepsilon x)|u_n|^p dx+ \frac{t^q}{2p} \int_{\mathbb{R}^N} V(\varepsilon x)|u_n|^q dx + C_1 t^p \int_{\Lambda_\varepsilon} (u_n^+)^p dx\nonumber\\ & + C_2 t^\nu \int_{\Lambda_\varepsilon} (u_n^+)^\nu dx\\ & \leqslant \frac{t^p}{2p} \int_{\mathbb{R}^N} V(\varepsilon x)|u_n|^p dx+ \frac{t^q}{2p} \int_{\mathbb{R}^N} V(\varepsilon x)|u_n|^q dx + \hat{C}_p t^{p} {\mathrm{dist}\,}(u_n, \partial S_\varepsilon^+)^{p} \nonumber\\ & + \hat{C}_\nu t^\nu {\mathrm{dist}\,}(u_n, \partial S_\varepsilon^+)^\nu, \end{align*}

where C ₁, C ₂, $\hat{C}_p$, and $\hat{C}_\nu$ are some positive constants. So, we have

(2.6)\begin{align} &\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, tu_n) \right) G(\varepsilon x, tu_n) dx\nonumber\\ \leqslant \ & \frac{t^p}{2p} \int_{\mathbb{R}^N} V(\varepsilon x)|u_n|^p dx+ \frac{t^q}{2p} \int_{\mathbb{R}^N} V(\varepsilon x)|u_n|^q dx + o_n(1), \end{align}

as $n \rightarrow \infty$. Moreover, for any t > 1, we infer that

(2.7)\begin{align} & \quad \frac{t^p}{p} \|u_n\|_{V_\varepsilon,p}^p + \frac{t^q}{q} \|u_n\|_{V_\varepsilon,q}^q - \frac{t^p}{2p} \int_{\mathbb{R}^N} V(\varepsilon x)|u_n|^p dx- \frac{t^q}{2p} \int_{\mathbb{R}^N} V(\varepsilon x)|u_n|^q dx\nonumber\\ & = \frac{t^p}{p} [u_n]_{s,p}^p + \frac{1}{2p} t^p \int_{\mathbb{R}^N} V(\varepsilon x) |u_n|^p dx + \frac{t^q}{q} [u_n]_{s,q}^q + \left(\frac{1}{q} - \frac{1}{2p}\right) t^q \int_{\mathbb{R}^N} V(\varepsilon x)|u_n|^q dx\nonumber\\ & \geqslant \frac{1}{2p} t^p \|u_n\|_{V_\varepsilon,p}^p + \left(\frac{1}{q} - \frac{1}{2p}\right) t^q \|u_n\|_{V_\varepsilon,q}^q\ \left(\text{since } 2p \gt q \right)\nonumber\\ & \geqslant \frac{1}{2p} t^p \|u_n\|_{V_\varepsilon,p}^q + \left(\frac{1}{q} - \frac{1}{2p}\right) t^q \|u_n\|_{V_\varepsilon,q}^q\ (\text{due to } q \gt p \ \text{and } 1 = \|u_n\|_{X_\varepsilon} \geqslant \|u_n\|_{V_\varepsilon,p}) \nonumber\\ & \geqslant \frac{1}{2^{q - 1}} \left(\frac{1}{q} - \frac{1}{2p}\right) t^p\ (\text{since } 2p \gt q \gt p \gt 1,\ t \gt 1). \end{align}

Recalling that the definition of m_ɛ and invoking relations (2.6) and (2.7), for all t > 1, we can deduce that

\begin{align*}\liminf_{n \rightarrow \infty} J_\varepsilon(m_\varepsilon(u_n)) \geqslant \liminf_{n \rightarrow \infty} J_\varepsilon(t u_n) \geqslant \frac{1}{2^{q - 1}} \left(\frac{1}{q} - \frac{1}{2p}\right) t^p.\end{align*}

The above inequality combined with the definition of J_ɛ and the arbitrariness of t > 1 means that

\begin{align*} \liminf_{n \rightarrow \infty} \left(\frac{1}{p}\|m_\varepsilon(u_n)\|_{V_\varepsilon,p}^p + \frac{1}{q} \|m_\varepsilon(u_n)\|_{V_\varepsilon,q}^q\right) \geqslant \liminf_{n \rightarrow \infty} J_\varepsilon(m_\varepsilon(u_n)) = + \infty, \end{align*}

and so $\|m_\varepsilon(u_n)\|_{X_\varepsilon} \rightarrow + \infty$ as $n \rightarrow \infty$. This proof is now complete.

Now, we introduce the functionals

\begin{align*}\hat{\psi}_\varepsilon : {X_\varepsilon^+} \mapsto \mathbb{R} \quad \ \text{and } \quad \psi_\varepsilon : S_\varepsilon^+ \mapsto \mathbb{R}\end{align*}

defined by $\hat{\psi}_\varepsilon(u) := J_\varepsilon(\hat{m}_\varepsilon(u))$ for any $u \in {X_\varepsilon^+}$ and $\psi_\varepsilon := \hat{\psi}_\varepsilon |_{S_\varepsilon^+}$.

Using lemma 2.2 and corollary 2.3 in the study by Szulkin and Weth [55], we deduce that the following result holds true.

Lemma 2.3. Assume that $(f_1)$–$(f_4)$ and $(V_1)$–$(V_2)$ are satisfied. Then,

1. (a) $\hat{\psi}_\varepsilon \in C^1({X_\varepsilon^+},\mathbb{R})$ and $\langle \hat{\psi}'_\varepsilon(u), v\rangle = \frac{\|\hat{m}_\varepsilon(u)\|_{X_\varepsilon}}{\|u\|_{X_\varepsilon}} \langle J_\varepsilon'(\hat{m}_\varepsilon(u)),v\rangle$ for all $u \in X_\varepsilon^+,$ all $v \in {X_\varepsilon};$

2. (b) $\psi_\varepsilon \in C^1(S_\varepsilon^+,\mathbb{R})$ and $\langle \psi'_\varepsilon(u),v\rangle = \|m_\varepsilon(u)\|_{X_\varepsilon} \langle J_\varepsilon'(m_\varepsilon(u)),v\rangle$ for all $u \in S_\varepsilon^+,$ all $v \in T_u(S_\varepsilon^+);$

3. (c) if $\{u_n\}_{n \in \mathbb{N}}$ is a Palais–Smale sequence for the functional $\psi_\varepsilon,$ then $\{m_\varepsilon(u_n)\}_{n \in \mathbb{N}}$ is a Palais–Smale sequence for the functional $J_\varepsilon.$ If $\{u_n\}_{n \in \mathbb{N}} \subset \mathcal{N}_\varepsilon$ is bounded Palais–Smale sequence for the functional $J_\varepsilon,$ then, $\{m_\varepsilon^{- 1}(u_n)\}_{n \in \mathbb{N}} \subset S_\varepsilon^+$ is a Palais–Smale sequence for $\psi_\varepsilon;$

4. (d) $u \in S_\varepsilon^+$ is a critical point of the functional ψ_ɛ if and only if $m_\varepsilon(u) \in \mathcal{N}_\varepsilon$ is a critical point of the functional $J_\varepsilon.$ Moreover, the corresponding critical values coincide and

   \begin{align*}\inf_{u \in S_\varepsilon^+} \psi_\varepsilon(u) = \inf_{u \in \mathcal{N}_\varepsilon}J_\varepsilon(u) = c_\varepsilon.\end{align*}

Lemma 2.4. The modified functional J_ɛ admits a Palais–Smale sequence $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ at the level $c_\varepsilon,$ that is, $J_\varepsilon(u_n) \rightarrow c_\varepsilon$ in $\mathbb{R}$ and $J_\varepsilon'(u_n) \rightarrow 0$ in ${X_\varepsilon}^*$ as $n \rightarrow \infty,$ where c_ɛ is given in lemma 2.2. Furthermore, there exists some constant ϑ > 0 $($independent of $\varepsilon,$ K, and $a)$ such that $c_\varepsilon \lt \vartheta$ for all ɛ sufficiently small.

Proof. In view of lemma 2.2, we only need to verify that J_ɛ possesses a mountain pass geometry, that is, the functional J_ɛ satisfies the following properties:

1. (i) there exist some constants $\rho_1 \gt 0$ and $\delta_1 \gt 0$ such that $J_\varepsilon(u) \geqslant \delta_1$ for $\|u\|_{X_\varepsilon} = \rho_1$;

2. (ii) there exists an element $ e \in {X_\varepsilon}$ with $\|e\|_{X_\varepsilon} \gt \rho_1$ such that $J_\varepsilon(e) \lt 0$.

(i) Arguing as in the proof of lemma 2.2 (a), we find some constants C > 0 and $C' \gt 0$ such that

\begin{align*}J_\varepsilon(u) \geqslant C\|u\|_{X_\varepsilon}^q - C' \|u\|_{X_\varepsilon}^{2 p}\end{align*}

for $\|u\|_{X_\varepsilon} = \rho_1 \in (0,1)$. So, using $2 p \gt q$, we obtain (i) if we take ρ ₁ small enough.

(ii) We choose a suitable function $\varphi \in C_0^\infty(\mathbb{R}^N)$ such that $\varphi \geqslant 0$, $\varphi \not\equiv 0$, and ${\mathrm{supp}}\, (\varphi) \subset \Lambda$. Then, for all ɛ sufficiently small, it is obvious that $G(\varepsilon x, \varphi) = F(\varphi)$ for all $x \in \mathbb{R}^N$. Hence, with arguments as in the proof of lemma 2.2 (a), there exist two constants

\begin{align*}C_\varphi = \frac{1}{2} \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(\varphi)\right)F(\varphi) dx \gt 0\end{align*}

and C > 0 such that

\begin{align*}J_\varepsilon(t \varphi) \leqslant Ct^q - C_\varphi t^{2p}\end{align*}

for all t > 1. Hence, (ii) holds true for $e = t \varphi$ and for some t > 1 sufficiently large.

According to the mountain pass theorem without the Palais–Smale condition (see the study by Brezis and Nirenberg [18]), we establish the existence of a Palais–Smale sequence $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ at the level c_ɛ. We recall that ${\mathrm{supp}}\, (\varphi) \subset \Lambda$, and then we can infer that there exists a constant ϑ > 0 such that $c_\varepsilon \lt \vartheta$ for all ɛ small enough, where ϑ is independent of ɛ, K, and a. This completes the proof of the lemma.

Lemma 2.5. Assume that $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ is the Palais–Smale sequence of J_ɛ at the level $c \leqslant \vartheta,$ where ɛ > 0 small enough. Then, for ɛ > 0 small enough, the sequence $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ is bounded and

\begin{equation*}\Arrowvert u_n\Arrowvert_{W^{s,p}(\mathbb{R}^N)}^p+\Arrowvert u_n\Arrowvert_{W^{s,q}(\mathbb{R}^N)}^q\leqslant2\left(\frac1{V_0}+1\right)\frac{pq(\vartheta+1)}{2p-q}\;\text{for all}\ n\in\mathbb{N}\;\text{large enough.}\end{equation*}

Proof. According to $(g_3)_{i,ii}$ and using hypothesis $(V_1)$, for any ɛ > 0 small enough, we deduce that

(2.8)\begin{align} \vartheta + o_n(1) \|u_n\|_{X_\varepsilon} & \geqslant c + o_n(1) \|u_n\|_{X_\varepsilon}\nonumber \\ & \geqslant J_\varepsilon(u_n) - \frac{1}{2 p} \langle J_\varepsilon'(u_n),u_n\rangle\nonumber\\ & \geqslant \left(\frac{1}{q} - \frac{1}{2 p}\right) \left( \|u_n\|_{V_\varepsilon,p}^p + \|u_n\|_{V_\varepsilon,q}^q\right)\nonumber\\ & + \frac{1}{2 p}\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, u_n)\right) \left( g(\varepsilon x, u_n) u_n - p G(\varepsilon x, u_n) \right) dx\nonumber\\ & \geqslant \min\left\{V_0, 1\right\} \left(\frac{1}{q} - \frac{1}{2 p}\right) \left( \|u_n\|_{W^{s,p}(\mathbb{R}^N)}^p + \|u_n\|_{W^{s,q}(\mathbb{R}^N)}^q\right). \end{align}

So, for ɛ > 0 small enough, we deduce that the sequence $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ is bounded and

\begin{equation*}\Arrowvert u_n\Arrowvert_{W^{s,p}(\mathbb{R}^N)}^p+\Arrowvert u_n\Arrowvert_{W^{s,q}(\mathbb{R}^N)}^q\leqslant2\left(\frac1{V_0}+1\right)\frac{pq(\vartheta+1)}{2p-q}\end{equation*}

for all $n \in \mathbb{N}$ large enough.

According to (2.8), we now define the following set:

\begin{align*}\mathcal{B} := \left\{u \in {X_\varepsilon} : \|u\|_{W^{s,p}(\mathbb{R}^N)}^p + \|u\|_{W^{s,q}(\mathbb{R}^N)}^q \leqslant 2\left(\frac{1}{V_0} +1\right)\left(\frac{(\vartheta + 1) p q}{2 p - q} + 1 \right)\right\},\end{align*}

where ϑ is given in lemma 2.4. Using the above notation, we can show the following estimate.

Lemma 2.6. Assume that $(f_1)$–$(f_4)$ hold true, $0 \lt \mu \lt sp$, and $\nu \lt (N - \mu) q / (N - sq),$ then there is a constant K > 0 such that

\begin{align*}\sup_{u \in \mathcal{B}} \left|\frac{1}{|x|^\mu} * G(\varepsilon x, u)\right|_{L^\infty(\mathbb{R}^N)} \lt \frac{K}{2} \ \text{for all}\ \varepsilon \gt 0.\end{align*}

Proof. Firstly, $(g_2)$ and $(f_1)$–$(f_2)$ imply that there exists a constant C > 0 (which is independent of ɛ) such that

(2.9)\begin{align} |G(\varepsilon x,t)| \leqslant |F(t)| \leqslant C(|t|^p + |t|^\nu) \ \text{for all}\ t \in \mathbb{R} \ \text{and for all}\ \varepsilon \gt 0. \end{align}

Therefore,

\begin{align*} \left|\frac{1}{|x|^\mu} * G(\varepsilon x, u)\right| & \leqslant \left|\frac{1}{|x|^\mu} * F(u)\right|\\ & \leqslant \int_{|x - y| \leqslant 1} \frac{|F(u(y))|}{|x - y|^\mu} dy + \int_{|x - y| \gt 1} \frac{|F(u(y))|}{|x - y|^\mu} dy\\ & \leqslant C \int_{|x -y| \leqslant 1} \frac{|u(y)|^p + |u(y)|^\nu}{|x - y|^\mu} dy \\ & \quad + C \int_{\mathbb{R}^N} \left(|u(y)|^p + |u(y)|^\nu\right) dy\ (\text{see (2.9)})\\ & \leqslant C \int_{|x -y| \leqslant 1} \frac{|u(y)|^p + |u(y)|^\nu}{|x - y|^\mu} dy \\ & \quad + C\ (\text{see (2.1) and use the definition of}\ \mathcal{B}) \end{align*}

for some constant C > 0 (which is independent of $x \in \mathbb{R}^N$ and ɛ > 0).

Let us choose

\begin{align*}t_1 \in \left(\frac{N}{N - \mu}, \frac{N}{N - sp}\right]\quad \ \text{and} \quad t_2 \in \left(\frac{N}{N - \mu},\frac{N q}{(N - sq) \nu}\right],\end{align*}

since $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ Then, combining the Hölder inequality, (2.1), and the definition of $\mathcal{B}$, we can easily see that

\begin{align*}\int_{|x -y| \leqslant 1} \frac{|u(y)|^p + |u(y)|^\nu}{|x - y|^\mu} dy \leqslant C \ \text{for all}\ x \in \mathbb{R}^N.\end{align*}

for some constant C > 0 (which is independent of $x \in \mathbb{R}^N$ and ɛ > 0). So, from the above information, we complete the proof of this lemma.

Now, we show that the modified functional J_ɛ satisfies the Palais–Smale condition.

Lemma 2.7. Let $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ Then, for all ɛ > 0 sufficiently small, the modified functional J_ɛ satisfies the Palais–Smale condition with $c \leqslant \vartheta$.

Proof. For all ɛ > 0 sufficiently small, let $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ be a Palais–Smale sequence of the functional J_ɛ at the level c. From lemma 2.5, we know that $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ is bounded for all ɛ > 0 small enough. So, passing to a subsequence, we may assume that there exists some $u \in {X_\varepsilon}$ such that $u_n \xrightarrow w u$ in X_ɛ, $u_n(x) \rightarrow u(x)$ a.e. in $\mathbb{R}^N$ and $u_n \rightarrow u$ in $L^r_{\mathrm{loc}}(\mathbb{R}^N)$ for all $r \in [1,q_s^*)$ as $n \rightarrow \infty$. We first assert that the following property is fulfilled.

Claim 1: The following properties hold up to a subsequence:

1. (a) $\lim_{n \rightarrow \infty}\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * \left(G(\varepsilon x,u_n) -G(\varepsilon x,u)\right)\right) g(\varepsilon x,u) \varphi dx = 0$ for all $\varphi \in C^\infty_0(\mathbb{R}^N)$;

2. (b) $\lim_{n \rightarrow \infty}\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x,u_n)\right) \left(g(\varepsilon x,u_n) - g(\varepsilon x,u) \right) \varphi dx = 0$ for all $\varphi \in C^\infty_0(\mathbb{R}^N)$;

3. (c) we have

   \begin{align*} &\lim_{n \rightarrow \infty} \int_{\mathbb{R}^N}\left(\frac{1}{|x|^\mu} * G(\varepsilon x, u_n)\right) g(\varepsilon x,u_n) \varphi dx\\ =\ & \int_{\mathbb{R}^N}\left(\frac{1}{|x|^\mu} * G(\varepsilon x, u)\right) g(\varepsilon x,u) \varphi dx \ \text{for all}\ \varphi \in C^\infty_0(\mathbb{R}^N). \end{align*}

(a) Note that

\begin{align*} \int_{\mathbb{R}^N} |G(\varepsilon x,u_n)|^{\frac{2 N}{2 N - \mu}} dx & \leqslant \int_{\mathbb{R}^N} \left|C\left(|u_n|^p + |u_n|^{\nu} \right)\right|^{\frac{2 N}{2 N - \mu}} dx \ (\text{see (2.9)})\\ & \leqslant C \int_{\mathbb{R}^N} \left(|u_n|^{\frac{2 N p}{2 N - \mu}} + |u_n|^{\frac{2 N \nu}{2 N - \mu}}\right) dx\\ & \leqslant C \left(\|u_n\|_{X_\varepsilon}^{2 N p /(2 N - \mu)} + \|u_n\|_{X_\varepsilon}^{2 N \nu /(2 N - \mu)} \right)\ (\text{see (2.1)})\\ & \leqslant C \ (\text{due to the boundedness of the sequence}\ \{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}). \end{align*}

On the other hand, since $u_n(x) \rightarrow u(x)$ a.e. in $\mathbb{R}^N$ as $n \rightarrow \infty$, we obtain

\begin{align*}G(\varepsilon x, u_n) \rightarrow G(\varepsilon x, u) \ \text{a.e. in}\ \mathbb{R}^N \ \text{as}\ n \rightarrow \infty.\end{align*}

From proposition 5.4.7 in the study by Willem [58], it follows that

(2.10)\begin{align} G(\varepsilon \cdot,u_n) \xrightarrow w G(\varepsilon \cdot, u) \ \text{in}\ L^{\frac{2 N}{2N - \mu}}(\mathbb{R}^N) \ \text{as}\ n \rightarrow \infty. \end{align}

Let us define

\begin{align*}H(w) := \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * w(x)\right) g(\varepsilon x,u) \varphi dx \ \text{for all}\ w \in L^{\frac{2 N}{2 N - \mu}}(\mathbb{R}^N).\end{align*}

Then, it follows from theorem 4.3 of [38], $(g_2)$, $(f_1)$–$(f_2)$, Hölder’s inequality, (2.1), and $u\in {X_\varepsilon}$ that H is linear bounded functional. Combining this with (2.10), we derive that (a) holds true.

(b) Using the boundedness of the sequence $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$, together with theorem 4.3 in the study by Lieb and Loss [38], $(g_2)$, $(f_1)$–$(f_2)$, and (2.1), we see that $G(\varepsilon \cdot,u_n)$ is also bounded in $L^{\frac{2 N}{2 N -\mu}}(\mathbb{R}^N)$. Then, applying the boundedness of the sequence $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ again and combining with the compact embeddings and Dominated Convergence Theorem, we deduce that (b) is also true.

(c) This result is a consequence of (a) and (b).

Next, we consider the following sequence:

\begin{align*}b_n(x,y) := \frac{|u_n(x) - u_n(y)|^{p - 2} (u_n(x) - u_n(y))}{|x - y|^{\frac{N + s p}{p'}}}\end{align*}

and we also introduce the following function

\begin{align*}b(x,y) := \frac{|u(x) - u(y)|^{p - 2} (u(x) - u(y))}{|x - y|^{\frac{N + s p}{p'}}},\end{align*}

where $p' = \frac{p}{p - 1}$. Then, $\{b_n\}_{n \in \mathbb{N}} \subset L^{p'}(\mathbb{R}^{2 N})$ is bounded and $b_n(x,y) \rightarrow b(x,y)$ a.e. in $\mathbb{R}^{2 N}$ as $n \rightarrow \infty$. Then, we deduce from proposition 5.4.7 in the study by Willem [58] that

(2.11)\begin{align} b_n \xrightarrow w b \ \text{in}\ L^{p'}(\mathbb{R}^{2 N}) \ \text{as } n \rightarrow \infty. \end{align}

On the other hand, for all $\varphi \in C^\infty_0(\mathbb{R}^N)$, we see that

\begin{align*}\frac{\varphi(x) - \varphi(y)}{|x - y|^{\frac{N + s p}{p}}} \in L^p(\mathbb{R}^{2 N}).\end{align*}

From this and (2.11), we have

\begin{align*} &\lim_{n \rightarrow \infty}\int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^{p - 2} (u_n(x) - u_n(y)) (\varphi(x) - \varphi(y))}{|x - y|^{N + s p}} dxdy\\ =\ & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u(x) - u(y)|^{p - 2} (u(x) - u(y)) (\varphi(x) - \varphi(y))}{|x - y|^{N + s p}} dxdy. \end{align*}

In a similar fashion, we also have

\begin{align*} &\lim_{n \rightarrow \infty}\int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^{q - 2} (u_n(x) - u_n(y)) (\varphi(x) - \varphi(y))}{|x - y|^{N + s q}} dxdy\\ =\ & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u(x) - u(y)|^{q - 2} (u(x) - u(y)) (\varphi(x) - \varphi(y))}{|x - y|^{N + s q}} dxdy. \end{align*}

Finally, it is easy to check that

\begin{align*} & \lim_{n \rightarrow \infty}\int_{\mathbb{R}^N} V(\varepsilon x,u_n) |u_n|^{p -2} u_n \varphi dx= \int_{\mathbb{R}^N} V(\varepsilon x,u) |u|^{p -2} u \varphi dx,\\ & \lim_{n \rightarrow \infty}\int_{\mathbb{R}^N} V(\varepsilon x,u_n) |u_n|^{q -2} u_n \varphi dx= \int_{\mathbb{R}^N} V(\varepsilon x,u) |u|^{q -2} u \varphi dx. \end{align*}

From the above information, together with the fact that $\langle J_\varepsilon'(u_n),\varphi \rangle \rightarrow 0$ as $n \rightarrow \infty$, it follows that $\langle J_\varepsilon'(u),\varphi \rangle = 0$ for all $\varphi \in C^\infty_0(\mathbb{R}^N)$. Using the density of $C^\infty_0(\mathbb{R}^N)$ in X_ɛ, we know that u is a critical point of the functional J_ɛ. Consequently, $\langle J_\varepsilon'(u),u\rangle = 0$.

In order to show that the Palais–Smale sequence satisfies the Palais–Smale condition, we need to establish the following property.

Claim 2: We have

\begin{align*} & \limsup_{n \rightarrow \infty} \int_{B_R^c} \left(\int_{\mathbb{R}^N} \left( \frac{|u_n(x) - u_n(y)|^p}{|x - y|^{N + s p}} + \frac{|u_n(x) - u_n(y)|^q}{|x - y|^{N + s q}} \right) dy + V(\varepsilon x) \left(|u_n|^p + |u_n|^q\right) \right) dx\\ &\rightarrow 0 \ \text{as } R \rightarrow + \infty. \end{align*}

We first observe that, for all ɛ > 0 small enough, there exists $ n_0 \in \mathbb{N}$ such that

\begin{align*}\sup_{n \geqslant n_0} \left|\frac{1}{|x|^\mu} * G(\varepsilon x, u_n)\right|_{L^\infty(\mathbb{R}^N)} \lt \frac{K}{2}.\end{align*}

For any R > 0, let $\eta_R \in C^\infty(\mathbb{R}^N)$ be such that $0 \leqslant \eta_R \leqslant 1$, $\eta_R = 0$ in $B_{\frac{R}{2}}$, $\eta_R = 1$ in $B_R^c$, and $|\nabla \eta_R| \leqslant C / R$ for some constant C > 0 (which is independent of R). Taking into account the boundedness of the sequence $\{\eta_R u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$, we see that $\langle J_\varepsilon'(u_n),\eta_R u_n \rangle \rightarrow 0$ as $n \rightarrow \infty$. It follows that

\begin{align*} & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^p \eta_R(x)}{|x - y|^{N + s p}} dxdy + \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^q \eta_R(x)}{|x - y|^{N + s q}} dxdy\\ +\ & \int_{\mathbb{R}^N} V(\varepsilon x) \left(|u_n|^p + |u_n|^q\right) \eta_R dx\\ = \ & o_n(1) - \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^{p - 2} (u_n(x) - u_n(y))(\eta_R(x) - \eta_R(y)) u_n(y)}{|x - y| ^{N + s p}} dxdy\\ - \ & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^{q - 2} (u_n(x) - u_n(y))(\eta_R(x) - \eta_R(y)) u_n(y)}{|x - y| ^{N + s q}} dxdy\\ + \ & \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x,u_n)\right) g(\varepsilon x,u_n) u_n \eta_R dx. \end{align*}

Fix ɛ > 0 small enough. Let R > 0 sufficiently large such that $\Lambda_\varepsilon \subset B_{\frac{R}{2}}$. Using the definitions of η_R and K, together with $(g_3)_{ii}$, for $n \geqslant n_0$ large enough, we have

(2.12)\begin{align} & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^p \eta_R(x)}{|x - y|^{N + s p}} dxdy + \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^q \eta_R(x)}{|x - y|^{N + s q}} dxdy\nonumber\\ + \ & \frac{1}{2} \int_{\mathbb{R}^N} V(\varepsilon x) \left(|u_n|^p + |u_n|^q\right) \eta_R dx\nonumber\\ \leqslant\ & o_n(1) - \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^{p - 2} (u_n(x) - u_n(y))(\eta_R(x) - \eta_R(y)) u_n(y)}{|x - y| ^{N + s p}} dxdy\nonumber\\ - \ & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^{q - 2} (u_n(x) - u_n(y))(\eta_R(x) - \eta_R(y)) u_n(y)}{|x - y| ^{N + s q}} dxdy. \end{align}

From the Hölder inequality and the boundedness of the sequence $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$, it follows that

(2.13)\begin{align} & \left| \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^{p - 2} (u_n(x) - u_n(y))(\eta_R(x) - \eta_R(y)) u_n(y)}{|x - y| ^{N + s p}} dxdy \right| \nonumber\\ \leqslant \ & C \left(\int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|\eta_R(x) - \eta_R(y)|^p |u_n(x)|^p}{|x - y|^{N + sp}} dxdy\right)^{\frac{1}{p}}\ (\text{for some constant } C \gt 0). \end{align}

In addition, by the definition of η_R, polar coordinates, and the boundedness of the sequence $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$, we have

(2.14)\begin{align} & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|\eta_R(x) - \eta_R(y)|^p |u_n(x)|^p}{|x - y|^{N + sp}} dxdy\nonumber\\ \leqslant \ & \int_{\mathbb{R}^N} \int_{|x - y| \gt R} \frac{|\eta_R(x) - \eta_R(y)|^p |u_n(x)|^p}{|x - y|^{N + sp}} dxdy \nonumber\\ &+ \int_{\mathbb{R}^N} \int_{|x - y| \leqslant R} \frac{|\eta_R(x) - \eta_R(y)|^p |u_n(x)|^p}{|x - y|^{N + sp}} dxdy\nonumber\\ \leqslant \ & C \int_{\mathbb{R}^N} \int_{|z| \gt R} \frac{|u_n(x)|^p}{|z|^{N + s p}} dxdz + \frac{C}{R^p} \int_{\mathbb{R}^N} \int_{|z| \leqslant R} \frac{|u_n(x)|^p}{|z|^{N + s p - p}} dxdz\nonumber\\ \leqslant \ & \frac{C}{R^{s p}} \int_{\mathbb{R}^N} |u_n|^p dx + \frac{C}{R^p} R^{- s p + p} \int_{\mathbb{R}^N} |u_n|^p dx\nonumber\\ \leqslant \ & \frac{C}{R^{s p}}. \end{align}

Using (2.13) and (2.14), we see that

(2.15)\begin{align} \left| \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^{p - 2} (u_n(x) - u_n(y))(\eta_R(x) - \eta_R(y)) u_n(y)}{|x - y| ^{N + s p}} dxdy \right| \leqslant \frac{C}{R^s}. \end{align}

Also, we have

(2.16)\begin{align} \left| \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|u_n(x) - u_n(y)|^{q - 2} (u_n(x) - u_n(y))(\eta_R(x) - \eta_R(y)) u_n(y)}{|x - y| ^{N + s q}} dxdy \right| \leqslant \frac{C}{R^s}. \end{align}

So, we deduce from (2.12), (2.15), and (2.16) that Claim 2 holds true.

Using Claim 2 and applying the locally compact embedding $X_\varepsilon \hookrightarrow L^p_{\mathrm{loc}}(\mathbb{R}^N)$, we can derive that $u_n \rightarrow u$ in $L^p(\mathbb{R}^N)$ as $n \rightarrow \infty$. In addition, we deduce from the interpolation inequality that $u_n \rightarrow u$ in $L^r(\mathbb{R}^N)$ as $n \rightarrow \infty$ for all $r \in [p,q_s^*)$. Then, from theorem 4.3 in the study by Lieb–Loss [38], $(g_2)$, $(f_1)$–$(f_2)$, the Dominated Convergence Theorem, and the boundedness of the sequence $\{u_n\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$, it follows that

(2.17)\begin{align} \lim_{n \rightarrow \infty}\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x,u_n)\right) g(\varepsilon x,u_n) u_n dx = \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x,u)\right) g(\varepsilon x,u) u dx. \end{align}

Also, we have

\begin{align*} & \langle J_\varepsilon'(u_n),u_n\rangle = o_n(1) \ \text{as } n \rightarrow \infty,\\ \Rightarrow\ \ & \|u_n\|_{V_\varepsilon,p}^p + \|u_n\|_{V_\varepsilon,q}^q = \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x,u_n)\right) g(\varepsilon x,u_n) u_n dx + o_n(1) \ \text{as } n \rightarrow \infty \end{align*}

and

\begin{align*} & \langle J_\varepsilon'(u),u\rangle = 0,\\ \Rightarrow\ \ & \|u\|_{V_\varepsilon,p}^p + \|u\|_{V_\varepsilon,q}^q = \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x,u)\right) g(\varepsilon x,u) u dx. \end{align*}

Hence, the above fact and (2.17) imply that

\begin{align*}\|u_n\|_{V_\varepsilon,p}^p + \|u_n\|_{V_\varepsilon,q}^q = \|u\|_{V_\varepsilon,p}^p + \|u\|_{V_\varepsilon,q}^q + o_n(1) \ \text{as } n \rightarrow \infty.\end{align*}

Furthermore, according to lemma 2.10 (a), (b), we see that

\begin{align*}\|u_n - u\|_{V_\varepsilon,p}^p = \|u_n\|_{V_\varepsilon,p}^p - \|u\|_{V_\varepsilon,p}^p + o_n(1)\end{align*}

and

\begin{align*}\|u_n - u\|_{V_\varepsilon,q}^q = \|u_n\|_{V_\varepsilon,q}^q - \|u\|_{V_\varepsilon,q}^q + o_n(1),\end{align*}

as $n \rightarrow \infty$. Thus, we conclude that

\begin{align*}\|u_n - u\|_{V_\varepsilon,p}^p + \|u_n - u\|_{V_\varepsilon,q}^q = o_n(1).\end{align*}

This fact means that $\|u_n - u\|_{X_\varepsilon} \rightarrow 0$ as $n \rightarrow \infty$. Now, we complete the proof of the lemma.

Lemma 2.8. Let $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ Assume that $(f_1)$–$(f_4)$ and $(V_1)$–$(V_2)$ are fulfilled, then for all ɛ > 0 small enough, problem (2.2) possesses a non-negative solution $u_\varepsilon \in {X_\varepsilon}.$

Proof. Using lemmas 2.4 and 2.7, we can employ the Mountain Pass Theorem to infer that for all ɛ > 0 sufficiently small, there exists $u_\varepsilon \in {X_\varepsilon}$ such that $J_\varepsilon'(u_\varepsilon) = 0$ and $J_\varepsilon(u_\varepsilon) = c_\varepsilon$. Furthermore, choosing $u_\varepsilon^- := \min\{u_\varepsilon,0\}$ and recalling that $g(\varepsilon \cdot,t) = 0$ for all $t \leqslant 0$ and $\langle J_\varepsilon'(u_\varepsilon),u_\varepsilon^- \rangle = 0$, we can infer that

\begin{align*} & \|u_\varepsilon^-\|_{V_\varepsilon,p}^p + \|u_\varepsilon^-\|_{V_\varepsilon,q}^q \leqslant 0,\\ \Rightarrow\ \ & u_\varepsilon^- = 0,\\ \Rightarrow\ \ & u_\varepsilon \geqslant 0 \ \text{and } u_\varepsilon \not\equiv 0. \end{align*}

This ends the proof of the lemma.

Corollary 2.9. Let $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ Then, for all ɛ > 0 sufficiently small, the functional ψ_ɛ satisfies the Palais–Smale condition at the level $c \leqslant \vartheta$ on $S_\varepsilon^+.$

Proof. Let $\{u_n\}_{n \in \mathbb{N}} \subset S_\varepsilon^+$ be a Palais–Samle sequence for the functional ψ_ɛ at the level c, that is,

\begin{align*}\psi_\varepsilon(u_n) \rightarrow c \ \text{in } \mathbb{R} \quad \ \text{and } \quad \psi_\varepsilon'(u_n) \rightarrow 0 \ \text{in }T_{u_n} (S_\varepsilon^+)^* \ \text{as } n \rightarrow \infty.\end{align*}

Using lemma 2.3 (c), we see that $\left\{m_\varepsilon(u_n)\right\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ is also a Palais–Samle sequence for the functional J_ɛ at the level c. Therefore, we can deduce from lemma 2.7 that the functional J_ɛ verifies the Palais–Smale condition. Hence, passing to a subsequence, we can find some $u \in S_\varepsilon^+$ such that $m_\varepsilon(u_n) \rightarrow m_\varepsilon(u)$ in X_ɛ as $n \rightarrow \infty$. Combining this fact with lemma 2.2 (c), we conclude that $u_n \rightarrow u$ in $S_\varepsilon^+$ as $n \rightarrow \infty$.

We end this section by showing the following result:

Lemma 2.10. Let $\left\{u_n\right\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$ be a sequence such that $u_n \xrightarrow w u$ in X_ɛ, and let $\mathcal{A}_t : \mathbb{R} \mapsto \mathbb{R}$ defined by $\mathcal{A}_t(\tau) = |\tau|^{t - 2} \tau$ $(t \in \left\{p,q\right\}).$ Setting $w_n = u_n - u,$ then for all $\varepsilon \geqslant 0$ and $n \in \mathbb{N}$ large enough, we have

1. (a) $\left([u_n]_{s,p}^p + [u_n]_{s,q}^q \right) - \left([w_n]_{s,p}^p + [w_n]_{s,q}^q \right) - \left([u]_{s,p}^p + [u]_{s,q}^q \right) = o_n(1);$

2. (b) $\left(\left|V(\varepsilon \cdot)^{1/p} u_n\right|_p^p + \left|V(\varepsilon \cdot)^{1/q} u_n\right|_q^q\right) - \left(\left|V(\varepsilon \cdot)^{1/p} w_n\right|_p^p + \left|V(\varepsilon \cdot)^{1/q} w_n\right|_q^q\right) -\left(\left|V(\varepsilon \cdot)^{1/p} u\right|_p^p + \left|V(\varepsilon \cdot)^{1/q} u\right|_q^q\right) = o_n(1);$

3. (c) $\left| \mathcal{A}_p(u_n)- \mathcal{A}_p(w_n) - \mathcal{A}_p(u) \right|_{{\frac{p}{p - 1}}}^{\frac{p}{p - 1}}+\left| \mathcal{A}_q(u_n)- \mathcal{A}_q(w_n) - \mathcal{A}_q(u) \right|_{{\frac{q}{q - 1}}}^{\frac{q}{q - 1}} = o_n(1);$

4. (d) $\int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \Bigg|\frac{\mathcal{A}_p(u_n(x) - u_n(y))}{|x - y|^{\frac{N + sp}{p / (p - 1)}}} - \frac{\mathcal{A}_p(w_n(x) - w_n(y))}{|x - y|^{\frac{N + sp}{p / (p - 1)}}} - \frac{\mathcal{A}_p(u(x) - u(y))}{|x - y|^{\frac{N + sp}{p / (p - 1)}}} \Bigg|^{\frac{p}{p - 1}}dxdy$

   $ + \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \Bigg|\frac{\mathcal{A}_q(u_n(x) - u_n(y))}{|x - y|^{\frac{N + sq}{q / (q - 1)}}} - \frac{\mathcal{A}_q(w_n(x) - w_n(y))}{|x - y|^{\frac{N + sq}{q / (q - 1)}}} - \frac{\mathcal{A}_q(u(x) - u(y))}{|x - y|^{\frac{N + sq}{q / (q - 1)}}} \Bigg|^{\frac{q}{q - 1}}dxdy$

   $ = o_n(1);$

5. (e) $\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(u_n)\right) F(u_n) dx - \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(w_n)\right) F(w_n) dx - \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(u)\right) F(u) dx = o_n(1)$

and for any $\varphi \in {X_\varepsilon}$ with $\|\varphi\|_{X_\varepsilon} \leqslant 1$, it holds

1. (f) $\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(u_n)\right) f(u_n) \varphi dx - \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(w_n)\right) f(w_n) \varphi dx - \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(u)\right) f(u) \varphi dx = o_n(1).$

Proof. We first point out that the proofs of (a) and (b) follow from the Brezis–Lieb Lemma (see [17]). Moreover, arguing as in the proof lemma 3.2 of Mercuri and Willem [43], we see that (c) holds true.

Next, we give the details of the proofs of (d), (e), and (f) for the convenience of readers.

(d) Case 1: $2 \leqslant t \in \left\{p,q\right\}$.

We deduce from the Mean Value Theorem, the Young inequality, and $t \geqslant 2$ that for any fixed σ > 0, there exists $C_\sigma \gt 0$ such that

(2.18)\begin{align} \left||a + b|^{t - 2}(a + b) - \left|a\right|^{t - 2} a\right| \leqslant \sigma |a|^{t - 1} + C_\sigma |b|^{t - 1} \quad \ \text{for all } a,\ b \in \mathbb{R}. \end{align}

In (2.18), we take

\begin{align*}a = \frac{w_n(x) - w_n(y)}{|x - y|^{\frac{N + s t}{t}}} \quad \ \text{and }\quad b = \frac{u(x) - u(y)}{|x - y|^{\frac{N + s t}{t}}},\end{align*}

and we have

\begin{align*} \Bigg| \frac{\mathcal{A}_t( u_n(x) - u_n(y))}{|x - y|^{\frac{N + s t}{t / (t - 1)}}} - \frac{\mathcal{A}_t(w_n(x) - w_n(y))}{|x - y|^{\frac{N + s t}{t / (t - 1)}}}\Bigg| & \leqslant\sigma \frac{|w_n(x) - w_n(y)|^{t - 1}}{|x - y|^{\frac{N + s t}{t / (t - 1)}}} \\ & \quad + C_\sigma \frac{|u(x) - u(y)|^{t - 1}}{|x - y|^{\frac{N + s t}{t / (t - 1)}}} . \end{align*}

Now, we introduce the mapping $H_{\sigma,n}^1 : \mathbb{R}^{2 N} \mapsto \mathbb{R}_+$ given by

\begin{align*} &H_{\sigma,n}^1(x,y) := \max\Bigg\{\left|\frac{\mathcal{A}_t(u_n(x) - u_n(y))}{|x - y|^{\frac{N + st}{t / (t - 1)}}} - \frac{\mathcal{A}_t(w_n(x) - w_n(y))}{|x - y|^{\frac{N + st}{t / (t - 1)}}} - \frac{\mathcal{A}_t(u(x) - u(y))}{|x - y|^{\frac{N + st}{t / (t - 1)}}}\right|\\ &- \sigma \frac{|w_n(x) - w_n(y)|^{t - 1}}{|x - y|^{\frac{N + s t}{t / (t - 1)}}},0 \Bigg\}. \end{align*}

Then, $H_{\sigma,n}^1(x,y) \rightarrow 0$ a.e. in $\mathbb{R}^{2 N}$ as $n \rightarrow \infty$ and

\begin{align*}0 \leqslant H_{\sigma,n}^1(x,y) \leqslant C \frac{|u(x) - u(y)|^{t - 1}}{|x - y|^{\frac{N + s t}{t / (t - 1)}}} \in L^{\frac{t}{t - 1}}(\mathbb{R}^{2 N})\end{align*}

for some constant C > 0. Hence, from the Dominated Convergence Theorem, we obtain

\begin{align*}\int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \left|H_{\sigma,n}^1\right|^{\frac{t}{t - 1}} dxdy \rightarrow 0 \ \text{in } \mathbb{R} \ \text{as } n \rightarrow \infty.\end{align*}

In addition, according to the definition of $H_{\sigma,n}^1$, we have

\begin{align*} & \left|\frac{\mathcal{A}_t(u_n(x) - u_n(y))}{|x - y|^{\frac{N + st}{(t / (t - 1))}}} - \frac{\mathcal{A}_t(w_n(x) - w_n(y))}{|x - y|^{\frac{N + st}{(t / (t - 1))}}} - \frac{\mathcal{A}_t(u(x) - u(y))}{|x - y|^{\frac{N + st}{(t / (t - 1))}}}\right|^\frac{t}{t - 1} \\ \leqslant\ & C \left(\sigma^\frac{t}{t - 1} \frac{|w_n(x) - w_n(y)|^t}{|x - y|^{N + s t}} + (H_{\sigma,n}^1(x,y))^\frac{t}{t - 1}\right) \end{align*}

for some constant C > 0. This implies that

\begin{align*} & \quad \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \left|\frac{\mathcal{A}_t(u_n(x) - u_n(y))}{|x - y|^{\frac{N + st}{t / (t - 1)}}} - \frac{\mathcal{A}_t(w_n(x) - w_n(y))}{|x - y|^{\frac{N + st}{t / (t - 1)}}} - \frac{\mathcal{A}_t(u(x) - u(y))}{|x - y|^{\frac{N + st}{t / (t - 1)}}} \right|^{\frac{t}{t - 1}}dxdy\\ & = o_n(1)\end{align*}

holds true for $n \in \mathbb{N}$ large enough.

Case 2: $1 \lt t \lt 2$.

Invoking lemma 3.1 in the study by Mercuri and Willem [43] and applying the Dominated Convergence Theorem, for $n \in \mathbb{N}$ large enough, we obtain that

\begin{align*} & \quad \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \left|\frac{\mathcal{A}_t(u_n(x) - u_n(y))}{|x - y|^{\frac{N + st}{t / (t - 1)}}} - \frac{\mathcal{A}_t(w_n(x) - w_n(y))}{|x - y|^{\frac{N + st}{t / (t - 1)}}} - \frac{\mathcal{A}_t(u(x) - u(y))}{|x - y|^{\frac{N + st}{t / (t - 1)}}} \right|^{\frac{t}{t - 1}}dxdy \\ & = o_n(1)\end{align*}

holds true.

Now, combining Case 1 and Case 2, we complete the proof of (d).

(e) Using the Mean Value Theorem, hypotheses $(f_1)$–$(f_2)$, and Young’s inequality, we can infer that for any σ > 0, there exists $C_\sigma \gt 0$ such that

\begin{align*}\left|F(u_n) - F(w_n) -F(u)\right|^{\frac{2 N }{2 N - \mu}} & \leqslant \sigma \left(|u_n|^{\frac{2 N p }{2 N - \mu}} + |u_n|^{\frac{2 N \nu}{2 N - \mu}} \right) \\ & \quad + C_\sigma\left(|u|^{\frac{2 N p }{2 N - \mu}} + |u|^{\frac{2 N \nu}{2 N - \mu}}\right).\end{align*}

Define the following mapping:

\begin{align*}H_{\sigma,n}^2 := \max\left\{\left|F(u_n) - F(w_n) -F(u)\right|^{\frac{2 N }{2 N - \mu}} - \sigma \left(|u_n|^{\frac{2 N p }{2 N - \mu}} + |u_n|^{\frac{2 N \nu}{2 N - \mu}} \right),0\right\}.\end{align*}

Then, arguing as in the proof of lemma 2.10 (d) and using the boundedness of the sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$, for $n \in \mathbb{N}$ sufficiently large, we can derive that

(2.19)\begin{align} \int_{\mathbb{R}^N}\left|F(u_n) - F(w_n) -F(u)\right|^{\frac{2 N }{2 N - \mu}} dx = o_n(1). \end{align}

On the other hand, we can deduce from theorem 4.3 in the study by Lieb and Loss [38] that

(2.20)\begin{align} \int_{\mathbb{R}^N}\left|\frac{1}{|x|^\mu} *\left( F(u_n) - F(w_n) - F(u)\right)\right|^{\frac{2 N }{\mu}} dx = o_n(1) \end{align}

for $n \in \mathbb{N}$ large enough.

We note that

\begin{align*} &\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(u_n)\right) F(u_n) dx - \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(w_n)\right) F(w_n) dx\\ =\ & \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * \left(F(u_n) - F(w_n)\right)\right) \left(F(u_n) - F(w_n)\right) dx \\ + \ & 2 \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * \left(F(u_n) - F(w_n)\right)\right) F(w_n) dx. \end{align*}

Using the above equality and taking into account that $F(w_n) \xrightarrow w 0$ in $L^{\frac{2 N }{2 N - \mu}}(\mathbb{R}^N)$ as $n \rightarrow \infty$, together with (2.19)–(2.20), we can show that (e) holds true.

(f) We first prove that

(2.21)\begin{align} \sup_{\|\varphi\|_{X_\varepsilon} \leqslant 1} \int_{\mathbb{R}^N} \left|\left(f(u_n) - f(w_n) - f(u)\right) \varphi\right|^{\frac{2 N }{2 N - \mu}} dx = o_n(1) \end{align}

for $n \in \mathbb{N}$ large enough.

For any fixed $0 \lt \sigma \lt 1$, we deduce from hypothesis $(f_1)$ that there exists $0 \lt \lambda_0 := \lambda_0(\sigma) \lt 1$ such that

(2.22)\begin{align} f(t) \leqslant \sigma |t|^{p - 1} \quad \ \text{for all } |t| \leqslant 2 \lambda_0. \end{align}

In addition, by hypothesis $(f_2)$, we can find $\lambda_1 := \lambda_1(\sigma) \gt 2$ such that

(2.23)\begin{align} |f(t)| \leqslant \sigma |t|^{\nu - 1} \quad \ \text{for all } |t| \geqslant \lambda_1 - 1. \end{align}

Next, by the continuity of f, there is a positive constant $\gamma :=\gamma(\sigma) \lt \lambda_0$ such that

(2.24)\begin{align} |f(t_1) - f(t_2)| \leqslant \lambda_0^{p - 1} \sigma \quad \ \text{for all } |t_1 - t_2| \leqslant \gamma,\ |t_1|,\, |t_2| \leqslant \lambda_1 + 1. \end{align}

Additionally, combining $(f_1)$ and $(f_2)$, we can conclude that there exists $C(\sigma) \gt 0$ such that

(2.25)\begin{align} f(t) \leqslant C(\sigma) |t|^{p - 1} + \sigma |t|^{\nu - 1} \quad\ \text{for all } t \in \mathbb{R}. \end{align}

We now estimate the following term:

\begin{align*}\int_{\mathbb{R}^N \setminus B_R}\left|(f(u_n) - f(w_n) - f(u))\varphi\right|^{\frac{2 N}{2 N - \mu}} dx.\end{align*}

Since $u \in {X_\varepsilon}$, together with (2.25), the Hölder inequality (2.1), and $(V_1)$, we know that there exists $R = R(\sigma) \gt 0$ such that

(2.26)\begin{align} & \quad \int_{\mathbb{R}^N \setminus B_R} \left|f(u) \varphi\right|^{\frac{2 N}{2 N - \mu}} dx \nonumber\\ & \leqslant C \left(\int_{\mathbb{R}^N \setminus B_R} |u|^{\frac{2 N (p - 1)}{2 N - \mu}} |\varphi|^{\frac{2 N}{2 N - \mu}} dx + \int_{\mathbb{R}^N \setminus B_R} |u|^{\frac{2 N (\nu - 1)}{2 N - \mu}} |\varphi|^{\frac{2 N}{2 N - \mu}} dx \right)\nonumber\\ &\leqslant C \left(\left(\int_{\mathbb{R}^N \setminus B_R} |u|^{\frac{2 Np}{2 N - \mu}} dx\right)^{\frac{p - 1}{p}} + \left(\int_{\mathbb{R}^N \setminus B_R} |u|^{\frac{2 N \nu}{2 N - \mu}} \right)^{\frac{\nu - 1}{\nu}} \right)\|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}}\nonumber\\ & \leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} \end{align}

for some constant C > 0.

Let us define

\begin{align*}D_{u_n}^1 := \left\{x \in \mathbb{R}^N \setminus B_R : |u_n(x)| \leqslant \lambda_0\right\}.\end{align*}

According to (2.22), the Hölder inequality, (2.1), $(V_1)$, and the boundedness of the sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset {X_\varepsilon}$, we have

(2.27)\begin{align} &\int_{D_{u_n}^1 \cap |u| \leqslant \gamma} |(f(u_n) - f(w_n))\varphi|^{\frac{2 N}{2 N- \mu}} dx \nonumber\\ \leqslant\ & \int_{D_{u_n}^1 \cap |u| \leqslant \gamma} \left( \sigma |u_n|^{p - 1} + \sigma |w_n|^{p - 1}\right)^\frac{2 N}{2 N - \mu}|\varphi|^{\frac{2 N}{2 N - \mu}} dx\nonumber\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} \end{align}

for some constant C > 0.

Set

\begin{align*}D_{u_n}^2 := \left\{x \in \mathbb{R}^N \setminus B_R : |u_n(x)| \geqslant \lambda_1\right\}.\end{align*}

Arguing as in (2.27), together with (2.23), we can obtain

(2.28)\begin{align} &\int_{D_{u_n}^2 \cap |u| \leqslant \gamma} |(f(u_n) - f(w_n))\varphi|^{\frac{2 N}{2 N- \mu}} dx \leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}}. \end{align}

for some positive constant C > 0.

Now, we introduce the following set:

\begin{align*}D_{u_n}^3 := \left\{x \in \mathbb{R}^N \setminus B_R : \lambda_0 \leqslant |u_n(x)| \leqslant \lambda_1\right\}.\end{align*}

It is easy to check that $|D_{u_n}^3| \lt + \infty$, since $u_n \in {X_\varepsilon}$. So, from (2.24), it follows that

(2.29)\begin{align} &\int_{D_{u_n}^3 \cap |u| \leqslant \gamma} |(f(u_n) - f(w_n))\varphi|^{\frac{2 N}{2 N- \mu}} dx \leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} \end{align}

for some constant C > 0.

Thus, we deduce from (2.27)–(2.29) that

(2.30)\begin{align} \int_{(\mathbb{R}^N \setminus B_R) \cap |u| \leqslant \gamma}\left|(f(u_n) - f(w_n))\varphi\right|^{\frac{2 N}{2 N - \mu}} dx \leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} \quad \ \text{for all } n \in \mathbb{N}. \end{align}

Thanks to $u \in {X_\varepsilon}$, we have $|(\mathbb{R}^N \setminus B_R) \cap \{|u| \geqslant \gamma\}| \rightarrow 0$ as $R \rightarrow \infty$. Using (2.25) again, we deduce that there exists $R := R(\sigma) \gt 0$ such that

\begin{align*} &\int_{(\mathbb{R}^N \setminus B_R) \cap |u| \geqslant \gamma} |(f(u_n) - f(w_n)) \varphi|^{\frac{2 N}{2 N - \mu}} dx\\ \leqslant \ & \int_{(\mathbb{R}^N \setminus B_R) \cap |u| \geqslant \gamma} \left(\sigma \left(|u_n|^{\nu - 1} + |w_n|^{\nu - 1}\right)|\varphi| + C(\sigma)\left(|u_n|^{p - 1} + |w_n|^{p - 1}\right)|\varphi|\right)^{\frac{2 N}{2 N - \mu}} dx\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} + C \int_{(\mathbb{R}^N \setminus B_R) \cap |u| \geqslant \gamma} \left(C(\sigma)\left(|u_n|^{p - 1} + |w_n|^{p - 1}\right)|\varphi|\right)^{\frac{2 N}{2 N - \mu}} dx\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} + (2C(\sigma))^{\frac{2 N}{2 N - \mu}} \int_{(\mathbb{R}^N \setminus B_R) \cap |u| \geqslant \gamma} \left(|u_n|^{\frac{2 N (p - 1)}{2 N - \mu}} + |w_n|^{\frac{2 N(p - 1)}{2 N - \mu}}\right)|\varphi|^{\frac{2 N}{2 N - \mu}} dx\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} + C(C (\sigma))^{\frac{2 N}{2 N - \mu}} \left(\int_{(\mathbb{R}^N \setminus B_R) \cap |u| \geqslant \gamma} |\varphi|^{\frac{2 N p}{2 N - \mu}} dx\right)^{\frac{1}{p}}\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} + C(C (\sigma))^{\frac{2 N}{2 N - \mu}} \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} |(\mathbb{R}^N \setminus B_R) \cap \{|u| \geqslant \gamma\}|^{\frac{2 s p - \mu}{p (2 N - \mu)}}\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}}, \end{align*}

where C is a positive constant. Combining this inequality with (2.26) and (2.30), we deduce that there is a constant C > 0 such that for all $n \in \mathbb{N}$,

(2.31)\begin{align} \int_{\mathbb{R}^N \setminus B_R} \left|(f(u_n) - f(w_n) - f(u)) \varphi\right|^{\frac{2 N}{2 N - \mu}} dx \leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}}. \end{align}

Recalling that $u_n \xrightarrow w u$ in X_ɛ as $n \rightarrow \infty$ and passing to a subsequence (still denoted by $\left\{u_n\right\}_{n \in \mathbb{N}}$), we can assume that $u_n \rightarrow u$ strongly in $L^{\frac{2 N p}{2 N - \mu}}(B_R)$ (since $0 \lt \mu \lt 2 sp$) and there is a function $d \in L^{\frac{2 N p}{2 N - \mu}}(B_R)$ such that $|u_n(x)|$, $|u(x)| \leqslant d(x)$ a.e. $x \in B_R$.

For $n \in \mathbb{N}$ large enough, we have

(2.32)\begin{align} \int_{B_R} |f(w_n) \varphi|^{\frac{2 N}{2 N - \mu}} & dx \leqslant \int_{B_R} \left( C(\sigma) |w_n)|^{p - 1} + \sigma |w_n|^{\nu - 1}\right)^{\frac{2 N}{2 N - \mu}} |\varphi|^{\frac{2 N}{2 N - \mu}}dx\nonumber\\ &\leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} + (2C(\sigma))^{\frac{2 N}{2 N - \mu}} \left(\int_{B_R}|w_n|^{\frac{2 N p}{2 N - \mu}}\right)^{\frac{p - 1}{p}} |\varphi|_{\frac{2 N p}{2 N - p}}^{\frac{2 N}{2 N - \mu}}\nonumber\\ & \leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}}, \end{align}

where C > 0 is some constant.

Set $D_{u_n}^4 :=\left\{x \in B_R : |u_n(x) - u(x)| \geqslant 1\right\}$. So, we get

\begin{align*} &\int_{D_{u_n}^4} |(f(u_n) - f(u))\varphi|^{\frac{2 N}{2 N - \mu}} dx \\ \leqslant \ & \int_{D_{u_n}^4} \left(C(\sigma) (|u_n|^{p - 1} + |u|^{p - 1}) +\sigma (|u_n|^{\nu - 1} + |u|^{\nu - 1})\right)^{\frac{2 N}{2 N - \mu}} |\varphi|^{\frac{2 N}{2 N - \mu}} dx\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} + C(2 C(\sigma))^{\frac{2 N}{2 N - \mu}} \left(\int_{D_{u_n}^4} d^{\frac{2 N p}{2 N - \mu}} dx\right)^\frac{p - 1}{p} \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}}, \end{align*}

where C > 0 is some constant. We observe that $|D_{u_n}^4| \rightarrow 0$ in $\mathbb{R}$ as $n \rightarrow \infty$, and then we can deduce that there exists some constant C > 0 such that

(2.33)\begin{align} \int_{D_{u_n}^4} |(f(u_n) - f(u))\varphi|^{\frac{2 N}{2 N - \mu}} dx \leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}}, \end{align}

provided that $n \in \mathbb{N}$ is sufficiently large.

In addition, from $u \in {X_\varepsilon}$, it follows that

\begin{align*}\left|\left\{x \in \mathbb{R}^N : |u(x)| \geqslant L\right\}\right| \rightarrow 0 \ \text{in } \mathbb{R}^N \ \text{as } L \rightarrow +\infty.\end{align*}

Using the above fact and invoking (2.25) again, we can infer that there exists $L :=L(\sigma) \gt 0$ such that

(2.34)\begin{align} &\int_{(B_R \setminus D_{u_n}^4) \cap \left\{x \in \mathbb{R}^N : |u(x)| \geqslant L\right\}} |(f(u_n) - f(u))\varphi|^{\frac{2 N}{2 N - \mu}} dx\nonumber\\ \leqslant \ & \int_{(B_R \setminus D_{u_n}^4) \cap \left\{x \in \mathbb{R}^N : |u(x)| \geqslant L\right\}} \left|\left(C(\sigma) (|u_n|^{p - 1} + |u|^{p - 1}) + \sigma(|u_n|^{\nu - 1} + |u|^{\nu - 1})\right)\varphi\right|^{\frac{2 N}{2 N - \mu}} dx\nonumber\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} + C(C (\sigma))^{\frac{2 N}{2 N - \mu}} \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} \left|(B_R \setminus D_{u_n}^4) \cap \left\{x \in \mathbb{R}^N : |u(x)| \geqslant L\right\}\right|^{\frac{2 s p - \mu}{p (2 N - \mu)}}\nonumber\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}}, \end{align}

where C > 0 is a constant. Additionally, we can deduce from the Dominated Convergence Theorem that

\begin{align*} \int_{(B_R \setminus D_{u_n}^4) \cap \left\{x \in \mathbb{R}^N : |u(x)| \leqslant L\right\}} |f(u_n) - f(u)|^{\frac{2 N p}{(2 N - \mu)(p - 1)}} dx = o_n(1) \end{align*}

for $n \in \mathbb{N}$ large enough.

It follows that

(2.35)\begin{align} \int_{(B_R \setminus D_{u_n}^4) \cap \left\{x \in \mathbb{R}^N : |u(x)| \leqslant L\right\}} |(f(u_n) - f(u))\varphi|^{\frac{2 N }{2 N - \mu}} dx \leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} \end{align}

for some constant C > 0.

Hence, we deduce from (2.32), (2.33), (2.34), and (2.35) that there exists some constant C > 0 such that

(2.36)\begin{align} \int_{B_R} \left|(f(u_n) - f(w_n) - f(u))\varphi\right|^{\frac{2 N}{2 N - \mu}} dx \leqslant C \sigma \|\varphi\|_{X_\varepsilon}^{\frac{2 N}{2 N - \mu}} \end{align}

for large enough $n \in \mathbb{N}$.

Putting together (2.31) and (2.36), we conclude that (2.21) holds true.

On account of the fact that $\frac{1}{|x|^\mu}*F(u) \in L^\frac{2 N}{\mu}(\mathbb{R}^N)$, for any $0 \lt \sigma \lt 1$, there exists $R_1 :=R_1(\sigma) \gt 0$ such that

\begin{align*} \left(\int_{\mathbb{R}^N \setminus B_{R_1}} \left|\frac{1}{|x|^\mu}*F(u)\right|^\frac{2 N}{\mu} dx\right)^\frac{\mu}{2 N} \lt \sigma. \end{align*}

Using the above all information and by a straightforward computation, for $n \in \mathbb{N}$ large enough, we can conclude that

(2.37)\begin{align} & \left|\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu}*F(u)\right) f(w_n) \varphi dx\right| \nonumber\\ \leqslant \ & \int_{B_{R_1}} \left|\left(\frac{1}{|x|^\mu}*F(u)\right) f(w_n) \varphi \right| dx + \int_{\mathbb{R}^N \setminus B_{R_1}} \left|\left(\frac{1}{|x|^\mu}*F(u)\right) f(w_n) \varphi \right| dx \nonumber\\ \leqslant \ & C_u \left(\int_{B_{R_1}} |f(w_n) \varphi|^\frac{2 N}{2 N - \mu} dx\right)^{\frac{2 N -\mu}{2 N}}\nonumber\\ +\ & \left(\int_{\mathbb{R}^N \setminus B_{R_1}} \left|\frac{1}{|x|^\mu}*F(u)\right|^\frac{2 N}{\mu} dx\right)^\frac{\mu}{2 N} \left(\int_{\mathbb{R}^N} |f(w_n) \varphi|^{\frac{2 N}{2N - \mu}} dx \right)^\frac{2 N - \mu}{2 N}\nonumber\\ \leqslant \ & C \sigma \|\varphi\|_{X_\varepsilon}, \end{align}

where C_u and C are positive constants.

Next, we prove the following relation:

(2.38)\begin{align} \left|\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu}*F(w_n)\right) f(u) \varphi dx \right| \leqslant C \sigma\|\varphi\|_{X_\varepsilon} \end{align}

for $n \in \mathbb{N}$ sufficiently large.

Indeed, it is easy to check that, for any $\sigma \in (0,1)$ and some $R_2 :=R(\sigma) \gt 0$, there exists some constant C > 0 such that

(2.39)\begin{align} \int_{B_{R_2}} |F(w_n)| \left(\int_{B_{R_2}} \frac{|f(u(x)) \varphi(x)|}{|x - y|^\mu} dx\right) dy \leqslant C \sigma \|\varphi\|_{X_\varepsilon}, \end{align}

(2.41)\begin{align} \int_{\mathbb{R}^N} |F(w_n)|\left(\int_{\mathbb{R}^N \setminus B_{R_2}}\frac{|f(u(x)) \varphi(x)|}{|x - y|^\mu} dx \right) dy \leqslant C \sigma \|\varphi\|_{X_\varepsilon} \end{align}

for $n \in \mathbb{N}$ large enough.

To prove (2.38), now we need to estimate the following part:

\begin{align*}\Phi_n :=\int_{\mathbb{R}^N \setminus B_{R_2}} |F(w_n)| \left(\int_{B_{R_2}} \frac{|f(u(x)) \varphi(x)|}{|x - y|^\mu} dx\right) dy.\end{align*}

To this end, we divide this into two parts for discussion.

($*$) If $f(u(x)) \varphi(x) = 0$ a.e. on $B_{R_2}$, so, we have

\begin{align*}\Phi_n \leqslant C \sigma \|\varphi\|_{X_\varepsilon} \ \text{for all } \sigma \gt 0 \ \text{and for some constant } C \gt 0.\end{align*}

($**$) If $|\left\{x \in B_{R_2} : f(u(x)) \varphi(x) \neq 0\right\}| \gt 0$, that is,

\begin{align*}\int_{B_{R_2}} |f(u) \varphi |^{\frac{6N}{6N - \mu}} dx \gt 0.\end{align*}

In addition, we can easily derive that

\begin{align*} \int_{B_{R_2}} |f(u)\varphi|^{\frac{6N}{6N - \mu}} dx \leqslant C \|\varphi\|_{X_\varepsilon}^{\frac{6N}{6N - \mu}}|B_{R_2}|^{\frac{2\mu}{6N - \mu}} \ \text{for some constant } C \gt 0.\end{align*}

Let us define

\begin{align*} d_\sigma := \left(\frac{|f(u)\varphi|_{L^{\frac{6N}{6N - \mu}}(B_{R_2})}}{\sigma \|\varphi\|_{X_\varepsilon}}\right)^{\frac{3}{\mu}} \ \text{and }\widehat{R}_2 := R_2 + \sigma^{-\frac{3}{\mu}} |B_{R_2}|^{\frac{6}{6N - \mu}} C^{\frac{6 N - \mu}{2 N\mu}}. \end{align*}

In the case ($**$), we can apply the above relations, theorem 4.3 in the study by Lieb and Loss [38], the Sobolev continuous embedding and the local compactness Sobolev embedding to infer that, for $n \in \mathbb{N}$ sufficiently large,

(2.42)\begin{align} \Phi_n & = \int_{B_{R_2}} |f(u) \varphi| \left(\int_{\mathbb{R}^N \setminus B_{R_2}} \frac{|F(w_n(y))|}{|x - y|^\mu} dy\right) dx\nonumber\\ & =\int_{B_{R_2}} |f(u) \varphi| \left(\int_{\mathbb{R}^N \setminus B_{R_2 + d_\sigma}} \frac{|F(w_n(y))|}{|x - y|^\mu} dy\right) dx\nonumber\\ & + \int_{B_{R_2}} |f(u) \varphi| \left(\int_{ {B_{R_2 + d_\sigma}} \setminus B_{R_2}} \frac{|F(w_n(y))|}{|x - y|^\mu} dy\right) dx\nonumber\\ & \leqslant C \frac{\sigma\|\varphi\|_{X_\varepsilon}}{|f(u)\varphi|_{L^{\frac{6N}{6N - \mu}}(B_{R_2})}}\int_{B_{R_2}} |f(u)\varphi| \left(\int_{\mathbb{R}^N \setminus B_{R_2 + d_\sigma}} \frac{|F(w_n(y))|}{|x - y|^{2\mu / 3}} dy\right) dx\nonumber\\ & + \int_{B_{R_2}} |f(u) \varphi| \left(\int_{ {B_{R_2 + d_\sigma}} \setminus B_{R_2}} \frac{|F(w_n(y))|}{|x - y|^\mu} dy\right) dx\nonumber\\ & \leqslant C \frac{\sigma\|\varphi\|_{X_\varepsilon}}{|f(u)\varphi|_{L^{\frac{6N}{6N - \mu}}(B_{R_2})}}\int_{B_{R_2}} |f(u)\varphi| \left(\int_{\mathbb{R}^N} \frac{|F(w_n(y))|}{|x - y|^{2\mu / 3}} dy\right) dx\nonumber\\ & + \int_{B_{R_2}} |f(u) \varphi| \left(\int_{ {B_{R_2 + d_\sigma}} \setminus B_{R_2}} \frac{|F(w_n(y))|}{|x - y|^\mu} dy\right) dx\nonumber\\ & \leqslant C \frac{\sigma\|\varphi\|_{X_\varepsilon}}{|f(u)\varphi|_{L^{\frac{6N}{6N - \mu}}(B_{R_2})}} |f(u)\varphi|_{L^{\frac{6N}{6N - \mu}}(B_{R_2})} \nonumber\\ & + \int_{B_{R_2}} |f(u) \varphi| \left(\int_{ {B_{\widehat{R}_2}}\setminus B_{R_2}} \frac{|F(w_n(y))|}{|x - y|^\mu} dy\right) dx\nonumber\\ & \leqslant C\sigma \|\varphi\|_{X_\varepsilon} + \int_{B_{R_2}} |f(u) \varphi| \left(\int_{ {B_{\widehat{R}_2}}\setminus B_{R_2}} \frac{|F(w_n(y))|}{|x - y|^\mu} dy\right) dx\nonumber\\ & \leqslant C \sigma\|\varphi\|_{X_\varepsilon}. \end{align}

Thus, relations (2.39), (2.41), and (2.42) and ($*$) imply that (2.38) holds true.

Finally, we point out that

\begin{align*} & \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(u_n)\right) f(u_n) \varphi dx - \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(w_n)\right) f(w_n) \varphi dx\\ =\ & \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * \left(F(u_n) - F(w_n)\right)\right) (f(u_n) - f(w_n))\varphi dx\\ + \ & \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * (F(u_n) - F(w_n))\right) f(w_n) \varphi dx \\ + \ & \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(w_n)\right) (f(u_n) - f(w_n)) \varphi dx. \end{align*}

Then, the above equality combined with (2.19), (2.20), (2.21), (2.37), and (2.38); theorem 4.3 in the study by Lieb and Loss [38]; the Hölder inequality; the boundedness of the sequence $\left\{u_n\right\}_{n \in\mathbb{N}} \subset {X_\varepsilon}$; and the fact that $\|\varphi\|_{X_\varepsilon} \leqslant 1$ yields that (f) is true. This proof is now complete.

3. The limit problem

To the best of our knowledge, there is no result about the nonlinear fractional (p, q)-Choquard problem. That is why we need to consider the following limit problem associated with problem (1.3):

(3.1)\begin{align} \left\{ \begin{array}{lll} (- \Delta)_p^s u + (- \Delta)_q ^s u + V_0 (|u|^{p - 2} u + |u|^{q - 2} u) = \left(\frac{1}{|x|^\mu} * F(u)\right) f(u) & \ \text{in }\mathbb{R}^N, \\ u \in W^{s,p}(\mathbb{R}^N) \cap W^{s,q}(\mathbb{R}^N) ,\ u \gt 0 & \ \text{in } \mathbb{R}^N. \end{array} \right\} \end{align}

It is worth pointing out that we denote by $X_{V_0}$ the space X_ɛ when ɛ = 0. For all $u \in {X_{V_0}}$, we introduce the corresponding energy functional associated with problem (3.1) defined by

\begin{align*}I_{V_0}(u) := \frac{1}{p} [u]_{s,p}^p + \frac{1}{q} [u]_{s,q}^q + V_0 \left(\frac{1}{p} |u|_p^p +\frac{1}{q} |u|_q^q\right) - \frac{1}{2} \int_{\mathbb{R}^N}\left(\frac{1}{|x|^\mu} * F(u)\right) F(u) dx.\end{align*}

It is obvious that the functional $I_{V_0}$ is well-defined and belongs to C ¹, with its differential given by

\begin{align*} \langle I_{V_0}'(u),v \rangle & = \langle u ,v \rangle_{s,p} + \langle u, v \rangle_{s,q} + \int_{\mathbb{R}^N} V_0 \left(|u|^{p - 2} u + |u|^{q - 2} u \right) v dx \\ & - \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(u)\right) f(u) v dx, \end{align*}

for all u, $v \in {X_{V_0}}$. So, it is easy to check that all the solutions of problem (3.1) correspond to critical points of the functional $I_{V_0}$.

Next, we denote by $\mathcal{M}_{V_0}$ the Nehari manifold associated with the functional $I_{V_0}$, that is,

\begin{align*}\mathcal{M}_{V_0} := \left\{u \in {X_{V_0}} \setminus \{0\} : \langle I_{V_0}'(u),u \rangle = 0\right\}.\end{align*}

Moreover, we define $c_{V_0} : =\inf_{u \in \mathcal{M}_{V_0}} I_{V_0}(u)$.

Now, we define the following sets:

\begin{align*}X_{V_0}^+ := \left\{u \in X_{V_0} : |{\mathrm{supp}\,} (u^+)| \gt 0\right\} \quad \ \text{and } \quad S_{V_0}^+ = S_{V_0} \cap {X_{V_0}^+},\end{align*}

where $S_{V_0}$ is the unit sphere of $X_{V_0}$. We point out that $S_{V_0}^+$ is also an incomplete $C^{1,1}$-manifold of codimension one and contained in $X_{V_0}^+$. Hence, $X_{V_0} = T_u (S_{V_0}^+)\bigoplus \mathbb{R} u$ for each $u \in S_{V_0}^+$, where

\begin{align*} T_u (S_{V_0}^+) := \left\{v \in {X_{V_0}} : \langle u, v \rangle_{s,p} + \langle u, v \rangle_{s,q} + \int_{\mathbb{R}^N} V_0\left(|u|^{p - 2} u + |u|^{q - 2} u \right) vdx= 0 \right\}. \end{align*}

With arguments as in the proof of lemma 2.2, we can deduce that the following property holds true.

Lemma 3.1. Assume that $(f_1)$–$(f_4)$ and $(V_1)$ are fulfilled, then we have the following properties:

1. (a) for all $u \in {X_{V_0}^+}$, there exists a unique $t_u \gt 0$ such that $t_u u \in \mathcal{M}_{V_0}.$ Furthermore, $\hat{m}_{V_0}(u) = t_u u$ is the unique maximum of $I_{V_0}(t u);$

2. (b) there exists τ > 0 independent of u such that $t_u \geqslant \tau$ for all $u \in S_{V_0}^+.$ Moreover, for each compact set $\mathcal{W} \subset S_{V_0}^+,$ there exists a constant $C_\mathcal{W} \gt 0$ such that $t_u \leqslant C_\mathcal{W}$ for all $u \in \mathcal{W};$

3. (c) the mapping $\hat{m}_{V_0} : {X_{V_0}^+} \mapsto \mathcal{M}_{V_0}$ is continuous, $m_{V_0} :=\hat{m}_{V_0} |_{S_{V_0}^+}$ is a homeomorphism between $S_{V_0}^+$ and $\mathcal{M}_{V_0},$ and the inverse of $m_{V_0}$ is given by $m_{V_0}^{-1}(u) := u / \|u\|_{X_{V_0}};$

4. (d) $c_{V_0} : = \inf_{u \in \mathcal{M}_{V_0}} I_{V_0}(u) \gt 0$, and $I_{V_0}$ is bounded below on $\mathcal{M}_{V_0}$ by some positive constant$;$

5. (e) let $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ If there exists a sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset S_{V_0}^+$ such that ${\mathrm{dist}\,}(u_n,\partial S_\varepsilon^+) \rightarrow 0$ as $n \rightarrow \infty,$ then $\|m_{V_0}(u_n)\|_{X_{V_0}} \rightarrow + \infty$ and $J_\varepsilon(m_\varepsilon(u_n)) \rightarrow + \infty$ as $n \rightarrow \infty;$

6. (f) $I_{V_0}$ is coercive on $\mathcal{M}_{V_0}.$

Now, let us define the mappings

\begin{align*}\hat{\psi}_{V_0} : {X_{V_0}^+} \mapsto \mathbb{R} \quad \ \text{and } \quad \psi_{V_0} : S_{V_0}^+ \mapsto \mathbb{R}\end{align*}

by $\hat{\psi}_{V_0}(u) := I_{V_0}(\hat{m}_{V_0}(u))$ for all $u \in {X_{V_0}^+}$ and $\psi_{V_0} := \hat{\psi}_{V_0} |_{S_{V_0}^+}$.

Lemma 3.2. Assume that $(f_1)$–$(f_4)$ and $(V_1)$ hold true. Then,

1. (a) $\hat{\psi}_{V_0} \in C^1({X_{V_0}^+},\mathbb{R})$ and $\langle \hat{\psi}'_{V_0}(u), v\rangle = \frac{\|\hat{m}_{V_0}(u)\|_{X_{V_0}}}{\|u\|_{X_{V_0}}} \langle I_{V_0}'(\hat{m}_{V_0}(u)),v\rangle$ for all $u \in {X_{V_0}^+},$ all $v \in {X_{V_0}};$

2. (b) $\psi_{V_0} \in C^1(S_{V_0}^+,\mathbb{R})$ and $\langle \psi'_{V_0}(u),v\rangle = \|m_{V_0}(u)\|_{X_{V_0}} \langle I_{V_0}'(m_{V_0}(u)),v\rangle$ for all $u \in S_{V_0}^+,$ all $v \in T_u(S_{V_0}^+);$

3. (c) if $\{u_n\}_{n \in \mathbb{N}}$ is a Palais–Smale sequence for $\psi_{V_0},$ then $\{m_{V_0}(u_n)\}_{n \in \mathbb{N}}$ is a Palais–Smale sequence for $I_{V_0}.$ If $\{u_n\}_{n \in \mathbb{N}} \subset \mathcal{M}_{V_0}$ is a bounded Palais–Smale sequence for $I_{V_0},$ then $\{m_{V_0}^{- 1}(u_n)\}_{n \in \mathbb{N}} \subset S_{V_0}^+$ is a Palais–Smale sequence for $\psi_{V_0};$

4. (d) $u \in S_{V_0}^+$ is a critical point of $\psi_{V_0}$ if and only if $m_{V_0}(u) \in \mathcal{M}_{V_0}$ is a critical point of $I_{V_0}.$ Moreover, the corresponding critical values coincide and

   \begin{align*}\inf_{u \in S_{V_0}^+} \psi_{V_0}(u) = \inf_{u \in \mathcal{M}_{V_0}}I_{V_0}(u) = c_{V_0}.\end{align*}

Remark 3.3. The following variational characterization for $c_{V_0}$ is fulfilled:

\begin{align*}c_{V_0} = \inf_{u \in \mathcal{M}_{V_0}} I_{V_0}(u) = \inf_{u \in {X_{V_0}^+}} \max_{t \gt 0} I_{V_0}(t u) = \inf_{u \in S_{V_0}^+} \max_{t \gt 0} I_{V_0}(t u).\end{align*}

Lemma 3.4. Assume that $\left\{u_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ is a Palais–Smale sequence of the functional $I_{V_0}$ at the level $c_{V_0}.$ Then, the sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ is bounded, and there exist a sequence $\left\{y_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ and some constants $R,$ α > 0 such that

\begin{align*}\liminf_{n \rightarrow \infty} \int_{B_R(y_n)} |u_n|^q dx \geqslant \alpha.\end{align*}

Proof. Arguing as in lemma 2.5, it is obvious to see that $\left\{u_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ is bounded. Now, arguing by contradiction, suppose that for any R > 0, the following relation

\begin{align*}\lim_{n \rightarrow \infty}\sup_{y \in \mathbb{R}^N} \int_{B_R(y)} |u_n|^q dx = 0\end{align*}

holds true. Then, using lemma 2.2 in the study by Alves, Ambrosio, and Isernia [1], we know that for all $r \in (p,q_s^*)$,

(3.2)\begin{align} u_n \rightarrow 0 \ \text{in } L^r(\mathbb{R}^N) \ \text{as } n \rightarrow \infty. \end{align}

So, we deduce from theorem 4.3 in the study by Lieb and Loss [38], hypotheses $(f_1)$–$(f_2)$, the boundedness of the sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$, and (3.2) that

\begin{align*}\lim_{n \rightarrow \infty}\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(u_n)\right) f(u_n) u_n dx = 0.\end{align*}

Moreover, by the boundedness of the sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$, we see that $\langle I_{V_0}'(u_n),u_n \rangle = o_n(1)$ as $n \rightarrow \infty$, that is,

\begin{align*}& \quad [u_n]_{s,p}^p + [u_n]_{s,q}^q + V_0 \left(|u_n|_p^p + |u_n|_q^q \right) \\ & = \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(u_n)\right) f(u_n) u_n dx + o_n(1) \ \text{as } n \rightarrow \infty.\end{align*}

This implies that $\|u_n\|_{X_{V_0}} = o_n(1)$ as $n \rightarrow \infty$. We get a contradiction since $I_{V_0}(u_n) \rightarrow c_{V_0} \gt 0$ in $\mathbb{R}$ as $n \rightarrow \infty$. The proof is now complete.

Theorem 3.5 Problem (3.1) has a positive ground state solution.

Proof. We deduce from a variant of the Mountain Pass Theorem without the Palais–Smale condition that there is a Palais–Smale sequence $\left\{u_n\right\}_{n\in \mathbb{N}} \subset {X_{V_0}}$ of the functional $I_{V_0}$ at the level $c_{V_0}$. As in the proof of lemma 2.5, we can conclude that the sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ is bounded. Then, we are able to deduce from lemma 3.4 that there exist a sequence $\left\{y_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ and some constants R, α > 0

\begin{align*}\liminf_{n \rightarrow \infty} \int_{B_R(y_n)} |u_n|^q dx \geqslant \alpha.\end{align*}

Set $v_n(x) : = u_n(x + y_n)$. Then, we have

\begin{align*}\int_{B_R(0)} |v_n|^q dx \geqslant \frac{\alpha}{2}.\end{align*}

It is easy to check that $I_{V_0}(v_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ and $I_{V_0}'(v_n) \rightarrow 0$ in ${X_{V_0}}^*$ as $n \rightarrow \infty$. Clearly, the sequence $\left\{v_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ is also bounded. Thus, we may suppose that there exists $0 \neq v \in {X_{V_0}}$ such that $v_n \xrightarrow w v$ (up to a subsequence) in ${X_{V_0}}$ as $n \rightarrow \infty$. Moreover, arguing as in lemma 2.7, we have $I_{V_0}'(v) = 0$. Due to v ≠ 0, we can derive that $v \in \mathcal{M}_{V_0}$, and so $I_{V_0}(v) \geqslant c_{V_0}$. On the other hand, from hypothesis $(f_3)$ and Fatou’s Lemma, we deduce that

\begin{align*}I_{V_0}(v) = I_{V_0}(v) - \frac{1}{q}\langle I_{V_0}'(v),v\rangle \leqslant \liminf_{n \rightarrow \infty} \left(I_{V_0}(v_n) - \frac{1}{q} \langle I_{V_0}'(v_n),v_n\rangle\right) = c_{V_0}.\end{align*}

So, we obtain $I_{V_0}(v) = c_{V_0}$.

We can also show that this ground state solution v is positive. Set $v^- :=\min\left\{v,0\right\}$, and then $v^- \in {X_{V_0}}$. Recalling that $f(t) = 0$ for $t \leqslant 0$ and using $\langle I_{V_0}'(v),v^- \rangle = 0$, we obtain

\begin{align*}\|v^-\|_{V_0,p}^p + \|v^-\|_{V_0,q}^q \leqslant 0.\end{align*}

This leads to $v^- = 0$, and so $v \geqslant 0$ on $\mathbb{R}^N$. Thus, $v \geqslant 0$ and $v \not \equiv 0$. Similar to the proof of lemma 6.1, we see that $v \in L^\infty(\mathbb{R}^N)$. In addition, from corollary 2.1 in the study by Ambrosio and Rădulescu [11], it follows that $v \in C^\sigma(\mathbb{R}^N)$ for some $\sigma \in (0,1)$. Now, it follows from the proof of theorem 1.1 (ii) in the study by Jarohs [33] that v > 0 on $\mathbb{R}^N$. This ends the proof.

Next, we introduce a compactness result for the autonomous problem, which will be very useful in the sequel.

Lemma 3.6. Let $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ Assume that $\left\{u_n\right\}_{n \in \mathbb{N}} \subset \mathcal{M}_{V_0}$ is a sequence satisfying $I_{V_0}(u_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ as $n \rightarrow \infty,$ then the sequence $\left\{u_n(\cdot + y_n)\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ possesses a convergent subsequence for some sequence $\left\{y_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N.$

Proof. On account of the fact that $\left\{u_n\right\}_{n \in \mathbb{N}} \subset \mathcal{M}_{V_0}$ and $I_{V_0}(u_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ as $n \rightarrow \infty$, lemma 3.1 (c), lemma 3.2 (d), and the definition of $c_{V_0}$ imply that

\begin{align*}w_n := m_{V_0}^{-1}(u_n) \in S_{V_0}^+ \ \text{for all } n \in \mathbb{N}\end{align*}

and

\begin{align*}\psi_{V_0}(w_n) = I_{V_0}(u_n) \rightarrow c_{V_0} = \inf_{w \in S_{V_0}^+} \psi_{V_0}(w) \ \text{in } \mathbb{R} \ \text{as } n \rightarrow \infty.\end{align*}

Now, we introduce the following map:

\begin{align*} \mathcal{I}(u) = \begin{cases} \ \psi_{V_0}(u) \quad & \ \text{if } u \in S_{V_0}^+,\\ \ + \infty\quad & \ \text{if } u \in \partial S_{V_0}^+. \end{cases} \end{align*}

It is worth pointing out the following essential facts:

1. (i) $(\overline{S_{V_0}^+}, \delta_{V_0})$, where $\delta_{V_0}(u,v) = \|u - v\|_{X_{V_0}}$, is a complete metric space;

2. (ii) $\mathcal{I} \in C(\overline{S_{V_0}^+},\mathbb{R} \cup \left\{+ \infty\right\})$, by lemma 3.1 (e);

3. (iii) $\mathcal{I}$ is bounded below, by lemma 3.2 (d).

We deduce from the Ekeland variational principle (see the study by Ekeland [28]) that there exists a Palais–Smale sequence $\left\{\hat{w}_n\right\}_{n \in \mathbb{N}} \subset S_{V_0}^+$ of the functional $\psi_{V_0}$ at the level $c_{V_0}$ such that $\|\hat{w}_n - w_n\|_{X_{V_0}} = o_n(1)$ as $n \rightarrow \infty$, that is, $\psi_{V_0}(\hat{w}_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ and $\psi_{V_0}'(\hat{w}_n) \rightarrow 0$ in ${T_{\hat{w}_n}(S_{V_0}^+)}^*$ as $n \rightarrow \infty$. Therefore, from lemma 3.2 (c), it follows that the sequence $\left\{m_{V_0}(\hat{w}_n)\right\}_{n \in \mathbb{N}} \subset \mathcal{M}_{V_0}$ is the Palais–Smale sequence of the functional $I_{V_0}$ at the level $c_{V_0}$. Then, we can conclude that the sequence $\left\{m_{V_0}(\hat{w}_n)\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ is the bounded Palais–Smale sequence at the level $c_{V_0}$. Now, let $v_n :=m_{V_0}(\hat{w}_n)$. According to theorem 3.5, we see that there exist a sequence $\left\{y_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ and $0 \neq \hat{v} \in {X_{V_0}}$ such that

(3.3)\begin{align} \begin{cases} \hat{v}_n(\cdot) :=v_n(\cdot + y_n) \xrightarrow w \hat{v} \quad & \ \text{in } {X_{V_0}} \ \text{as } n \rightarrow \infty,\\ I_{V_0}(\hat{v}_n) = c_{V_0} + o_n(1),\ I_{V_0}'(\hat{v}_n) = o_n(1) \quad &\ \text{as } n \rightarrow \infty,\\ I_{V_0}(\hat{v}) = c_{V_0},\ I_{V_0}'(\hat{v}) = 0. \end{cases} \end{align}

Set $\widetilde{v}_n := \hat{v}_n - \hat{v}$. Applying lemma 2.10, the Hölder inequality, and (3.3), we can deduce that

\begin{align*}I_{V_0}(\widetilde{v}_n) = o_n(1) \quad \ \text{and }\quad I_{V_0}'(\widetilde{v}_n) = o_n(1) \ \text{as } n \rightarrow \infty.\end{align*}

Consequently, $\widetilde{v}_n \rightarrow 0$ in ${X_{V_0}}$ as $n \rightarrow \infty$, that is, $\hat{v}_n \rightarrow \hat{v}$ in ${X_{V_0}}$ as $n \rightarrow \infty$. From lemma 3.1 (c) and (3.3), we infer that $u_n(\cdot + y_n) \rightarrow \hat{v} \in \mathcal{M}_{V_0}$ as $n \rightarrow \infty$. So, we now complete the proof.

4. The barycenter map

Next, we need to establish a relationship between the topology of M and the number of positive solutions for problem (2.2). To this end, we first choose δ > 0 such that

(4.1)\begin{align} M_\delta := \left\{x \in \mathbb{R}^N : {\mathrm{dist}\,} (x,M) \leqslant \delta\right\} \subset \Lambda \end{align}

and take $\eta \in C^\infty([0, +\infty),[0,1])$ non-increasing satisfying $\eta(t) = 1$ for $t \in (0,\delta / 2)$, $\eta(t) = 0$ for $t \in [\delta, + \infty)$, and $|\eta'(t)| \leqslant C$ for some constant C > 0.

Let ω be a positive ground state solution to the autonomous problem (3.1). For any $y \in M$, we define the following function:

\begin{align*}\Psi_{\varepsilon,y}(x) := \eta (|\varepsilon x - y|) \omega\left(\frac{\varepsilon x - y}{\varepsilon}\right)\end{align*}

with the unique number $t_\varepsilon \gt 0$ satisfying

\begin{align*}\max_{t \geqslant 0} J_\varepsilon(t \Psi_{\varepsilon,y}) = J_\varepsilon(t_\varepsilon \Psi_{\varepsilon,y}),\end{align*}

and let us consider the mapping $\Phi_\varepsilon : M \mapsto \mathcal{N_\varepsilon}$ defined by

\begin{align*}\Phi_\varepsilon(y) := t_\varepsilon \Psi_{\varepsilon,y}.\end{align*}

Lemma 4.1. The mapping $\Phi_\varepsilon$ verifies the following property:

\begin{align*}\lim_{\varepsilon \rightarrow 0} J_\varepsilon(\Phi_\varepsilon(y)) = c_{V_0} \ \text{uniformly in } y \in M.\end{align*}

Proof. Arguing by contradiction, we may assume that there exist $\delta_0 \gt 0$, $\left\{y_n\right\}_{n \in \mathbb{N}} \subset M$, and $\varepsilon_n \rightarrow 0$ in $\mathbb{R}$ as $n \rightarrow \infty$ such that

(4.2)\begin{align} \left|J_{\varepsilon_n}(\Phi_{\varepsilon_n}(y_n)) - c_{V_0} \right| \geqslant \delta_0. \end{align}

We first note that for each $ n \in \mathbb{N}$ and for all $ z \in B_{\frac{\delta}{\varepsilon_n}}$, $ \varepsilon_n z\in B_\delta$, and hence $\varepsilon_n z + y_n \in B_\delta(y_n) \subset M_\delta \subset \Lambda$.

Applying the change of variable $z = (\varepsilon_n x - y_n) / {\varepsilon_n}$ and recalling that G = F in Λ and $\eta(t) = 0$ for $t \geqslant \delta$, we have

(4.3)\begin{align} J_{\varepsilon_n}(\Phi_{\varepsilon_n}(y_n)) & = \frac{{t_{\varepsilon_n}^p}}{p} \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n}, p}^p + \frac{{t_{\varepsilon_n}^q}}{q} \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n}, q}^q\nonumber\\ & - \frac{1}{2}\int_{\mathbb{R}^N}\left(\frac{1}{|x|^\mu} * G(\varepsilon_n x, t_{\varepsilon_n} \Psi_{\varepsilon_n,y_n})\right)G(\varepsilon_n x, t_{\varepsilon_n} \Psi_{\varepsilon_n,y_n}) dx\nonumber\\ & = \frac{{t_{\varepsilon_n}^p}}{p} \left([\eta(|\varepsilon_n \cdot|) \omega]_{s,p}^p + \int_{\mathbb{R}^N} V(\varepsilon_n z + y_n)(\eta(|\varepsilon_n z|) \omega(z))^p dz\right)\nonumber\\ & + \frac{{t_{\varepsilon_n}^q}}{q} \left([\eta(|\varepsilon_n \cdot|) \omega]_{s,q}^q + \int_{\mathbb{R}^N} V(\varepsilon_n z + y_n)(\eta(|\varepsilon_n z|) \omega(z))^q dz\right)\nonumber\\ & - \frac{1}{2}\int_{\mathbb{R}^N} \left(\frac{1}{|z|^\mu} * F(t_{\varepsilon_n} \eta(|\varepsilon_n z|) \omega(z))\right) F(t_{\varepsilon_n} \eta(|\varepsilon_n z|) \omega(z)) dz. \end{align}

Next, we prove that the sequence $\left\{t_{\varepsilon_n}\right\}_{n \in \mathbb{N}} \subset \mathbb{R}$ verifies $t_{\varepsilon_n} \rightarrow 1$ in $\mathbb{R}$ as $n \rightarrow \infty.$ Using the definition of $t_{\varepsilon_n}$, we see that $t_{\varepsilon_n} \Psi_{\varepsilon_n,y_n} \in \mathcal{N}_{\varepsilon_n}$, that is,

(4.4)\begin{align} & t_{\varepsilon_n}^{p - q} \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n},p}^p + \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n},q}^q \nonumber\\ =\ & \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(t_{\varepsilon_n} \Psi_{\varepsilon_n,y_n})\right) \frac{f(t_{\varepsilon_n} \Psi_{\varepsilon_n,y_n}) \Psi_{\varepsilon_n,y_n}}{t_{\varepsilon_n}^{q - 1}} dx\nonumber\\ & (\text{on account of the fact that } g = f ,\ G = F\ \text{on } \Lambda)\nonumber\\ =\ & \int_{\mathbb{R}^N} \left(\frac{1}{|z|^\mu} * F(t_{\varepsilon_n} \eta(|\varepsilon_n z|) \omega(z))\right) \frac{f(t_{\varepsilon_n} \eta(|\varepsilon_n z|) \omega(z)) \eta(|\varepsilon_n z|) \omega(z)}{t_{\varepsilon_n}^{q - 1}} dz. \end{align}

Taking into account $\eta (|x|) = 1$ for $x \in B_{\frac{\delta}{2}}$ and $B_{\frac{\delta}{2}} \subset B_{\frac{\delta}{\varepsilon_n}}$ for $n \in \mathbb{N}$ large enough, we deduce from (4.4) that

\begin{align*}t_{\varepsilon_n}^{p - q} \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n},p}^p + \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n},q}^q \geqslant \int_{B_{\frac{\delta}{2}}} \int_{B_{\frac{\delta}{2}}} \frac{F(t_{\varepsilon_n} \omega(x)) f(t_{\varepsilon_n} \omega(z)) \omega(z)}{t_{\varepsilon_n}^{q - 1}|z - x|^\mu} dxdz.\end{align*}

On the other hand, ω is a continuous and positive function in $\mathbb{R}^N$, so there exists $\bar{z} \in \mathbb{R}^N$ such that

\begin{align*}\omega(\bar{z}) = \min_{z \in \overline{B}_{\frac{\delta}{2}}} \omega(z) \gt 0.\end{align*}

Thus, we deduce from hypothesis $(f_4)$ that

(4.5)\begin{align} t_{\varepsilon_n}^{p - q} \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n},p}^p + \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n},q}^q \geqslant \delta^{- \mu} \left|B_{\frac{\delta}{2}}\right|^2 \frac{F(t_{\varepsilon_n} \omega(\bar{z})) f(t_{\varepsilon_n} \omega(\bar{z}))}{\left(t_{\varepsilon_n} \omega(\bar{z})\right)^{q - 1}} \omega(\bar{z})^q. \end{align}

According to the Dominated Convergence Theorem and lemma 2.2 in the study by Ambrosio [7], we can deduce that

(4.6)\begin{align} \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n},t} \rightarrow \|\omega\|_{V_0,t} \in (0,+ \infty) \ \text{in } \mathbb{R} \ \text{as } n \rightarrow \infty \quad\ \text{for } t \in \left\{p,q\right\} \end{align}

and

(4.7)\begin{align} \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(\Psi_{\varepsilon_n,y_n})\right) F(\Psi_{\varepsilon_n,y_n}) dx \rightarrow \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(\omega)\right) F(\omega) dx \ \text{in } \mathbb{R} \ \text{as } n \rightarrow \infty. \end{align}

Therefore, if $t_{\varepsilon_n} \rightarrow + \infty$, we can conclude that the left-hand side of (4.5) satisfies the following property:

(4.8)\begin{align} \lim_{n \rightarrow \infty}\left(t_{\varepsilon_n}^{p - q} \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n},p}^p + \|\Psi_{\varepsilon_n,y_n}\|_{V_{\varepsilon_n},q}^q\right) = \|\omega\|_{V_0,q}^q, \end{align}

since q > p. On the other hand, we deduce from hypothesis $(f_3)$ that

(4.9)\begin{align} \lim_{n \rightarrow \infty} \frac{F(t_{\varepsilon_n} \omega(\bar{z}))}{(t_{\varepsilon_n} \omega(\bar{z}))^{\frac{q}{2}}} = \lim_{n \rightarrow \infty} \frac{f(t_{\varepsilon_n} \omega(\bar{z}))}{(t_{\varepsilon_n} \omega(\bar{z}))^{\frac{q}{2} - 1}} = + \infty. \end{align}

From (4.5), (4.8), and (4.9) we get a contradiction. Then, we pass to a subsequence and assume that there exists t ₀ such that $t_{\varepsilon_n} \rightarrow t_0 \geqslant 0$. In addition, applying (4.4) and (4.6) and combining with $(f_1)$–$(f_2)$, we deduce that $t_0 \gt 0$.

In (4.4), we pass to the limit as $n \rightarrow \infty$, and then we can use (4.6) and the Dominated Convergence Theorem to infer that

(4.10)\begin{align} t_0^{p - q} \|\omega\|_{V_0,p}^p + \|\omega\|_{V_0,q}^q = \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(t_0 \omega)\right) \frac{f(t_0 \omega)}{t_0^{q - 1}} \omega dx. \end{align}

Due to $\omega \in \mathcal{M}_{V_0}$, we derive that

(4.11)\begin{align} \|\omega\|_{V_0,p}^p + \|\omega\|_{V_0,q}^q = \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(\omega)\right) \frac{f(\omega)}{t_0^{q - 1}} \omega dx. \end{align}

From the fact that the functions $\frac{F(t)}{t^{\frac{q}{2}}}$ and $\frac{f(t)}{t^{\frac{q}{2} - 1}}$ are increasing for t > 0, together with (4.10) and (4.11), it follows that $t_0 = 1$.

In (4.3), we pass to the limit as $n \rightarrow \infty$, and together with (4.7), we have

\begin{align*}\lim_{n \rightarrow \infty} J_{\varepsilon_n}(\Phi_{\varepsilon_n}(y_n)) = I_{V_0}(\omega) = c_{V_0}.\end{align*}

This is a contradiction, since (4.2). So, we complete the proof of the lemma.

Let us consider the function $h : \mathbb{R}^+ \mapsto \mathbb{R}^+$ such that $h(\varepsilon) := \sup_{y \in M} \left|J_\varepsilon(\Phi_\varepsilon(y)) - c_{V_0}\right|$ for all ɛ > 0. Then, we define the following subset of $\mathcal{N}_\varepsilon$:

\begin{align*}\hat{\mathcal{N}}_\varepsilon := \left\{u \in \mathcal{N}_\varepsilon : J_\varepsilon(u) \leqslant c_{V_0} + h(\varepsilon)\right\}.\end{align*}

From lemma 4.1, it follows that $h(\varepsilon) \rightarrow 0$ as $\varepsilon \rightarrow 0$. Additionally, we deduce from the definition of h that $\Phi_\varepsilon(y) \in \hat{\mathcal{N}}_\varepsilon$ for any $y \in M$ and ɛ > 0, and so $\hat{\mathcal{N}}_\varepsilon \neq \emptyset$.

For any δ > 0 given by (4.1), let us choose $\rho := \rho(\delta) \gt 0$ such that $M_\delta \subset B_\rho$. Then, we introduce the map $\zeta : \mathbb{R}^N \mapsto \mathbb{R}^N$ defined by

\begin{align*} \zeta(x) := \begin{cases} x \quad & \ \text{if } |x| \lt \rho,\\ \frac{\rho x}{|x|} \quad & \ \text{if } |x| \geqslant \rho. \end{cases} \end{align*}

Now, we introduce the following barycenter map $\beta_\varepsilon : \mathcal{N}_\varepsilon \mapsto \mathbb{R}^N$ defined by

\begin{align*}\beta_\varepsilon(u) := \frac{\int_{\mathbb{R}^N}\zeta(\varepsilon x) \left(|u|^p + |u|^q\right) dx}{\int_{\mathbb{R}^N} \left(|u|^p + |u|^q\right) dx}\end{align*}

for all $u \in \mathcal{N}_\varepsilon$.

Then, from the above information, we can give the following lemma.

Lemma 4.2. The function β_ɛ has the following property:

\begin{align*}\lim_{\varepsilon \rightarrow 0} \beta_\varepsilon(\Phi_\varepsilon(y)) = y \ \text{uniformly in } y \in M.\end{align*}

Proof. Arguing by contradiction, assume that there exist $\delta_0 \gt 0$, $\left\{y_n\right\}_{n \in \mathbb{N}} \subset M$, and $\varepsilon_n \rightarrow 0$ in $\mathbb{R}$ as $n \rightarrow \infty$ such that

(4.12)\begin{align} \left|\beta_{\varepsilon_n}(\Phi_{\varepsilon_n}(y_n)) - y_n\right| \geqslant \delta_0. \end{align}

Employing the definitions of $\Phi_{\varepsilon_n}$, $\beta_{\varepsilon_n}$, and ζ and using the change of variable $z = (\varepsilon_n x - y_n) / \varepsilon_n$, we can conclude that

\begin{align*}\beta_{\varepsilon_n}(\Phi_{\varepsilon_n}(y_n)) = y_n + \frac{\int_{\mathbb{R}^N} \left(\zeta(\varepsilon_n z + y_n) - y_n \right) \left(|\eta(|\varepsilon_n z|) \omega(z)|^p + |\eta(|\varepsilon_n z|) \omega(z)|^q\right) dz}{\int_{\mathbb{R}^N} \left(|\eta(|\varepsilon_n z|) \omega(z)|^p + |\eta(|\varepsilon_n z|) \omega(z)|^q\right) dz}.\end{align*}

On account of $\left\{y_n\right\}_{n \in \mathbb{N}} \subset M \subset M_\delta$, combining the Dominated Convergence Theorem, we can derive that

\begin{align*}\lim_{n \rightarrow \infty} |\beta_{\varepsilon_n}(\Phi_{\varepsilon_n}(y_n)) - y_n| = 0,\end{align*}

which contradicts relation (4.12). This proof is now complete.

Lemma 4.3. Let $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ Assume that the sequences $\left\{\varepsilon_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}$ and $\left\{u_n\right\}_{n \in \mathbb{N}} \subset \mathcal{N}_{\varepsilon_n}$ satisfy $\varepsilon_n \rightarrow 0$ in $\mathbb{R}$ and $J_{\varepsilon_n}(u_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ as $n \rightarrow \infty$; then, there is a sequence $\left\{\hat{y}_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ such that the sequence $\left\{\hat{u}_n(x) := u_n(x + \hat{y}_n)\right\}_{n \in \mathbb{N}}$ admits a subsequence which converges in ${X_{V_0}}.$ Furthermore, the sequence $ \left\{y_n := \varepsilon_n \hat{y}_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ has a subsequence $\left\{y_n\right\}_{n \in \mathbb{N}}$ $($still denoted by itself $)$ such that $y_n \rightarrow y_0 \in M$ as $n \rightarrow \infty.$

Proof. It is easy to verify that $\left\{u_n\right\}_{n \in \mathbb{N}} \subset X_{V_0}$ is bounded. Then, using lemma 3.4, we know that there exist a sequence $\left\{\hat{y}_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ and some constants $R,$ α > 0 such that

\begin{align*}\liminf_{n \rightarrow \infty} \int_{B_R(\hat{y}_n)} |u_n|^q dx \geqslant \alpha.\end{align*}

Let $\hat{u}_n(x) := u_n(x + \hat{y}_n)$. Consequently, $\left\{\hat{u}_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ is bounded, and so passing to a subsequence, we can suppose that there exists some $0 \neq \hat{u} \in {X_{V_0}}$ such that $\hat{u}_n \xrightarrow w \hat{u}$ in ${X_{V_0}}$ as $n \rightarrow \infty$. Let $t_n \gt 0$ be such that $\hat{v}_n := t_n \hat{u}_n \in \mathcal{M}_{V_0}$, and let $y_n :=\varepsilon_n \hat{y}_n$. Thus, we deduce that

\begin{align*} c_{V_0}& \leqslant I_{V_0}(\hat{v}_n) \ (\text{from the definition of } c_{V_0})\\ & \leqslant \frac{1}{p} [\hat{v}_n]_{s,p}^p + \frac{1}{q} [\hat{v}_n]_{s,q}^q + \int_{\mathbb{R}^N} V(\varepsilon_n x + y_n) \left(\frac{1}{p} |\hat{v}_n|^p + \frac{1}{q}|\hat{v}_n|^q\right) dx\\ & - \frac{1}{2}\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(\hat{v}_n)\right) F(\hat{v}_n) dx\\ & \leqslant \frac{t_n^p}{p} [u_n]_{s,p}^p + \frac{t_n^q}{q} [u_n]_{s,q}^q + \int_{\mathbb{R}^N} V(\varepsilon_n x) \left(\frac{t_n^p}{p}|u_n|^p + \frac{t_n^q}{q}|u_n|^q\right) dx\\ & - \frac{1}{2}\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon x, t_n u_n)\right) G(\varepsilon x, t_n u_n) dx \ (\text{by } (g_2))\\ & = J_{\varepsilon_n}(t_n u_n) \leqslant J_{\varepsilon_n}(u_n)\ (\text{since } u_n \in \mathcal{N}_{\varepsilon_n})\\ & = c_{V_0} + o_n(1) \ \text{as } n \rightarrow \infty. \end{align*}

This implies that $I_{V_0}(\hat{v}_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ as $n \rightarrow \infty$ and $\hat{v}_n \in \mathcal{M}_{V_0}.$ Clearly, the sequence $\left\{\hat{v}_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ is bounded. Therefore, up to a subsequence if necessary, still denoted by itself, we may assume that there exists $\hat{v} \in {X_{V_0}}$ such that $\hat{v}_n \xrightarrow w \hat{v}$ in ${X_{V_0}}$ as $n \rightarrow \infty$. It is easy to see that the sequence $\left\{t_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}$ is bounded, and it holds that $t_n \rightarrow t_0 \geqslant 0$ as $n \rightarrow \infty$. Indeed, $t_0 \gt 0$. Otherwise, $t_0 = 0$, so, we infer from the boundedness of the sequence $\left\{\hat{v}_n\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ that $\|\hat{v}_n\|_{X_{V_0}} = t_n \|\hat{u}_n\|_{X_{V_0}} \rightarrow 0$ in $\mathbb{R}$ as $n \rightarrow \infty$, and so $I_{V_0}(\hat{v}_n) \rightarrow 0$ in $\mathbb{R}$ as $n \rightarrow \infty$, but this is impossible, since $c_{V_0} \gt 0$. Thus, $t_0 \gt 0$. We deduce from the uniqueness of the weak limit that $\hat{v} = t_0 \hat{u}$ and $\hat{u} \neq 0$. Then, from lemma 3.6, it follows that

(4.13)\begin{align} \hat{v}_n \rightarrow \hat{v} \ \text{in } {X_{V_0}} \ \text{as } n \rightarrow \infty, \end{align}

and so $\hat{u}_n \rightarrow \hat{u}$ in ${X_{V_0}}$ as $n \rightarrow \infty$. Moreover, $I_{V_0}(\hat{v}) = c_{V_0}$ and $\langle I_{V_0}'(\hat{v}), \hat{v} \rangle = 0$.

Next, we shall prove that the sequence $\left\{y_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ has a subsequence, still denoted by itself, such that $y_n \rightarrow y_0 \in M$ as $n \rightarrow \infty$. We first show the boundedness of the sequence $\left\{y_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$. Otherwise, the sequence $\left\{y_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ is not bounded. So, we may assume that there exists a subsequence, still denoted by itself, such that $|y_n| \rightarrow + \infty$ in $\mathbb{R}$ as $n \rightarrow \infty$. Then, we choose R > 0 large enough such that $\Lambda \subset B_R$, we may suppose that $|y_n| \gt 2 R$ for $n \in \mathbb{N}$ sufficiently large, and so for all $x \in B_{R / {\varepsilon_n}}$ we have

\begin{align*}|\varepsilon_n x + y_n| \geqslant |y_n| - |\varepsilon_n x| \gt R.\end{align*}

On account of the facts that $u_n \in \mathcal{N}_{\varepsilon_n}$ and $J_{\varepsilon_n}(u_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ as $n \rightarrow \infty$, thus, for $n \in \mathbb{N}$ large enough, we have that $u_n \in \mathcal{B}$. Consequently, from lemma 2.6, it follows that

\begin{align*}\left|\frac{1}{|x|^\mu} * G(\varepsilon_n x, u_n)\right|_{L^\infty(\mathbb{R}^N)} \lt \frac{K}{2} \ \text{for } n \in \mathbb{N} \ \text{large enough}.\end{align*}

Therefore, for $n \in \mathbb{N}$ large enough, we obtain

\begin{align*} \|\hat{u}_n\|_{V_0,p}^p + \|\hat{u}_n\|_{V_0,q}^q & \leqslant \frac{K}{2} \int_{\mathbb{R}^N} g(\varepsilon_n x + y_n, \hat{u}_n) \hat{u}_n dx\\ & \leqslant \frac{K}{2} \int_{B_{R / {\varepsilon_n}}} \widetilde{f}(\hat{u}_n) \hat{u}_n dx + \frac{K}{2} \int_{B_{R / {\varepsilon_n}}^c} f(\hat{u}_n) \hat{u}_n dx\\ & \leqslant \frac{K}{2} \int_{B_{R / {\varepsilon_n}}} \frac{V_0}{K} \left(|\hat{u}_n|^p + |\hat{u}_n|^q \right) dx + \frac{K}{2} \int_{B_{R / {\varepsilon_n}}^c} f(\hat{u}_n) \hat{u}_n dx,\\ &\quad (\text{on account of the fact that } \widetilde{f}(\hat{u}_n) \hat{u}_n \\ & \qquad \leqslant \frac{V_0}{K} \left(|\hat{u}_n|^p + |\hat{u}_n|^q \right) \ \text{on } B_{R / {\varepsilon_n}}). \end{align*}

Recalling that $\hat{u}_n \rightarrow \hat{u}$ in ${X_{V_0}}$ as $n \rightarrow \infty$ and using the Dominated Convergence Theorem, we obtain

\begin{align*}\int_{B_{R / {\varepsilon_n}}^c} f(\hat{u}_n) \hat{u}_n dx = o_n(1) \ \text{as } n \rightarrow \infty.\end{align*}

So, we have

\begin{align*}\frac{1}{2} \left (\|\hat{u}_n\|_{V_0,p}^p + \|\hat{u}_n\|_{V_0,q}^q \right) \leqslant o_n(1) \ \text{as } n \rightarrow \infty.\end{align*}

Using $\hat{u}_n \rightarrow \hat{u} \neq 0$ in ${X_{V_0}}$ as $n \rightarrow \infty$ again, we see that this is a contradiction.

Now, we get the boundedness of the sequence $\left\{y_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$. Passing to a subsequence (still denoted by $\left\{y_n\right\}_{n \in \mathbb{N}}$), we may assume that there exists $y_0 \in \mathbb{R}^N$ such that $y_n \rightarrow y_0 \in \overline{\Lambda}$ as $n \rightarrow \infty$. In fact, if $y_0 \not \in \overline{\Lambda}$, then we can find some constant r > 0 such that $y_n \in B_{r / 2}(y_0) \subset \overline{\Lambda}^c$. Arguing as before, we can reach a contradiction. So, $y_0 \in \overline{\Lambda}$.

It remains to show that $V(y_0) = V_0$. Arguing by contradiction, again we may assume that $V(y_0) \gt V_0$. From (4.13), together with Fatou’s lemma and the invariance of $\mathbb{R}^N$ by translations, it follows that

\begin{align*} c_{V_0} & = I_{V_0}(\hat{v})\\ & \lt \liminf_{n \rightarrow \infty} \Bigg( \frac{1}{p} [\hat{v}_n]_{s,p}^p + \frac{1}{q} [\hat{v}_n]_{s,q}^q + \int_{\mathbb{R}^N} V(\varepsilon_n x + y_n) \left(\frac{1}{p}|\hat{v}_n|^p + \frac{1}{q}|\hat{v}_n|^q\right) dx\\ & - \frac{1}{2}\int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * F(\hat{v}_n)\right) F(\hat{v}_n) dx\Bigg)\\ & \leqslant \liminf_{n \rightarrow \infty}J_{\varepsilon_n}(t_n u_n) \leqslant \liminf_{n \rightarrow \infty}J_{\varepsilon_n}(u_n) = c_{V_0}. \end{align*}

This is impossible. So, from hypothesis $(V_2)$, it follows that $y_0 \in M$. This proof is now complete.

Lemma 4.4. For any $\delta \gt 0,$ we have

\begin{align*}\lim_{\varepsilon \rightarrow 0} \sup_{u \in \hat{\mathcal{N}}_\varepsilon} {\mathrm{dist}\,}(\beta_\varepsilon(u),M_\delta) = 0.\end{align*}

Proof. Let $\varepsilon_n \rightarrow 0$ in $\mathbb{R}$ as $n \rightarrow \infty$, then we can find a sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset \hat{\mathcal{N}}_{\varepsilon_n}$ such that

\begin{align*}\sup_{u \in \hat{\mathcal{N}}_{\varepsilon_n}} \inf_{y \in M_\delta} |\beta_{\varepsilon_n}(u_n) - y| = \inf_{y \in M_\delta} |\beta_{\varepsilon_n}(u_n) - y| + o_n(1) \ \text{as } n \rightarrow \infty.\end{align*}

Noting that $\left\{u_n\right\}_{n \in \mathbb{N}} \subset \hat{\mathcal{N}}_{\varepsilon_n} \subset \mathcal{N}_{\varepsilon_n}$, we infer that

\begin{align*}c_{V_0} \leqslant c_{\varepsilon_n} \leqslant J_{\varepsilon_n}(u_n) \leqslant c_{V_0} + h(\varepsilon_n),\end{align*}

and so

\begin{align*}\lim_{n \rightarrow \infty} J_{\varepsilon_n}(u_n) = c_{V_0}.\end{align*}

Then, we deduce from lemma 4.3 that for $n \in \mathbb{N}$ large enough, there exists some sequence $\left\{\hat{y}_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ such that $y_n = \varepsilon_n \hat{y}_n \in M_\delta$. So, we have

\begin{align*}\beta_{\varepsilon_n}(u_n) = y_n + \frac{\int_{\mathbb{R}^N} \left(\zeta(\varepsilon_n z + y_n) - y_n\right) \left(|u_n(z + \hat{y}_n)|^p + |u_n(z + \hat{y}_n)|^q\right) dz}{\int_{\mathbb{R}^N} \left(|u_n(z + \hat{y}_n)|^p + |u_n(z + \hat{y}_n)|^q\right) dz}.\end{align*}

Taking into account the facts that $\left\{\hat{u}_n(\cdot + \hat{y}_n)\right\}_{n \in \mathbb{N}} \subset {X_{V_0}}$ has a convergent subsequence and $\varepsilon_n z + y_n \rightarrow y_0 \in M$ as $n \rightarrow \infty$, we derive that $\beta_{\varepsilon_n}(u_n) = y_n + o_n(1)$ in $\mathbb{R}^N$ as $n \rightarrow \infty$. So, there exists a sequence $\left\{y_n\right\}_{n \in \mathbb{N}} \subset M_\delta$ such that

\begin{align*}\lim_{n \rightarrow \infty} |\beta_{\varepsilon_n}(u_n) - y_n| = 0.\end{align*}

This ends the proof of the lemma.

5. Multiple solutions for problem (2.2)

In this section, we shall establish a relationship between the topology of M and the number of solutions for problem (2.2). Since $\mathcal{N}_\varepsilon$ is not a C ¹ submanifold of the space X_ɛ, we cannot use directly the standard Ljusternik–Schnirelmann theory, but we can bypass this difficulty by applying the abstract results in the study by Szulkin & Weth [55].

Theorem 5.1 Let $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq).$ Assume that $(f_1)$–$(f_4)$ and $(V_1)$–$(V_2)$ are fulfilled, then for any δ > 0 satisfying $M_\delta \subset \Lambda,$ there exists $\hat{\varepsilon}_\delta \gt 0$ such that for any $\varepsilon \in (0,\hat{\varepsilon}_\delta)$ problem (2.2) possesses at least $cat_{M_\delta}(M)$ positive solutions.

Proof. For any fixed ɛ > 0, we introduce the mapping $\alpha_\varepsilon : M \mapsto S_\varepsilon^+$ defined by

\begin{align*}\alpha_\varepsilon(y) := m_\varepsilon^{- 1}(\Phi_\varepsilon(y)) \ \text{for all } y \in M.\end{align*}

Consequently, we deduce from lemma 4.1 that

(5.1)\begin{align} \lim_{\varepsilon \rightarrow 0} \psi_\varepsilon(\alpha_\varepsilon(y)) = \lim_{\varepsilon \rightarrow 0} J_\varepsilon(\Phi_\varepsilon(y)) = c_{V_0} \ \text{uniformly in } y \in M. \end{align}

Let us introduce the following function:

\begin{align*}h'(\varepsilon) : = \sup_{y \in M} |\psi_\varepsilon(\alpha_\varepsilon(y)) - c_{V_0}|.\end{align*}

According to (5.1), we see that $h'(\varepsilon) \rightarrow 0$ in $\mathbb{R}$ as $\varepsilon \rightarrow 0$. Also, we define the following set:

\begin{align*}\hat{S}_\varepsilon^+ := \left\{\omega \in S_\varepsilon^+ : \psi_\varepsilon(\omega) \leqslant c_{V_0} + h'(\varepsilon)\right\}.\end{align*}

Clearly, for all $y \in M$ and ɛ > 0, $\psi_\varepsilon(\alpha_\varepsilon(y)) \in \hat{S}_\varepsilon^+$, and so $\hat{S}_\varepsilon^+ \neq \emptyset$.

From lemma 4.1, lemma 2.2 (c), lemma 4.4, and lemma 4.2, it follows that there exists $\hat{\varepsilon} = \hat{\varepsilon}_\delta \gt 0$ such that the following diagram is well-defined:

\begin{align*}M \stackrel{\Phi_\varepsilon}{\longmapsto} \Phi_\varepsilon(M) \stackrel{m_\varepsilon^{- 1}}{\longmapsto} \alpha_\varepsilon(M) \stackrel{m_\varepsilon}{\longmapsto} \Phi_\varepsilon(M) \stackrel{\beta_\varepsilon}{\longmapsto} M_\delta \ \text{for any } \varepsilon \in (0, \hat{\varepsilon}).\end{align*}

Using lemma 4.2 and decreasing $\hat{\varepsilon}$ if necessary, for all $y \in M$, we have $\beta_\varepsilon(\Phi_\varepsilon(y)) = y + l(\varepsilon,y)$, where $|l(\varepsilon,y)| \leqslant \delta / 2$ uniformly in $y \in M$ and for all $\varepsilon \in (0,\hat{\varepsilon})$. Hence, $H(t,y) := y + (1- t) l(\varepsilon,y)$ for $(t,y) \in [0,1] \times M$ is homotopy between $\beta_\varepsilon \circ \Phi_\varepsilon = \left(\beta_\varepsilon \circ m_\varepsilon\right) \circ \left(m_\varepsilon^{- 1} \circ \Phi_\varepsilon\right)$ and the inclusion map $id : M \mapsto M_\delta$. This means that

(5.2)\begin{align} cat_{\alpha_\varepsilon(M)} \alpha_\varepsilon(M) \geqslant cat_{M_\delta}(M). \end{align}

Additionally, let us choose a function $h'(\varepsilon) \gt 0$ such that $h'(\varepsilon) \rightarrow 0$ in $\mathbb{R}$ as $\varepsilon \rightarrow 0$ and such that $c_{V_0} + h'(\varepsilon)$ is not a critical level for the functional J_ɛ. Using corollary 2.9 and theorem 27 in the study by Szulkin and Weth [55], for ɛ > 0 sufficiently small, we can deduce that ψ_ɛ possesses at least $cat_{\alpha_\varepsilon(M)} \alpha_\varepsilon(M)$ critical points on $\hat{S}_\varepsilon^+$. So, lemma 2.3 and (5.2) imply that the functional J_ɛ has at least $cat_{M_\delta}(M)$ critical points in $\hat{\mathcal{N}}_\varepsilon$. This proof is now complete.

6. Proof of theorem 1.2

The main idea is to show that the solutions obtained in theorem 5.1 verify the following estimate:

\begin{align*}\text{for } \varepsilon \gt 0 \ \text{sufficiently small, } u_\varepsilon(x) \leqslant a \ \text{for all } x \in \Lambda_\varepsilon^c.\end{align*}

Then, we can deduce from this fact that these solutions are indeed solutions of the original problem (1.3). To this end, we shall treat the regularity of non-negative solutions to problem (2.2). More precisely, we first establish the following result inspired by Moser [48] and Ambrosio and Rădulescu [11].

Lemma 6.1. Let $0 \lt \mu \lt sp$ and $\nu \lt (N - \mu) q / (N - sq)$, and let $\varepsilon_n \rightarrow 0$ in $\mathbb{R}$ as $n \rightarrow \infty$ and $u_n \in \hat{\mathcal{N}}_{\varepsilon_n}$ be a solution to problem (2.2). Then, $J_{\varepsilon_n}(u_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ as $n \rightarrow \infty,$ and there exists a sequence $\left\{\hat{y}_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ such that $\hat{u}_n(\cdot) := u_n(\cdot + \hat{y}_n) \in L^\infty(\mathbb{R}^N)$ and $|\hat{u}_n|_{L^\infty(\mathbb{R}^N)} \leqslant C$ for all $n \in \mathbb{N},$ for some constant C > 0. Furthermore,

(6.1)\begin{align} \hat{u}_n(x) \rightarrow 0 \ \text{as } |x| \rightarrow + \infty \ \text{uniformly in } n \in \mathbb{N}. \end{align}

Proof. On account of the fact that $u_n \in \hat{\mathcal{N}}_{\varepsilon_n}$, arguing as in the proof of lemma 4.4, we see that $J_{\varepsilon_n}(u_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ as $n \rightarrow \infty$. Then, from lemma 4.3, it follows that there exists a sequence $\left\{\hat{y}_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ such that $\hat{u}_n(\cdot) := u_n(\cdot + \hat{y}_n) \rightarrow \hat{u}(\cdot) \in {X_{V_0}}$ and $y_n :=\varepsilon_n \hat{y}_n \rightarrow y_0 \in M$ as $n \rightarrow \infty$. For any L > 0 and β > 1, we introduce the function

\begin{align*}\psi(\hat{u}_n) := \hat{u}_n \hat{u}_{n,L}^{q (\beta - 1)} \in X_{\varepsilon_n}, \ \text{where } \hat{u}_{n, L} := \min\left\{\hat{u}_n,L\right\}.\end{align*}

Taking $\psi(\hat{u}_n)$ as test function, we have

\begin{align*} & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|\hat{u}_n(x) - \hat{u}_n(y)|^{p - 2}(\hat{u}_n(x) - \hat{u}_n(y))(\psi(\hat{u}_n(x)) - \psi(\hat{u}_n(y)))}{|x - y|^{N + sp}} dxdy\\ + \ & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|\hat{u}_n(x) - \hat{u}_n(y)|^{q - 2}(\hat{u}_n(x) - \hat{u}_n(y))(\psi(\hat{u}_n(x)) - \psi(\hat{u}_n(y)))}{|x - y|^{N + sq}} dxdy\\ + \ & \int_{\mathbb{R}^N} V(\varepsilon_n x + y_n) |\hat{u}_n|^{p - 2} \hat{u}_n \psi(\hat{u}_n) dx + \int_{\mathbb{R}^N} V(\varepsilon_n x + y_n) |\hat{u}_n|^{q - 2} \hat{u}_n \psi(\hat{u}_n) dx\\ =\ & \int_{\mathbb{R}^N} \left(\frac{1}{|x|^\mu} * G(\varepsilon_n x + y_n, \hat{u}_n)\right) g(\varepsilon_n x + y_n,\hat{u}_n) \psi(\hat{u}_n) dx. \end{align*}

Additionally, applying the boundedness of the sequence $\left\{\hat{u}_n\right\}_{n \in \mathbb{N}} \subset X_{\varepsilon_n}$ and arguing as in the proof of lemma 2.6, we can deduce that there exists $C_0 \gt 0$ such that

\begin{align*}\sup_{n \in \mathbb{N}} \left|\frac{1}{|x|^\mu} * G(\varepsilon_n x + y_n, \hat{u}_n)\right|_{L^\infty(\mathbb{R}^N)} \leqslant C_0.\end{align*}

According to the hypotheses on g, we see that for any σ > 0 there exists $C_\sigma \gt 0$ such that

\begin{align*}|g(x,t)| \leqslant \sigma |t|^{p - 1} + C_\sigma |t|^{q_s^* - 1} \ \text{for all } (x,t) \in \mathbb{R}^N \times \mathbb{R}.\end{align*}

Choosing $\sigma \in (0,V_0/{C_0})$, together with the above inequalities, we can infer that

(6.2)\begin{align} & \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|\hat{u}_n(x) - \hat{u}_n(y)|^{p - 2}(\hat{u}_n(x) - \hat{u}_n(y))(\psi(\hat{u}_n(x)) - \psi(\hat{u}_n(y)))}{|x - y|^{N + sp}} dxdy \nonumber\\ & + \int_{\mathbb{R}^N}\int_{\mathbb{R}^N} \frac{|\hat{u}_n(x) - \hat{u}_n(y)|^{q - 2}(\hat{u}_n(x) - \hat{u}_n(y))(\psi(\hat{u}_n(x)) - \psi(\hat{u}_n(y)))}{|x - y|^{N + sq}} dxdy\nonumber\\ \leqslant \ & C \int_{\mathbb{R}^N} |\hat{u}_n|^{q_s^*}\hat{u}_{n,L}^{q (\beta - 1)} dx \end{align}

for some constant C > 0.

Let us define the following functions:

\begin{align*}\varphi(t) := \frac{|t|^q}{q} \quad \ \text{and } \quad\Upsilon(t) := \int^t_0 (\psi' (\tau))^{\frac{1}{q}} d\tau.\end{align*}

We first observe that ψ is an increasing function, and hence

(6.3)\begin{align} (a - b)(\psi(a) - \psi(b)) \geqslant 0 \quad\ \text{for all } a,\ b \in \mathbb{R}. \end{align}

Then, we can infer from (6.3) and the Jensen inequality that

(6.4)\begin{align} \varphi'(a - b)(\psi(a) - \psi(b)) \geqslant |\Upsilon(a) - \Upsilon(b)|^q \quad\ \text{for all } a,\ b \in \mathbb{R}. \end{align}

We also point out that

(6.5)\begin{align} \Upsilon(\hat{u}_n) \geqslant \frac{1}{\beta} \hat{u}_n \hat{u}_{n,L}^{\beta - 1}. \end{align}

So, putting together with (6.2), (6.3), (6.4), and (6.5) and using the Sobolev embedding, we infer that there exists some constant C > 0 such that

(6.6)\begin{align} |\hat{u}_n \hat{u}_{n,L}^{\beta - 1}|_{q_s^*}^q \leqslant C \beta^q \int_{\mathbb{R}^N} \hat{u}_n^{q_s^*}\hat{u}_{n,L}^{q(\beta - 1)} dx. \end{align}

Choose $\beta = \frac{q_s^*}{q}$, and let R > 0 large enough. Combining $\hat{u}_n \rightarrow \hat{u}$ in ${X_{V_0}}$ as $n \rightarrow \infty$ with the Hölder inequality, we can conclude that there exists some constant C > 0 such that

\begin{align*}\left(\int_{\mathbb{R}^N} \left(\hat{u}_n \hat{u}_{n,L}^{\frac{q_s^* - q}{q}}\right)^{q_s^*} dx\right)^{\frac{q}{q_s^*}} & \leqslant C \beta^q \int_{\mathbb{R}^N} R^{q_s^* - q} \hat{u}_n^{q_s^*} dx \\ & \quad + C \epsilon \left(\int_{\mathbb{R}^N} \left(\hat{u}_n \hat{u}_{n,L}^{\frac{q_s^* - q}{q}}\right)^{q_s^*} dx\right)^{\frac{q}{q_s^*}}.\end{align*}

Then, we choose a fixed $\epsilon \in (0,1 / C)$ and infer that

\begin{align*}\left(\int_{\mathbb{R}^N} \left(\hat{u}_n \hat{u}_{n,L}^{\frac{q_s^* - q}{q}}\right)^{q_s^*} dx\right)^{\frac{q}{q_s^*}} \leqslant C \beta^q \int R^{q_s^* - q} \hat{u}_n^{q_s^*} dx \lt + \infty.\end{align*}

In the above inequality, we pass to the limit as $L \rightarrow + \infty$ and we obtain $\hat{u}_n \in L^{\frac{{q_s^*}^2}{q}}(\mathbb{R}^N)$.

Thanks to $0 \leqslant \hat{u}_{n,L} \leqslant \hat{u}_n$, then in (6.6), we pass to the limit as $L \rightarrow + \infty$ and we have

\begin{align*}|\hat{u}_n|_{\beta q_s^*}^{\beta q} \leqslant C \beta^q \int_{\mathbb{R}^N} \hat{u}_n^{q_s^* + q(\beta - 1)} dx.\end{align*}

This means that

\begin{align*}\left(\int_{\mathbb{R}^N} \hat{u}_n^{\beta q_s^*} dx\right)^{\frac{1}{q_s^*(\beta - 1)}} \leqslant (C^{1 / q} \beta)^{\frac{1}{\beta - 1}} \left(\int_{\mathbb{R}^N} \hat{u}_n^{q_s^* + q(\beta - 1)} dx\right)^{\frac{1}{q(\beta - 1)}}.\end{align*}

For $1 \leqslant m \in \mathbb{N}$, let us define

\begin{align*}q_s^* + q(\beta_{m + 1} - 1) = \beta_m q_s^*\quad \ \text{and } \quad \beta_1 = \frac{q_s^*}{q}.\end{align*}

It follows that

\begin{align*}\beta_{m + 1} = \beta_1^m(\beta_1 - 1) + 1,\end{align*}

and so

\begin{align*}\lim_{m \rightarrow \infty} \beta_m = + \infty.\end{align*}

Let us define

\begin{align*}T_m : =\left(\int_{\mathbb{R}^N} \hat{u}_n^{\beta_m q_s^*} dx\right)^{\frac{1}{q_s^*(\beta_m - 1)}}.\end{align*}

Then, we have

\begin{align*}T_{m + 1} \leqslant (C^{1 / q} \beta_{m + 1})^{\frac{1}{\beta_{m + 1} - 1}} T_m.\end{align*}

Consequently, using a standard iteration argument, we have

\begin{align*}T_{m+1} \leqslant \prod_{k=1}^m(C^{1 / q} \beta_{k + 1})^{\frac{1}{\beta_{k + 1} - 1}} T_1 \leqslant \overline{C} T_1, \ \text{where } \overline{C} \ \text{is independent of } m.\end{align*}

In the above inequality, we pass to the limit as $m \rightarrow \infty$ and then we infer that $|\hat{u}_n|_{L^\infty(\mathbb{R}^N)} \leqslant C$ uniformly in $n \in \mathbb{N}$.

Next, let us define

\begin{align*}\kappa_n := - V(\varepsilon_n x + y_n)\left(\hat{u}_n^{p - 1} + \hat{u}_n^{q - 1}\right) + \left(\frac{1}{|x|^\mu} * G(\varepsilon_n x + y_n,\hat{u}_n)\right) g(\varepsilon_n x + y_n,\hat{u}_n).\end{align*}

We point out that $\hat{u}_n$ satisfies the following equation:

\begin{align*}(- \Delta)_p^s \hat{u}_n + (- \Delta)_q^s \hat{u}_n = \kappa_n \quad\ \text{in } \mathbb{R}^N.\end{align*}

From the growth hypotheses on g, corollary 2.1 in the study by Ambrosio & Rădulescu [11], $\hat{u}_n \rightarrow \hat{u}$ in ${X_{V_0}}$ as $n \rightarrow \infty$, and the uniformly boundedness of the sequence $\left\{\hat{u}_n\right\}_{n \in \mathbb{N}}$ in $L^\infty(\mathbb{R}^N) \cap {X_{V_0}}$, we can conclude that $\hat{u}_n(x) \rightarrow 0$ in $\mathbb{R}$ as $|x| \rightarrow +\infty$ uniformly with respect to $n \in \mathbb{N}$. This ends the proof of the lemma.

Proof of theorem 1.2 completed

We first choose δ > 0 small enough such that $M_\delta \subset \Lambda$. Then, we claim that there exists $\bar{\varepsilon}_\delta \gt 0$ such that for any $\varepsilon \in (0,\bar{\varepsilon}_\delta)$ and any solution $u_\varepsilon \in \hat{\mathcal{N}}_\varepsilon$ of problem (2.2), we have

(6.7)\begin{align} |u_\varepsilon|_{L^\infty(\Lambda_\varepsilon^c)} \lt a. \end{align}

Otherwise, we may assume that there exists a subsequence $\left\{\varepsilon_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}$ such that $\varepsilon_n \rightarrow 0$ as $n \rightarrow \infty$, $u_{\varepsilon_n} \in \hat{\mathcal{N}}_{\varepsilon_n}$ such that $J_{\varepsilon_n}'(u_{\varepsilon_n})=0$ and

(6.8)\begin{align} |u_{\varepsilon_n}|_{L^\infty(\Lambda_{\varepsilon_n}^c)} \geqslant a. \end{align}

But we see that $J_{\varepsilon_n}(u_{\varepsilon_n}) \rightarrow c_{V_0}$ in $\mathbb{R}$ as $n \rightarrow \infty$. Thus, from lemma 4.3, it follows that there exists a sequence $\left\{\hat{y}_n\right\}_{n \in \mathbb{N}} \subset \mathbb{R}^N$ such that $\hat{u}_n(\cdot) :=u_{\varepsilon_n}(\cdot + \hat{y}_n) \rightarrow \hat{u}(\cdot)$ in ${X_{V_0}}$ and $\varepsilon_n \hat{y}_n \rightarrow y_0 \in M$ as $n \rightarrow \infty$.

Next, we choose r > 0 such that $B_r(y_0) \subset B_{2r}(y_0) \subset \Lambda$, and so $B_{\frac{r}{\varepsilon_n}}\left(\frac{y_0}{\varepsilon_n}\right) \subset \Lambda_{\varepsilon_n}$. Furthermore, for n large enough, we can deduce that $\Lambda_{\varepsilon_n}^c \subset B_{\frac{r}{\varepsilon_n}}^c(\hat{y}_n)$. In addition, from (6.1), we see that $\hat{u}_n(x) \rightarrow 0$ as $|x| \rightarrow + \infty$ uniformly in $n \in \mathbb{N}$. Therefore, we can find R > 0 such that $\hat{u}_n(x) \lt a$ for any $|x| \geqslant R$, $n \in \mathbb{N}$. Consequently, $u_{\varepsilon_n}(x) \lt a$ for any $x \in B_R^c(\hat{y}_n)$, $n \in \mathbb{N}$. Moreover, for $n \in \mathbb{N}$ sufficiently large, we know that

\begin{align*}\Lambda_{\varepsilon_n}^c \subset B_{\frac{r}{\varepsilon_n}}^c(\hat{y}_n) \subset B_R^c(\hat{y}_n).\end{align*}

Thus, we infer that $u_{\varepsilon_n}(x) \lt a$ for any $x \in \Lambda_{\varepsilon_n}^c$ and for all $n \in \mathbb{N}$ large enough, which contradicts relation (6.8).

Fix $\varepsilon \in (0,\varepsilon_\delta)$, where $\varepsilon_\delta := \min\{\hat{\varepsilon}_\delta, \bar{\varepsilon}_\delta\}$. From theorem 5.1, we can see that problem (2.2) has at least $\text{cat}_{M_\delta}(M)$ nontrivial solutions. Now, we use u_ɛ to denote one of these solutions, and so $u_\varepsilon \in \hat{\mathcal{N}}_\varepsilon$. Then, using (6.7) and recalling that the definitions of g and G, we can also infer that u_ɛ is a solution of problem (1.3). So, problem (1.3) possesses at least $\text{cat}_{M_\delta}(M)$ nontrivial solutions.

Finally, we establish the behaviour of the maximum points of solutions to problem (1.3). Let us choose $\varepsilon_n \rightarrow 0$ and consider a sequence $\left\{u_n\right\}_{n \in \mathbb{N}} \subset X_{\varepsilon_n}$ of solutions for problem (1.3) as before. From $(g_1)$, it follows that there exists a positive constant $\iota \lt a$ such that

(6.9)\begin{align} g(\varepsilon x,t) t \leqslant \frac{V_0}{K}\left(t^p + t^q\right) \ \text{for any } x \in \mathbb{R}^N,\ t \in [0,\iota], \end{align}

since $u_n \in \mathcal{N}_{\varepsilon_n}$ and $J_{\varepsilon_n}(u_n) \rightarrow c_{V_0}$ in $\mathbb{R}$ as $n \rightarrow \infty$. Then, for $n \in \mathbb{N}$ sufficiently large, we have that $u_n \in \mathcal{B}$. Consequently, from lemma 2.6, it follows that

(6.10)\begin{align} \left|\frac{1}{|x|^\mu} * G(\varepsilon_n x, u_n)\right|_{L^\infty(\mathbb{R}^N)} \lt \frac{K}{2} \ \text{for } n \in \mathbb{N} \ \text{large enough}. \end{align}

Arguing as before, there exists R > 0 such that

(6.11)\begin{align} |u_n|_{L^\infty(B_R^c(\hat{y}_n))} \lt \iota. \end{align}

Furthermore, up to a subsequence, we may assume that

(6.12)\begin{align} |u_n|_{L^\infty(B_R(\hat{y}_n))} \geqslant \iota. \end{align}

Otherwise, if relation (6.12) does not hold, we deduce from (6.11) that $|u_n|_{L^\infty(\mathbb{R}^N)} \lt \iota$. Using $u_n \in \mathcal{N}_{\varepsilon_n}$ again and (6.9)–(6.10), we have

\begin{align*} \|u_n\|_{V_{\varepsilon_n},p}^p + \|u_n\|_{V_{\varepsilon_n},q}^q &\leqslant \int_{\mathbb{R}^N}\left(\frac{1}{|x|^\mu} * G(\varepsilon_n x,u_n)\right) g(\varepsilon_n x, u_n) u_n dx,\\ & \leqslant \frac{V_0}{2} \int_{\mathbb{R}^N} \left(|u_n|^p + |u_n|^q\right) dx\ \ \text{for all } n \in \mathbb{N} \ \text{large enough}. \end{align*}

This implies that $\|u_n\|_{X_{\varepsilon_n}} = 0$ for all $n \in \mathbb{N}$ sufficiently large, which is a contradiction. Consequently, relation (6.12) holds true. On account of (6.11) and (6.12), we can infer that if p_n is a global maximum point of u_n and $p_n = \hat{y}_n + q_n$ for some $q_n \in B_R$, $\varepsilon_n p_n \rightarrow y_0 \in M$ as $n \rightarrow \infty$; then using the continuity of the potential V we see that $V(\varepsilon_n p_n) \rightarrow V(y_0) = V_0$ in $\mathbb{R}$ as $n \rightarrow \infty$.

This proof is nowcomplete. $\square$