On a class of critical Schrödinger-Poisson systems
involving the \((p,q)\)-Laplacian
June 01, 2026
This paper investigates a class of Schrödinger-Poisson systems in \(\mathbb{R}^3\) featuring the \((p,q)\)-Laplacian operator and a combination of critical and subcritical nonlinearities in the Schrödinger equation while the \(m\)-Laplacian and a power type nonlinearity in the Poisson’s one. We consider both the attractive and repulsive cases, which correspond to different signs in front of the nonlocal term. While most existing literature relies on auxiliary functionals or specialized techniques to overcome the lack of compactness and ensure the boundedness of Palais-Smale sequences, we employ a direct variational approach. By applying the Mountain Pass Theorem and concentration compactness principles, we establish the existence of positive solutions. A careful analysis is conducted to identify the parameter ranges for which the Mountain Pass level falls within the compactness threshold, despite the technical challenges posed by the unbalanced growth of the operator and the nonlocal interaction.
In this paper we investigate existence results of quasilinear Schrödinger-Poisson systems in \(\mathbb{R}^3\) driven by the \((p,q)\)-Laplacian operator with a critical nonlinearity perturbed by a subcritical term, namely \[\label{maineq}_\pm} \begin{cases} -\Delta_pu-\Delta_qu+(|u|^{p -2}+|u|^{q -2})u\pm\lambda \phi |u|^{\vartheta-2}u=\beta {\mathcal{W}}|u|^{ \tau -2}u+{\mathcal{K}}|u|^{p^* -2}u \,\, &\text{in } \, \mathbb{R}^3\\ -\Delta_m\phi =|u|^{\vartheta} &\text{in }\, \mathbb{R}^3 \end{cases}\tag{1}\] where \(1<q\le p<3, \Delta_su=div(|\nabla u|^{s-2}\nabla u)\) is the \(s\)-Laplacian operator with \(s>1\), \(\,\lambda, \beta>0\), \(1<m<3\), \(p\le \tau < p^*\), with \(p^*=3p/(3-p)\) critical Sobolev exponent in dimension \(N=3\), and \[\label{defthetamain} \max\left\{1,\frac{q}{(m^*)'}\right\}<\vartheta<\frac{p^*}{(m^*)'}.\tag{2}\] While the nontrivial weights \({\mathcal{K}}={\mathcal{K}}(x),{\mathcal{W}}={\mathcal{W}}(x)\) satisfy \[\label{K} 0\le {\mathcal{K}}\in L^\infty(\mathbb{R}^3)\cap C(\mathbb{R}^3),\tag{3}\] \[\label{W} 0\le {\mathcal{W}}\in L^{\eta}(\mathbb{R}^3), \quad \eta=\frac{p^*}{p^*-\tau}\quad \text{and} \quad {\mathcal{W}}>0 \quad \text{in}\quad \Omega_{\mathcal{W}}\subset\mathbb{R}^3, \quad |\Omega_{\mathcal{W}}|>0.\tag{4}\]
As indicated by the choice of sign in 1 , we investigate two distinct classes of Schrödinger-Poisson systems. The specific sign adopted introduces unique analytical challenges in each case. Broadly speaking, when the nonlocal term appears with a negative sign, it acts in opposition to the norm induced by the operator on the left-hand side and is, therefore, treated as a nonlinearity. Conversely, when the nonlocal term is positive, it reinforces the norm while opposing the nonlinearity. Although both problems remain variational, the functional geometry is significantly altered by this change in sign, necessitating distinct methodological approaches that will be detailed below.
Semilinear Schrödinger-Poisson systems of the form \[\label{Lapla} \begin{cases} -\Delta u+u\pm \phi u=f(x,u) \,\, &\text{in} \,\, \mathbb{R}^3,\\ -\Delta\phi =u^2 &\text{in }\mathbb{R}^3, \end{cases}\tag{5}\] describe the interaction of charged particles with an electromagnetic or gravitational field generated by the particles themselves. Here, the unknown \(u\) arises from the standing wave ansatz \(\psi(x,t) = e^{-it} u(x)\) for the Schrödinger equation where \(\psi\) is the unknown, while \(\phi\) represents the potential.
If the particles move in their own gravitational field, where the field is generated by the particle probability density via the classical Newton field equation, we arrive to \((SP_-)\). Conversely, if the particles are electrically charged, the long-range electrostatic interaction can be effectively modeled by a potential term, we end up with \((SP_+)\), see [1].
In both cases, the term \(\phi u\) in the first equation is nonlocal since, from the second equation, \(\phi\) can be represented as the convolution of \(u^2\) with the fundamental solution of the Laplacian. Finally, the nonlinear term \(f(x,u)\) models the mutual interactions between particles. When \(f(x,u) \equiv 0\), system 5 reduces to the so called Schrödinger-Maxwell system.
Pioneering works concerning 5 with the negative sign include those by Benci and Fortunato [2], [3], who investigated an eigenvalue-type problem. Subsequently, the existence of positive radial solutions for 5 with superlinear and subcritical power-type nonlinearities was studied in [4], see also [5], [6] for investigations into the case \(\lambda \to 0\). Due to the variational structure of 5 , the aforementioned papers employ a variational approach, so that solutions are obtained as critical points of the associated energy functional. Regarding non-radial solutions, we highlight the work of Azzollini and Pomponio [7], in which the existence of a ground state solution is established via a concentration-compactness argument on suitable measures to restore compactness.
We specifically highlight the work of Vaira [1], who investigated the system \[\label{eqvar}\begin{cases} -\Delta u+u\pm\lambda K(x)\phi u=a(x)|u|^{k-2}u&\qquad \text{in}\,\,\mathbb{R}^3\\ -\Delta\phi=K(x) u^2&\qquad \text{in}\,\,\mathbb{R}^3 \end{cases}\tag{6}\] for both sign cases, assuming \(K\) and \(a\) are nonnegative real functions satisfying suitable conditions. The author proves the existence of positive ground states, i.e. solutions with minimal energy, across the entire range \(k \in (2, 6(=2^*))\) when the nonlocal term in 6 carries a negative sign. Conversely, when the nonlocal term is positive, the competition between this term and the nonlinearity leads to substantially different scenarios depending on \(k\). Consequently, the analysis in [1] is restricted to \(k \in (4,6)\). For further results, see also [8], where 6 is treated with the positive sign.
Owing to their robust physical foundations, systems analogous to 5 and their generalizations have been subjects of extensive investigation over the past several decades. Schrödinger-Poisson systems featuring more general nonlinearities have been addressed in numerous studies [4], [9]–[12]. Furthermore, the case of the critical (conformal) dimension has been explored starting with the seminal work [13] and followed by [14], [15], employing a diverse range of analytical approaches.
Motivated by the physical arguments presented above, this paper focuses on the three-dimensional case, although the mathematical framework developed herein can be extended to any dimension \(N \geq 2\) with suitable modifications. Specifically, we investigate generalizations of 5 and 6 involving operators with unbalanced growth, such as the \((p,q)\)-Laplacian, which arises in models of nonlinear elasticity [16] and in the study of solitary waves for elementary particles [17], [18]. Furthermore, the \((2,4)\)-Laplacian and its extensions appear as approximations of the Born–Infeld operator (see [19], [20]).
It is worth noting that the presence of a nonlinear operator in the second equation of 1 introduces significant analytical challenges, beginning with the Poisson equation itself. Indeed, in the standard Laplacian case, the Lax-Milgram Theorem provides a representation formula for \(\phi\). Conversely, when \(m \neq 2\), an explicit expression for \(\phi\) is generally unavailable. Nevertheless, the uniqueness of the nonnegative solution to the Poisson equation, as established in Proposition 3 below, is sufficient to recover the key properties listed in Proposition 4.
Regarding the nonlinearity considered in the first equation of 1 , it can be viewed as a critical term perturbed by a subcritical one, both involving nontrivial weights. Nonlinearities of this type are of significant physical interest appearing, for instance, in the Yamabe problem or in the search for extremal functions for isoperimetric inequalities, and were first introduced by Brézis and Nirenberg [21] for the Laplacian case. Subsequently, Guedda and Véron [22] extended several results from [21] to quasilinear equations driven by the \(p\)-Laplacian operator \[-\Delta_p u=\lambda u^{p-1}+u^{p^*-1}\quad \text{in} \,\, \Omega,\qquad u>0 \quad \text{in} \,\, \Omega, \qquad u=0 \quad \text{on} \,\, \partial\Omega,\] where \(\Omega\subset \mathbb{R}^N\) is a bounded open set whose boundary is a \(C^2\) submanifold of codimension \(1\). In the same work, the authors established a general strong comparison principle for the \(p\)-Laplacian, extending the results of [23], along with an extension of the Pohozaev identity [24]. On the other hand, the pure critical case in \(\mathbb{R}^N\) was considered in [25], where Guedda and Véron investigated positive radial solutions to \(-\Delta_p u = u^{k-1}\) for \(p < k \le p^*\), obtaining, among other things, that the best Sobolev constant for the embedding \(W^{1,p}_0\hookrightarrow L^{p^*}\) is achieved in \(\mathbb{R}^N\), see [22] and also providing a complete classification of isolated singularities in the subcritical case.
Moreover, in [26], the same authors characterized the structure of the set of solutions for the eigenvalue problem with a potential in the one-dimensional case. For Brézis-Nirenberg type problems involving the \(p\)-Laplacian in the entire Euclidean space, we refer to [27], [28], while the non-homogeneous case of the \((p,q)\)-Laplacian is addressed in [29], [30].
In contrast, the study of Schrödinger-Poisson systems with critical nonlinearities appears to be a much more recent field of investigation. Key contributions include [31], which addresses the case \(p=q=2\) in \((\mathcal{P}_+)\), [32] for the case of fractional operators, and [33] for the \(p\)-Laplacian operator. We also refer to [34] where multiplicity and concentration of normalized solutions of a double critical Schrödinger–Poisson system involving the fractional \(p\)-Laplacian is studied. More recently, the \((p,q)\)-Laplacian was investigated in [35] looking for nodal solutions. Regarding \((\mathcal{P}_-)\), the nonlocal critical case was explored in [36], while the zero-mass case for the \(p\)-Laplacian was studied in [37]. Furthermore, for the \((p,q)\)-Laplacian, we mention [38], where a fixed-point theory approach was employed to establish the existence of a positive solution to 1 (with \(m=2, \vartheta=q\)), highlighting the crucial role of the perturbation term in their argument.
A central difficulty in the variational study of classical Schrödinger-Poisson systems is ensuring the boundedness of Palais-Smale \((PS)\) sequences for the energy functional. Consequently, the majority of literature relies on specialized variational techniques that circumvent the need for a priori boundedness of all \((PS)\) sequences. These methods, introduced by Jeanjean [39] and widely adopted in various contexts [40], [41], typically involve constructing a specific bounded \((PS)\) sequence that satisfies an additional condition related to an auxiliary functional. Alternatively, Jeanjean and Le Coz [42] developed another method based on an auxiliary truncated energy functional, which has been used in several settings [43], [44].
One of the primary objectives of the present paper is to provide a comprehensive study of 1 for both sign cases. Unlike previous works described above, we employ a classical mountain pass approach. Although technically demanding, this allows us to identify the specific intervals where the standard mountain pass geometry is preserved and the boundedness, together with compactness, of the \((PS)\) sequences can be directly established.
While the Schrödinger-Poisson system is receiving increasing interest, the difficulties arising from the nonlinear nature of both equations in 1 mean that, to the best of our knowledge, this paper represents the first attempt to treat this type of problem variationally using a technique that differs from the standard ones. In this regard, we mention [38], where \(\vartheta=q\) and \(m=2\) in \((\mathcal{P}_-)\) with the addition of a perturbation term, and [35], where nodal solutions to \((\mathcal{P}_+)\) were established for \(\vartheta=q\) and \(m=2\). Furthermore, in [45], quasilinear operators appear in both equations in \((\mathcal{P}_+)\), but the authors consider a subcritical nonlinearity. Notably, we investigate both signs of the nonlocal term, a feature that, outside of the Laplacian case [1], remains largely unexplored. Beyond the mathematical generality of the \((p,q)\)-operator, our analysis is specifically designed to include the \(q=2, p=4\) case, establishing a direct link to the physical foundations of Born-Infeld theory.
Our main result is the following
Theorem 1. Assume 3 and 4 . Then, there exists \(\, \beta^*>0\) such that
problem \(({\mathcal{P}}_+)\) admits a nontrivial weak solution with positive energy for all \(\beta>\beta^*\) and \(\lambda>0\), provided that \[\label{cond43} \vartheta m'\le\tau<p^*\quadand\quad \max\left\{1,\frac{q}{(m^*)'},\frac{p}{m'}\right\}<\vartheta<\frac{p^*}{m'};\qquad{(1)}\] or \[\label{cond4343} p<\tau<p^*\quadand\quad\max\left\{1,\dfrac{q}{(m^*)'}\right\}<\vartheta<\min\left\{\dfrac{p^*}{(m^*)'},\dfrac{p}{m'}\right\};\qquad{(2)}\]
problem \(({\mathcal{P}}_-)\) admits a nontrivial weak solution with positive energy for all \(\beta>\beta^*\) and \(\lambda>0\) provided that \[\label{cond-} p<\tau<\min\{\vartheta m',p^*\}\quadand\quad\max\left\{1,\dfrac{q}{(m^*)'},\dfrac{p}{m'}\right\}<\vartheta<\dfrac{p^*}{(m^*)'}.\qquad{(3)}\]
The proof of Theorem 1 employs variational methods, primarily utilizing the Mountain Pass Theorem. We begin by establishing the appropriate functional framework and observing that 1 can be reduced to a single equation involving a nonlocal term. This reduction is typically achieved through representation theorems, as seen in the semilinear case where the Laplacian operator is involved, see [1]. See also the papers by Mitidieri et al. [46]–[48] where the authors consider semilinear equations, inequalities or systems associated to general classes of differential operators, such as the Kohn laplacian on the Heisenberg group, differential inequalities on Carnot groups or polyharmonic systems.
The presence of the quasilinear \(m\)-Laplacian in 1 complicates the process. Specifically, it precludes an explicit expression for \(\phi_u\) for a fixed \(u\), unlike the standard Laplacian case. Nevertheless, we report existence, uniqueness, and other essential properties that allow the argument to proceed.
As a second step, the boundedness of any Palais-Smale sequences at every energy level \(c\) is established. This represents the first significant challenge, as the behavior of the nonlocal term varies depending on the range of the parameter \(\vartheta\). Consequently, several cases must be analyzed separately, while also accounting for different ranges of \(\tau\). Next, we verify that the energy functional satisfies the Mountain Pass geometry under more relaxed assumptions on the parameters. Given the intrinsic lack of compactness in 1 due to the presence of the critical Sobolev’s exponent and the ambient space \(\mathbb{R}^3\), we must restore it via the Palais-Smale condition by employing concentration-compactness principles: the analysis of concentration at points follows Lions [49], while concentration at infinity is addressed by Ben-Naoum et al. [50]. The final and most delicate step in ensuring the existence of a critical point is to demonstrate that the Mountain Pass level lies within the range where the compactness condition is satisfied. It should be noted that, while the general conditions on \(\tau\) and \(\vartheta\) are stated in Theorem 1, each preliminary lemma provides more refined requirements. In some cases, these conditions prove to be less restrictive than those in ?? , ?? , and ?? .
Although this strategy follows a classical variational framework, its implementation introduces numerous technical assumptions on the parameters \(\tau\) and \(\vartheta\) of the subcritical and nonlocal terms, respectively.
The paper is organized as follows. Section 2 provides an overview of standard concepts and results, such as properties for the Poisson equation, concentration compactness principles and the Mountain Pass Theorem. In Section 3 we introduce the functional framework giving the foundation to implement the variational approach. Finally, Section 4, starting from the properties of the energy functional and its behavior, is devoted to the rigorous proofs of the main result of the paper, Theorem 1.
In this section, we recall the main definitions and properties of Sobolev spaces together with classical results. Then, we give a brief outline of the regularity properties of the functionals associated with our main problems.
We begin by reviewing some notation. Since this paper focuses on the three-dimensional case, we shall maintain this assumption throughout the section, although results such as Sobolev embeddings or concentration compactness principles hold in higher dimensions.
We indicate with \(B_r(x)\) the \(\mathbb{R}^3\)-ball of center \(x\in\mathbb{R}^3\) and radius \(r>0\), omitting \(x\) when it is the origin. Let \((X, \|\cdot\|)\) be a Banach space, we indicate with \(X'\) its dual, while \(\langle \cdot, \cdot \rangle\) stands for the duality brackets for the pair \((X,X')\). Given two Banach spaces \(X,Y\), the continuous embedding of \(X\) into \(Y\) is indicated by \(X \hookrightarrow Y\); if the embedding is compact, we write \(X \hookrightarrow \hookrightarrow Y\).
A sequence \((u_n)_n\in X\) strongly converges to \(u\) when \(u_n \to u\) in \(X\). If the convergence is in weak sense then \(u_n \rightharpoonup u\) in \(X\).
Given any measurable set \(\Omega \subset \mathbb{R}^3\) and \(r \in [1,+\infty]\), \(L^r(\Omega)\) stands for the standard Lebesgue space, whose norm will be indicated with \(\| \cdot \|_{L^r(\Omega)}\) or simply \(\| \cdot \|_r\) when \(\Omega=\mathbb{R}^3\). Moreover, we denote \(r':=r/(r-1)\) as the conjugated index of \(r\).
We will also make use of the Sobolev space \(W^{1,r}(\mathbb{R}^3)\), which is the closure of smooth functions with compact support, i.e. \(C^{\infty}_c(\mathbb{R}^3)\), with respect to the norm \[\|u\|_{1,r}=\|u\|_r+\|\nabla u\|_r.\]
The continuous embeddings of Sobolev spaces are clearly described in Sobolev-Gagliardo-Nirenberg’s Theorem stated below.
Theorem 1 ([51]). Let \(1 \le p <3\). Then \[W^{1,p}(\mathbb{R}^3) \hookrightarrow L^{p^*}(\mathbb{R}^3),\] where \(p^*=\frac{3p}{3-p}\) is the critical Sobolev’s exponent. In particular, \[W^{1,p}(\mathbb{R}^3) \hookrightarrow L^{s}(\mathbb{R}^3), \,\, \forall s \in [p,p^*].\]
Since we are working with the \((p,q)\)-Laplacian operator in 1 , then we will define \((X, \|\cdot\|)\) as \[X=W^{1,p}(\mathbb{R}^3)\cap W^{1,q}(\mathbb{R}^3),\qquad\qquad \|u\|:=\|u\|_{1,p}+\|u\|_{1,q}.\] From Theorem 1, we have the following embedding \[\label{embX} X\hookrightarrow L^s(\mathbb{R}^3), \,\, \forall s \in [q,p^*].\tag{7}\]
In some cases we will need to use a larger space, so called homogeneous Sobolev space or Beppo Levi space, given by \(D^{1,m}(\mathbb{R}^3)\) where \(1<m<3\), defined as the closure of \(C^\infty_c(\mathbb{R}^3)\) with respect to the norm \[\|u\|_{D^{1,m}(\mathbb{R}^3)}:=\|\nabla u\|_m.\] If \(1<m<3\), then \(D^{1,m}(\mathbb{R}^3)\) is reflexive, separable Banach space and we indicate with \(D^{-1,m'}(\mathbb{R}^3)\) the dual of \(D^{1,m}(\mathbb{R}^3)\). The continuous embedding of the \(D^{1,m}(\mathbb{R}^3)\) is known as Sobolev’s theorem, here reported for completeness.
Theorem 2 ([52]). Let \(1<m<3\). Then \[D^{1,m}(\mathbb{R}^3) \hookrightarrow L^{m^*}(\mathbb{R}^3),\] and the best constant in the Sobolev inequality will be indicated with \(S^{-1/m}\), where \[\label{S} S= \inf_{u\in D^{1,m}(\mathbb{R}^3,\mathbb{R})\setminus\{0\}} \frac{\|\nabla u\|_{1,m}^m}{\|u\|_{m^*}^m}\qquad{(4)}\]
According to which, one has \[D^{1,m}(\mathbb{R}^3) = \left\{u\in L^{m^*}(\mathbb{R}^3): \, |\nabla u|\in L^m(\mathbb{R}^3)\right\}.\]
Now we will focus our attention on the Poisson equation in 1 , starting with an existence and uniqueness result and ending with properties of such solution, following [12].
Proposition 3. Let \(1<q<p<3\), \(u\in X\) and \(\vartheta\) satisfies 2 . Then there is a unique nonnegative solution \(\phi_u\in D^{1,m}(\mathbb{R}^3)\) to \[\label{probphi} -\Delta_m \phi_=|u|^\vartheta\quad \text{in}\,\, \mathbb{R}^3.\qquad{(5)}\]
We do not report its proof, as the one of the Proposition below, since they come straightforwardly from Propositions 2.1 and 2.2 in [12] by using 7 .
It is hard to give an explicit expression of \(\phi_u\). We can prove the following properties of \(\phi_u\) via the uniqueness of the solution of ?? .
Proposition 4. For \(u \in X\) and \(\vartheta\) satisfy 2 , the solution \(\phi_u\) of ?? given by Proposition 3 has the following properties.
It holds \[\int_{\mathbb{R}^3} \left( \frac{1}{m} |\nabla \phi_u|^m - |u|^\vartheta \phi_u \right) dx = \min_{\phi \in D^{1,m}(\mathbb{R}^3)} \int_{\mathbb{R}^3} \left( \frac{1}{m} |\nabla \phi|^m - |u|^\vartheta \phi \right) dx, \quad \phi_u \geq 0.\]
For \(t > 0\), \(\phi_{tu} = t^{\frac{\vartheta}{m-1}} \phi_u\) and \(\phi_{u_t}(x) = t^{\frac{k\vartheta - m}{m}} \phi_u(tx)\), where \(u_t(x) = t^k u(tx)\). Moreover \(\phi_{u(\cdot + y)} = \phi_u(\cdot + y)\) for any \(y \in \mathbb{R}^3\).
It holds \[\|\phi_u\|_{D^{1,m}} \leq C \|u\|^{\frac{\vartheta}{m-1}}, \quad \text{where C does not depend on } u.\]
If \(u_n \to u\) in \(X\), then \(\phi_{u_n} \to \phi_u\) in \(D^{1,m}(\mathbb{R}^3)\) and \[\int_{\mathbb{R}^3} \phi_{u_n} |u_n|^{\vartheta-2} u_n \varphi dx \to \int_{\mathbb{R}^3} \phi_u |u|^{\vartheta-2} u \varphi dx, \quad \forall \varphi \in X.\]
If \(u_n \to u\) in \(X\), then \(\phi_{u_n} \to \phi_u\) in \(D^{1,m}(\mathbb{R}^3)\).
To address the lack of compactness proper of problems with a critical nonlinearity in the entire Euclidean space \(\mathbb{R}^3\), we introduce the following two lemmas. These tools are specifically designed to handle concentration of compactness at points and at infinity, respectively.
Lemma 5 ([53]). Let \(1\leq p<3\). Suppose \((u_n) \subseteq D^{1,p}(\mathbb{R}^3)\) to be such that \(u_n \rightharpoonup u\) in \(D^{1,p}(\mathbb{R}^3)\), and both \(|\nabla u_n|^p \rightharpoonup \mu\), \(|u_n|^{p^*} \stackrel{*}{\rightharpoonup} \nu\) in the sense of measures, for some \(u\in D^{1,p}(\mathbb{R}^3)\) and \(\mu,\nu\) bounded non-negative measures on \(\mathbb{R}^3\). Then there exist some at most countable set \(J\), a family of distinct points \((x_j)_{j\in J}\subseteq \mathbb{R}^3\), and two families of numbers \((\nu_j)_{j\in J}, (\mu_j)_{j\in J}\subseteq (0,+\infty)\) fulfilling \[\label{ineqmeasures} \nu=|u|^{p^*}+\sum_{j\in J} \nu_j \delta_{x_j},\qquad \mu\geq |\nabla u|^p+\sum_{j\in J} \mu_j \delta_{x_j},\qquad S\nu_j^{p/p^*} \leq \mu_j \quad for all \;\; j\in J,\qquad{(6)}\] where \(S\) is the best constant in the Sobolev inequality defined in ?? with \(m=p\).
Note that Lemma 5 requires the tight convergence of the measures involving the critical Sobolev exponent, but the proof of this condition reveals to be rather difficult and technical. Thus, Ben-Naoum et al. established a version of the Lemma 5 known as escape to infinity principle, where the concentration at infinity is enclosed in the parameters \(\nu_\infty\) and \(\mu_\infty\).
Lemma 6 ([50]). Let \(1\leq p<3\). Suppose that \((u_n) \subseteq D^{1,p}(\mathbb{R}^3)\) is bounded and define \[\label{munuinf} \nu_\infty := \lim_{R\to+\infty} \limsup_{n\to\infty} \int_{B_R^c} |u_n|^{p^*} dx, \quad \mu_\infty := \lim_{R\to+\infty} \limsup_{n\to\infty} \int_{B_R^c} |\nabla u_n|^p dx.\qquad{(7)}\] Then, it holds \(S\nu_\infty^{p/p^*} \leq \mu_\infty\) and \[\label{nunuinfty} \limsup_{n\to\infty} \int_{\mathbb{R}^3} |u_n|^{p^*} dx = \int_{\mathbb{R}^3} \, {\rm d}\nu + \nu_\infty, \qquad \limsup_{n\to\infty} \int_{\mathbb{R}^3} |\nabla u_n|^p dx = \int_{\mathbb{R}^3} \, {\rm d}\mu + \mu_\infty,\qquad{(8)}\] where \(\nu,\mu\) are as in Lemma 5.
We will use Lemmas 5–6 to avoid concentration both at points, i.e. \(\nu_j=\mu_j=0\) for all \(j\in J\), and at infinity, i.e. \(\nu_\infty=\mu_\infty=0\).
We end the present section by introducing the following version of the Mountain Pass Theorem.
Theorem 2 ([54]). Let \((V,\| .\|_V)\) be a Banach space and consider \(F\in C^1(V)\). We assume that
(i) \(F(0)=0\),
(ii) There exist \(\alpha,R>0\) such that \(F(u)\geq\alpha\) for all \(u\in V\), with \(\|u\|_V=R\),
(iii) There exists \(v_0\in V\) such that \(\limsup_{t\to \infty}F(tv_0)<0\).
Let \(t_0>0\) be such that \(\|t_0v_0\|_V>R\) and \(F(t_0v_0)<0\) and let \[c:=\inf_{\gamma\in\Gamma}\,\sup_{t\in [0,1]}F(\gamma(t)),\] where \[\Gamma:=\{\gamma\in C^0([0,1],V) : \gamma(0)=0 and \gamma(1)=t_0v_0\}.\] Then, there exists a Palais Smale sequence at level \(c\), that is a sequence \((u_n)_n\subset V\) such that \[\lim_{n\to \infty}F(u_n)=c\quad and \quad\lim_{n\to \infty}F'(u_n)= 0\quadstrongly in V'.\]
Now we establish the variational framework of 1 .
By Proposition 3, for any \(u\in X\), there is an unique nonnegative solution \(\phi_u\in D^{1,m}(\mathbb{R}^3)\) to ?? . Thus the map \(\Phi:X \to D^{1,m}(\mathbb{R}^3)\) defined as \(\Phi(u)=\phi_u,\) is well-defined from the above argument. Now we define the functional \({\mathcal{F}}: X \times D^{1,m}(\mathbb{R}^3) \to \mathbb{R}\) as \[\begin{align} {\mathcal{F}}(u, \phi):= \frac{1}{p}\|u\|_{1,p}^p+\frac{1}{q}\|u\|_{1,q}^q&\pm \frac{\lambda}{\vartheta}{\int_{\mathbb{R}^3}} \phi |u|^{\vartheta}dx\mp\frac{\lambda}{m \vartheta}{\int_{\mathbb{R}^3}} |\nabla \phi|^{m}dx\\ &-\frac{\beta}{\tau}{\int_{\mathbb{R}^3}} {\mathcal{W}}(x)|u|^{\tau}dx-\frac{1}{p^*}{\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u|^{p^*}dx, \end{align}\] for any \(v \in X\) and \(\phi \in D^{1,m}(\mathbb{R}^3)\). By standard arguments, the functional \({\mathcal{F}}\) is of class \(C^1\) and its critical points are weak solutions of 1 . In particular, its partial derivatives are \[\begin{align} \partial_u{\mathcal{F}}(u,\phi)[v]=&{\int_{\mathbb{R}^3}} |\nabla u|^{p-2}\nabla u \nabla v dx+ {\int_{\mathbb{R}^3}} |u|^{p-2}uv dx+{\int_{\mathbb{R}^3}} |\nabla u|^{q-2}\nabla u \nabla v dx+ {\int_{\mathbb{R}^3}} |u|^{q-2}uv dx \\ &\pm \lambda{\int_{\mathbb{R}^3}} \phi |u|^{\vartheta-2}uv dx- \beta{\int_{\mathbb{R}^3}} {\mathcal{W}}(x) |u|^{\tau -2}u v dx- {\int_{\mathbb{R}^3}} {\mathcal{K}}(x) |u|^{p^* -2}u v dx, \end{align}\] \[\partial_{\phi}{\mathcal{F}}(u,\phi)[\xi]= \pm\frac{\lambda}{\vartheta}{\int_{\mathbb{R}^3}} |u|^{\vartheta} \xi dx\mp\frac{\lambda}{\vartheta}{\int_{\mathbb{R}^3}} |\nabla\phi|^{m-2}\nabla\phi \nabla \xi dx.\] Let us define \[{\mathcal{J}}(u)={\mathcal{F}}(u,\Phi(u)),\] then since \(\phi_u\) solves ?? , then \({\mathcal{J}}\) takes the form \[\begin{align} {\mathcal{J}}(u)=\frac{1}{p}\|u\|_{1,p}^p&+\frac{1}{q}\|u\|_{1,q}^q\pm \frac{\lambda}{ \vartheta m'}{\int_{\mathbb{R}^3}} \phi_u |u|^{\vartheta}dx\\&-\frac{\beta}{\tau}{\int_{\mathbb{R}^3}} {\mathcal{W}}(x)|u|^{\tau}dx-\frac{1}{p^*}{\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u|^{p^*}dx. \end{align}\]
Although our objective is to study problem 1 through a unified approach, it will become necessary at certain stages to distinguish between the two cases. For the sake of clarity, we define the functionals corresponding to the plus and minus signs as \(J_+\) and \(J_-\), respectively.
\[\label{defJ43}\begin{align} {\mathcal{J}}_+(u)=\frac{1}{p}\|u\|_{1,p}^p&+\frac{1}{q}\|u\|_{1,q}^q+ \frac{\lambda}{ \vartheta m'}{\int_{\mathbb{R}^3}} \phi_u |u|^{\vartheta}dx\\&-\frac{\beta}{\tau}{\int_{\mathbb{R}^3}} {\mathcal{W}}(x)|u|^{\tau}dx-\frac{1}{p^*}{\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u|^{p^*}dx, \end{align}\tag{8}\] \[\label{defJ-}\begin{align} {\mathcal{J}}_-(u)=\frac{1}{p}\|u\|_{1,p}^p&+\frac{1}{q}\|u\|_{1,q}^q- \frac{\lambda}{ \vartheta m'}{\int_{\mathbb{R}^3}} \phi_u |u|^{\vartheta}dx\\&-\frac{\beta}{\tau}{\int_{\mathbb{R}^3}} {\mathcal{W}}(x)|u|^{\tau}dx-\frac{1}{p^*}{\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u|^{p^*}dx. \end{align}\tag{9}\]
Clearly, \({\mathcal{J}}\in C^1(X, \mathbb{R})\) and for any \(u \in X\), \[{\mathcal{J}}'(u)=\partial_u({\mathcal{F}}(u,\Phi(u))+\partial_{\phi}({\mathcal{F}}(u,\Phi(u))\Phi'(u).\] However, recalling that \[\partial_{\phi}{\mathcal{F}}(u,\Phi(u))=0,\] we can write \[\begin{align} {\mathcal{J}}'(u)[v]= &{\int_{\mathbb{R}^3}} |\nabla u|^{p-2}\nabla u \nabla v dx+ {\int_{\mathbb{R}^3}} |u|^{p-2}uv dx+{\int_{\mathbb{R}^3}} |\nabla u|^{q-2}\nabla u \nabla v dx+ {\int_{\mathbb{R}^3}} |u|^{q-2}uv dx \\ &\pm \lambda{\int_{\mathbb{R}^3}} \phi_u |u|^{\vartheta -2}uv dx - \beta{\int_{\mathbb{R}^3}} {\mathcal{W}}(x) |u|^{\tau-2}u v dx- {\int_{\mathbb{R}^3}} {\mathcal{K}}(x) |u|^{p^*-2}u v dx, \end{align}\] for any \(v \in X\), see Proposition 2.3 in [12]. Therefore to look for weak solutions of 1 , it suffices to look for critical points of the functional \({\mathcal{J}}\).
With the variational framework now established, we can delve into the core of the proof of Theorem 1. We begin with taking Palais Smale sequences associated to the functional \({\mathcal{J}}\) and first proving their boundedness. However, due to the presence of the nonlocal term, this is possible only in some cases.
Lemma 7. Assume 2 , 3 , 4 and \(p< \tau <p^*\).
Let \((u_n)_n\) be a \((PS)_c\) sequence for \({\mathcal{J}}_+\) in \(X\), with \(c\in \mathbb{R}\) under the assumptions \[\label{bound:PS:J43} \max\{p,\vartheta m'\}<\tau<p^*\qquad \max\left\{1,\frac{q}{(m^*)'}\right\}<\vartheta<\frac{p^*}{m'}\qquad{(9)}\] or \[\label{bound:PS:J432} \tau=\vartheta m' \qquad \max\left\{1,\frac{q}{(m^*)'},\frac{p}{m'}\right\}<\vartheta<\frac{p^*}{m'}\qquad{(10)}\] Then, \((u_n)_n\) is bounded in \(X\).
Let \((u_n)_n\) be a \((PS)_c\) sequence for \({\mathcal{J}}_-\) in \(X\), with \(c\in \mathbb{R}\) under the assumptions ?? or ?? . Then, \((u_n)_n\) is bounded in \(X\).
Take any \((u_n)_n\) \((PS)_c\) sequence for \({\mathcal{J}}\) in \(X\), with \(c\in \mathbb{R}\). By definition \[{\mathcal{J}}(u_n)\to \, c \quadand\quad {\mathcal{J}}'(u_n)\to 0 \,\,inX',\] as \(n\to\infty\). Thus, \[\label{PSbound}\begin{align} c+o(1)&={\mathcal{J}}(u_n)-\frac{1}{\tau}{\mathcal{J}}'(u_n)u_n\\&=\left(\frac{1}{p}-\frac{1}{\tau}\right)\|u_n\|_{1,p}^p+\left(\frac{1}{q}-\frac{1}{\tau}\right)\|u_n\|_{1,q}^q\pm \frac{\lambda}{ \vartheta}\left(\frac{1}{m'}-\frac{\vartheta}{\tau}\right){\int_{\mathbb{R}^3}} \phi_{u_n} |u_n|^{\vartheta}dx\\&\qquad-\left(\frac{1}{p^*}-\frac{1}{\tau}\right){\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u_n|^{p^*}dx\\&\ge \left(\frac{1}{p}-\frac{1}{\tau}\right)(\|u_n\|_{1,p}^p+\|u_n\|_{1,q}^q)\pm \frac{\lambda}{ \vartheta}\left(\frac{1}{m'}-\frac{\vartheta}{\tau}\right){\int_{\mathbb{R}^3}} \phi_{u_n} |u_n|^{\vartheta}dx, \end{align}\tag{10}\] where we have used \(\tau<p^*\), \(q\le p\) and 3 .
Both for \({\mathcal{J}}_+\) or \({\mathcal{J}}_-\), by employing the relative assumptions ?? ?? and ?? ?? , we obtain the nonnegativity of the nonlocal term.
So that 10 gives \[\begin{align}c+o(1)\ge&\left(\frac{1}{p}-\frac{1}{\tau}\right)(\|u_n\|_{1,p}^p+\|u_n\|_{1,q}^q). \end{align}\] Letting \(\|u_n\|=\|u_n\|_{1,p}+\|u_n\|_{1,q}\to\infty\), the conclusion follows easily by contradiction.
Remark 8. Note that the assumptions on \(\tau\) and \(\vartheta\) in Lemma 7 are necessary to ensure the positivity of the nonlocal term’s coefficients for both \({\mathcal{J}}_+\) and \({\mathcal{J}}_-\). This allows the term to be bounded from below by zero. Note that adopting a negative coefficient would make it impossible to compare the operator’s norm with the \(L^{p^*}\) norm of the critical term. This difficulty persists even when using a weighting factor different from \(1/\tau\) and estimating the nonlocal term by 11 .
Next, we verify the Mountain Pass geometry and study the Palais-Smale sequence given by Theorem 2, recovering the compactness necessary to ensure the existence of a solution.
Lemma 9. Assume 4 , 3 , 2 , \(p\le \tau <p^*\). Then, properties
there exists \(\rho>0\) and \(\delta>0\) so that \({\mathcal{J}}(u)\ge \delta\) for any \(u \in X\) with \(\|u\|=\rho\),
there exists \(v \in X\) such that \(\|v\|>\rho\) and \({\mathcal{J}}(v)<0\)
hold for \({\mathcal{J}}_+\) if \[\max\left\{1,\frac{q}{(m^*)'}\right\}<\vartheta<\frac{p^*}{m'}.\] While they hold for \({\mathcal{J}}_-\) if \[\label{theta95new} \max\left\{1,\frac{q}{(m^*)'}, \frac{p}{m'}\right\}<\vartheta<\frac{p^*}{(m^*)'}.\qquad{(11)}\]
(i) We divide the proof in two cases:
Using Hölder’s inequality we infer that
\[\begin{align}{\mathcal{J}}_+(u) \ge \frac{1}{p}(\|u\|_{1,p}^p+\|u\|_{1,q}^q)-\frac{\beta}{\tau}\|{\mathcal{W}}\|_{\eta}\|u\|_{p^*}^\tau-\frac{1}{p^*}\|{\mathcal{K}}\|_{\infty}\|u\|_{p^*}^{p^*} \end{align}\]
Now, taking \(\|u\|\le 1\), then \(\|u\|_{1,p},\|u\|_{1,q}\le 1\) so that, being \(q\le p\) we have \(\|u\|_{1,q}^q\ge \|u\|_{1,q}^p\) and by the standard inequality \((a+b)^r\le \max\{1,2^{r-1}\}(a^r+b^r)\) for all \(a,b,r>0\), also by Theorem 1 we reach \[\begin{align}{\mathcal{J}}_+(u)&\gtrsim \|u\|^p-\frac{\beta}{\tau}\|{\mathcal{W}}\|_{\eta}\|u\|^\tau-\frac{1}{p^*}\|{\mathcal{K}}\|_{\infty}\|u\|^{p^*} \\ &= \|u\|^p\biggl(1-\frac{\beta}{\tau}\|{\mathcal{W}}\|_{\eta}\|u\|^{\tau-p}\frac{1}{p^*}\|{\mathcal{K}}\|_{\infty}\|u\|^{p^*-p}\bigr). \end{align}\]
First note that, by using Holder’s inequality, Proposition 4, Theorem 2, 7 by 2 , we get \[\label{disphi}\begin{align} {\int_{\mathbb{R}^3}}& \phi_{u_n} |u_n|^{\vartheta} dx\le \biggl({\int_{\mathbb{R}^3}} |\phi_{u_n}|^{m^*} dx \biggr)^{\frac{1}{m^*}}\biggl ( {\int_{\mathbb{R}^3}} |u_n|^{(m^*)'\vartheta}dx\biggr) ^{\frac{1}{(m^*)'}} \\ &\le C \|\phi_{u_n}\|_{D^{1,m}} \|u_n\|^\vartheta \le C \|u_n\|^{\frac{\vartheta}{m-1}} \|u_n\|^\vartheta = C\|u_n\|^{\vartheta m'}. \end{align}\tag{11}\] Thus, applying Theorem 1 and 11 , we infer that \[{\mathcal{J}}_-(u)\gtrsim \frac{1}{p}(\|u\|_{1,p}^p+\|u\|_{1,q}^q)- \frac{\lambda}{ \vartheta}\frac{1}{m'}\|u\|^{\vartheta m'}-\frac{\beta}{\tau}\|{\mathcal{W}}\|_{\eta}\|u\|^\tau-\frac{1}{p^*}\|{\mathcal{K}}\|_{\infty}\|u\|^{p^*}\]
Now, taking \(\|u\|\le 1\), as above, then \(\|u\|_{1,q}^q\ge \|u\|_{1,q}^p\) and by the standard inequality \((a+b)^r\le \max\{1,2^{r-1}\}(a^r+b^r)\) for all \(a,b,r>0\), we reach \[\begin{align}{\mathcal{J}}_-(u)&\gtrsim \|u\|^p- \frac{\lambda}{ \vartheta m'}\|u\|^{\vartheta m'}-\frac{\beta}{\tau}\|{\mathcal{W}}\|_{\eta}\|u\|^\tau-\frac{1}{p^*}\|{\mathcal{K}}\|_{\infty}\|u\|^{p^*} \\ &= \|u\|^p\biggl(1- \frac{\lambda}{ \vartheta m'}\|u\|^{\vartheta m'-p}-\frac{\beta}{\tau}\|{\mathcal{W}}\|_{\eta}\|u\|^{\tau-p}\frac{1}{p^*}\|{\mathcal{K}}\|_{\infty}\|u\|^{p^*-p}\bigr). \end{align}\]
Since \(\tau>p\) and \(\vartheta >p/m'\) in ?? , then in both cases above there exists a sufficiently small norm \(\|u\|:=\rho (<1)\) and \(\delta >0\) such that \({\mathcal{J}}(u)\geq \delta\) for every \(u\) satisfying \(\|u\|=\rho\).
(ii) For any \(u\in X\setminus \{0\}\), by Proposition 4, we get \[{\mathcal{J}}(tu)=\frac{t^p}{p}\|u\|_{1,p}^p+\frac{t^q}{q}\|u\|_{1,q}^q \pm t^{m'\vartheta} \frac{\lambda}{m' \vartheta }{\int_{\mathbb{R}^3}} \phi_{u} |u|^{\vartheta} dx -\frac{t^{\tau}}{\tau} {\int_{\mathbb{R}^3}} {\mathcal{W}}|u|^{\tau} dx-\frac{t^{p^*}}{p^*}{\int_{\mathbb{R}^3}} {\mathcal{K}}|u|^{p^*}.\]
Now, recalling that \({\mathcal{K}}\) is nontrivial and satisfy 3 , we have two different situations:
Since \(q<p\le\tau<p^*\) and assuming \(\vartheta<p^*/m'\), then \({\mathcal{J}}_+(tu)\to-\infty\) as \(t\to\infty\).
Since \(q<p\le\tau<p^*\), then \({\mathcal{J}}_-(tu)\to-\infty\) as \(t\to\infty\).
Thus, in both cases, there exists \(t_u >0\) large such that \({\mathcal{J}}(t_u u)<0\). Consequently, (ii) holds with \(v=t_u u\).
Consider for all \(u\in X\setminus\{0\}\) \[\Gamma_u:=\{\gamma\in C^0([0,1],X)\,:\, \gamma(0)=0 and \gamma(1)=v\},\] where \(v\) is given in (ii)-Lemma 9 with \({\mathcal{J}}(v)<0\) and \(\|v\|>\rho\). Then for all \(\gamma\in \Gamma_u\), it holds \[\|\gamma(0)\|=0,\qquad \|\gamma(1)\|>\rho,\qquad \gamma \text{ continuous}.\] Thus, then exists \(\bar t\in (0,1)\) such that \(\|\gamma(\bar t)\|=\rho\) for all \(\gamma\in\Gamma_u\) implying \[\max_{t\in[0,1]}{{\mathcal{J}}(\gamma(t))}\geq \delta >0,\] by (i)-Lemma 9. Thus, \[c_u:=\inf_{\gamma\in\Gamma_u}\,\sup_{t\in [0,1]}{\mathcal{J}}(\gamma(t))>0.\] Then the hypotheses of Theorem 2 are satisfied, yielding the existence of a Palais Smale sequence at level \(c_u\). In order to avoid ambiguity, when necessary, we consider \[\label{cu43-}\begin{align} &c_u^+:=\inf_{\gamma\in\Gamma_u}\,\sup_{t\in [0,1]}{\mathcal{J}}_+(\gamma(t)),\qquad c_u^-:=\inf_{\gamma\in\Gamma_u}\,\sup_{t\in [0,1]}{\mathcal{J}}_-(\gamma(t)) \end{align}\tag{12}\]
In what follows, we establish the compactness properties for the functional \({\mathcal{J}}\). This analysis requires a case-by-case approach based on the sign of the nonlocal term. Specifically, we treat \({\mathcal{J}}_+\) and \({\mathcal{J}}_-\) as defined in 8 and 9 .
Lemma 10. Assume 4 , 3 , 2 and \(p< \tau <p^*\). Define \[\label{Csegnato} c_{PS}:=\frac{S^{3/p}}{3\|{\mathcal{K}}\|_\infty^{3/p^*}}\qquad{(12)}\] \[\label{Csegnato39} c_{PS}':=\left(\frac{1}{\vartheta m'}-\frac{1}{p^*}\right)\frac{S^{3/p}}{3\|{\mathcal{K}}\|_\infty^{3/p^*}}\quad \text{with}\quad \vartheta m'<p^*\qquad{(13)}\] and note that \(c_{PS}'<c_{PS}\).
Then, \({\mathcal{J}}_+\) satisfies the \((PS)_c\) condition for every
\(c<c_{PS}'\) if ?? holds
\(c<c_{PS}\) if ?? holds
and \({\mathcal{J}}_-\) satisfies the \((PS)_c\) condition for every
Remark 11. Note that while for \({\mathcal{J}}_-\) the conditions in Lemma 10 and Lemma 10 coincide, conditions ?? and ?? are stronger than ?? and ?? . Consequently, Lemma 7 applies, ensuring the boundedness of \((PS)\) sequences.
Let \((u_n)_n\) be a \((PS)_c\) sequence in \(X\), so that, by Lemma 7, then \((u_n)_n\) is bounded in \(X\), indeed assumptions on \(\tau,\theta\) above are stronger than ?? , ?? , ?? . Since \(X\) is a reflexive Banach space, then, by Banach-Alaoglu’s Theorem, there exists \(u\in X\) such that, up to subsequences, we get \(u_{n}\rightharpoonup u\) in \(X\). Since \(\nabla u_{n}\rightharpoonup \nabla u\) in \(L^{p}(\mathbb{R}^3)\cap L^{q}(\mathbb{R}^3)\), the sequence of measures \((|\nabla u_n|^p dx+|\nabla u_n|^q dx)_n\) is bounded and \(|\nabla u_{n}|^{p}dx+|\nabla u_n|^q dx\rightharpoonup \mu\). Analogously, \(|u_n|^{p^*}dx\rightharpoonup \nu,\) where \(\mu, \nu\) are bounded nonnegative measures on \(\mathbb{R}^3\).
By Proposition 5 and Lemma 6, there exist at most countable set \(J\), a family \((x_j)_{j\in J}\) of distinct points in \(\mathbb{R}^3\) and two families \((\nu_j)_{j\in J}, \,(\mu_j)_{j\in J}\in ]0,\infty[\) such that ?? , ?? hold, with \(\nu_\infty\), \(\mu_\infty\) defined in ?? , satisfying \[\label{64622} S\nu_{j}^{p/p^{*}}\le\mu_{j}, \qquad S\nu_{\infty}^{p/p^*}\le \mu_\infty.\tag{13}\]
Take a standard cut-off function \(\psi\in C_{c}^{\infty}(\mathbb{R}^3)\), such that \(0\le\psi\le1\) in \(\mathbb{R}^3\), \(\psi=0\) for \(|x|>1\), \(\psi=1\) for \(|x|\le 1/2\). For each index \(j\in J\) and each \(0<\varepsilon<1\), define \[\psi_{\varepsilon}(x)=\psi\left(\frac{x-x_{j}}{\varepsilon}\right).\]
Since \({\mathcal{J}}'(u_{n})\psi\to0\) being \((u_{n})_n\) a \((PS)_c\) sequence, we have, as \(n\to\infty\), \[\begin{align} {\int_{\mathbb{R}^3}} &|\nabla u_n|^{p-2}\nabla u_n \nabla \psi dx+ {\int_{\mathbb{R}^3}} |u_n|^{p-2}u_n\psi dx+{\int_{\mathbb{R}^3}} |\nabla u_n|^{q-2}\nabla u_n \nabla \psi dx+ {\int_{\mathbb{R}^3}} |u_n|^{q-2}u_n\psi dx \\ &\pm \lambda{\int_{\mathbb{R}^3}} \phi_{u_n} |u_n|^{\vartheta -2}u_n\psi dx - \beta{\int_{\mathbb{R}^3}} {\mathcal{W}}(x) |u_n|^{\tau-2}u_n \psi dx- {\int_{\mathbb{R}^3}} {\mathcal{K}}(x) |u_n|^{p^*-2}u_n \psi dx=o(1). \end{align}\] Now, choosing \(\psi=\psi_\varepsilon u_n\), which is still bounded, we obtain \[\label{dis1}\begin{align} {\int_{\mathbb{R}^3}} &\bigl(|\nabla u_n|^p+|\nabla u_n|^q\bigr)\psi_\epsilon dx+ {\int_{\mathbb{R}^3}} u_n\bigl(|\nabla u_n|^{p-2}+ |\nabla u_n|^{q-2}\bigr)\nabla u_n \nabla \psi_\epsilon dx\\ &+ {\int_{\mathbb{R}^3}} (|u_n|^p+|u_n|^q)\psi_\epsilon dx\pm \lambda{\int_{\mathbb{R}^3}} \phi_{u_n} |u_n|^\vartheta \psi_\epsilon dx \\ &- \beta{\int_{\mathbb{R}^3}} {\mathcal{W}}(x) |u_n|^\tau\psi_\epsilon dx- {\int_{\mathbb{R}^3}} {\mathcal{K}}(x) |u_n|^{p^*} \psi_\epsilon dx=o(1). \end{align}\tag{14}\] Applying Hölder inequality \[\label{6464} \begin{align} \biggl|\int_{\mathbb{R}^3}&{u_n}|\nabla u_n|^{p-2}\nabla u_n\cdot\nabla \psi_\varepsilon dx\biggr|\le \| u_n\|^{p-1} \biggl(\int_{B_{\varepsilon}(x_j)}|u_n|^p|\nabla \psi_\varepsilon|^pdx\biggr)^{1/p}. \end{align}\tag{15}\] Furthermore, \(u_n\to u\) in \(L^{p}_\mathrm{loc}(\mathbb{R}^3)\), by compactness, yielding, up to subsequences \[\label{aeconv} u_n(x)\to u(x)\qquad \text{a.e. \,\, in}\quad \omega=\overline{B}_{\varepsilon}(x_{j})\tag{16}\] and there exists \(g\in L^p(\omega, \mathbb{R})\) such that \(|u_{n}(x)|\le g(x)\) a.e.in \(\omega\). Thus, \(|u_n(x)| |\nabla\psi_{\varepsilon}(x)|\le C g(x)\) a.e.in \(\omega\), as well as in \(\mathbb{R}^3\) being \(\psi\in C_{c}^{\infty}(\mathbb{R}^3)\), and in turn, Lebesgue Dominated Convergence Theorem gives \[\label{conv95grad} |u_n\nabla\psi_{\varepsilon}|\to|u\nabla\psi_{\varepsilon}| \;\textrm{in} \;L^{p}(\mathbb{R}^3).\tag{17}\] Consequently, by 17 and Hölder’s inequality with exponents \(3/(3-p)\) and \(3/p\), we obtain \[\begin{align} \lim_{n\to\infty}&\int_{B_{\varepsilon}(x_{j})}|u_n|^{p}|\nabla \psi_\varepsilon|^p dx\\ &\le \left(\int_{B_{\varepsilon}(x_{j})}|\nabla\psi_{\varepsilon}|^{3}dx\right)^{p/3} \left(\int_{B_{\varepsilon}(x_{j})}|u|^{p^*}dx\right)^{p/p^*}\le C\left(\int_{B_{\varepsilon}(x_{j})}|u|^{p^{*}}dx \right)^{p/p^{*}}, \end{align}\] where we used that \(|\nabla \psi_\varepsilon|\le C \varepsilon^{-1}\) and \(|B(x_i,\varepsilon)| \le C'\varepsilon^3\). In turn, using the boundedness of \((u_n)_n\in X\) by Lemma 7, then 15 gives \[\limsup_{n\to\infty}\biggl|\int_{\mathbb{R}^3}{u_n}|\nabla u_n|^{p-2}\nabla u_n\cdot\nabla \psi_\varepsilon dx\biggr|\le C \left(\int_{B_{\varepsilon}(x_{j})}|u|^{p^{*}}dx \right)^{1/p^{*}}.\] Arguing in the same way when \(p=q\) and letting \(\varepsilon\to 0\) from \(u\in L^{p^*}(\mathbb{R}^3)\cap L^{q^*}(\mathbb{R}^3)\), we get \[\lim_{\varepsilon\to 0}\limsup_{n\to\infty}\biggl|\int_{\mathbb{R}^3}u_n(|\nabla u_n|^{p-2}+|\nabla u_n|^{q-2})\nabla u_n\nabla \psi_\varepsilon dx\biggr|=0.\] Moreover, by 16 \[\biggl|{\int_{\mathbb{R}^3}} (|u_n|^p+|u_n|^q)\psi_\epsilon dx\biggr|\le \int_{B_\varepsilon(x_j)}(|u_n|^p+|u_n|^q) dx=o(1),\] \[\biggl|{\int_{\mathbb{R}^3}} {\mathcal{W}}(x) |u_n|^\tau\psi_\epsilon dx\biggr|=o(1),\] as \(n\to\infty\) and \(\varepsilon\to0\) since \(u_n\to u\) in \(L^{s}_\mathrm{loc}(\mathbb{R}^3)\) for all \(1<s<p^*\), \(u\in X\) and by using the Lebesgue Dominated Convergence Theorem.
Furthermore, by 11 \[\label{stimaphiu} \biggl|{\int_{\mathbb{R}^3}} \phi_u |u_n|^\vartheta \psi_\epsilon dx\biggr|\le C\|u_n\|_{W^{1,p}(B_\varepsilon(x_j))}^{\vartheta m'}=o(1)\tag{18}\] as \(n\to\infty\) and \(\varepsilon\to0\) by using the Lebesgue Dominated Convergence Theorem.
Then, from 14 and the inequalities above, we can conclude for \(n\) large \[\label{nabla95le95K95ustar}\int_{\mathbb{R}^3} (|\nabla u_n|^p +|\nabla u_n|^q)\psi_\varepsilon dx \le\int_{\mathbb{R}^3}{\mathcal{K}}(x)|u_n|^{p^*}\psi_\varepsilon dx+o(1),\tag{19}\] yelding for \(\varepsilon\to0\) \[\label{64623} \mu_{j}\le {\mathcal{K}}(x_{j})\nu_{j},\tag{20}\] since \({\mathcal{K}}\in C(\mathbb{R}^3)\) and by ?? . Consequently, either \(\nu_j=0\) and then also \(\mu_j=0\), or \(\nu_j>0\). We claim that the latter case cannot occur for each \(j\in J\), with \(J\) given in Lemma 5. First observe that combining 20 and 13 , we have \[\label{sxj} S\leq {\mathcal{K}}(x_j)\nu_j^{p/3},\tag{21}\] that is concentration of the measure \(\nu\) can occur only at those points \(x_j\) where \({\mathcal{K}}(x_j)>0\). Consequently, from 13 and 20 the measure \(\mu\) can concentrate at points in which the measure \(\nu\) can. In turn, zeros for \({\mathcal{K}}\) cannot belong to \(X_J=\{x_j: j\in J\}\).
Let \(J_1=\{j\in J: {\mathcal{K}}(x_j)>0\}\), we claim that \[\label{J1empty}J_1=\emptyset.\tag{22}\] If not, for any \(j\in J_1\), then 21 implies \[\label{J2} \nu_{j}\ge \left( \frac{S}{{\mathcal{K}}(x_{j})}\right)^{3/p}\ge \left( \frac{S}{\|{\mathcal{K}}\|_\infty}\right)^{3/p},\tag{23}\] which in particular gives that \(|J_1|<\infty\) being \(\nu\) a bounded measure; indeed, from ?? and 23 , we get \[\infty>\int_{\mathbb{R}^3}d\nu=\|u\|_{p^*}^{p^*}+\int_{\mathbb{R}^3}\sum_{j\in J_1}\nu_j\delta_{x_j}dx+\nu_\infty \ge \|u\|_{p^*}^{p^*}+\left(\frac{S}{\|K\|_\infty}\right)^{3/p}|J_1|+\nu_\infty.\] Now, we show that 23 cannot occur. Now we divide the proof into cases
Case 1. If \(\vartheta m'\le\tau<p^*\,\) and \(\, \max\{1,q/(m^*)',p/m'\}<\vartheta<p^*/m'\), then \[\label{epn243-}\begin{align}c+o(1)&={\mathcal{J}}(u_n)-\frac{1}{\vartheta m'}{\mathcal{J}}'(u_n)u_n\\&=\left(\frac{1}{p}-\frac{1}{\vartheta m'}\right)\|u_n\|_{1,q}^p+\left(\frac{1}{q}-\frac{1}{\vartheta m'}\right)\|u_n\|_{1,q}^q\\&\qquad-\beta \left(\frac{1}{\tau}-\frac{1}{\vartheta m'}\right){\int_{\mathbb{R}^3}} {\mathcal{W}}|u_n|^\tau dx+\left(\frac{1}{\vartheta m'}-\frac{1}{p^*}\right){\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u_n|^{p^*}dx \\&\ge \left(\frac{1}{\vartheta m'}-\frac{1}{p^*}\right){\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u_n|^{p^*}dx\ge \left(\frac{1}{\vartheta m'}-\frac{1}{p^*}\right)\int_{B_{\varepsilon}(x_{j})} {\mathcal{K}}(x)|u_n|^{p^*}dx, \end{align}\tag{24}\] for any \(\varepsilon>0\). In particular, 24 holds both for \({\mathcal{J}}_+\) and for \({\mathcal{J}}_-\). Consequently, inserting 23 in 24 and letting \(n\to\infty\) and \(\varepsilon\to0\), both for \({\mathcal{J}}_+\) and for \({\mathcal{J}}_-\), we obtain \[c\ge \left(\frac{1}{\vartheta m'}-\frac{1}{p^*}\right) \nu_j {\mathcal{K}}(x_j)\ge \left(\frac{1}{\vartheta m'}-\frac{1}{p^*}\right)\frac{S^{3/p}}{\|{\mathcal{K}}\|_\infty^{3/p^*}}(=c_{PS}'),\] yielding a contradiction, so that the claim 22 is proved. However, this case cannot occur for \({\mathcal{J}}_-\) since Lemma 7 does not hold if \(\vartheta m'\le\tau<p^*\), cfr ?? \(_1\).
Case 2. If \(p\le\tau<p^*\), then \[\begin{align}c+o(1)&={\mathcal{J}}(u_n)-\frac{1}{p}{\mathcal{J}}'(u_n)u_n\\&=\left(\frac{1}{q}-\frac{1}{p}\right)\|u_n\|_{1,q}^q\pm \frac{\lambda}{ \vartheta}\left(\frac{1}{m'}-\frac{\vartheta}{p}\right){\int_{\mathbb{R}^3}} \phi_{u_n} |u_n|^{\vartheta}dx\\&\qquad+\beta\left(\frac{1}{p}-\frac{1}{\tau}\right){\int_{\mathbb{R}^3}} {\mathcal{W}}(x)|u_n|^{\tau}dx+\frac{1}{3}{\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u_n|^{p^*}dx \\&\ge \frac{1}{3}{\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u_n|^{p^*}dx\pm \frac{\lambda}{ \vartheta}\left(\frac{1}{m'}-\frac{\vartheta}{p}\right){\int_{\mathbb{R}^3}} \phi_{u_n} |u_n|^{\vartheta}dx, \end{align}\] so that, if \(\,\max\{1,q/(m^*)'\}<\vartheta<\min\{p^*/(m^*)',p/m'\}\), we have \[c +o(1)={\mathcal{J}}_+(u_n)-\frac{1}{p}{\mathcal{J}}_+'(u_n)u_n\ge \frac{1}{3}{\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u_n|^{p^*}dx\] or, if \(\max\{1,q/(m^*)',p/m'\}<\vartheta<p^*/(m^*)'\), we have \[c +o(1)={\mathcal{J}}_-(u_n)-\frac{1}{p}{\mathcal{J}}_-'(u_n)u_n\ge \frac{1}{3}{\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u_n|^{p^*}dx.\] In turn, arguing as in Case 1, both for \({\mathcal{J}}_+\) and for \({\mathcal{J}}_-\), thanks to 23 it holds \[c\ge \frac{1}{3} \nu_j {\mathcal{K}}(x_j)\ge \frac{S^{3/p}}{3\|{\mathcal{K}}\|_\infty^{3/p^*}}(=c_{PS}),\] yielding a contradiction, so that also in Case 2 the claim 22 is proved
We have so obtained that in both cases concentration cannot occur at finite points.
It remains to show that the concentration of \(\nu\) cannot occur at infinity, namely \(\nu_\infty=0\). We use the same idea employed to prove 20 , but with the following cutoff function \(\psi_{R}\in C^{\infty}(\mathbb{R}^3)\) such that \(0\le\psi_{R}\le1\) in \(\mathbb{R}^{3}\), \(\psi_R(x)=0\) for \(|x|<R\) and \(\psi_{R}(x)=1\) for \(|x|>2R\). In this way, we obtain 19 with \(\psi_\varepsilon\) replaced by \(\psi_R\), that is \[\int_{B_R^c} (|\nabla u_n|^p +|\nabla u_n|^q)\psi_R dx \le\int_{B_R^c}{\mathcal{K}}(x)|u_n|^{p^*}\psi_R dx+o(1)\] yielding \(\mu_\infty\le \|{\mathcal{K}}\|_\infty\nu_\infty\) since \[\lim_{R\to\infty}\limsup_{n\to\infty}\left\{\int_{B_R^c}{\mathcal{K}}|u_{n}|^{p^{*}}\psi_{R}dx\right\} \le\|{\mathcal{K}}\|_{\infty}\nu_{\infty},\] by ?? , which gives \(\mu_\infty\le \|{\mathcal{K}}\|_\infty\nu_\infty\). Moreover, recalling 13 we arrive at \(S\le \nu_{\infty}^{p/3}\|{\mathcal{K}}\|_\infty\). Arguing as in Case 1 and Case 2 above with \(B_\varepsilon(x_j)\) replaced by \(B_R^c\), we get again a contradiction.
In turn, \(\nu_\infty=\mu_\infty=0\) is in force. Consequently, recalling also that \(\nu_i=\mu_i=0\), by ?? we end up with \[\lim_{n\to\infty}\int_{\mathbb{R}^3}|u_{n}|^{p^{*}}dx=\int_{\mathbb{R}^3}|u|^{p^{*}}dx,\] that is \(\|u_n\|_{p^*}\to\|u\|_{p^*}\) as \(n\to\infty\), which combined with \(u_n(x)\to u(x)\) a.e. in \(\mathbb{R}^3\), the latter obtained by 16 and an exhaustion process applied to a.e. convergence on compact sets in \(\mathbb{R}^3\), implies \(\|u_n-u\|_{p^*}\to 0\) by Brezis Lieb Lemma in [55]. It remains to prove \[\label{claimfin} \|u_n-u\|\to0, \quad \text{as}\,\,\,n\to\infty,\tag{25}\] which is equivalent to prove \[\int_{\mathbb{R}^3}|\nabla (u_n-u)|^p+|u_n-u|^pdx, \,\, \int_{\mathbb{R}^3}|\nabla (u_n-u)|^q + |u_n-u|^q\to 0 \quadas\quad n\to\infty.\] To this aim, since \((u_n)_n\) is a \((PS)_c\) sequence, we have \[\label{g-11}\begin{align} o(1)&=\langle {\mathcal{J}}'(u_n)-{\mathcal{J}}'(u),u_n-u\rangle= \int_{\mathbb{R}^3} \bigl(|\nabla u_n|^{p-2}\nabla u_n-|\nabla u|^{p-2}\nabla u)(\nabla (u_n-u)\bigr)dx\\ &+\int_{\mathbb{R}^3} \bigl(|\nabla u_n|^{q-2}\nabla u_n-|\nabla u|^{q-2}\nabla u)(\nabla (u_n-u)\bigr)dx\\ &+\int_{\mathbb{R}^3}(|u_n|^{p-2}u_n-|u|^{p-2}u)(u_n-u)dx+\int_{\mathbb{R}^3} (|u_n|^{q-2}u_n-|u|^{q-2}u)(u_n-u)dx \\ &\pm \lambda\int_{\mathbb{R}^3} \phi_u(|u_n|^{\vartheta-2}u_n-|u|^{\vartheta-2}u)(u_n-u)dx \\&-\beta\int_{\mathbb{R}^3}{\mathcal{W}}(x)(|u_n|^{\tau-2}u_n-|u|^{\tau-2}u)(u_n-u)dx \\&-\int_{\mathbb{R}^3}{\mathcal{K}}(x)(|u_n|^{p^*-2}u_n-|u|^{p^*-2}u)(u_n-u)dx. \end{align}\tag{26}\] By Lemma 7, using Hölder’s and Schwarz’s inequality, the \(L^{p^*}\) function estimates of \((u_n)_n\) and the convergence of \((u_n)_n\) in \(L^{p^*}(\mathbb{R}^3)\) we get
\[\begin{align}\\ &\le \|{\mathcal{K}}\|_\infty \int_{\mathbb{R}^3} \left(|u_n|^{p^*-1}+|u|^{p^*-1}\right)|u_n-u|dx\\ & \le \|{\mathcal{K}}\|_\infty \left(\|u_n\|_{p^*}^{p^*-1}+\|u\|_{p^*}^{p^*-1}\right)\|u_n-u\|_{p^*}=o(1). \end{align}\] Similarly, we have \[\begin{align} &\left|\int_{\mathbb{R}^3}{\mathcal{W}}(x)(|u_n|^{\tau-2}u_n-|u|^{\tau-2}u)(u_n-u)dx\right|=o(1), \end{align}\] and arguing as in 18 \[\left|\int_{\mathbb{R}^3} \phi_u(|u_n|^{\vartheta-2}u_n-|u|^{\vartheta-2}u)(u_n-u)dx\right|=o(1).\] Thus, 26 reduces to \[\begin{align} o(1)&=\langle {\mathcal{J}}'(u_n)-{\mathcal{J}}'(u),u_n-u\rangle\\ &=\int_{\mathbb{R}^3} \bigl(|\nabla u_n|^{p-2}\nabla u_n-|\nabla u|^{p-2}\nabla u)(\nabla (u_n-u)\bigr)dx\\ &+\int_{\mathbb{R}^3} \bigl(|\nabla u_n|^{q-2}\nabla u_n-|\nabla u|^{q-2}\nabla u)(\nabla (u_n-u)\bigr)dx\\ &+\int_{\mathbb{R}^3}(|u_n|^{p-2}u_n-|u|^{p-2}u)(u_n-u)dx+\int_{\mathbb{R}^3} (|u_n|^{q-2}u_n-|u|^{q-2}u)(u_n-u)dx, \end{align}\] so that by Simon’s inequality, see [56], valid for all \(s>1\) and \(a,b\in\mathbb{R}^3\) \[|a\!-\!b|^{s}\!\lesssim\!\!\begin{cases}(|a|^{s-2}a- |b|^{s-2}b)(a-b)&\phantom{1<\,}s\!\geq \!2;\\ \!\left(|a|^{s-2}a- |b|^{s-2}b)(a-b\!\right)\!^{\frac{s}{2}}\!\left(|a|^s+|b|^s\right)^{\frac{2-s}{2}} &1\!<\!s\!<\!2,\end{cases}\] as in pag 713 in [57], then 25 is in force concluding the proof of the lemma.
The final step consists in showing that the mountain pass levels \(c_u^{\pm}\), defined in 12 , remain below the thresholds \(c_{PS}\) and \(c_{PS}'\) from ?? and ?? . Below these levels, the functional \({\mathcal{J}}\) satisfies the Palais-Smale condition, as established in the previous lemma, thereby restoring compactness.
To this aim, from now on we denote, for each \(\lambda, \beta>0\), \[\label{clambda43-}\begin{align} &\hat{c}_+ := \inf_{u\in X\setminus\{0\}} \max_{t\ge 0} {\mathcal{J}}_+(tu),\qquad \hat{c}_- := \inf_{u\in X\setminus\{0\}} \max_{t\ge 0} {\mathcal{J}}_-(tu). \end{align}\tag{27}\] Note that \(\hat{c}_{\pm}\geq c_u^{\pm}\) since \({\mathcal{J}}_\pm(tu)<0\) for \(u\in X\setminus\{0\}\) and \(t\) large by the structure of \({\mathcal{J}}_\pm\), as observed in Theorem 4.2 in [58].
Lemma 12. Assume 3 , 4 , 2 and \(p\le \tau <p^*\). Let \(c_{PS}\) and \(\hat{c}_{\pm}\) defined in ?? and 27 , respectively. Then, there exists \(\lambda^*, \beta^*>0\) such that
\(0< \hat{c}_+<c_{PS}\) for all \(\beta>\beta^*\) and \(\lambda>0\) provided that \[\max\left\{1,\frac{q}{(m^*)'}\right\}<\vartheta<\frac{p^*}{m'}.\]
\(0< \hat{c}_-<c_{PS}\)
for all \(\beta>\beta^*\) and \(\lambda>0\) provided that \[p \le\tau< \min\{\vartheta m', p^*\}\quad\text{and}\quad \max\left\{1,\dfrac{q}{(m^*)'},\dfrac{p}{m'}\right\}<\vartheta<\dfrac{p^*}{(m^*)'}\]
for all \(\lambda>\lambda^*\) and \(\beta>0\) provided that \[\max\{\vartheta m',p\}\le \tau<p^*\quad\text{and}\quad \max\left\{1,\dfrac{q}{(m^*)'}\right\}<\vartheta<\dfrac{p^*}{m'}.\]
Remark 13. Before proceeding with the proof, we present the following observations:
We point out that the case \(\tau=\vartheta m'\) is handled both in \((a)\) and \((b)\), yielding the validity of \(0< \hat{c}_-<c_{PS}\) either for \(\beta\) large or for \(\lambda\) large.
As it will be clear from the proof below, since the upper bound for \(\hat{c}_+\) is obtained by a limit procedure, then Lemma 12 continues to be valid with \(c_{PS}\) replaced by \(c_{PS}'\).
In view of ?? , case (b) is excluded due to the incompatibility of the ranges for \(\tau\). Consequently, existence in Theorem 1 is established only for sufficiently large values of \(\beta\).
Take the open set \(\Omega_{\mathcal{W}}\) where \({\mathcal{W}}\) is positive by 4 . Let \(u_0\in X\setminus \{0\}\) with \(|supp(u_0)\cap\Omega_{{\mathcal{W}}}|>0\) such that \(u_0\ge0\) and \(\|\nabla u_0\|_q>0\). Take any \(t\ge 0\) \[\begin{align} {\mathcal{J}}(tu_0)=\frac{t^p}{p}\|u_0\|_{1,p}^p&+\frac{t^q}{q}\|u_0\|_{1,q}^q\pm \frac{\lambda}{ \vartheta m'}t^{\vartheta m'}{\int_{\mathbb{R}^3}} \phi_{u_0} |u_0|^{\vartheta}dx\\&-\beta\frac{t^\tau}{\tau}{\int_{\mathbb{R}^3}} {\mathcal{W}}(x)|u_0|^{\tau}dx-\frac{t^{p^*}}{p^*}{\int_{\mathbb{R}^3}} {\mathcal{K}}(x)|u_0|^{p^*}dx. \end{align}\] Now, from \(q<p\le \tau<p^*\) and either by 2 in case \({\mathcal{J}}_-\) or by \(\max\{1,q/(m^*)'\}<\vartheta<p^*/m'\) in case \({\mathcal{J}}_+\), it follows that \({\mathcal{J}}(tu_0)\to 0^+\) as \(t\to0^+\) being positive the coefficient of the lower order term in \(t\), while \({\mathcal{J}}(tu_0)\to-\infty\) as \(t\to\infty\), being negative the coefficient of the higher order term. Thus, there exists \(t_{\lambda,\beta}>0\) such that \[\max_{t\ge 0}{\mathcal{J}}(tu_0)={\mathcal{J}}(t_{\lambda,\beta} u_0).\] In particular, being \({\mathcal{J}}\in C^1\), we get \[\label{dtjt0}\begin{align}0=\frac{d}{dt}\Bigl[ {\mathcal{J}}(tu_0)\Bigr]_{t=t_{\lambda,\beta}}&=t_{\lambda,\beta}^{p-1}\|\nabla u_0\|_p^p+t_{\lambda,\beta}^{q-1}\|\nabla u_0\|_q^q \pm \lambda t^{\vartheta m'-1}{\int_{\mathbb{R}^3}} \phi_{u_0} u_0^{\vartheta}dx\\&\quad -\beta t_{\lambda,\beta}^{\tau-1}\int_{\mathbb{R}^3}{\mathcal{W}}(x)u_0^{\tau}\, dx -t_{\lambda,\beta}^{p^*-1}\int_{\mathbb{R}^3}{\mathcal{K}}(x) u_0^{p^*} \,dx.\end{align}\tag{28}\] Now we divide the proof into cases.
Case \({\mathcal{J}}_+\): In this case 28 is equivalent to \[\label{primozero} \frac{\|\nabla u_0\|_p^p}{t_{\lambda,\beta}^{\tau -p}}+\frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\tau-q}} -t_{\lambda,\beta}^{p^*-\tau}\int_{\mathbb{R}^3}{\mathcal{K}}(x) u_0^{p^*} \,dx + \frac{\lambda}{t_{\lambda,\beta}^{\tau-\vartheta m'}}{\int_{\mathbb{R}^3}} \phi_{u_0} u_0^{\vartheta}dx=\beta \int_{\mathbb{R}^3}{\mathcal{W}}(x)u_0^{\tau}\, dx\tag{29}\] for every \(\lambda, \beta>0\). Since the support of \(u_0\) is contained in \(\Omega_{\mathcal{W}}\), the right hand side of 29 is positive and it goes to \(\infty\) if \(\beta\to\infty\). Thus, also the left hand side of 29 must go to \(\infty\) if \(\beta\to\infty\) and for all \(\lambda\). Being \(q<p\le \tau<p^*\), necessarily \(t_{\lambda,\beta}\to 0^+\) as \(\beta\to\infty\) and for all \(\lambda>0\) since \[\begin{align} \frac{\|\nabla u_0\|_p^p}{t_{\lambda,\beta}^{\tau -p}}&+\frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\tau-q}} -t_{\lambda,\beta}^{p^*-\tau}\int_{\mathbb{R}^3}{\mathcal{K}}(x) u_0^{p^*} \,dx + \frac{\lambda}{t_{\lambda,\beta}^{\tau-\vartheta m'}}{\int_{\mathbb{R}^3}} \phi_{u_0} u_0^{\vartheta}dx\\& \ge \frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\tau-q}} -t_{\lambda,\beta}^{p^*-\tau}\int_{\mathbb{R}^3}{\mathcal{K}}(x) u_0^{p^*} \,dx \sim \frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\tau-q}} \to\infty, \end{align}\] as \(\beta\to\infty\). From \({\mathcal{J}}(t_{\lambda,\beta} u_0)\to 0^+\) as \(t_{\lambda,\beta}\to0^+\) or equivalently when \(\beta\to\infty\), we can conclude that there exists \(\beta^*>0\) such that for all \(\beta>\beta^*\) and \(\lambda>0\) \[\label{maxj43} \max_{t\ge0}{\mathcal{J}}_+(tu_0)={\mathcal{J}}_+(t_{\lambda,\beta} u_0)<c_{PS}.\tag{30}\] By the definition of \(\hat{c}_+\), we get \(\hat{c}_+<c_{PS}\) for all \(\beta>\beta^*\) and \(\lambda>0\).
Case \({\mathcal{J}}_-\): We split the the interval for \(\tau\) in two disjoint intervals.
If \(p \le\tau< \vartheta m'\) and \(\max\{1,q/(m^*)',p/m'\}<\vartheta<p^*/(m^*)'\), we use 29 with \(\lambda\) replaced with \(-\lambda\), to deduce that \(t_{\lambda,\beta}\to 0^+\) as \(\beta\to\infty\) and for all \(\lambda>0\) since \[\begin{align} \frac{\|\nabla u_0\|_p^p}{t_{\lambda,\beta}^{\tau -p}}&+\frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\tau-q}} -t_{\lambda,\beta}^{p^*-\tau}\int_{\mathbb{R}^3}{\mathcal{K}}(x) u_0^{p^*} \,dx - t_{\lambda,\beta}^{\vartheta m'-\tau}\lambda{\int_{\mathbb{R}^3}} \phi_{u_0} u_0^{\vartheta}dx\\& \ge \frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\tau-q}} -t_{\lambda,\beta}^{p^*-\tau}\int_{\mathbb{R}^3}{\mathcal{K}}(x) u_0^{p^*} \,dx- t_{\lambda,\beta}^{\vartheta m'-\tau}\lambda{\int_{\mathbb{R}^3}} \phi_{u_0} u_0^{\vartheta}dx \sim \frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\tau-q}} \to\infty, \end{align}\] as \(\beta\to\infty\). Thus, arguing as in 30 replacing \({\mathcal{J}}_+\) with respectively \({\mathcal{J}}_-\), we get \(\hat{c}_-<c_{PS}\) for all \(\beta>\beta^*\) and \(\lambda>0\).
While, if \(\vartheta m'\le \tau<p^*\) and \(\max\{1,q/(m^*)'\}<\vartheta<p^*/m'\), we wite 28 as follows \[\label{primozero39}\begin{align} \frac{\|\nabla u_0\|_p^p}{t_{\lambda,\beta}^{\vartheta m' -p}}&+\frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\vartheta m'-q}} -t_{\lambda,\beta}^{p^*-\vartheta m'}\int_{\mathbb{R}^3}{\mathcal{K}}(x) u_0^{p^*} \,dx \\&-\beta t_{\lambda,\beta}^{\tau-\vartheta m'}\int_{\mathbb{R}^3}{\mathcal{W}}(x)u_0^{\tau}dx= \lambda {\int_{\mathbb{R}^3}} \phi_{u_0} |u_0|^{\vartheta}dx \end{align}\tag{31}\] for every \(\beta,\lambda>0\). Being \(u_0\) is nontrivial, then the right hand side of 31 is positive and goes to \(\infty\) if \(\lambda\to\infty\). Thus, also the left hand side of 31 must go to \(\infty\) if \(\lambda\to\infty\), this occurs if \(t_{\lambda,\beta}\to 0^+\) as \(\lambda\to\infty\) being \(\vartheta>q/(m^*)'>q/m'\) and \(\|\nabla u_0\|_q>0\) since \[\begin{align} \frac{\|\nabla u_0\|_p^p}{t_{\lambda,\beta}^{\vartheta m' -p}}&+\frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\vartheta m'-q}} -t_{\lambda,\beta}^{p^*-\vartheta m'}\int_{\mathbb{R}^3}{\mathcal{K}}(x) u_0^{p^*} \,dx -\beta t_{\lambda,\beta}^{\tau-\vartheta m'}\int_{\mathbb{R}^3}{\mathcal{W}}(x)u_0^{\tau}dx\\& \ge \frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\vartheta m'-q}} -t_{\lambda,\beta}^{p^*-\vartheta m'}\int_{\mathbb{R}^3}{\mathcal{K}}(x) u_0^{p^*} \,dx -\beta t_{\lambda,\beta}^{\tau-\vartheta m'}\int_{\mathbb{R}^3}{\mathcal{W}}(x)u_0^{\tau}dx \sim \frac{\|\nabla u_0\|_q^q}{t_{\lambda,\beta}^{\vartheta m'-q}} \to\infty \end{align}\] as \(\lambda\to\infty\). From \({\mathcal{J}}(t_{\lambda,\beta} u_0)\to 0^+\) as \(t_{\lambda,\beta}\to0^+\) or equivalently when \(\lambda\to\infty\), we can conclude that there exists \(\lambda^*>0\) such that for all \(\lambda>\lambda^*\) and \(\beta>0\) \[\max_{t\ge0}{\mathcal{J}}_-(tu_0)={\mathcal{J}}_-(t_{\lambda,\beta} u_0)<c_{PS}.\] By the definition of \(\hat{c}_-\), we get \(\hat{c}_-<c_{PS}\) for all \(\lambda>\lambda^*\) and \(\beta>0\).
Finally, we have all the ingredients to conclude the proof of the main theorem of the paper.
Proof of Theorem 1. Under the different assumptions on \(\tau,\vartheta\) in the statement of Theorem 1, Lemma 9 implies that the energy functional \({\mathcal{J}}\) has the mountain pass geometry. Thus, Theorem 2 can be applied giving the existence of a Palais-Smale sequence \((u_n)_n\subset X\) at level \(c_u\). By employing the boundedness of such sequence given by Lemma 7, we proved a compactness assumption in Lemma 10 in terms of the validity of the Palais-Smale condition for suitable levels, and finally Lemma 12 confirms that \((u_n)_n\subset X\) converges, up to subsequences, to a nontrivial function \(u\in X\) having positive energy and for which it holds \({\mathcal{J}}'(u_n)\varphi=0\) for all \(\varphi\in X\), i.e. \(u\) is a weak solution to 1 .
The authors are members of the Gruppo Nazionale per l’Analisi Matematica, la Probabilità e le loro Applicazioni (GNAMPA) of the Istituto Nazionale di Alta Matematica (INdAM). L.B is partially supported by INdAM-GNAMPA Project 2026 titled Structural degeneracy and criticality in (sub)elliptic PDEs (E53C25002010001) and by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Project-ID 258734477 - SFB 1173.