Upper bounds and asymptotics in a quantitative version of the Oppenheim conjecture.

Alex Eskin^{∗}, Gregory Margulis^{†}and Shahar Mozes^{‡}

April 20, 1997

1 Introduction

Let Q be an indefinite nondegenerate quadratic form in n variables. Let L_{Q} = Q(ℤ^{n}) denote the set of values of Q at integral points. The Oppenheim conjecture, proved by G.A. Margulis (cf. [Mar2, Mar3, Mar4]) states that if n ≥ 3, and Q is not proportional to a form with rational coefficients, then L_{Q} is dense. Since it was conjectured in 1929, the Oppenheim conjecture enjoyed attention and many studies, mostly using analytic number theory methods. See [Mar5], [Bor2] and [Rat6] for references and background.

The proof given in [Mar 2-4] uses a connection, noticed by M.S. Raghunathan, between the Oppenheim conjecture and questions concerning closures of orbits of certain subgroups. Indeed it is not difficult to show that the Oppenheim conjecture (or, more precisely a slightly weaker form of this conjecture) is equivalent to the statement that any relatively compact orbit of SO(2, 1) in SL(3, ℝ)/SL(3, ℤ) is compact; in explicit form in the literature this equivalence appears in [Mar 2-4]. (We recently realized that in implicit form this relation already appears in section 10 of an old paper of Cassels and Swinnerton-Dyer [CSD]). In the mid-seventies M.S. Raghunathan noticed that the Oppenheim conjecture would follow from a conjecture about closures of orbits of unipotent subgroups (see [Dan2]). The Raghunathan conjecture states that if G is a connected Lie group, Γ a lattice in G, and U a unipotent subgroup of G then for any x ∈ G/Γ, there exists a closed subgroup L = L(x) containing U such that the closure of the orbit Ux coincides with Lx. In the literature it was first stated in the paper [Dan2] and in a more general form in [Mar3] (when the subgroup U is not necessarily unipotent but generated by unipotent elements).

∗Research partially supported by an NSF postdoctoral fellowship and by BSF grant 94-00060/1

†Research partially supported by NSF grants DMS-9204270 and DMS-9424613

‡Research partially supported by the Israel Science foundation and by BSF grant 94-00060/1

Raghunathan’s conjecture was eventually proved in full generality by M. Ratner (see [Rat4]). Earlier it was known in the following cases: (a) G is reductive and U is horospherical (see [Dan2]); (b) G = SL(3, ℝ) and U = {u(t)} is a one-parameter unipotent subgroup of G such that u(t) − I has rank 2 for all t ≠ 0, where I is the identity matrix (see [DM2]); (c) G is solvable (see [Sta1] and [Sta2]). We remark that the proof given in [Dan2] is restricted to horospherical U and the proof given in [Sta1] and [Sta2] cannot be applied for nonsolvable G. However the proof in [DM2] together with the methods developed in [Mar 2-4] and [DM1] suggest an approach for proving the Raghunathan conjecture in general by studying the minimal invariant sets, and the limits of orbits of sequences of points tending to a minimal invariant set. This program was being actively pursued at the time Ratner’s results were announced (cf. [Sha3]).

Ratner’s proof of the Raghunathan conjecture is based on her uniform distribution theorem (see [Rat4]). This theorem states that if {u_{t}} is a one-parameter unipotent subgroup and x ∈ G/Γ then there exist a closed subgroup L ⊂ G containing U such that the orbit Lx is closed, and an L-invariant Borel probability measure μ on G/Γ supported on Lx such that for any bounded continuous function f on G,

lim_{T →∞} \frac{1}{T} ∫_{0}^{T} f(u_{t}x) dt = ∫_{G/Γ} f dμ.

Note that the uniform distribution theorem can be considered as the quantitative version of Raghunathan’s conjecture for one-parameter unipotent subgroups.

The proof of the uniform distribution theorem is mostly based on another basic result: Ratner’s measure classification theorem (“Raghunathan’s measure conjecture”). This theorem, conjectured by S.G. Dani in [Dan2], asserts that all finite, ergodic, U-invariant measures on G/Γ are “algebraic”. This deep result, which is the hardest part of Ratner’s proof of the Raghunathan conjecture, proved to be an important and useful tool for the study of (unipotent) flows on homogeneous spaces. In addition to Ratner’s original proof of the measure classification theorem given in [Rat1, Rat2, Rat3], we refer the reader also to a different and shorter proof given by G.A. Margulis and G.M. Tomanov (see [MT1] for the case where G is algebraic and [MT2] for a simple reduction to that case). An alternative approach to the proof of the uniform distribution theorem for the case of a regular unipotent subgroup, also using Ratner’s measure classification theorem, is given in [Sha1].

Subsequently, using both Ratner’s measure classification theorem, as well as their methods of studying unipotent flows on homogeneous spaces via “linearization”, S.G. Dani and G.A. Margulis proved in [DM3] a refined version of the uniform distribution theorem (see Theorem 4.1 below). This theorem is the main ergodic result used in this paper.

It should be noted that Dani’s theorem on the behavior of orbits of unipotent flows at infinity (see [Dan3] and [Dan4]) is essentially used in [DM1, DM2, DM3], and [Rat4]. For the case where G = SL(n, ℝ) and Γ = SL(n, ℤ) this theorem is essentially equivalent to the following assertion: for any lattice Λ in ℝ^{n}, any unipotent one-parameter subgroup {u_{t}} of SL(n, ℝ) and any ϵ > 0, one can find δ > 0 such that the set

Ω(Λ, δ) = {t ≥ 0 | ∥u_{t}z∥ > δ for all z ∈ Λ − {0}}  (1.1)

has density greater then 1 − ϵ on every interval [0, T]. In a weak form this assertion was proved in [Mar1] (namely it was proved in [Mar1] that Ω(Λ, δ) is unbounded for some δ = δ(Λ) > 0). Some related questions will be considered in detail below.

Finally, we note that a characteristic property of the unipotent flow, (which distinguishes it from e.g. the geodesic flow) is its close connection to “slow growing” functions like polynomials of bounded degree. In fact most of the results mentioned above for unipotent flows are also true for polynomial maps: see [Sha2]. For further results in the p-adic and S-arithmetic settings see [Rat5] and [MT1].

By “quantitative Oppenheim conjecture” we mean^{1} the study of the distribution of values of a quadratic form, e.g. the number of lattice points in a subset Ω of ℝ^{n} such that a given quadratic form has values belonging to some given interval, for example determining the asymptotics of the number V_{(a,b)}^{Q}(T) of integral points v in a ball of radius T with a < Q(v) < b. In [DM3] S.G. Dani and G.A. Margulis obtained asymptotically exact lower bounds for V_{(a,b)}^{Q}(T) and for some more general quantities. In particular they proved that if L is a compact family of indefinite non-degenerate quadratic forms in n ≥ 3 variables which consists of forms not proportional to rational forms, then

lim inf_{T →∞} inf_{Q∈L} \frac{V_{(a,b)}^{Q}(T)}{Vol(\{v ∈ ℝ^{n} | a < Q(v) < b, ∥v∥ < T \})} ≥ 1,

where Vol(·) is the Lebesgue measure on ℝ^{n}. For a single Q, the fact that the lower limit is positive had been obtained earlier by S.G. Dani and S. Mozes and also independently by M. Ratner (both unpublished). The proof of the lower bounds in [DM3] is based on the following identity which is immediate from the definitions:

1An announcement of the results of this paper appeared in [EMM]

∫_{T}^{ℓT} ∫_{B} ∑_{v∈gℤ^{n}} ψ(u_{t}kv) dσ(k) dt = ∫_{T}^{ℓT} ∫_{B} ψ̃(u_{t}kgΓ) dσ(k) dt,  (1.2)

where {u_{t}} is a certain one-parameter unipotent subgroup, B is a Borel subset of the maximal compact subgroup K of SO(p, q), σ is the normalized Haar measure on K, ψ is an integrable function on ℝ^{n} − {0}, Γ = SL(n, ℤ) and

ψ̃(gΓ) = ∑_{v∈gℤ^{n}} ψ(v).

The left hand side in (1.2) gives a good approximation for V_{(a,b)}^{Q}(T) for an appropriate choice of ψ and g and the right hand side can be estimated, uniformly when gΓ belongs to certain compact subsets of SL(n, ℝ)/Γ, using the abovementioned version from [DM3] of the uniform distribution theorem.

The function ψ̃ on SL(n, ℝ)/SL(n, ℤ) is unbounded for any non-negative nonzero continuous function ψ on ℝ^{n}. Therefore we can not use (1.2) and the uniform distribution theorem to get the asymptotic of V_{(a,b)}^{Q}(T), because this theorem is proved (and in general true) only for bounded continuous functions. On the other hand, as it was done in [DM3] one can get lower bounds by considering bounded continuous functions f ≤ ψ̃ and applying the uniform distribution theorem to f.

It would be possible to obtain the asymptotic of V_{(a,b)}^{Q}(T) from (1.2) and the uniform distribution theorem if we knew that for some δ > 0 and any continuous nonnegative function ψ on ℝ^{n} with compact support,

lim sup_{T →∞} \frac{1}{T} ∫_{0}^{T} ∫_{K} ψ̃(u_{t}kgΓ)^{1+δ} dσ(k) dt < ∞.  (1.3)

Using methods of the papers [Dan1], [Dan4] and [Mar1] it is possible to prove a stronger version of (1.1), i.e. that there exists θ = θ(n) > 0 (and this θ can be effectively computed) such that

lim sup_{T →∞} \{sup_{g∈L} \frac{1}{T} ∫_{0}^{T} ψ̃(u_{t}gΓ)^{θ} dt\} < ∞

for any compact subset L ⊂ SL(n, ℝ) and any unipotent subgroup {u_{t}} ⊂ SL(n, ℝ). But these methods are certainly not enough to prove (1.3).

An earlier, technically complex, argument based in part on the ideas of [Dan1],

[Dan4] and [Mar1] allowed us to prove (1.3) for the special case (p, q) = (3, 1) and δ near zero. Here we present a different argument which uses a new idea for handling the complications arising from the involved structure at infinity of SL(n, ℝ)/SL(n, ℤ). Using this we essentially prove (1.3) for all 0 ≤ δ < 1 in the case where p ≥ 3 and q ≥ 1 (recall that K is a maximal compact subgroup of SO(p, q)). If (p, q) = (2, 2) or (p, q) = (2, 1) then (1.3) is in general not true even for δ = 0 but it becomes true for δ = 0 if we replace in (1.3) the factor 1/T by 1/(T log T). Actually, for technical reasons, in the sequel we consider not the integrals from (1.3) but integrals of the form

∫_{K} ψ̃(a_{t}k)^{1+δ} dσ(k),

where {a_{t}} is a certain diagonalizable subgroup of SO(p, q).

2 Statement of Results

Let ρ be a continuous positive function on the sphere {v ∈ ℝ^{n} | ∥v∥ = 1}, and let Ω = {v ∈ ℝ^{n} | ∥v∥ < ρ(v/∥v∥)}. We denote by T Ω the dilate of Ω by T. Define the following set:

V_{(a,b)}^{Q}(ℝ) = {x ∈ ℝ^{n} | a < Q(x) < b}

We shall use V_{(a,b)} = V_{(a,b)}^{Q} when there is no confusion about the form Q. Also let V_{(a,b)}(ℤ) = V_{(a,b)}^{Q}(ℤ) = {x ∈ ℤ^{n} | a < Q(x) < b}. The set T Ω ∩ ℤ^{n} consists of O(T^{n}) points, Q(T Ω ∩ ℤ^{n}) is contained in an interval of the form [−μT^{2}, μT^{2}], where μ > 0 is a constant depending on Q and Ω. Thus one might expect that for any interval [a, b], as T → ∞,

|V_{(a,b)}(ℤ) ∩ T Ω| ∼ c_{Q,Ω}(b − a)T^{n−2}.  (2.1)

where c_{Q,Ω} is a constant depending on Q and Ω. This may be interpreted as “uniform distribution” of the sets Q(ℤ^{n} ∩ T Ω) in the real line. Our main result is that (2.1) holds if Q is not proportional to a rational form, and has signature (p, q) with p ≥ 3, q ≥ 1. We also determine the constant c_{Q,Ω}.

If Q is an indefinite quadratic form in n variables, Ω is as above and (a, b) is an interval, we will show that there exists a constant λ = λ_{Q,Ω} so that as T → ∞,

Vol(V_{(a,b)}(ℝ) ∩ T Ω) ∼ λ_{Q,Ω}(b − a)T^{n−2}.  (2.2)

We show that λ_{Q,Ω} as in (2.2) serves as the desired constant c_{Q,Ω}. Thus, our main result is the following:

Theorem 2.1. Let Q be an indefinite quadratic form of signature (p, q), with p ≥ 3 and q ≥ 1. Suppose Q is not proportional to a rational form. Then for any interval (a, b), as T → ∞,

|V_{(a,b)}(ℤ) ∩ T Ω| ∼ λ_{Q,Ω}(b − a)T^{n−2},

where n = p + q, and λ_{Q,Ω} is as in (2.2).

Only the upper bound in this formula is new: the asymptotically exact lower bound was proved in [DM3].

If the signature of Q is (2, 1) or (2, 2) then no universal formula like (2.1) holds. In fact, we have the following theorem:

Theorem 2.2. Let Ω_{0} be the unit ball, and let q = 1 or 2. Then for every ϵ > 0 and every interval (a, b) there exists a quadratic form Q of signature (2, q) not proportional to a rational form, and a constant c > 0 such that for an infinite sequence T_{j} → ∞,

|V_{(a,b)}(ℤ) ∩ T Ω_{0}| > cT_{j}^{q}(log T_{j})^{1−ϵ}.

The case q = 1, b ≤ 0 of Theorem 2.2 was noticed by P. Sarnak and worked out in detail in [Bre].

While an estimate as in (2.1) does not hold in the case of signatures (2, 1) and (2, 2), we can still establish an upper bound of the form T^{q} log T. This upper bound is effective, and is uniform over compact sets in the set of quadratic forms. We also give an effective uniform upper bound for the case p ≥ 3.

Theorem 2.3. Let O(p, q) denote the space of quadratic forms of signature (p, q) and discriminant ±1, let n = p + q, (a, b) be an interval, and let D be a compact subset of O(p, q). Let ν be a continuous positive function on the unit sphere and let Ω = {v ∈ ℝ^{n} | ∥v∥ < ν(v/∥v∥)}. Then, if p ≥ 3 there exists a constant c depending only on D, (a, b) and Ω such that for any Q ∈ D and all T > 1,

|V_{(a,b)}(ℤ) ∩ T Ω| < cT^{n−2}.

If p = 2 and q = 1 or q = 2, then there exists a constant c > 0 depending only on D, (a, b) and Ω such that for any Q ∈ D and all T > 2,

|V_{(a,b)} ∩ T Ω ∩ ℤ^{n}| < cT^{n−2} log T.

Also, for the (2, 1) and (2, 2) cases, we have the following “almost everywhere” result:

Theorem 2.4. For almost all quadratic forms Q of signature (p, q) = (2, 1) or (2, 2)

|V_{(a,b)}(ℤ) ∩ T Ω| ∼ λ_{Q,Ω}(b − a)T^{n−2},

where n = p + q, and λ_{Q,Ω} is as in (2.2).

Theorem 2.4 can be proved using a recent general result of Nevo and Stein [NS]; in §3.6 we will present a self-contained argument which suffices for the application. In [Sar], P. Sarnak proved that an analogous asymptotic formula holds for almost all forms within a specific two-parameter family; this family arises in problems related to Quantum Chaos.

Finally, following [DM3] we give a “uniform” version of Theorem 2.1:

Theorem 2.5. Let D be a compact subset of O(p, q), with p ≥ 3. Let n = p + q, and let Ω be as in Theorem 2.3. Then for every interval [a, b] and every θ > 0, there exists a finite subset P of D such that each Q ∈ P is a scalar multiple of a rational form and for any compact subset F of D − P there exists T_{0} such that for all Q in F and T ≥ T_{0},

(1 − θ)λ_{Q,Ω}(b − a)T^{n−2} ≤ |V_{(a,b)}(ℤ) ∩ T Ω|, ≤ (1 + θ)λ_{Q,Ω}(b − a)T^{n−2}

where λ_{Q,Ω} is as in (2.2).

As in Theorem 2.1 only the upper bound is new; the asymptotically exact lower bound, which holds even for SO(2, 1) and SO(2, 2), was proved in [DM3].

Remark 2.6. If we consider |V_{(a,b)}(ℝ)∩T Ω∩P(ℤ^{n})| instead of |V_{(a,b)}(ℤ)∩T Ω| (where P(ℤ^{n}) denotes the set of primitive lattice points), then Theorem 2.1 and Theorem 2.5 hold provided one replaces λ_{Q,Ω} by λ′_{Q,Ω} = λ_{Q,Ω}/ζ(n), where ζ is the Riemann zeta function.

Remark 2.7. Theorem 2.1 and Theorem 2.5, as well as lower bounds in [DM3] and even the proof of the Oppenheim conjecture in [Mar2-4], are not effective. In the case of Theorem 2.5 it means that we do not know how to determine effectively P and T_{0}. In the case of the Oppenheim conjecture, it means that the proof given in [Mar2-4] does not give an algorithm for determining T such that {v ∈ ℤ^{n} | a < Q(v) < b, ∥v∥ < T, v ≠ 0} ≠ ∅; this T should be determined in terms of Diophantine approximation properties of the coefficients of Q. The proof in [Mar2-4] is not effective because, in particular, it implicitly uses the fact that the closure of any relatively compact orbit F x contains a minimal closed F -invariant subset; this fact has no effective version. The proof of the lower bound in [DM3] and the proofs of Theorem 2.1 and Theorem 2.5 are not effective because these proofs use the uniform distribution theorem and the proof of this theorem uses different facts from ergodic theory which have no effective versions (in particular the fact that the measure \frac{1}{T} ∫_{0}^{T} u_{t}δ_{x} dt converges to the set of invariant measures). It seems to be a very difficult problem to give effective versions of Theorem 2.1, Theorem 2.5 and results from [Mar2-4],[DM1-3],[Rat4].

The paper is organized as follows. In §3 we connect the study of integral values of quadratic forms to estimates of certain integrals over orbits of semisimple groups. We state the results we need in §3.3 and prove the theorems stated in §2 modulo these results in §3.4-§3.6. In §3.7 we discuss “counterexamples”, i.e Theorem 2.2.

In the rest of the paper we prove the theorems stated in §3.3. The main result, Theorem 3.5 is naturally a combination of a result for functions with compact support, Theorem 4.4 and an integrability estimate, Theorem 3.2. In §4 we derive Theorem 4.4 from [DM3, Theorem 3] (restated as Theorem 4.1). In §5, which contains most of the new ideas of this paper, we prove Theorem 3.2, and its analogue for the SO(2, 1) and SO(2, 2) cases, Theorem 3.3. We then complete the proof of Theorem 3.5.

Acknowledgments: The authors would like to thank P. Sarnak and A. Nevo for useful conversations, and also G. Tomanov and the referee for pointing out many inaccuracies in the initial version of this paper.

3 Integral Values of quadratic forms

In this section we relate the counting problems of §2 to certain integral expressions involving the orthogonal group of the quadratic form and the space of lattices SL(n, ℝ)/SL(n, ℤ). Roughly this is done as follows. Let f be a bounded function on ℝ^{n} − {0} vanishing outside a compact subset. For g ∈ SL(n, ℝ) let

f̃(g) = ∑_{v∈ℤ^{n}} f(gv).  (3.1)

The proof is based on the identity of the form

∫_{K} f̃(a_{t}kg) dk = ∑_{v∈ℤ^{n}} ∫_{K} f(a_{t}kgv) dk,  (3.2)

obtained by integrating (3.1). In (3.2) {a_{t}} is a certain diagonal subgroup of the orthogonal group of Q, and K is a maximal compact subgroup of the orthogonal group of Q. Then for an appropriate function f and an appropriate element g ∈ G, the right hand side is then related to the number of lattice points v ∈ [e^{t}/2, e^{t}]∂Ω with a < Q(v) < b. For a precise statement see Proposition 3.7 below. We then establish the asymptotics of the left-hand side using the ergodic theory of unipotent flows and some other techniques.

3.1 Lattices and the function α.

Let Δ be a lattice in ℝ^{n}. We say that a subspace L of ℝ^{n} is Δ-rational if L ∩ Δ is a lattice in L. For any Δ-rational subspace L, we denote by d_{Δ}(L) or simply by d(L) the volume of L/(L ∩ Δ). Let us note that d(L) is equal to the norm of u_{1} ∧ ⋯ ∧ u_{ℓ} in the exterior power ⋀^{ℓ}(ℝ^{n}) where ℓ = dim L, (u_{1}, ⋯, u_{ℓ}) is a basis over ℤ of L∩Δ, and the norm on ⋀(ℝ^{n}) is induced from the Euclidean norm on ℝ^{n}. If L = {0} we write d(L) = 1. A lattice is Δ unimodular if d_{Δ}(ℝ^{n}) = 1. The space of unimodular lattices is canonically identified with SL(n, ℝ)/SL(n, ℤ).

Let us introduce the following notation:

α_{i}(Δ) = sup \{\frac{1}{d(L)}| L is a Δ-rational subspace of dimension i \}, 0 ≤ i ≤ n,
α(Δ) = max_{0≤i≤n} α_{i}(Δ).  (3.3)

Lemma 3.1 ([Sch, Lemma 2]). Let f be a bounded function on ℝ^{n} vanishing outside a compact subset. Then there exists a positive constant c = c(f) such that

f̃(Δ) < cα(Δ),

for any lattice Δ in ℝ^{n}. Here f̃ is the function on the space of lattices defined in (3.1).

3.2 Quadratic Forms

Let n ≥ 3, and let p ≥ 2. We denote n − p by q, and assume q > 0. Let {e_{1}, e_{2}, … e_{n}} be the standard basis of ℝ^{n}. Let Q_{0} be the quadratic form defined by

Q_{0} (∑_{i=1}^{n} v_{i}e_{i}) = 2v_{1}v_{n} + ∑_{i=2}^{p} v_{i}^{2} − ∑_{i=p+1}^{n−1} v_{i}^{2} for all v_{1}, …, v_{n} ∈ ℝ.  (3.4)

It is straightforward to verify that Q_{0} has signature (p, q). Let G = SL(n, ℝ), the group of n × n matrices of determinant 1. For each quadratic form Q and g ∈ G, let Q^{g} denote the quadratic form defined by Q^{g}(v) = Q(gv) for all v ∈ ℝ^{n}. Recall that O(p, q) denotes the set of quadratic forms of signature (p, q) and discriminant ±1. By the well known classification of quadratic forms over ℝ, for each Q ∈ O(p, q) there exists g ∈ G such that Q = Q_{0}^{g}. For any quadratic form Q let SO(Q) denote the special orthogonal group corresponding to Q; namely {g ∈ G | Q^{g} = Q}. Let H = SO(Q_{0}). Then the map H\G → O(p, q) given by Hg → Q_{0}^{g} is a homeomorphism.

For t ∈ ℝ, let a_{t} be the linear map so that a_{t}e_{1} = e^{−t}e_{1}, a_{t}e_{n} = e^{t}e_{n}, and a_{t}e_{i} = e_{i}, 2 ≤ i ≤ n − 1. Then the one-parameter group {a_{t}} is contained in H. Let K̂ be the subgroup of G consisting of orthogonal matrices, and let K = H ∩ K̂. It is easy to check that K is a maximal compact subgroup of H, and consists of all h ∈ H leaving invariant the subspace spanned by {e_{1} + e_{n}, e_{2}, …, e_{p}}. We denote by m the normalized Haar measure on K.

3.3 The main theorems

In this subsection we state without proof the main theorems used to prove the results stated in §2. The proofs of the theorems in this subsection will be given in §4 and §5. We keep the notation of the previous two subsections.

The main ergodic result which we use, namely Theorem 4.1 (i.e. [DM3, Theorem 3]) holds for bounded continuous functions, but not for arbitrary continuous functions from L^{1}(G/Γ), (see the remark following Theorem 4.1). However, for a non-negative bounded continuous function f on ℝ^{n}, the function f̃ defined in (3.1) is non-negative, continuous, and L^{1} but unbounded (it is in L^{s}(G/Γ) for 1 ≤ s < n, where G = SL(n, ℝ), and Γ = SL(n, ℤ)). Thus in order to carry out the integrals in (3.2) and prove the upper bounds in the theorems of §2 we need to examine carefully the situation at the “cusp” of G/Γ, i.e outside of compact sets.

By Lemma 3.1, the function f̃(g) on the space of unimodular lattices G/Γ is majorized by the function α(g). The function α is more convenient since it is invariant under the left action of the maximal compact subgroup K̂ of G, and its growth rate at infinity is known explicitly. The main integrability estimate we will use is the following theorem, proved in §5:

Theorem 3.2. If p ≥ 3, q ≥ 1 and 0 < s < 2, or if p = 2, q ≥ 1 and 0 < s < 1, then for any lattice Δ in ℝ^{n}

sup_{t>0} ∫_{K} α(a_{t}kΔ)^{s} dm(k) < ∞.

The upper bound is uniform as Δ varies over compact sets in the space of lattices.

This result can be interpreted as follows. For a lattice Δ in G/Γ and for h ∈ H, let f(h) = α(hΔ). Since α is left K̂-invariant, f is a function on the symmetric space X = K\H. Theorem 3.2 is the statement that if p ≥ 3, then the averages of f^{s}, 0 < s < 2 over the sets Ka_{t}K in X remain bounded as t → ∞, and the bound is uniform as one varies Δ over compact sets. We remark that in the case q = 1, the rank of X is 1, and the sets Ka_{t}K are metric spheres of radius t, centered at the origin.

In the case p = 2, the analogue of the assertion in Theorem 3.2 fails even for s = 1, and the best possible result is the following theorem which states that the average of α over large spheres in X grows at most linearly with the radius of the sphere. This yields the cases p = 2, q = 1, and p = 2, q = 2 in Theorem 2.3.

Theorem 3.3. If p = 2 and q = 1 or 2, then for any lattice Δ in ℝ^{n},

sup_{t>1} \frac{1}{t} ∫_{K} α(a_{t}kΔ) dm(k) < ∞.  (3.5)

The upper bound is uniform as Δ varies over compact sets in the space of lattices.

To prove Theorem 2.1 one may use the following theorem:

Theorem 3.4. Suppose p ≥ 3, q ≥ 1. Let ϕ be a continuous function on G/Γ (the space of lattices in ℝ^{n} with determinant 1). Assume that for some s, 0 < s < 2 and some C > 0,

|ϕ(Δ)| < Cα(Δ)^{s}, for all Δ ∈ G/Γ.

Let x_{0} ∈ G/Γ be a unimodular lattice such that Hx_{0} is not closed. Let ν be any continuous function on K. Then

lim_{t→+∞} ∫_{K} ϕ(a_{t}kx_{0})ν(k) dm(k) = ∫_{K} ν dm ∫_{G/Γ} ϕ(y) dμ(y).

Theorem 3.4 is proved by combining Theorem 3.2 with the results of §4. To prove Theorem 2.5 we will need the following generalization of Theorem 3.4:

Theorem 3.5. Suppose p ≥ 3, q ≥ 1. Let ϕ, ν be as in Theorem 3.4, and let C be any compact set in G/Γ. Then for any ϵ > 0 there exist finitely many points x_{1}, …, x_{ℓ} ∈ G/Γ such that (i) The orbits Hx_{1}, …, Hx_{ℓ} are closed and have finite H-invariant measure.

(ii) For any compact subset F of C − ⋃_{1≤i≤ℓ} Hx_{i}, there exists t_{0} > 0, so that for all x ∈ F and t > t_{0},

|∫_{K} ϕ(a_{t}kx)ν(k) dm(k) − ∫_{G/Γ} ϕ dμ ∫_{K} ν dm| ≤ ϵ.  (3.6)

Theorem 3.5 will be proved in §5 by combining Theorem 4.4 with Theorem 3.2.

3.4 A Lemma about vectors in ℝ^{n}.

We denote the standard inner product on ℝ^{n} by ⟨·, ·⟩. Let ℝ_{+}^{n} denote the set {x ∈ ℝ^{n} : ⟨x, e_{1}⟩ > 0}. Let c_{p,q} = 2^{(n−2)/2}/(γ_{p−1}γ_{q−1}), where for m ∈ ℕ, γ_{m} denotes the volume of S^{m}. We use the following lemma about vectors in ℝ^{n}:

Lemma 3.6. Let f be a continuous function of compact support on ℝ_{+}^{n} and let

J_{f}(r, ζ) = \frac{1}{r^{n−2}} ∫_{ℝ^{n−2}} f (r, x_{2}, …, x_{n−1}, x_{n}) dx_{2} … dx_{n−1},

where in the integral x_{n} = (ζ −Q_{0}(0, x_{2}, …, x_{n−1}, 0))/(2r), so that Q_{0}(r, x_{2}, …, x_{n}) = ζ. Let ν be a non-negative continuous function on S^{n−1}. Then for every ϵ > 0 there exists T_{0} > 0 such that for every t with e^{t} > T_{0}, and every v ∈ ℝ^{n} with ∥v∥ > T_{0},

|c_{p,q}e^{(n−2)t} ∫_{K} f(a_{t}kv)ν(k^{−1}e_{1}) dm(k) − J_{f} (∥v∥e^{−t}, Q_{0}(v)) ν (\frac{v}{∥v∥})| < ϵ, (3.7)

Proof. Let E denote the support of f. Let c_{1} = inf_{v∈E}⟨v, e_{1}⟩, c_{2} = sup_{v∈E}⟨v, e_{1}⟩. From the definition of a_{t} and the fact that a_{t} ∈ H = SO(Q_{0}), it follows that f(a_{t}w) = 0, unless

|⟨w, e_{1}⟩⟨w, e_{n}⟩| ≤ β  (3.8)
c_{1} ≤ ⟨w, e_{1}⟩e^{−t} ≤ c_{2}  (3.9)
π′(w) ∈ π′(E),  (3.10)

where β depends only on E and π′ denotes the projection onto the span of e_{2}, …, e_{n−1}. For w satisfying (3.8) and (3.9) we have ⟨w, e_{n}⟩ = O(e^{−t}). This, together with (3.10) and (3.9) imply that if f(a_{t}w) ≠ 0 and t is large, then

∥w∥ = ⟨w, e_{1}⟩ + O(e^{−t}).  (3.11)

Hence if f(a_{t}kv) ≠ 0 and t is large, then kv/∥v∥ must be near e_{1}. Now the uniform continuity of ν implies that for every δ > 0 there exists t_{0} > 0 so that for every v with ∥v∥ > t_{0} and every k in the support of the function k ↦ f(a_{t}kv),

|ν(k^{−1}e_{1}) − ν(v/∥v|)| < δ.  (3.12)

Note that if (3.8), (3.9) and (3.10) hold, then

⟨a_{t}w, e_{1}⟩ = ⟨w, e_{1}⟩e^{−t} = e^{−t}∥w∥ + O(e^{−2t}),  (3.13)

by (3.11). Also

⟨a_{t}w, e_{i}⟩ = ⟨w, e_{i}⟩,  2 ≤ i ≤ n − 1.  (3.14)

Finally,

\begin{aligned}⟨a_{t}w, e_{n}⟩ & = (Q_{0}(w) − Q_{0}(0, ⟨w, e_{2}⟩, …, ⟨w, e_{n−1}⟩, 0))/(2⟨a_{t}w, e_{1}⟩) \\ & = (Q_{0}(w) − Q_{0}(0, ⟨w, e_{2}⟩, …, ⟨w, e_{n−1}⟩, 0))/(2e^{−t}∥w∥) + O(e^{−t}). \end{aligned} (3.15)

Hence, using (3.13), (3.14) and (3.15) applied with w = kv, and the uniform continuity of f, we see that for any δ > 0 there exists t_{0} > 0 so that if t > t_{0} then on the support of the function k ↦ f(a_{t}kv),

|f(a_{t}kv) − f (∥v∥e^{−t}, ⟨kv, e_{2}⟩, …, ⟨kv, e_{n−1}⟩, v_{n})| < δ,  (3.16)

where v_{n} is determined by Q_{0}(∥v∥e^{−t}, ⟨kv, e_{2}⟩, …, ⟨kv, e_{n−1}⟩, v_{n}) = Q_{0}(v).

It is convenient to change basis by letting f_{1} = (e_{1} + e_{n})/\sqrt{2}, f_{n} = (e_{1} − e_{n})/\sqrt{2}, f_{j} = e_{j}, 2 ≤ j ≤ n − 1. In this basis K ≅ SO(p) × SO(q) consists of orthogonal matrices preserving the subspace L_{1} spanned by f_{1}, … f_{p}, and the subspace L_{2} spanned by f_{p+1}, … f_{n}. For i = 1, 2, let π_{i} denote orthogonal projection onto L_{i}. Write r_{i} = ∥π_{i}(v)∥; then the orbit Kv is the product of two spheres r_{1}S^{p−1} × r_{2}S^{q−1}, where S^{p−1} denotes the unit sphere in L_{1} and S^{q−1} denotes the unit sphere in L_{2}. We claim that f(a_{t}kv) = 0 for all k ∈ K unless both r_{1} and r_{2} are comparable to ∥v∥. Indeed, suppose w ∈ Kv is such that f(a_{t}w) ≠ 0. Then from (3.8) and (3.9) it follows that ⟨w, e_{n}⟩ = O(e^{−t}). Now if we write w = ∑ w_{i}f_{i}, then w_{1} = 2^{−1/2}⟨w, e_{1}⟩ + O(e^{−t}), w_{n} = 2^{−1/2}⟨w, e_{1}⟩ + O(e^{−t}), and for 2 ≤ i ≤ n − 1, w_{i} = O(1). Hence ∥π_{1}(w)∥ = w_{1} + O(e^{−t}), and ∥π_{2}(w)∥ = w_{n} + O(e^{−t}), It follows that for i = 1, 2,

r_{i} = ∥π_{i}(w)∥ = 2^{−1/2}⟨w, e_{1}⟩ + O(e^{−t}) = 2^{−1/2}∥w∥ + O(e^{−t}),  (3.17)

where the last estimate follows from (3.11).

The equation (3.10) implies that if f(a_{t}kv) ≠ 0, then kv is within a bounded distance from r_{1}f_{1} +r_{2}f_{n}. As ∥v∥ increases, so do the r_{i}, and the the normalized Haar measure on r_{1}S^{p−1} near r_{1}f_{1} (resp. r_{2}S^{q−1} near r_{2}f_{2}) tends to (1/γ_{p−1})r_{1}^{1−p}dw_{2} … dw_{p} (resp. (1/γ_{q−1})r_{2}^{1−q}dw_{p+1} … dw_{n−1}). Now the lemma follows from combining this fact with (3.16), (3.12) and (3.17).

3.5 Passage to the space of lattices.

Proof of Theorem 2.3. Choose a compact set C ⊂ G so that every quadratic form in D is a scalar multiple of Q_{0}^{x} for some x ∈ C. There exists a constant β so that for every x ∈ C and every v ∈ ℝ^{n}, β^{−1}∥v∥ ≤ ∥xv∥ ≤ β∥v∥. Let ϵ > 0 be an arbitrarly small number, and let x ∈ C be arbitrary. Choose a continuous non-negative function f on ℝ_{+}^{n} of compact support so that J_{f} ≥ 1 + ϵ on [β^{−1}, 2β] × [a, b]. Then if v ∈ ℝ^{n} satisfies e^{t} ≤ ∥v∥ ≤ 2e^{t}, a ≤ Q_{0}(xv) ≤ b, then J_{f}(∥xv∥e^{−t}, Q_{0}(xv)) ≥ 1 + ϵ. Then by Lemma 3.6, for sufficiently large t,

c_{p,q}e^{(n−2)t} ∫_{K} f(a_{t}kxv) dm(k) ≥ 1 if e^{t} ≤ ∥v∥ ≤ 2e^{t} and a ≤ Q_{0}(xv) ≤ b. (3.18)

Then summing over v ∈ ℤ^{n} satisfying the condition of (3.18), we get

|V_{(a,b)}(ℤ) ∩ [e^{t}, 2e^{t}]S^{n−1}| ≤ ∑_{v∈ℤ^{n}} c_{p,q}e^{(n−2)t} ∫_{K} f(a_{t}kxv) dm(k)
= c_{p,q}e^{(n−2)t} ∫_{K} f̃(a_{t}kx) dm(k),

where f̃ is defined in §3.1. By Lemma 3.1, f̃(g) ≤ c_{1}α(g) for some constant c_{1}. Hence if p ≥ 3 then by Theorem 3.2, ∫_{K} f̃(a_{t}kx) dm(k) is bounded for all t, uniformly for x ∈ C. Thus if p ≥ 3,

|V_{(a,b)}(ℤ) ∩ [T, 2T]S^{n−1}| ≤ c_{2}T^{n−2}

for some constant c_{2} > 0, and we have let T = e^{t}. If (p, q) = (2, 1) or (p, q) = (2, 2), then by Theorem 3.3,

|V_{(a,b)}(ℤ) ∩ [T, 2T]S^{n−1}| ≤ c_{3}T^{n−2} log T,

for some constant c_{3} > 0. Now the theorem follows from the compactness of Ω and an easy geometric series argument.

Proposition 3.7. Let f be a non-negative continuous function of compact support on ℝ_{+}^{n}, and ν a positive continuous function on S^{n−1}.

(i) Suppose p ≥ 3. Then for every ϵ > 0 and every x ∈ G there exists t_{0} > 0 so that for t > t_{0},

|e^{−(n−2)t} ∑_{v∈ℤ^{n}}(J_{f})(∥xv∥e^{−t}, Q_{0}(xv))ν(xv/∥xv∥) − c_{p,q} ∫_{K} f̃(a_{t}kx)ν(k^{−1}e_{1}) dm(k)| < ϵ. (3.19)

The constant t_{0} can be chosen uniformly as x varies over a compact subset of G.

(ii) If (p, q) = (2, 1) or (2, 2) then the equation (3.19) still holds for a fixed x ∈ G, any ϵ > 0 and t is sufficiently large, under the following additional assumption: there exists a non-negative continuous function f_{+} of compact support on ℝ_{+}^{n} with the support of f contained in the interior of the support of f_{+}, and

sup_{t>0} ∫_{K} f̃_{+}(a_{t}kx) dm(k) < ∞.

Proof. Suppose p ≥ 3. Since J_{f} has compact support, the number of non-zero terms in the sum on the left hand side of (3.19) is bounded by ce^{(n−2)t} because of Theorem 2.3. Then applying Lemma 3.6 with xv instead of v and ϵ/c instead of ϵ and summing over v ∈ ℤ^{n} we get (3.19). This proves (i).

In view of the proof of (i), to prove (ii) we need only show that the number of non-zero terms in the sum on the left hand side of (3.19) is O(e^{(n−2)t}). Note that the support of J_{f} is contained in the interior of the support of J_{f_{+}}; hence there exists ρ > 0 such that on the support of J_{f}, J_{f_{+}} > ρ > 0.

We apply Lemma 3.6 with ν = 1, xv instead of v, f_{+} instead of f, and ϵ < ρ/3, and sum over the set S = {v ∈ ℤ^{n} : J_{f}(∥xv∥e^{−t}, Q_{0}(xv)) ≠ 0}. We get

∑_{v∈S} J_{f_{+}}(∥xv∥e^{−t}, Q_{0}(xv)) ≤ c_{p,q}e^{(n−2)t} ∫_{K} f̃_{+}(a_{t}kx) dm(k) + ϵ|S|,  (3.20)

where |S| denotes the cardinality of S. By the assumption on f_{+} the integral in (3.20) is bounded, hence the right hand side of (3.20) is bounded by ϵ|S| + O(e^{(n−2)t}). Since each term in the sum in (3.20) is of size at least ρ, we get ρ|S| < ϵ|S| + O(e^{(n−2)t}). Since ρ > 2ϵ, this implies that |S| is indeed O(e^{(n−2)t}). This completes the proof of (ii).

Lemma 3.8.

(i) Suppose h is a continuous function of compact support on ℝ^{n} − {0} × ℝ. Then

lim_{T →∞} \frac{1}{T^{n−2}} ∫_{ℝ^{n}} h(v/T, Q_{0}(v)) dv =

= c_{p,q} ∫_{K} ∫_{−∞}^{∞} ∫_{0}^{∞} h(rk^{−1}e_{1}, ζ)r^{n−2}(dr/2r) dζ dm(k). (3.21)

The convergence is uniform as h varies over equicontinuous families of functions vanishing outside a given compact set.

(ii) Let ρ be a continuous positive function on the sphere, and let Ω = {v ∈ ℝ^{n} |∥v∥ < ρ(v/∥v∥)}. Then there exists a constant λ = λ_{Q,Ω} so that as T → ∞,

Vol(V_{(a,b)}^{Q}(ℝ) ∩ T Ω) ∼ λ_{Q,Ω}(b − a)T^{n−2}.

Proof. Write v = ∑_{i=1}^{n} v_{i}f_{i}, where the {f_{i}} is the basis defined in the proof of

Lemma 3.6. As in Lemma 3.6, let r_{1} = (∑_{i=1}^{p} v_{i}^{2})^{1/2}, r_{2} = (∑_{i=p+1}^{n} v_{i}^{2})^{1/2}. Then

Q_{0}(v) = r_{1}^{2} − r_{2}^{2}, and ∥v∥ = (r_{1}^{2} + r_{2}^{2})^{1/2}. Using polar coordinates (r_{1}, ω_{1}) and (r_{2}, ω_{2}) on the spaces span(f_{1}, …, f_{p}) and span(f_{p+1}, …, f_{n}) respectively, we get, for any function h of compact support on ℝ^{n} − {0} × ℝ,

∫_{ℝ^{n}} h(v/T, Q_{0}(v)) dv =

∫_{0}^{∞} ∫_{0}^{∞} ∫_{S^{p−1}×S^{q−1}} h (\frac{r_{1}ω_{1} + r_{2}ω_{2}}{T} , r_{1}^{2} − r_{2}^{2}) r_{1}^{p−1}r_{2}^{q−1} dω_{1} dω_{2} dr_{1} dr_{2}.

Since h has compact support in ℝ^{n} − {0}, for large T the integrand is zero unless r_{1} and r_{2} are comparable to T. Using the compactness of the support of h in the second variable we get that the integrand is zero unless r_{2} = r_{1} + O(1/T). Letting ζ = r_{1}^{2} − r_{2}^{2}, we see that dr_{1} dr_{2} = (1 + O(1/T))(dr_{1}/2r_{1})dζ and, (r_{1}ω_{1} + r_{2}ω_{2})/T = r_{1}(ω_{1} + ω_{2})/T + O(1/T). Hence

lim_{T →∞} \frac{1}{T^{n−2}} ∫_{ℝ^{n}} h (v/T, Q_{0}(v)) dv =

= lim_{T →∞} \frac{1}{T^{n−2}} ∫_{S^{p−1}×S^{q−1}} ∫_{−∞}^{∞} ∫_{0}^{∞} h (\frac{r_{1}(ω_{1} + ω_{2})}{T}, ζ) r_{1}^{n−2} \frac{dr_{1}}{2r_{1}} dζdω_{1} dω_{2}

= c_{p,q} ∫_{K} ∫_{−∞}^{∞} ∫_{0}^{∞} h(rk^{−1}e_{1}, ζ)r^{n−2}(dr/2r) dζ dm(k), where in the last line we have written (ω_{1} + ω_{2})/2^{1/2} = k^{−1}e_{1} and made the change of variable r = 2^{1/2}r_{1}/T. This proves (i).

We now prove (ii). Since we may write Q = Q_{0}^{x} and replace Ω by x^{−1}Ω, we may assume Q = Q_{0}. Let χ(v, ζ) denote the characteristic function of [1/2, 1]∂Ω×[a, b]. We claim that (3.21) holds for h = χ. Indeed, for any m > 1, we may choose continuous compactly supported functions h_{m}^{+} and h_{m}^{−}, such that 0 ≤ h_{m}^{−} ≤ χ ≤ h_{m}^{+} ≤ 1, h_{m}^{+} vanishes outside [1/2 − 1/m, 1 + 1/m]∂Ω × [a − 1/m, b + 1/m] and h_{m}^{−} = χ on [1/2 + 1/m, 1 − 1/m]∂Ω × [a + 1/m, b − 1/m]. Let L(h) denote the right hand side of (3.21). By part (i),

L(h_{m}^{−}) ≤ lim inf_{T →∞} \frac{1}{T^{n−2}} ∫_{ℝ^{n}} χ(v/T, Q_{0}(v)) dv ≤ lim sup_{T →∞} \frac{1}{T^{n−2}} ∫_{ℝ^{n}} χ(v/T, Q_{0}(v)) dv ≤ L(h_{m}^{+}).

From the definition of L(·) it is clear that lim_{m→∞} L(h_{m}^{+}) = lim_{m→∞} L(h_{m}^{−}) = L(χ). Hence (3.21) indeed holds for h = χ. Hence,

lim_{T →∞} \frac{1}{T^{n−2}} Vol(V_{(a,b)}^{Q_{0}}(ℝ) ∩ [T/2, T]∂Ω) = L(χ) = c(b − a).

Now (ii) follows from an easy geometric series argument.

Remark. One can verify that:

λ_{Q,Ω} = ∫_{L∩Ω} \frac{dA}{∥∇Q∥},

where L is the lightcone Q = 0 and dA is the area element on L.

Lemma 3.9. Suppose f is a continuous function of compact support on ℝ_{+}^{n}, ν a non-negative continuous function on S^{n−1}. Let h(v, ζ) = J_{f}(∥v∥, ζ)ν(v/∥v∥). Then

lim_{T →∞} \frac{1}{T^{n−2}} ∫_{ℝ^{n}} h(v/T, Q_{0}(v)) dv = c_{p,q} ∫_{G/Γ} f̃∫_{K} ν(k^{−1}e_{1}) dm(k).

The convergence is uniform as ν varies over equicontinuous families of functions and f varies over equicontinuous families of functions vanishing outside a given compact set in ℝ_{+}^{n}.

Proof. The Siegel integral formula is the identity:

∫_{G/Γ} f̃(g) dg = ∫_{ℝ^{n}} f(v) dv.  (3.22)

Also using the change of variable (x_{1}, …, x_{n}) → (x′_{1}, x′_{2}, …, x′_{n−1}, ζ), where x′_{j} = x_{j}, 1 ≤ j ≤ n − 1 and ζ = Q_{0}(x_{1}, …, x_{n}), we get (writing r for x′_{1}):

∫_{ℝ^{n}} f(v) dv = ∫_{−∞}^{∞} ∫_{0}^{∞} J_{f}(r, ζ)r^{n−2}(dr/2r) dζ.  (3.23)

Now the lemma follows from (3.22), (3.23) and Lemma 3.8 part (i), with h(v, ζ) = J_{f}(∥v∥, ζ)ν(v/∥v∥).

Proof of Theorems 2.1 and 2.5. We note that Theorem 2.1 is a special case of Theorem 2.5, and so we only prove the latter. By rescaling, we may assume that all the forms in D have discriminant 1. Let C be a compact subset of G such that every Q ∈ D is of the form Q_{0}^{x} for some x in C.

Let C denote the space of continuous functions on ℝ^{n} − {0} × ℝ vanishing outside a fixed compact set, with the topology of uniform convergence. Then by Lemma 3.8 part (i), the functional L on C given by

L(h) = lim_{T →∞} \frac{1}{T^{n−2}} ∫_{ℝ^{n}} h(v/T, Q_{0}(v)) dv

is continuous. From the proof of part (ii) of Lemma 3.8 it follows that if χ is the characteristic function of [1/2, 1]∂Ω × [a, b], then for every ϵ > 0 and every x ∈ C there exist functions h_{x,+} and h_{x,−} in C such that for all v, ζ

h_{x,−}(xv, ζ) ≤ χ(v, ζ) ≤ h_{x,+}(xv, ζ),  (3.24)

and

|L(h_{x,+}) − L(h_{x,−})| < ϵ.  (3.25)

Let L denote the space of finite linear combinations of functions on ℝ^{n} × ℝ of the form J_{f}(∥v∥, ζ)ν(v/∥v∥) where f is a continuous compactly supported function on ℝ_{+}^{n}, and ν a non-negative continuous function on S^{n−1}. By the definition it easily follows that every continuous compactly supported function on ℝ^{+} × ℝ vanishing outside a fixed compact set is of the form J_{f} for f a continuous compactly supported function on ℝ_{+}^{n}; indeed the mapping f ↦ J_{f} is essentially integration along the leaves of the foliation whose leaves are the hyperboloids Q_{0} = const. Then by the Stone-Weierstrass Theorem (cf. [Rud]) L is dense in the space C. Hence, using the compactness of C, we may assume that all h_{x,+}, h_{x,−} belong to some finite collection H of functions in L. By Proposition 3.7, Theorem 3.2 and Lemma 3.9 there exist points x_{1}, … x_{ℓ} in G/Γ so that Hx_{i} is closed for each i, and for each compact subset F of C − ⋃_{i=1}^{ℓ} Hx_{i} there exists t_{0} > 0 so that for all h ∈ H, and all x ∈ F

|e^{−(n−2)t} ∑_{v∈ℤ^{n}} h(xve^{−t}, Q_{0}(xv)) − L(h)| < ϵ,  (3.26)

and

|e^{−(n−2)t} ∫_{ℝ^{n}} h(xve^{−t}, Q_{0}(xv)) − L(h)| ≤ ϵ.  (3.27)

Combining (3.24), (3.25), (3.26) and (3.27) with a suitable ϵ, we see that for every θ > 0 there exist finitely many points x_{1}, …, x_{m} with Hx_{i} closed for each i, so that for every compact subset F of C − ⋃_{i=1}^{m} Hx_{i} there exists t_{0} > 0 such that for every x ∈ F and every t > t_{0},

(1 − θ) ∫_{ℝ^{n}} χ(ve^{−t}, Q_{0}(xv)) dv ≤ ∑_{v∈ℤ^{n}} χ(ve^{−t}, Q_{0}(xv)) ≤ (1 + θ) ∫_{ℝ^{n}} χ(ve^{−t}, Q_{0}(xv)) dv,

i.e.

(1 − θ) Vol(V_{(a,b)}^{Q}(ℝ) ∩ Ω_{t}) ≤ |V_{(a,b)}^{Q}(ℤ) ∩ Ω_{t}| ≤ (1 + θ) Vol(V_{(a,b)}^{Q}(ℝ) ∩ Ω_{t}),

where Ω_{t} = [e^{t}/2, e^{t}]∂Ω.

Let P = D ∩ {ℝQ_{0}^{x_{1}}, …, ℝQ_{0}^{x_{m}}} (for a quadratic form Q, ℝQ means the multiples of Q). By [DM3, Proposition 9.3] all the quadratic forms in P are multiples of rational forms, and for every compact subset F of D − P there exists a compact subset F of C − ⋃_{1≤i≤m} Hx_{i} such that every quadratic form Q ∈ F is a multiple of Q_{0}^{x} for some x ∈ F.

This implies that for sufficiently large T, for every Q ∈ F,

(1 − θ) Vol(V_{(a,b)}^{Q}(ℝ) ∩ T Ω) ≤ |V_{(a,b)}^{Q}(ℤ) ∩ T Ω| ≤ (1 + θ) Vol(V_{(a,b)}^{Q}(ℝ) ∩ T Ω).

To finish the proof of the theorem it only remains to remark that the asymptotics of the volume Vol(V_{(a,b)}^{Q}(ℝ) ∩ T Ω) are of the form (2.2) by Lemma 3.8 (ii).

Remark. To prove Theorem 2.1 and Theorem 2.5 for the counting function of primitive lattice points defined in Remark 2.6, one may argue as above but instead of f̃ one may use

f̂(g) = ∑_{v primitive^{v∈ℤ^{n}−0}} f(gv).

With this definition the analogue of (3.22) is

∫_{G/Γ} f̂(g) dg = \frac{1}{ζ(n)} ∫_{ℝ^{n}} f(v) dv.

3.6 Convergence a.e. in the (2, 1) and (2, 2) cases

Lemma 3.10. The function α on the space of unimodular lattices in ℝ^{m}, SL(m, ℝ)/SL(m, ℤ), belongs to every L^{μ}, 1 ≤ μ < m.

Proof. Let N denote the set of upper triangular matrices, and K the set of orthogonal matrices. For β ∈ ℝ, let A_{β} = {diag(a_{1}, …, a_{m}) : a_{i}/a_{i+1} < β}. One can choose a compact set Ω ⊂ N and a number β > 0 so that the Siegel set: 𝔖 = KA_{β}Ω contains a fundamental domain for the action of Γ on G (see [Bor1, p. 34] for a proof). Hence it is enough to show that the integral of α^{μ} over 𝔖 is finite. Note that for g = kan ∈ 𝔖, α(g) ≤ Cα(a). By its definition, α(a) = max_{1≤j≤m−1} α_{j}(a), where α_{j}(a) = ∏_{1≤i≤j} a_{i}^{−1}. It is convenient to pass to the Lie algebra by writing for 1 ≤ i ≤ m − 1, a_{i}/a_{i+1} = e^{−β_{i}}. Then on A_{β}, β_{i} ≥ − log β for all i. A straightforward calculation (or [BMP]) shows that α_{j} = exp ∑_{r=1}^{m−1} c_{jr}β_{r}, where the coefficients c_{jr} satisfy c_{jr} = c_{rj}, c_{jj} = j(n − j)/m, and c_{ij} ≤ c_{ii} for all i, j. By [Kna, §V.6], the measure of integration in KAN coordinates is e^{−2ρ} (∏_{r} dβ_{r}) dn dk, where 2ρ = ∑_{r=1}^{m−1} r(m − r)β_{r}. Hence for each j, 1 ≤ j ≤ m − 1, and 1 ≤ μ < m,

∫_{𝔖} α_{j}^{μ} = ∫_{K} ∫_{log A_{β}} ∫_{Ω} α_{j}(kan)^{μ} e^{−2ρ} dn dβ_{1} … dβ_{m−1} dk

≤ C′ ∏_{r=1}^{m−1} ∫_{− log β}^{∞} e^{(c_{jr}μ−r(m−r))β_{r}} dβ_{r} < ∞.

Since α = max_{j} α_{j}, the lemma follows.

In view of Lemma 3.10 and the results subsection §3.5 Theorem 2.4 can also be derived from the general pointwise ergodic theorem of [NS]. We choose to present a self-contained argument which uses additional properties of the function α.

We keep the notation of the previous subsections. For g ∈ G, we denote by d(g, 1) the distance in the symmetric space K̂\G between the point K̂g and the origin K̂. For 1 ≤ r ≤ ∞, we denote the L^{r} norm of a function f by ∥f∥_{r}.

Theorem 3.11 (Howe-Moore). Let G be a connected simple Lie group with finite center, and let Γ be a lattice in G. Let K̂ denote the maximal compact subgroup of G. Then there exists a constant λ > 0, so that the following holds: Let f_{1}, f_{2} ∈ L^{2}(G/Γ) be orthogonal to the constant function. Suppose either both f_{1} and f_{2} are smooth of compact support, or both f_{1} and f_{2} are left K̂ invariant. Then there exists a constant C > 0, depending on f_{1}, f_{2} so that

∫_{G/Γ} f_{1}(gx)f_{2}(x) dx ≤ Ce^{−λd(g,1)}.

Proof. See [How],[KM, Theorem A.5], [KS].

Lemma 3.12. Suppose t > 0, and let x, y, e be points in ℍ^{2} with d(x, e) = d(y, e) = t. Let θ = θ_{e}(x, y) denote the visual angle between x and y as viewed from e. Then for any β with 0 < β < 1 there exists t_{0} > 0 such that if θ ≥ e^{−βt} and t > t_{0}, then d(x, y) ≥ βt.

Proof. The hyperbolic law of cosines [Bea] yields

cosh(d(x, y)) = 1 + sinh^{2} t · (1 − cos θ)
= 1 + 2(sinh^{2} t) sin^{2}(θ/2).

Hence, (assuming e^{−βt} < π),

d(x, y) ≥ cosh^{−1} (1 + 2(sinh^{2} t) sin^{2}(e^{−βt}/2)) .  (3.28)

Now the desired estimate easily follows from the fact that the right hand side of (3.28) is asymptotic to (2 − β)t as t → ∞.

Proposition 3.13. Suppose (p, q) = (2, 1) or (2, 2). Let ϕ be a continuous function on G/Γ ≈ the space of lattices in ℝ^{n} with determinant 1. Assume that for some s, 0 < s < n/2 and some C > 0,

|ϕ(Δ)| < Cα(Δ)^{s}, for all Δ ∈ G/Γ.

Let ν be any bounded function on K. Then for almost all x_{0} ∈ G/Γ,

lim_{t→+∞} ∫_{K} ϕ(a_{t}kx_{0})ν(k) dm(k) = ∫_{K} ν dm ∫_{G/Γ} ϕ(y) dμ(y).

Proof. Any continuous function on G/Γ can be approximated in C(G/Γ) from above and below by smooth functions. Hence we may assume the function ϕ is smooth. We may also assume ν is positive.

Note that for any g ∈ G and any lattice Δ, \frac{α_{i}(gΔ)}{α_{i}(Δ)} ≤ ∥ ⋀^{i}(g)∥. Thus log α is Lipschitz, hence uniformly continuous.

Let ϵ > 0 be arbitrary. Let A(r) = {x ∈ G/Γ : α(x) > r}. Choose a smooth nonnegative function g_{r} on G/Γ such that g_{r}(x) = 1 if x ∈ A(r + 1), g_{r}(x) = 0 if x ∉ A(r) and 0 ≤ g_{r}(x) ≤ 1 if x ∈ A(r) − A(r + 1). Let ϕ_{1} = ϕg_{r} and ϕ_{2} = ϕ − ϕg_{r} so that ϕ = ϕ_{1} + ϕ_{2}. By Lemma 3.10 we can choose θ > 0 so that α^{s+θ} ∈ L^{2}(G/Γ). Then ϕ_{1}(x) ≤ Cα(x)^{s}g_{r}(x) = Cα(x)^{s+θ}g_{r}(x)α(x)^{−θ} ≤ Cr^{−θ}α(x)^{s+θ}. Let ψ(x) = Cr^{−θ}α(x)^{s+θ}, so that |ϕ_{1}| ≤ ψ and ψ ∈ L^{2}(G/Γ). The function log α^{s+θ} is uniformly continuous, hence there exists a constant M > 0 so that for all x ∈ G/Γ and all τ ∈ [0, 1], α^{s+θ}(a_{τ}x) ≤ Mα^{s+θ}(x), hence ψ(a_{τ}x) ≤ Mψ(x). By choosing r sufficiently large we may make ∥ψ∥_{1} ≤ ϵ/6M.

For any function f on G/Γ, let (A_{t}f)(x) = ∫_{K} f(a_{t}kx)ν(k) dm(k). Suppose f ∈ L^{2} is orthogonal to the constants, and f is either smooth of compact support or left Kinvariant. Then by Theorem 3.11, there exists λ > 0 and C = C_{f} > 0, so that for any g ∈ G,

|∫_{G/Γ} f(gx)f(x) dx| ≤ C_{f}e^{−λd(g,1)}.  (3.29)

Now

∥A_{t}f∥_{2}^{2} = ∫_{G/Γ} ∫_{K×K} f(a_{t}k_{1}x)f(a_{t}k_{2}x)ν(k_{2})ν(k_{1}) dm(k_{2}) dm(k_{1}) dx =

= ∫_{K} (∫_{G/Γ} f(a_{t}ka_{t}^{−1}x)f(x) dx) (ν ∗ ν̌)(k) dm(k), (3.30)

where ν̌ is the “inverted” function ν̌(k) = ν(k^{−1}), and ν ∗ ν̌ is the convolution (ν ∗ ν̌)(k) = ∫_{K} ν(kk′)ν(k′) dm(k′).

Fix β with 0 < β < 1, and let U = {k ∈ K : d(a_{t}ka_{t}^{−1}, 1) ≤ βt}. By Lemma 3.12, m(U) ≤ e^{−βt}, Hence, by the Cauchy-Schwartz inequality,

|∫_{U} ∫_{G/Γ} f(a_{t}ka_{t}^{−1}x)f(x) dx (ν ∗ ν̌)(k) dm(k)| ≤ m(U) max(ν ∗ ν̌)∥f∥_{2}^{2}

≤ e^{−βt} max(ν ∗ ν̌)∥f∥_{2}^{2}. (3.31)

If k ∉ U, then d(a_{t}ka_{t}^{−1}, 1) ≥ βt, and thus, by (3.29),

|∫_{K−U} ∫_{G/Γ} f(a_{t}ka_{t}^{−1}x)f(x) dx(ν ∗ ν̌)(k) dm(k)| ≤ C_{f}e^{−λβt} max(ν ∗ ν̌). (3.32)

Combining (3.30) (3.31) and (3.32) we get for sufficiently large t,

∥A_{t}f∥_{2}^{2} ≤ (e^{−βt}∥f∥_{2}^{2} + C_{f}e^{−λβt}) max(ν ∗ ν̌) < e^{−λ′t},  (3.33)

where λ′ is independent of t. Hence

|\{x ∈ G/Γ : A_{t}ψ(x) > \frac{ϵ}{3M} \}| ≤ |\{x ∈ G/Γ : |A_{t}ψ − ∫ ψ ∫ ν| > \frac{ϵ}{6M} \}|

≤ (\frac{6M}{ϵ} )^{2} e^{−λ′t}, (3.34)

where |·| denotes Haar measure on G/Γ. Note that A_{t+τ}ψ < MA_{t}ψ, for all 0 ≤ τ ≤ 1. Thus, since |ϕ_{1}| ≤ ψ,

|\{x ∈ G/Γ : |A_{t+τ}ϕ_{1}(x)| > \frac{ϵ}{3} for some τ ∈ [0, 1]\}| ≤ (\frac{6M}{ϵ} )^{2} e^{−λ′t}. (3.35)

Since ϕ_{2} is uniformly continuous, there exists δ > 0 such that |A_{t+τ}ϕ_{2}(x)−A_{t}ϕ_{2}(x)| < ϵ/3 for all t > 0, x ∈ G/Γ and τ ∈ [0, δ]. Hence using (3.33),

|\{x ∈ G/Γ : |A_{t+τ}ϕ_{2} − ∫ ϕ_{2} ∫ ν| > \frac{2ϵ}{3} for some τ ∈ [0, δ]\}| < (\frac{3}{ϵ})^{2} e^{−λ′t}.

(3.36)

Let

E_{t} = {x ∈ G/Γ : |(A_{t+τ}ϕ)(x) − ∫ ϕ ∫ ν| > ϵ for some τ ∈ [0, δ]}.

Then, combining (3.35) and (3.36) we get

|E_{t}| ≤ Ce^{−λ′t},

where C is independent of t. Hence,

| ⋂_{m=1}^{∞} ⋃_{n=m}^{∞} E_{nδ}| = 0,

which is equivalent to the statement of the proposition.

Remark. One can easily modify the proof to handle signatures other than (2, 1) and (2, 2), and s with 0 ≤ s < n.

Proof of Theorem 2.4. Theorem 2.4 follows from Proposition 3.13, Proposition 3.7 part (ii) and Lemma 3.9; the argument is similar to the argument used in the proof of Theorem 2.1 on page 18.

3.7 Counterexamples.

In this subsection we prove Theorem 2.2. For α > 0, let Q_{α}(x_{1}, x_{2}, x_{3}) denote the quadratic form x_{1}^{2} + x_{2}^{2} − α^{2}x_{3}^{2}, and let Q̂_{α}(x_{1}, x_{2}, x_{3}, x_{4}) denote the quadratic form x_{1}^{2} + x_{2}^{2} − α^{2}(x_{3}^{2} + x_{4}^{2}). The quadratic forms we will construct to prove Theorem 2.2 will be of the form Q_{α} if n = 3, and Q̂_{α} if n = 4.

The following lemma is well known, and can be deduced from the properties of the divisor function:

Lemma 3.14. Suppose α is rational. Then

|{x ∈ ℤ^{3} | Q_{α}(x) = 0, ∥x∥ ≤ T }| ∼ c_{α}T log T,  (3.37)

and

|{x ∈ ℤ^{4} | Q̂_{α}(x) = 0, ∥x∥ ≤ T }| ∼ c_{α}T^{2} log T.  (3.38)

Lemma 3.15. Suppose [a, b] = [1/4, 1] or [a, b] = [−1, −1/4]. Then for every ϵ > 0 and S > 0, the set of β ∈ ℝ for which there exists a T > S such that

|V_{(a,b)}^{Q_{β}}(ℤ) ∩ T Ω_{0})| ≥ T (log T)^{1−ϵ}

is dense. Similarly, the set of β ∈ ℝ for which there exists a T > S such that

|V Q̂_{β} ; (a,b)(ℤ) ∩ T Ω_{0})| ≥ T^{2} (log T)^{1−ϵ}

is dense.

Proof of Lemma 3.15. For α rational and T > 0, let L(α, T) denote the set of x ∈ ℤ^{3} such that Q_{α}(x) = 0 and T/2 ≤ ∥x∥ ≤ T. Then by (3.37) for sufficiently large T (depending on α), the cardinality of L(α, T) is at least (c_{α}/4)T log T > T (log T)^{1−ϵ}. Note that if (x_{1}, x_{2}, x_{3}) ∈ L(α, T), then

\frac{T^{2}}{4(1 + α^{2})} ≤ x_{3}^{2} ≤ \frac{T^{2}}{1 + α^{2}} ,  (3.39)

so that if

β^{2} = α^{2} − \frac{1 + α^{2}}{T^{2}} ,  (3.40)

then for x ∈ L(α, T),

Q_{β}(x) = Q_{α}(x) + \frac{1 + α^{2}}{T^{2}} x_{3}^{2}

satisfies (1/4) ≤ Q_{β}(x) ≤ 1 by (3.39). Hence L(α, T) ⊂ V_{(1/4,1)}^{Q_{β}}(ℤ) ∩ T Ω_{0}, and hence the cardinality of the latter is at least T (log T)^{1−ϵ}. Similarly, if we choose

β^{2} = α^{2} + \frac{1 + α^{2}}{T^{2}} ,  (3.41)

then the cardinality of V_{(−1,−1/4)}^{Q_{β}}(ℤ)∩T Ω_{0}) is at least T (log T)^{1−ϵ}. The set of numbers β satisfying (3.40) or (3.41) for rational α and sufficiently large T is clearly dense. The proof for the (2, 2) case is identical.

Remark. The above proof can be interpreted geometrically as follows: The light cone of the form Q_{β} is slightly wider (resp. narrower) then the light cone of Q_{α}, so that the integral points L(α, T) which are on the light cone of Q_{α} are inside (resp. outside) the light cone of Q_{β}.

Proof of Theorem 2.2. Since the interval [a, b] must intersect either the positive reals or the negative reals, we may assume after passing to a subset and rescaling that e.g. [a, b] = [1/8, 2] or [a, b] = [−2, −1/8]. We assume the former for notational simplicity; the latter case is proved identically. For a given S > 0 let U_{S} denote the set of γ ∈ ℝ for which there exists β ∈ ℝ and T > S with

|V_{(1/4,1)}^{Q_{β}}(ℤ) ∩ T Ω_{0}| ≥ T (log T)^{1−ϵ},  (3.42)

and

|β − γ| < T^{−3}.  (3.43)

Then U_{S} is open and dense by Lemma 3.15. By the Baire category theorem, the intersection ∩_{j=1}^{∞}U_{2^{j}} is not empty; in fact it is of second category so uncountable. Let γ be an irrational point in this intersection. Then there exist infinite sequences β_{j}, T_{j} satisfying (3.42) and (3.43), with β replaced by β_{j} and T replaced by T_{j}. The equation (3.43) implies that for ∥x∥ < T_{j}, |Q_{β_{j}}(x) − Q_{γ}(x)| < c/T_{j} < 1/8, so that

V_{(1/8,2)}^{Q_{γ}}(ℤ) ∩ T_{j}Ω_{0} contains V_{(1/4,1)}^{Q_{β_{j}}}(ℤ) ∩ T_{j}Ω_{0}. Thus the cardinality of the former is at least T_{j}(log T_{j})^{1−ϵ} by (3.42).

Remarks. We have actually proved that the set of γ ∈ ℝ for which Theorem 2.2 holds for Q_{γ} is of second category in the Baire sense. Also from the proof it is clear that in the statement of Theorem 2.2, (log T)^{1−ϵ} can be replaced by log T/ν(T) where ν(T) is any unbounded increasing function. One can also obtain results analogous to Theorem 2.2 for the function |L′_{Q}(T Ω_{0}, [a, b])| counting primitive lattice points, but the construction for the SO(2, 1) case must be modified: instead of (3.37) one can use the fact that for rational α, the number of integral points on the one-sheeted hyperboloid x^{2} + y^{2} − α^{2}z^{2} = 1 with x^{2} + y^{2} + z^{2} < T^{2} grows like T log T. (However the number of integral points x on any rational two-sheeted hyperboloid satisfying ∥x∥ < T is asymptotic to a constant times T). We omit the details.

4 Some results on the equidistibution of translates

Consider the following situation: Let G be a connected Lie group, Γ a lattice in G, H a connected semisimple subgroup of G, and K a maximal compact subgroup of H. In this section we study the question of whether large spheres in H (and more generally, sets of the form a_{t}Kx where x ∈ G and A = {a_{s} | s ∈ ℝ} is a suitable oneparameter subgroup of H) are equidistributed in the quotient G/Γ. In our application, G = SL(n, ℝ), Γ = SL(n, ℤ), H ≅ SO(p, q) where p ≥ q ≥ 1, p + q = n, and K ⊂ H is isomorphic to SO(p) × SO(q). Theorem 4.4 below states that if one restricts to bounded continuous test functions ϕ, then under certain conditions on H and K and A (which are satisfied for the application) the answer is yes, as long Hx is not closed. (the case where Hx is closed corresponds to a quadratic form which is a multiple of a rational form). Furthermore, Theorem 4.4 implies that if one fixes a test function ϕ, then the convergence is uniform on compact sets away from a finite number of closed H orbits.

The following theorem of Dani and Margulis answers the analogous question for averages along orbits of one-parameter unipotent groups, instead of over spheres in a semisimple subgroup H. The “uniformity” of the convergence as one varies the base point x is crucial for the application.

For any closed subgroups H and W of G let X(H, W) = {g ∈ G | W g ⊂ gH}.

Theorem 4.1 ([DM3], Theorem 3). Let G be a connected Lie group and let Γ be a lattice in G. Let μ be the G-invariant probability measure on G/Γ. Let U = {u_{t}} be an Ad-unipotent one-parameter subgroup of G and let ϕ be a bounded continuous function on G/Γ. Let D be a compact subset of G/Γ and let ϵ > 0 be given. Then there exist finitely many proper closed subgroups H_{1} = H_{1}(ϕ, D, ϵ), ⋯, H_{k} = H_{k}(ϕ, D, ϵ) such that H_{i} ∩Γ is a lattice in H_{i} for all i, and compact subsets C_{1} = C_{1}(ϕ, D, ϵ), ⋯, C_{k} = C_{k}(ϕ, D, ϵ) of X(H_{1}, U), ⋯, X(H_{k}, U) respectively, for which the following holds: For any compact subset F of D − ⋃_{1≤i≤k} C_{i}Γ/Γ there exists a T_{0} ≥ 0 such that for all x ∈ F and T > T_{0}

|\frac{1}{T} ∫_{0}^{T} ϕ(u_{t}x) dt − ∫_{G/Γ} ϕ dμ| < ϵ.

Remark. It can be easily checked that in Theorem 4.1 the assumption that ϕ is bounded can not be weakened to ϕ ∈ L^{1}, even if we assume that log ϕ is uniformly continuous. For example if G = SL(2, ℝ), Γ = SL(2, ℤ), and U = \{u_{t}\} = ( \begin{matrix}1 & t \\ 0 & 1 \end{matrix} ) the function α of §3.1 belongs to L^{2−ϵ} for every ϵ > 0; yet there exists a point x ∈ G such that the orbit Ux is dense in G/Γ, but the averages

\frac{1}{T} ∫_{0}^{T} α(u_{t}x) dt

are unbounded. Since in our application we need to integrate an unbounded function in L^{1}, this presents a difficulty, which is handled in §5.

In the rest of this section, we deduce Theorem 4.4, from Theorem 4.1. This relies on the the well known fact that in the hyperbolic plane large circles are well approximated by horocyles. Theorem 4.4 immediately implies Theorem 3.5 for functions of compact support.

Lemma 4.2. Let K be a compact group with the normalized Haar measure m acting continuously on a locally compact second countable space X. Let A ⊂ X be a closed subset and F ⊂ X a compact subset. Assume that

m({k ∈ K | kx ∈ A}) = 0

for any x ∈ F. Then for any ϵ > 0 one can find an open subset U_{ϵ} ⊂ X containing A such that

m({k ∈ K | kx ∈ U_{ϵ}}) < ϵ,

for any x ∈ F.

Proof. Since m({k ∈ K | kx ∈ A}) = 0, x ∈ F, there exist a continuous function f_{x} on X, an open subset W_{x}, A ⊂ W_{x} ⊂ X, and a neighborhood V_{x} of x such that 0 ≤ f_{x}(y) ≤ 1 for any y ∈ X, f_{x}(w) = 0 for any w ∈ W_{x} and

∫_{K} f_{x}(kv) dm(k) > 1 − ϵ

for any v ∈ V_{x}. The subset F is compact. Therefore one can find a finite number of points x_{1}, ⋯, x_{k} ∈ F such that F ⊂ ⋃_{1≤i≤k} V_{x_{i}}. Let us denote the maximum of functions f_{i}, 1 ≤ i ≤ k, by f. Then 0 ≤ f(y) ≤ 1 for any y ∈ X, f(w) = 0 for any

w ∈ U_{ϵ} =^{def} ⋂_{1≤i≤k} W_{x_{i}} and

∫_{K} f(kx)dm(k) > 1 − ϵ

for any x ∈ F. It remains to notice now that

m({k ∈ K | kx ∈ U_{ϵ}}) ≤ 1 − ∫_{K} f(kx) dm(k),

because χ_{Uϵ} ≤ 1 − f where χ_{Uϵ} denotes the characteristic function of U_{ϵ}.

Theorem 4.3. (cf. [DM3, Proposition 9.4]) Let G, Γ, μ, U = {u_{t}}, ϕ, D and ϵ > 0 be the same as in Theorem 4.1. Let H be a connected semisimple subgroup of G and let K be a maximal compact subgroup of H with the normalized Haar measure m. Assume that the following conditions are satisfied:

(a) H has finite center and has no compact factors;

(b) U ⊂ H and U is not contained in any proper normal subgroup of H;

(c) H is a maximal proper connected subgroup of G and H has finite index in the normalizer N_{G}(H).

Then there exist finitely many points x_{1}, ⋯, x_{ℓ} ∈ G/Γ such that

(I) the orbits Hx_{1}, ⋯, Hx_{ℓ} are closed and have finite H-invariant measure;

(II) for any compact subset F of D − ⋃_{1≤i≤ℓ} Hx_{i} there exists T_{0} such that for all x ∈ F and T > T_{0}

m\{k ∈ K| |\frac{1}{T} ∫_{0}^{T} ϕ(u_{t}kx) dt − ∫_{G/Γ} ϕ dμ| > ϵ\} < ϵ,  (4.1)

and, consequently,

|\frac{1}{T} ∫_{0}^{T} ∫_{K} ϕ(u_{t}kx)q(k) dt dm(k) − ∫_{G/Γ} ϕ dμ ∫_{K} q dm| ≤

≤ ∫_{K} |q(k)| |\frac{1}{T} ∫_{0}^{T} ϕ(u_{t}kx) dt − ∫_{G/Γ} ϕ dμ| dm(k) ≤

≤ ϵ sup_{k∈K} |q(k)| (1 + 2 sup_{y∈G/Γ} |ϕ(y)|),

(4.2)

for any bounded measurable function q on K.

Proof. Let us consider subgroups H_{1} = H_{1}(ϕ, KD, ϵ), ⋯, H_{k} = H_{k}(ϕ, KD, ϵ) and subsets C_{1} = C_{1}(ϕ, KD, ϵ), ⋯, C_{k} = C_{k}(ϕ, KD, ϵ) defined in Theorem 4.1, and let us define subsets

Y_{i} = {g ∈ G | Kg ⊂ X(H_{i}, U)}, 1 ≤ i ≤ k.

The subgroup S generated by ⋃_{k∈K} k^{−1}Uk is normalized by U ∪ K. Since H is semisimple with finite center, any connected subgroup L of H containing K can be represented as L = H′K where H′ is a connected normal subgroup of H. Therefore conditions (a) and (b) imply that S = H. Let y ∈ Y_{i}. Then Uky ⊂ kyH_{i}, or equivalently, k^{−1}Uk ⊂ yH_{i}y^{−1} for any k ∈ K. This and the equality S = H imply that H ⊂ yH_{i}y^{−1}. Now using condition (c) we get

H = yH_{i}^{o}y^{−1} for any y ∈ Y_{i},  1 ≤ i ≤ k,  (4.3)

where H_{i}^{o} denotes the connected component of identity of H_{i}. We deduce from (4.3) that y_{1}^{−1}y_{2} ∈ N_{G}(H) for any y_{1}, y_{2} ∈ Y_{i}. Now using (4.3) and condition (c) we get that Y_{i} can be covered by a finite number of right cosets of H. Hence there exist finitely many points x_{1}, ⋯, x_{ℓ} ∈ G/Γ such that

⋃_{1≤i≤k} Y_{i}Γ/Γ ⊂ ⋃_{1≤j≤ℓ} Hx_{j}.  (4.4)

Since H_{i} ∩ Γ is a lattice in H_{i} we get from (4.3) that Hx_{j}, 1 ≤ j ≤ ℓ, carries finite H-invariant measure. Thus it remains to check property (II).

Since X(H_{i}, U) is a real analytic submanifold of G and the group K is connected we have that m({k ∈ K | kg ∈ X(H_{i}, U)γ}) = 0 for any g ∈ G − Y_{i}γ, γ ∈ Γ. Therefore,

m({k ∈ K | kg ∈ ⋃_{1≤i≤k} X(H_{i}, U)Γ}) = 0

for any g ∈ G − ⋃_{1≤i≤k} Y_{i}Γ. But C_{i} ⊂ X(H_{i}, U) and F ⊂ D − ⋃_{1≤j≤ℓ} Hx_{j}. This and (4.4) imply that

m({k ∈ K | kx ∈ ⋃_{1≤i≤k} C_{i}Γ/Γ}) = 0

for any x ∈ F. Now applying Lemma 4.2 we get that there exists an open subset W ⊂ G/Γ, W ⊃ ⋃_{1≤i≤k} C_{i}Γ/Γ such that m({k ∈ K | kx ∈ W }) < ϵ for any x ∈ F. It remains to notice that since C_{i} = C_{i}(ϕ, KD, ϵ) there exists T_{0} ≥ 0 such that

|\frac{1}{T} ∫_{0}^{T} ϕ(u_{t}y) dt − ∫_{G/Γ} ϕ dμ| < ϵ

for all y ∈ KD − W and T > T_{0}.

Theorem 4.4. Let G be a connected Lie group, Γ a lattice in G, μ the G-invariant probability measure on G/Γ, ϕ a bounded continuous function on G/Γ, D a compact subset of G/Γ, H a connected semisimple subgroup of G, K a maximal compact subgroup of H with the normalized Haar measure m, and A = {a_{s} | s ∈ ℝ} a one-parameter subgroup of H. Assume that H satisfies conditions (a) and (c) of Theorem 4.3. Let us assume also that

(a) A is not contained in any proper normal subgroup of H; (b) there exists a continuous homomorphism with finite kernel ρ : SL_{2}(ℝ) → H

such that ρ(D) = A and ρ(SO(2)) ⊂ K where D = \{d_{t} = ( \begin{matrix}t & 0 \\ 0 & t^{−1} \end{matrix} ) |t > 0\}.

Then for any ϵ > 0 and any bounded measurable function q on K there exist finitely many points x_{1}, ⋯, x_{ℓ} ∈ G/Γ such that

(I) the orbits Hx_{1}, ⋯, Hx_{ℓ} are closed and have finite H-invariant measure;

(II) for any compact subset F of D − ⋃_{1≤i≤ℓ} Hx_{i} there exists S_{0} > 0 such that for all x ∈ F and s > S_{0}

| ∫_{K} ϕ(a_{s}kx)q(k) dm(k) − ∫_{G/Γ} ϕ dμ ∫_{K} q dm| ≤ ϵ.

Proof. Since ϕ is bounded and continuous and q is bounded, there exist bounded uniformly continuous functions ϕ′_{n} and ϕn^{′′} on G/Γ and q_{n} on K such that ϕ′_{n} ≤ ϕ ≤

ϕn^{′′}, lim_{n→∞} ∫_{G/Γ}(ϕn^{′′} − ϕ′_{n}) dμ = 0 and lim_{n→∞} ∫_{K} |q_{n} − q| dm(k) = 0. Hence we can assume

that ϕ and q are uniformly continuous (with respect to the metric induced by a left invariant metric on G).

Let us denote ( \begin{matrix}1 & t \\ 0 & 1 \end{matrix} ) by u_{t} and ( \begin{matrix}0 & −1 \\ 1 & 0 \end{matrix} ) by w. Then simple calculations show that

d_{t} = b_{t}u_{t}k_{t}w,

where

b_{t} = \frac{1}{\sqrt{1 + t^{−2}}} (\begin{matrix}1 & 0 \\ −t^{−1} & 1 + t^{−2} \end{matrix}) k_{t} = \frac{1}{\sqrt{1 + t^{−2}}} (\begin{matrix}1 & t^{−1} \\ −t^{−1} & 1 \end{matrix}) .

Then k_{t} ∈ SO(2) and

lim_{t→+∞} b_{t} = lim_{t→+∞} k_{t} = e.  (4.5)

Let us denote ρ(d_{t}), ρ(b_{t}), ρ(u_{t}), ρ(k_{t}) and ρ(w) by d′_{t}, b′_{t}, u′_{t}, k′_{t} and w′ respectively. Then for any t > 0 and x ∈ G/Γ

\begin{matrix}∫_{K} ϕ(d′_{t}kx)q(k) dm(k) = ∫_{K} ϕ(b′_{t}u′_{t}k′_{t}w′kx)q(k) dm(k) = \\ = ∫_{K} ϕ(b′_{t}u′_{t}k′_{t}w′kx)q(k) dm(k′_{t}w′k) = \\ = ∫_{K} ϕ(b′_{t}u′_{t}kx)q(w^{′−1}k_{t}^{′−1}k) dm(k). \end{matrix}  (4.6)

It follows from (4.6) that for any positive r and t

\begin{matrix}| ∫_{K} ϕ(d′_{t}kx)q(k) dm(k) − ∫_{K} ϕ(u′_{rt} kx)q(w^{′−1}k)dm(k)| ≤ \\ ≤ & | ∫_{K} (ϕ(d′_{rt}kx) − ϕ(d′_{t}kx))q(k) dm(k)| + \\ + & | ∫_{K} (ϕ(d′_{rt}kx) − ϕ(u′_{rt}kx))q(w^{′−1}k) dm(k)| = \\ = & | ∫_{K} (ϕ(d′_{r}d′_{t}kx) − ϕ(d′_{t}kx))q(k) dm(k)| + \\ + & | ∫_{K} ϕ(b′_{rt}u′_{rt}kx)q(w^{′−1}k_{rt}^{′−1}k) − ϕ(u′_{rt}kx)q(w^{′−1}k) dm(k)|. \end{matrix}  (4.7)

The functions ϕ and q are uniformly continuous. Therefore we can deduce from (4.5) and (4.7) the existence of T_{1} > 0 and α > 0 such that

| ∫_{K} ϕ(d′_{t}kx)q(k) dm(k) − ∫_{K} ϕ(u′_{rt}kx)q(w^{′−1}k) dm(k)| ≤ \frac{ϵ}{2},

whenever t > T_{1} and |r − 1| < α. Hence

| ∫_{K} ϕ(d′_{t}kx)q(k) dm(k) − \frac{1}{αT} ∫_{T}^{(1+α)T} ∫_{K} ϕ(u′_{t}kx)q(w^{′−1}k) dm(k) dt| ≤ \frac{ϵ}{2},  (4.8)

if T > T_{1}. Let us denote the one-parameter subgroup {u′_{t}|t ∈ ℝ} by U. Any proper normal subgroup of ρ(SL_{2}(ℝ)) is finite. But A is not contained in any proper normal subgroup of H and U ⊂ ρ(SL(2, ℝ)). Hence U is not contained in any proper normal subgroup of H. Thus conditions (a) - (c) of Theorem 4.3 are satisfied. Let x_{1}, ⋯, x_{ℓ} ∈ G/Γ be points defined as in Theorem 4.3. Then property (I) is true and for any compact subset F of D − ⋃_{1≤i≤ℓ} Hx_{i} there exists T_{2} > 0 such that for all x ∈ F and T > T_{2}

|\frac{1}{αT} ∫_{T}^{(1+α)T} ∫_{K} ϕ(u′_{t}kx)q(w^{′−1}k) dm(k) − ∫_{G/Γ} ϕ dμ ∫_{K} q dm| ≤ \frac{ϵ}{2}  (4.9)

(we use that ∫_{K} q(w^{′−1}k) dm(k) = ∫_{K} q(k) dm(k)). Combining (4.8) and (4.9) we get

for all x ∈ F and t > max{T_{1}, T_{2}}

| ∫_{K} ϕ(ρ(d_{t})kx)q(k) dm(k) − ∫_{G/Γ} ϕ dμ ∫_{K} q dm| ≤ ϵ.  (4.10)

Let ρ′(y) = ρ(wyw^{−1}), y ∈ SL_{2}(ℝ). Then ρ′(d_{t}) = a_{−s} if ρ(d_{t}) = a_{s}. Now it remains to recall that ρ(D) = A and to apply (4.10) to the homomorphism ρ and ρ′.

The following theorem is the specialization of Theorem 4.4 to the case considered in §3.

Theorem 4.5. Let {a_{s}|s ∈ ℝ} be a self-adjoint one-parameter subgroup of SO(2, 1). Let p ≥ 2 and q ≥ 1. Denote p + q by n, SO(p, q) by H, SL(n, ℝ) by G, and SO(p) × SO(q) by K. Let Γ be a lattice in G and let ϕ be any continuous function f on G/Γ vanishing outside a compact set. Then for any ϵ > 0 and any bounded measurable function q on K and every compact subset D of G/Γ there exist finitely many points x_{1}, ⋯, x_{ℓ} ∈ G/Γ such that

(I) the orbits Hx_{1}, ⋯, Hx_{ℓ} are closed and have finite H-invariant measure;

(II) for any compact subset F of D − ⋃_{1≤i≤ℓ} Hx_{i} there exists S_{0} > 0 such that for all x ∈ F and s > S_{0}

| ∫_{K} ϕ(a_{s}kx)q(k) dm(k) − ∫_{G/Γ} ϕ dμ ∫_{K} q dm| ≤ ϵ.

Proof. Note that H^{0} ≅ SO(p, q)^{0} is a maximal proper connected subgroup of SL(n, ℝ). Also if H_{1} ⊂ H^{0} is the subgroup of H^{0} fixing {e_{3}, …, e_{n−1}}, then H_{1} ≅ SO(2, 1), {a_{t}} ⊂ H_{1}, and there is a homomorphism ρ from SL(2, ℝ) to H_{1} ⊂ H^{0} so that ρ ( \begin{matrix}e^{t/2} & 0 \\ 0 & e^{−t/2} \end{matrix} ) = a_{t}. Hence the conditions (a) and (b) of Theorem 4.4 are satisfied, with H^{0} in place of H. Thus the conclusions of Theorem 4.4 hold for H^{0}, hence for H.

5 The upper bounds

In this section we prove Theorem 3.2 and Theorem 3.3. We recall the notation from §3: G is SL(n, ℝ), Γ = SL(n, ℤ), K̂ ≅ SO(n) is a maximal compact subgroup of G, H ≅ SO(p, q) ⊂ G, K = H ∩ K̂ is a maximal compact subgroup of H, and X is the symmetric space K\H. From its definition (3.3), the function α(Δ) is the maximum over 1 ≤ i ≤ n of K̂ invariant functions α_{i}(Δ). The main idea of the proof is to show that the α_{i}^{s} satisfy a certain system of integral inequalities which imply the desired bounds.

If p ≥ 3 and 0 < s < 2, or if (p, q) = (2, 1) or (2, 2) and 0 < s < 1, we show that for any c > 0 there exist t > 0, and ω > 1 so that the the functions α_{i}^{s} satisfy the following system of integral inequalities in the space of lattices:

A_{t}α_{i}^{s} ≤ c_{i}α_{i}^{s} + ω^{2} max_{0<j≤min(n−i,i)} \sqrt{α_{i+j}^{s}α_{i−j}^{s}},  (5.1)

where A_{t} is the averaging operator (A_{t}f)(Δ) = ∫_{K} f(a_{t}kΔ), and c_{i} ≤ c (Lemma 5.7). If (p, q) = (2, 1) or (2, 2) and s = 1, then (5.1) also holds (for suitably modified functions α_{i}), but some of the constants c_{i} cannot be made smaller than 1.

Let f_{i}(h) = α_{i}(hΔ), so that each f_{i} is a function on the symmetric space X. When one restricts to an orbit of H, (5.1) becomes:

A_{t}f_{i}^{s} ≤ c_{i}f_{i}^{s} + ω^{2} max_{0<j≤min(n−i,i)} \sqrt{f_{i+j}^{s}f_{i−j}^{s}}.  (5.2)

If rank X = 1, then (A_{t}f)(h) can be interpreted as the average of f over the sphere of radius 2t in X, centered at h. In §5.4 we show that if the f_{i} satisfy (5.2) then for any ϵ > 0, the function f = f_{ϵ,s} = ∑_{0≤i≤n} ϵ^{i(n−i)}f_{i}^{s} satisfies the scalar inequality:

A_{t}f ≤ cf + b,  (5.3)

where t, c and b are constants. This inequality is studied in §5.3. We show that if c is sufficiently small, then (5.3) for a fixed t together with the uniform continuity of log f imply that (A_{r}f)(1) is bounded as a function of r, which is the conclusion of Theorem 3.2. If c = 1, which will occur in the SO(2, 1) and SO(2, 2) cases, then (5.3) implies that (A_{r}f)(1) is growing at most linearly with the radius. In §5.4, we complete the proof of Theorem 3.2, and also prove Theorems 3.3 and 3.5.

Throughout the proof we consider the functions α(g)^{s} for 0 < s < 2 even though for the application to quadratic forms we only need s = 1 + δ. This yields a better integrability result, and is also necessary for the proof of Theorem 3.4 and Theorem 3.5.

5.1 Averages of the functions 1/d_{i}^{s} over spheres.

Recall that the function d_{i} is the norm of a certain vector in the exterior power ⋀^{i}(ℝ^{n}). In this subsection it is convenient to work with norms of vectors in arbitrary finite dimensional representations.

Lemma 5.1. Let V be a finite dimensional real inner product space, A a self-adjoint linear transformation of V, K a closed connected subgroup of the orthogonal group O(V), and Q a closed subset of the unit sphere {v ∈ V | ∥v∥ = 1}. Assume that the only eigenvalues of A are −1, 0 and 1 and denote by W^{−}, W^{0} and W^{+} the eigenspaces corresponding respectively to −1, 0 and 1. Let ℓ be a positive integer, and assume that the following conditions are satisfied:

(a) Kv ⊄ W^{−} + W^{0} for any v ∈ Q;

(b) For any nonzero v ∈ W^{−} the codimension of the subspace {x ∈ Lie (K) | xv ∈ W^{−}} in the Lie algebra Lie (K) of K is at least ℓ or, equivalently, there exist a ℓ-dimensional subspace L_{v} ⊂ Lie (K) such that xv ∉ W^{−} for any nonzero x ∈ L_{v}.

Then for any s, 0 < s < ℓ,

lim_{t→+∞} sup_{v∈Q} ∫_{K} \frac{dm(k)}{∥ exp(tA)kv∥^{s}} = 0,  (5.4)

where m is the normalized Haar measure on K.

Proof. Let p : V → W^{+} ⊕ W^{0} and p^{+} : V → W^{+} denote the natural orthogonal projections. For any v ∈ V and r ≥ 0, let us consider the following subsets of K:

D(v, r) = {k ∈ K | ∥p(kv)∥ ≤ r} and
D^{+}(v, r) = {k ∈ K | ∥p^{+}(kv)∥ ≤ r}.

It follows from the condition (b) that there exists ϵ > 0 so that the differential of function f_{v}(k) = p(kv) has rank at least ℓ at every point k ∈ K with ∥p(kv)∥ < ϵ∥v∥. It then follows from the implicit function theorem (or e.g. [Spi, Vol I, Theorem II.9] ) that around every point k with ∥p(kv)∥ < ϵ∥v∥ there exist coordinate systems for K and W^{+} ⊕ W^{0} such that in these coordinates the function f_{v} has the form f_{v}(u_{1}, …, u_{m}) = (u_{1}, u_{2}, …, u_{ℓ}, ψ_{ℓ+1}(u), …, ψ_{m}(u)) where the ψ_{i} are smooth functions, and that this coordinate system depends smoothly on v. Then by compactness,

C =^{def} sup_{∥v∥=1, r>0} \frac{m(D(v, r))}{r^{ℓ}} < ∞.  (5.5)

Since D^{+}(v, 0) is an algebraic subvariety in K and K is connected we get from the condition (a) that m(D^{+}(v, 0)) = 0 for every v ∈ Q. This and the compactness of Q easily imply

lim_{r→0} sup_{v∈Q} m(D^{+}(v, r)) = 0.  (5.6)

It is clear that

∥ exp(tA)kv∥^{s} ≥ max{∥p(kv)∥^{s}, e^{ts}∥p^{+}(kv)∥^{s}}.  (5.7)

In view of (5.5) and (5.7) we have

∫_{D(v,r)−D(v,r/2)} \frac{dm(k)}{∥ exp(tA)kv∥^{s}} ≤ \frac{Cr^{ℓ}}{(r/2)^{s}} = (2^{−s}C)r^{ℓ−s},  (5.8)

for any v ∈ V, ∥v∥ = 1, and any r > 0. But m(D(v, 0)) = 0. Therefore for any v ∈ V, ∥v∥ = 1, and any r > 0

∫_{D(v,r)} \frac{dm(k)}{∥ exp(tA)kv∥^{s}} =

= ∫_{D(v,0)} \frac{dm(k)}{∥ exp(tA)kv∥^{s}} + ∑_{n=0}^{∞} ∫_{D(v,2^{−n}r)−D(v,2^{−n−1}r)} \frac{dm(k)}{∥ exp(tA)kv∥^{s}}

≤ \frac{C}{2^{−s} − 2^{−ℓ}} r^{ℓ−s}.

(5.9)

Using (5.7) we get for any v ∈ V, r > 0 and r_{1} > 0

∫_{K−D(v,r)} \frac{dm(k)}{∥ exp(tA)kv∥^{s}}

≤ ∫_{K−D^{+}(v,r_{1})} \frac{dm(k)}{∥ exp(tA)kv∥^{s}} + ∫_{D^{+}(v,r_{1})−D(v,r)} \frac{dm(k)}{∥ exp(tA)kv∥^{s}} ≤

≤ e^{−ts}r_{1}^{−s} + m(D^{+}(v, r_{1}))r^{−s}.

(5.10)

Combining (5.6), (5.9) and (5.10) we easily get (5.4).

Lemma 5.2. Let V, A, K, Q, W^{−}, W^{0} and W^{+} be the same as in Lemma 5.1. Assume that Kv ⊄ W^{0} for any v ∈ Q and there exists a closed self-adjoint subgroup H_{1} of GL(V) with the following properties:

(a) the Lie algebra Lie (H_{1}) of H_{1} contains A;

(b) H_{1} is locally isomorphic to SL(2, ℂ) or, equivalently, to SO(3, 1);

(c) H_{1} ∩ K is a maximal compact subgroup of H_{1}.

Then (5.4) holds for any s satisfying 0 < s < 2.

Proof. After a suitable normalization, we may assume that the only eigenvalues of Ad A are −λ, 0 and λ, where λ = 2 if H_{1} ≅ SL(2, ℂ) and λ = 1 if H_{1} ≅ SO(3, 1). Then in view of (a) and (b)

Lie(H_{1}) = 𝔥^{−} ⊕ 𝔥^{0} ⊕ 𝔥^{+},

where

𝔥^{−} = {x ∈ Lie(H_{1}) | [A, x] = −λx},  (5.11)
𝔥^{0} = {x ∈ Lie(H_{1}) | [A, x] = 0},  (5.12)
𝔥^{+} = {x ∈ Lie(H_{1}) | [A, x] = λx}.  (5.13)

We also have that 𝔥^{−}, 𝔥^{0} and 𝔥^{+} are subalgebras of Lie (H_{1}) and that for any nonzero x ∈ 𝔥^{+} there exists a unique α(x) ∈ 𝔥^{−} with [x, α(x)] = A. Note that x, A and α(x) span a Lie algebra isomorphic to sℓ_{2}(ℝ) = Lie(SL(2, ℝ)). Now using standard arguments about representations of sℓ_{2}(ℝ) we get that for any v ∈ W^{−}+W^{0}, v ∉ W^{−},

{x ∈ Lie(H_{1}) | xv ∈ W^{−} + W^{0}} = 𝔥^{−} ⊕ 𝔥^{0}.  (5.14)

The intersection of the Lie algebra of any compact self-adjoint subgroup of H_{1} with 𝔥^{−} ⊕ 𝔥^{0} belongs to 𝔥^{0}. Therefore

Lie(K) ∩ (𝔥^{−} ⊕ 𝔥^{0}) ⊂ 𝔥^{0}.  (5.15)

Since Kv ⊄ W^{0} for any v ∈ Q we get from (5.14) and (5.15) that Kv ⊄ W^{−} +W^{0} for any v ∈ Q. Thus the condition (a) of Lemma 5.1 is satisfied. On the other hand since dim(K ∩ H_{1}) = 3 and dim(Lie(K) ∩ 𝔥^{0}) = 1 we can deduce from (5.14) and (5.15) that the condition (b) of Lemma 5.1 is also satisfied with ℓ = 2. Now we apply Lemma 5.1 and get (5.4).

Lemma 5.3. Let H^{0} denote the connected component of the identity of the group SO(p, q) and let K = SO(p)×SO(q) be the maximal compact subgroup of H^{0}. Assume that (p, q) ≠ (2, 2). Then for any nonzero proper subspace L of ℝ^{p+q} the subgroup

H_{L} =^{def} {h ∈ H^{0} | h(kL) = kL for any k ∈ K}

is contained in K.

Proof. Since the set KL is invariant under K we have that K normalizes H_{L}. Thus a subgroup F generated by K ∪ H_{L} normalizes H_{L}. But K is a maximal proper subgroup of H^{0} (because (p, q) ≠ (2, 2), H^{0} is simple). Therefore if H_{L} ⊄ K then F = H^{0} that is H_{L} is a normal subgroup of H^{0}. Since H^{0} is simple, either H_{L} = H^{0} or H_{L} ⊂ {±E}. But since H^{0} acts irreducibly on ℝ^{p+q} the first case is not possible. Thus H_{L} ⊂ K.

As in §3.2, we realize SO(p, q) as SO(Q_{0}) where Q_{0} is the quadratic form defined by (3.4). Let H_{1} ≅ SO(2, 1) denote the subgroup of H = SO(Q_{0}) fixing {e_{3}, …, e_{n−1}}. We refer to the embedding H_{1} ↪ H as the standard embedding of SO(2, 1) into SO(p, q).

Proposition 5.4. Let {a_{t} | t ∈ ℝ} be a self-adjoint one-parameter subgroup of SO(2, 1). Let p and q be positive integers and let 0 < i < p + q. Let

F (i) = {x_{1} ∧ x_{2} ∧ ⋯ ∧ x_{i} | x_{1}, x_{2}, ⋯, x_{i} ∈ ℝ^{p+q}} ⊂ ^{⋀i}(ℝ^{p+q}).

Then, if p ≥ 3, or if p = 2, q = 2 and i ≠ 2, then for any s, 0 < s < 2,

lim_{t→∞} sup_{v∈F (i), ∥v∥=1} ∫_{K} \frac{dm(k)}{∥a_{t}kv∥^{s}} = 0.  (5.16)

where K = SO(p) × SO(q) and SO(2, 1) is embedded into SO(p, q). If p = 2 and q = 1, or if p = 2, q = 2 and i = 2, then (5.16) holds for any s, 0 < s < 1.

Proof. Let ρ_{i} denote the i-th exterior representation of SO(p, q) in ⋀^{i}(ℝ^{p+q}). Since {a_{t}} ⊂ SO(2, 1) we can assume that a_{t} = exp tA where A is a self-adjoint linear transformation with the only eigenvalues −1, 0 and 1 and such that the multiplicities of eigenvalues −1 and 1 is equal to 1. Then the only eigenvalues of ρ_{i}(A) are also −1, 0 and 1. Let W_{i}^{−}, W_{i}^{0} and W_{i}^{+} denote the corresponding eigenspaces of ρ_{i}(A). It is clear that for any t ≠ 0

W_{i}^{0} = {v ∈ ^{⋀i}(ℝ^{p+q}) | a_{t}v = v}.  (5.17)

Suppose (p, q) ≠ (2, 2). It follows from Lemma 5.3 that for any t ≠ 0 and any v ∈ F (i), v ≠ 0,

a_{t} ∉ {h ∈ H | h(kv) = kv for any k ∈ K}.  (5.18)

Combining (5.17) and (5.18) we get that Kv ⊄ W_{i}^{0} for any v ∉ F (i), v ≠ 0. If p ≥ 3 it remains now to apply Lemma 5.2. If p = 2 and q = 1, then one can apply Lemma 5.1 directly with ℓ = 1.

For the SO(2, 2) assertions, recall that SO(2, 2) is locally isomorphic to SL(2, ℝ)× SL(2, ℝ), via the action of SL(2, ℝ)×SL(2, ℝ) on M_{2}(ℝ) ≅ ℝ^{4} given by (h_{1}, h_{2})·v = h_{1}^{t}vh_{2}. This action preserves the determinant which is a quadratic form on M_{2}(ℝ) of signature (2, 2). If we choose a basis so that the quadratic form det has the form (3.4), the standard embedded SO(2, 1) becomes locally isomorphic to the diagonal subgroup

of SL(2, ℝ) × SL(2, ℝ), and a_{t} becomes ( \begin{matrix}e^{t/2} & 0 \\ 0 & e^{−t/2} \end{matrix} ) × ( \begin{matrix}e^{t/2} & 0 \\ 0 & e^{−t/2} \end{matrix} ). Hence if i = 1, then

the spaces W^{−} and W^{0} of Lemma 5.1 consist of matrices of the forms ( \begin{matrix}0 & 0 \\ 0 & ∗ \end{matrix} ), and ( \begin{matrix}0 & ∗ \\ ∗ & 0 \end{matrix} ), respectively. Thus (a) and (b) of Lemma 5.1 are satisfied, with ℓ = 2. The assertion for i = 3 follows by duality. To obtain the assertion for i = 2, recall that the the representation ρ_{2} of SO(2, 2) splits into two three dimensional representations. Each invariant subspace is fixed by one of the SL(2, ℝ) factors, while the other acts via the three dimensional representation of SL(2, ℝ), hence fixes a quadratic form of signature (2, 1). If we write K = K_{1} × K_{2} and denote the projections onto the irreducible subspaces by π_{1} and π_{2}, then for any c > 0, 0 < s < 1, and for sufficiently large t,

∫_{K} \frac{dm(k)}{∥a_{t}kv∥^{s}} ≤ ∫_{K} min (\frac{1}{∥π_{1}(a_{t}kv)∥^{s}} , \frac{1}{∥π_{2}(a_{t}kv)∥^{s}} ) dm(k)  (5.19)
≤ min (∫_{K_{1}} \frac{dm(k_{1})}{∥π_{1}(a_{t}k_{1}v)∥^{s}} , ∫_{K_{2}} \frac{dm(k_{2})}{∥π_{2}(a_{t}k_{2}v)∥^{s}} )  (5.20)
≤ min (\frac{c}{∥π_{1}(v)∥^{s}} , \frac{c}{∥π_{2}(v)∥^{s}} ) by the assertion for SO(2, 1) (5.21)
≤ \frac{2^{s}c}{∥v∥^{s}} .  (5.22)

Since c is arbitrary, the assertion follows.

In the next lemma we obtain an analogous result for the case (p, q) = (2, 1), s = 1.

Lemma 5.5. Let H ≅ SO(2, 1) be the orthogonal group of the quadratic form x^{2} + y^{2} − z^{2}. Let {a_{t} | t ∈ ℝ} be a self-adjoint one-parameter subgroup of H, and let K = H ∩ O(3) denote the maximal compact of H. We define another norm ∥ · ∥^{∗} on ℝ^{3} by

∥(x, y, z)∥^{∗} = max(\sqrt{x^{2} + y^{2}}, |z|).  (5.23)

Then, for any v ∈ ℝ^{3}, v ≠ 0, and any t > 0,

∫_{K} \frac{dm(k)}{∥a_{t}kv∥^{∗}} ≤ \frac{1}{∥v∥^{∗}} .  (5.24)

Proof. The group K ≅ SO(2) acts by rotations in the xy plane and fixes the z axis. The norm ∥·∥^{∗} is equivalent to the standard Euclidean norm ∥·∥, and is K-invariant. Thus the function f_{v}(h) = 1/∥hv∥^{∗} is well defined on K\H ≅ ℍ^{2}. Note that the cosets K(a_{t}K) ⊂ ℍ^{2} are metric circles centered at the origin. The Laplace operator (i.e the Casimir of H) is given by

Δ = −(x^{2} + y^{2} − z^{2}) (\frac{∂^{2}}{∂x^{2}} + \frac{∂^{2}}{∂y^{2}} − \frac{∂^{2}}{∂z^{2}} ) + E^{2} + E,

where E is the Euler operator

E = x\frac{∂}{∂x} + y \frac{∂}{∂y} + z \frac{∂}{∂z}

(see [HT, Chapter III, Eq. (2.3.4)]. A direct calculation shows that if x^{2} +y^{2} −z^{2} > 0, then

Δf_{v} = Δ (\frac{1}{\sqrt{x^{2} + y^{2}}} ) = −\frac{x^{2} + y^{2} + z^{2}}{(x^{2} + y^{2})^{3/2}} < 0.  (5.25)

The equation (5.25) also implies that if x^{2} + y^{2} − z^{2} = 0, then

Δf_{v} = Δ (\frac{1}{\sqrt{x^{2} + y^{2}}} ) = 0.

Finally, if x^{2} + y^{2} − z^{2} < 0, then |z| > 0, and

Δf_{v} = Δ (\frac{1}{|z|}) = (x^{2} + y^{2} − z^{2})\frac{2}{|z|^{3}} < 0.

Hence f_{v} is always superharmonic with respect to the hyperbolic metric on ℍ^{2}. Thus, the averages of f_{v} over circles are bounded by the value at the center, which implies (5.24), (c.f. [Hel, Ch. IV, Proposition 2.5].

5.2 A system of inequalities

Lemma 5.6. For any two Δ-rational subspaces L and M

d(L)d(M) ≥ d(L ∩ M)d(L + M).  (5.26)

Proof. Let π : ℝ^{n} → ℝ^{n}/(L ∩ M) denote the natural projection. Then d(L) = d(π(L))d(L∩M), d(M) = d(π(M))d(L∩M) and d(L+M) = d(π(L+M))d(L∩M). On the other hand the inequality (5.26) is equivalent to the inequality

\frac{d(L)}{d(L ∩ M)} \frac{d(M)}{d(L ∩ M)} ≥ \frac{d(L + M)}{d(L ∩ M)}.

Therefore replacing L, M and L + M by π(L), π(M) and π(L + M) we can assume that L ∩ M = {0}. Let (e_{1}, ⋯, e_{ℓ}), ℓ = dim L, and (e_{ℓ+1}, ⋯, e_{ℓ+m}), m = dim M, be bases in L and M respectively. Then

d(L)d(M) = ∥e_{1} ∧ ⋯ ∧ e_{ℓ}∥ ∥e_{ℓ+1} ∧ ⋯ ∧ e_{ℓ+m}∥ ≥ ∥e_{1} ∧ ⋯ ∧ e_{ℓ} ∧ e_{ℓ+1} ∧ ⋯ ∧ e_{ℓ+m}∥ ≥ d(L + M) (5.27)

that proves (5.26) (the second inequality in (5.27) is true because (L∩Δ)+(M ∩Δ) ⊂ (L + M) ∩ Δ.

Lemma 5.7. Let {a_{t} | t ∈ ℝ} be a self-adjoint one-parameter subgroup of SO(2, 1). Let p and q be positive integers, and denote p + q by n. Denote SO(p) × SO(q) by K. Suppose p ≥ 3, q ≥ 1 and 0 < i < n, or p = 2, q = 2 and i = 1 or 3. Then for any s, 0 < s < 2, and any c > 0 there exist t > 0 and ω > 1 such that for any lattice Λ in ℝ^{n}

∫_{K} α_{i}(a_{t}kΛ)^{s} dm(k) < \frac{c}{2}α_{i}(Λ)^{s} + ω^{2} max_{0<j≤min\{n−i,i\}} (\sqrt{α_{i+j}(Λ)α_{i−j}(Λ)} )^{s}. (5.28)

If p = 2, q = 1 and i = 1, 2, or if p = 2, q = 2 and i = 2, then for any s, 0 < s < 1, and any c > 0 there exist t > 0 and ω > 1 such that (5.28) holds.

Proof. Fix c > 0. In view of Proposition 5.4 one can find t > 0 such that

∫_{K} \frac{dm(k)}{∥a_{t}kv∥^{s}} < \frac{c}{2},

whenever v ∈ F (i), ∥v∥ = 1. It follows that

∫_{K} \frac{dm(k)}{∥a_{t}kv∥^{s}} < \frac{c}{2} · \frac{1}{∥v∥^{s}} ,  (5.29)

for any v ∈ F (i), v ≠ 0. Let Λ be a lattice in ℝ^{n}. There exists a Λ-rational subspace L_{i} of dimension i such that

\frac{1}{d_{Λ}(L_{i})} = α_{i}(Λ).  (5.30)

The inequality (5.29) implies

∫_{K} \frac{dm(k)}{d_{a_{t}kΛ}(a_{t}kL_{i})^{s}} < \frac{c}{2} \frac{1}{d_{Λ}(L_{i})^{s}} .  (5.31)

Let ω = max_{0<j<n} ∥ ⋀^{j}(a_{t})∥. (In fact ω = e^{t}). We have that

ω^{−1} ≤ \frac{∥a_{t}v∥}{∥v∥} ≤ ω, 0 < j < n, v ∈ F (j).  (5.32)

Let us denote the set of Λ-rational subspaces L of dimension i with d_{Λ}(L) < ω^{2}d_{Λ}(L_{i}) by Ψ_{i}. We get from (5.32) that for a Λ-rational i-dimensional subspace L ∉ Ψ_{i}

d_{a_{t}kΛ}(a_{t}kL) > d_{a_{t}kΛ}(a_{t}kL_{i}),  k ∈ K.  (5.33)

It follows from (5.31), (5.33) and the definition of α_{i} that

∫_{K} α_{i}(a_{t}kΛ)^{s} dm(k) < \frac{c}{2}α_{i}(Λ)^{s} if Ψ_{i} = \{L_{i}\}.  (5.34)

Assume now that Ψ_{i} ≠ {L_{i}}. Let M ∈ Ψ_{i}, M ≠ L_{i}. Then dim(M +L_{i}) = i+j, j > 0. Now using (5.30), (5.32) and Lemma 5.6 we get that for any k ∈ K

α_{i}(a_{t}kΛ) < ωα_{i}(Λ) = \frac{ω}{d_{Λ}(L_{i})} < \frac{ω^{2}}{\sqrt{d_{Λ}(L_{i})d_{Λ}(M)}}

≤ \frac{ω^{2}}{\sqrt{d_{Λ}(L_{i} ∩ M)d_{Λ}(L_{i} + M)}}

≤ ω^{2}\sqrt{α_{i+j}(Λ)α_{i−j}(Λ)}.

(5.35)

Hence if Ψ_{i} ≠ {L_{i}}

∫_{K} α_{i}(a_{t}kΛ)^{s} dm(k) ≤ ω^{2} max_{0<j≤min\{n−i,i\}} (\sqrt{α_{i+j}(Λ)α_{i−j}(Λ)} )^{s}.  (5.36)

Combining (5.34) and (5.36) we get that for any lattice Λ ⊂ ℝ^{n}, (5.28) holds.

In the rest of this subsection we obtain similar systems of inequalities for the SO(2, 1) and SO(2, 2) cases, with s = 1. For H = SO(2, 1), Δ a lattice in ℝ^{3}, and L a Δ-rational subspace of ℝ^{3}, let d_{Δ}^{∗}(L) = ∥e_{1} ∧ … e_{ℓ}∥^{∗} where (e_{1}, … e_{l}) is a basis for Δ ∩ L. (The norm ∥ · ∥^{∗} defined in (5.23) on ℝ^{3} = ⋀^{1}(ℝ^{3}) can be extended to ⋀^{2}(ℝ^{3}) by duality.) For 1 ≤ i ≤ 2, let

α_{i}^{∗}(Δ) = sup \{\frac{1}{d_{Δ}^{∗}(L)}|L is a Δ-rational subspace of dimension i \}.  (5.37)

Clearly for any Δ,

(1/2)α_{i}(Δ) < α_{i}^{∗}(Δ) < 2α_{i}(Δ).  (5.38)

Lemma 5.8. Let {a_{t} | t ∈ ℝ} be a self-adjoint one-parameter subgroup of H = SO(2, 1), and denote SO(2) by K. Then there exist t_{0} > 0 and ω > 1, such that for any t < t_{0}, for any unimodular lattice Λ in ℝ^{3}, and 1 ≤ i ≤ 2,

∫_{K} α_{i}^{∗}(a_{t}kΛ) dm(k) < α_{i}^{∗}(Λ) + ω^{2}\sqrt{α_{3−i}(Λ)}.  (5.39)

Proof. The argument is identical to the proof of Lemma 5.7 except that one uses Lemma 5.5 instead of Proposition 5.4.

Now let H = SO(2, 2). The space V = ⋀^{2}(ℝ^{4}) splits as a direct sum V_{1} ⊕ V_{2} of two invariant subspaces, where on each V_{i}, H preserves a quadratic form Q_{i} of signature (2, 1). We define on each V_{i} a Euclidean norm ∥ · ∥_{i}^{∗} by (5.23) (adapted to Q_{i}). Let π_{i} denote the orthogonal projections from V to V_{i}. Now let Δ be a lattice in ℝ^{4}, and let L be a two-dimensional Δ-rational subspace of ℝ^{4}. For 1 ≤ i ≤ 2, let

d_{Δ}^{i,#}(L) = ∥π_{i}(e_{1} ∧ e_{2})∥_{i}^{∗},  (5.40)

where {e_{1}, e_{2}} is a basis over ℤ for Δ ∩ L. Then let

α_{2}^{#}(Δ) = sup_{L} \{min (\frac{1}{d_{Δ}^{1,#}(L)}, \frac{1}{d_{Δ}^{2,#}(L)})\} .  (5.41)

The supremum is taken over Δ-rational two dimensional subspaces L. By construction, for any Δ,

C^{−1}α_{2}^{#}(Δ) < α_{2}(Δ) < Cα_{2}^{#}(Δ),  (5.42)

where C is an absolute constant.

Lemma 5.9. Let {a_{t} | t ∈ ℝ} be a self-adjoint one-parameter subgroup of SO(2, 1), where SO(2, 1) is diagonally embedded in H = SO(2, 2), under its local identification with SL(2, ℝ) × SL(2, ℝ). Denote SO(2) × SO(2) by K, and the maximal compact of SO(2, 1) by K̃. Then there exist t_{0} > 0 and ω > 1, such that for any t < t_{0} and for any unimodular lattice Λ in ℝ^{4},

∫_{K̃} α_{2}^{#}(a_{t}k̃Λ) dm(k̃) < α_{2}^{#}(Λ) + ω^{2}\sqrt{α_{1}(Λ)α_{3}(Λ)}.  (5.43)

Proof. The group K̃ is diagonally embedded in K. Recall that each SO(2, 2) invariant subspace V_{i} of ⋀^{2}(ℝ^{4}) is fixed pointwise by one of the SL(2, ℝ) factors, while the other fixes a quadratic form of signature (2, 1). Thus, for 1 ≤ i ≤ 2, the inequalities:

∫K̃ \frac{dm(k̃)}{∥π_{i}(a_{t}k̃v)∥_{i}^{∗}} ≤ \frac{1}{∥π_{i}(v)∥_{i}^{∗}}  (5.44)

follow immediately from Lemma 5.5. Hence,

∫K̃ min (\frac{1}{∥π_{1}(a_{t}k̃v)∥_{1}^{∗}} , \frac{1}{∥π_{2}(a_{t}k̃v)∥_{2}^{∗}} ) dm(k) ≤ min (∫K̃ \frac{dm(k̃)}{∥π_{1}(a_{t}k̃v)∥_{1}^{∗}} , ∫K̃ \frac{dm(k̃)}{∥π_{2}(a_{t}k̃v)∥_{2}^{∗}} )

≤ min (\frac{1}{∥π_{1}(v)∥_{1}^{∗}} , \frac{1}{∥π_{2}(v)∥_{2}^{∗}} ) .  (5.45)

The rest of the proof is identical to that of Lemma 5.7 except that (5.45) is used in place of Proposition 5.4.

5.3 Coarsely Superharmonic Functions.

Let n ∈ ℕ^{+} and let D_{n}^{+} denote the set of diagonal matrices d(λ_{1}, ⋯, λ_{n}) ∈ GL_{n}(ℝ) with λ_{1} ≥ λ_{2} ≥ ⋯ ≥ λ_{n} > 0. For any g ∈ GL_{n}(ℝ), consider the Cartan decomposition g = k_{1}(g)d(g)k_{2}(g), k_{1}(g), k_{2}(g) ∈ K = O_{n}(ℝ), d(g) ∈ D_{n}^{+} and denote by λ_{1}(g) ≥ λ_{2}(g) ≥ ⋯ ≥ λ_{n}(g) the eigenvalues of d(g).

Lemma 5.10. For every ϵ > 0 there exists a neighborhood U of e in O_{n}(ℝ) such that

|\frac{λ_{i}(d_{1}kd_{2})}{λ_{i}(d_{1})λ_{i}(d_{2})} − 1| < ϵ  (5.46)

for any d_{1}, d_{2} ∈ D_{n}^{+}, k ∈ U and 1 ≤ i ≤ n.

Proof. Let (e_{1}, ⋯, e_{n}) be the standard orthonormal basis in ℝ^{n}. If k ∈ O_{n}(ℝ) and ⟨ke_{1}, e_{1}⟩ > 1 − ϵ then

∥d_{1}kd_{2}e_{1}∥ > (1 − ϵ)λ_{1}(d_{1})λ_{1}(d_{2}).  (5.47)

On the other hand, for any g ∈ GL_{n}(ℝ).

λ_{1}(g) = ∥g∥ ≥ ∥ge_{1}∥.  (5.48)

Since ∥d_{1}kd_{2}∥ ≤ ∥d_{1}∥ ∥d_{2}∥ it follows from (5.47) and (5.48) that

1 ≥ \frac{λ_{1}(d_{1}kd_{2})}{λ_{1}(d_{1})λ_{1}(d_{2})} > 1 − ϵ,  (5.49)

if ⟨ke_{1}, e_{1}⟩ > 1 − ϵ. Analogously considering the representation of GL_{n}(ℝ) in the i-th exterior product ⋀^{i}(ℝ^{n}) of ℝ^{n} we get that

1 ≥ \frac{(λ_{1}λ_{2} ⋯ λ_{i})(d_{1}kd_{2})}{(λ_{1}λ_{2} ⋯ λ_{i})(d_{1}d_{2})} > 1 − ϵ,  (5.50)

if k ∈ O_{n}(ℝ) and ⟨⋀^{i}(k)(e_{1} ∧⋯∧e_{i}), e_{1} ∧⋯∧e_{i}⟩ > 1−ϵ. It is clear that there exists a neighborhood U of identity in O_{n}(ℝ) such that ⟨⋀^{i}(k)(e_{1} ∧ ⋯ ∧ e_{i}), e_{1} ∧ ⋯ ∧ e_{i}⟩ > \sqrt{1 − ϵ} for every k ∈ U and 1 ≤ i ≤ n. But

λ_{i}(g) = \frac{(λ_{1}λ_{2} ⋯ λ_{i})(g)}{(λ_{1}λ_{2} ⋯ λ_{i−1})(g)}.

Therefore (5.46) follows from (5.50).

Lemma 5.11. (cf. the “wavefront lemma” [EM, Theorem 3.1]) Let H be a selfadjoint connected reductive subgroup of GL_{n}(ℝ), let K = O_{n}(ℝ) ∩ H be a maximal compact subgroup of H and let {a_{t} | t ∈ ℝ} be a self-adjoint one-parameter subgroup of H. Then for every neighborhood V of e in H there exists a neighborhood U of e in K such that

a_{t}Ua_{s} ⊂ KV a_{t}a_{s}K  (5.51)

for any t ≥ 0 and s ≥ 0.

Proof. Conjugating a_{t} by an element of K we can assume that {a_{t} | t ≥ 0} ⊂ D_{n}^{+}. It is easy to see that there exists ϵ > 0 such that h_{1} ∈ V h_{2} whenever h_{1}, h_{2} ∈ D_{n}^{+} and |\frac{λ_{i}(h_{1})}{λ_{i}(h_{2})} − 1| < ϵ for every 1 ≤ i ≤ n. Take a neighborhood U such that (5.46) is satisfied. Then (5.51) is true for this U.

Proposition 5.12. Let H be a self-adjoint reductive subgroup of GL_{n}(ℝ), let K = O_{n}(ℝ) ∩ H, let m denote the normalized measure on K, and let A = {a_{t} | t ∈ ℝ} be a self-adjoint one-parameter subgroup of H. Let F be a family of strictly positive functions on H having the following properties:

(a) The logarithms log f for f ∈ F are equicontinuous with respect to a leftinvariant uniform structure on H or, equivalently, for any ϵ > 0 there exists a neighborhood V (ϵ) of 1 in H such that for any f ∈ F,

(1 − ϵ)f(h) < f(uh) < (1 + ϵ)f(h)

for any h ∈ H and u ∈ V (ϵ);

(b) The functions f ∈ F are left K-invariant, that is f(Kh) = f(h), h ∈ H,

(c) sup_{f∈F} f(1) < ∞.

Then there exists 0 < c = c(F) < 1 such that for any t > 0 and b > 0 there exists B = B(t, b) < ∞ with the following property: If f ∈ F and

∫_{K} f(a_{t}kh) dm(k) < cf(h) + b  (5.52)

for any h ∈ KAK ⊂ H, then

∫_{K} f(a_{τ}k) dm(k) < B

for any τ > 0.

Proof. Fix f ∈ F, and let

f̃(h) = ∫_{K} f(hk) dm(k).

Properties (a), (b), (c) of the function f imply that f̃ has the same properties. Hence it suffices to show that the conclusion of the proposition holds for f̃. Therefore we can assume that

f(KhK) = f(h), h ∈ H,  (5.53)

and we have to prove that

sup_{τ>0} f(a_{τ}) < B < ∞.  (5.54)

It follows from property (a) that

\frac{1}{2}f(h) < f(uh) < 2f(h), h ∈ H, u ∈ V = V (\frac{1}{2}).  (5.55)

According to Lemma 5.11 there exists a neighborhood U of 1 in H such that a_{t}Ua_{τ} ∈ KV a_{t}a_{τ}K for any t ≥ 0 and τ ≥ 0. Then we get from (5.53) and (5.55) that

∫_{K} f(a_{t}ka_{τ}) dm(k) ≥ ∫_{U∩K} f(a_{t}ka_{τ}) dm(k) > \frac{1}{2}m(U ∩ K)f(a_{t}a_{τ}).  (5.56)

Suppose for some t > 0 and b > 0

∫_{K} f(a_{t}kh) dm(k) < \frac{1}{4}m(U ∩ K)f(h) + b, h ∈ H.  (5.57)

It follows from (5.56) and (5.57) that for some b′ > 0,

f(a_{t}a_{τ}) < \frac{1}{2}f(a_{τ}) + b′, for all τ > 0.  (5.58)

Using induction on ℓ we get from (5.58) that

f(a_{ℓt}) < 2 max{f(1), b′}, ; ℓ ∈ ℕ^{+}.  (5.59)

Since {a_{r} | 0 ≤ r ≤ t} belongs to V^{i} for some i where V^{1} = V, V^{i} = V V^{i−1}, it follows that sup_{h∈H, 0≤r≤t} \frac{f(a_{r}h)}{f(h)} < ∞. Therefore (5.59) and property (c) imply (5.54).

Lemma 5.13. Let H = SL(2, ℝ), K = O(2) ∩ H, and let {a_{t} | t ∈ ℝ} be a selfadjoint one-parameter subgroup of H. Suppose f is a strictly positive continuous function on H with the following properties:

(a) The logarithm of f is uniformly continuous with respect to a left-invariant uniform structure on H, or equivalently, for any ϵ > 0 there exists a neighborhood V (ϵ) of 1 in H such that

(1 − ϵ)f(h) < f(uh) < (1 + ϵ)f(h)

for any h ∈ H and u ∈ V (ϵ);

(b) f is left K-invariant, i.e f(Kh) = f(h)

(c) There exist t_{0} > 0 and b > 0, such that for 0 ≤ t < t_{0} and any h ∈ H,

∫_{K} f(a_{t}kh) dm(k) < f(h) + b.  (5.60)

(d) f(1) < β.

Then there exists B > 0 depending only on b, β and t_{0} and the modulus of continuity of log f, so that for all t > 1,

∫_{K} f(a_{t}k) dm(k) < Bt.  (5.61)

Proof. Let

f̂(h) = ∫_{K} f(hk) dm(k).

Then f̂ also has properties (a), (b), (c) and (d). Thus we may assume f(KhK) = f(h). Let ρ(h) be defined by ρ(Ka_{t}K) = log cosh \frac{t}{2}, where t > 0. Then f and ρ may be interpreted as radially symmetric functions on K\H = ℍ^{2}. Then Δρ = c_{0}, where Δ is the Laplace operator on ℍ^{2}, and c_{0} is an absolute constant (depending on the normalizations). Hence, for a fixed τ ∈ [0, t_{0}],

∫_{K} ρ(a_{τ}kh) dm(k) = ρ(h) + c_{1},  (5.62)

where c_{1} depends only on τ. Let u(h) = f(h) − (b/c_{1})ρ(h). Then (5.60) and (5.62) imply that

∫_{K} u(a_{τ}kh) dm(k) ≤ u(h).  (5.63)

The estimate (5.63) implies that the minimum of u(Kh) on the disk of radius t centered at the origin is attained within a distance τ of the boundary. Since f(1) = β, u(1) < β − (b/c_{1}) log cosh 0 ≤ β, hence u(Kh) < β for some point Kh on the circle of radius t_{1}, where t − τ < t_{1} < t. But u is radially symmetric, hence u(Kh) < β on the entire circle of radius t_{1}. Then

∫_{K} f(a_{t_{1}}k) dm(k) < β + \frac{b}{c_{1}} t_{1}.

To get (5.61) for arbitrary t it remains to use again property (a).

5.4 Averages over large spheres.

In this subsection we complete the proofs of Theorem 3.2, Theorem 3.3 and Theorem 3.5.

Proof of Theorem 3.2. Define functions f_{0}, f_{1}, ⋯, f_{n} on H = SO(p, q) by the following equalities

f_{i}(h) = α_{i}(hΔ), h ∈ H, 0 ≤ i ≤ n.

Since α(a_{t}kΔ)^{s} = max_{0≤i≤n} f_{i}(a_{t}k)^{s} < ∑_{0≤i≤n} f_{i}(a_{t}k)^{s} it is enough to show that

sup_{t>0, 0≤i≤n} ∫_{K} f_{i}^{s}(a_{t}k) dm(k) < ∞.  (5.64)

Let A_{t} denote the averaging operator defined by

(A_{t}f)(h) = ∫_{K} f(a_{t}kh) dm(k), h ∈ H.

As in Proposition 5.4, let

F (i) = {x_{1} ∧ x_{2} ∧ ⋯ ∧ x_{i} | x_{1}, x_{2}, ⋯, x_{i} ∈ ℝ^{n}} ⊂ ^{⋀i}(ℝ^{n}).

Since ∥Kv∥ = ∥v∥ and \frac{∥hv∥}{∥v∥} ≤ ∥ ⋀^{i}(h)∥, for any v ∈ F (i) and h ∈ H, each f_{i} has properties (a) and (b) of Proposition 5.12. Applying Lemma 5.7 to Λ = hΔ we see that for any i, 0 < i < n, and h ∈ H

A_{t}f_{i}^{s} < \frac{c}{2}f_{i}^{s} + ω^{2} max_{0<j≤min\{n−i,i\}} \sqrt{f_{i+j}^{s}f_{i−j}^{s}}.  (5.65)

Let us denote q(i) = i(n−i). Then by direct computations 2q(i)−q(i+j)−q(i−j) = 2j^{2}. Therefore we get from (5.65) that for any i, 0 < i < n, and any positive ϵ < 1

A_{t}(ϵ^{q(i)}f_{i}^{s}) < \frac{c}{2}ϵ^{q(i)}f_{i}^{s} + ω^{2} max_{0<j≤min\{n−i,i\}} ϵ^{q(i)− \frac{q(i+j)+q(i−j)}{2}} \sqrt{ϵ^{q(i+j)}f_{i+j}^{s}ϵ^{q(i−j)}f_{i−j}^{s}}

≤ \frac{c}{2}ϵ^{q(i)}f_{i}^{s} + ϵω^{2} max_{0<j≤min\{n−i,i\}} \sqrt{ϵ^{q(i+j)}f_{i+j}^{s}ϵ^{q(i−j)}f_{i−j}^{s}}.

(5.66)

Consider the linear combination

f_{ϵ,s} = ∑_{0≤i≤n} ϵ^{q(i)} f_{i}^{s}.

The function f_{ϵ,s} then also has properties (a) and (b) of Proposition 5.12. Since ϵ^{q(i)}f_{i}^{s} < f_{ϵ,s}, f_{0} = 1 and f_{n} = 1/d(Δ), the inequalities (5.66) imply the following inequality:

A_{t}f_{ϵ,s} < 1 + d(Δ)^{−s} + \frac{c}{2}f_{ϵ,s} + nϵω^{2}f_{ϵ,s}.  (5.67)

Taking ϵ = \frac{c}{2nω^{2}} we see that (5.52) from Proposition 5.12 also holds. Furthermore property (a) and (5.52) of Proposition 5.12 hold with the same constants for any unimodular lattice Δ ∈ ℝ^{n}. Since f_{ϵ,s}(1) ≤ nα(Δ)^{s}, f_{ϵ,s}(1) is uniformly bounded as Δ varies over a compact set C of unimodular lattices. Hence the family F of functions f_{ϵ,s} obtained as Δ varies over C satisfies all the conditions of Proposition 5.12. Since α_{i}(hΔ)^{s} = f_{i}(h)^{s} ≤ ϵ^{−q(i)}f_{ϵ,s}(h), Proposition 5.12 implies that there exists a constant B > 0 so that for each i, all t > 0, and all Δ ∈ C,

∫_{K} α_{i}(a_{t}kΔ)^{s} dm(k) < B.

From this the theorem follows.

Proof of Theorem 3.3. Suppose p = 2 and q = 1. Let C be a compact set of lattices in ℝ^{3}; by rescaling we may assume that C consists of unimodular lattices. Applying Theorem 3.2 with s = 1/2, we see that there is a constant B_{1} > 0, so that for any t ≥ 0 and i = 1, 2, and any Δ ∈ C,

∫_{K} α_{i}(a_{t}kΔ)^{1/2} dm(k) < B_{1}.  (5.68)

For i = 1, 2, let f_{i}(h) = α_{i}^{∗}(hΔ) (where α_{i}^{∗} is defined in (5.37)). By Lemma 5.8 there exist t_{0} > 0 and ω > 1 such that for 0 ≤ t < t_{0}, any h ∈ H and any unimodular lattice Δ,

∫_{K} f_{i}(a_{t}kh) dm(k) < f_{i}(h) + ω^{2}α_{3−i}(hΔ)^{1/2}.  (5.69)

Let f̃_{i}(h) = ∫_{K} f_{i}(hk) dm(k). Then writing (5.69) with h replaced by hk′ and integrating over k′ ∈ K we get

∫_{K} f̃_{i}(a_{t}kh) dm(k) < f̃_{i}(h) + ω^{2} ∫_{K} α_{3−i}(hk′Δ)^{1/2} dm(k′).  (5.70)

Since we may write h = ka_{s}k′′ for k, k′′ ∈ K and the α_{i} are left K-invariant, the integral on the right hand side of (5.70) is bounded by (5.68). Thus, for 0 ≤ t < t_{0} and any h ∈ H,

∫_{K} f̃_{i}(a_{t}kh) dm(k) < f̃_{i}(h) + b.

where b = ω^{2}B_{1}. Thus for i = 1, 2, f̃_{i} satisfies all the conditions of Lemma 5.13, with constants independent of the choice of Δ ∈ C. Hence for i = 1, 2 and t > 1,

∫_{K} f_{i}(a_{t}k) dm(k) = ∫_{K} f̃_{i}(a_{t}k) dm(k) < B_{2}t,  (5.71)

where B_{2} is independent of t or the choice of Δ ∈ C. Now (3.5) follows from (5.71) and (5.38).

Now suppose p = 2 and q = 2. Let C be a compact set of unimodular lattices in ℝ^{4}. Applying Theorem 3.2 with s = 1/2, we see that there is a constant B_{4} > 0, so that for any t ≥ 0 and i = 1, 2, 3,

(A_{t}α_{i}^{1/2})(Δ) < B_{4}.  (5.72)

(recall that for any function f on G/Γ, (A_{t}f)(Δ) = ∫_{K} f(a_{t}kΔ) dm(k)). For i = 1, and i = 3, let f_{i}(h) = α_{i}(hΔ). Then Lemma 5.7 implies that for any c > 0 there exist t > 0 and ω > 0 such that for any h ∈ H and i = 1, 3,

(A_{t}f_{i})(h) < \frac{c}{2}f_{i}(h) + ω^{2}α_{2}(hΔ)^{1/2}.  (5.73)

For i = 1, 3, let f̃_{i}(h) = ∫_{K} f_{i}(hk) dm(k). Writing (5.73) with hk′ instead of h and integrating over k′ ∈ K we get that for all h ∈ H and i = 1, 3,

(A_{t}f̃_{i})(h) < \frac{c}{2}f̃_{i}(h) + ω^{2} ∫_{K} α_{2}(hk′Δ)^{1/2} dm(k′).  (5.74)

Let A = {a_{τ} | τ ∈ ℝ}, and suppose h ∈ KAK. Then, since α_{2} is left-K invariant the integral on the right hand side of (5.74) is bounded by (5.72). Hence for any c > 0 there exist t > 0 and b′ > 0 so that for all h ∈ KAK,

(A_{t}f̃_{i})(h) < \frac{c}{2}f̃_{i}(h) + b′,

where b′ = ω^{2}B_{1}. Then by Proposition 5.12, there exists B_{5} > 0 so that for all t > 0, and i = 1, 3,

f̃_{i}(Ka_{t}K) = ∫_{K} f_{i}(a_{t}k) dm(k) < B_{5},

i.e for all t > 0 and i = 1, 3,

∫_{K} α_{i}(a_{t}kΔ) dm(k) < B_{5}.  (5.75)

Since the constants in conditions (a)-(c) and equation (5.52) of Proposition 5.12 can be chosen uniformly as Δ varies over C, the constant B_{5} does not depend on the choice of Δ ∈ C.

Now let f_{2}(h) = α_{2}^{#}(hΔ). By Lemma 5.9, there exist t_{0} > 0 and ω > 0 so that for any 0 < t < t_{0} and any h ∈ H,

(Ã_{t}f_{2})(h) < f_{2}(h) + ω^{2}\sqrt{α_{1}(hΔ)α_{3}(hΔ)} ≤ f_{2}(h) + ω^{2}(α_{1}(hΔ) + α_{3}(hΔ)). (5.76)

where (Ã_{t}f)(h) = ∫K̃ f(a_{t}k̃h) dm(k) and K̃ denotes the maximal compact of H̃ = SO(2, 1), which is naturally (i.e diagonally) embedded in SO(2, 2) ≅ SL(2, ℝ) × SL(2, ℝ). If we let

f̃_{2}(h) = ∫_{K} f_{2}(hk) dm(k),

then using (5.76) with hk in place of h and integrating over k ∈ K we get

(Ã_{t}f̃_{2})(h) < f̃_{2}(h) + ω^{2} ∫_{K} [α_{1}(hkΔ) + α_{3}(hkΔ)] dm(k).  (5.77)

Since α_{1} and α_{3} are left K-invariant, and H̃ ⊂ KAK, (5.77) and (5.75) imply that for h ∈ H̃,

(Ã_{t}f̃_{2})(h) < f̃_{2}(h) + b′′,

where b′′ = 2ω^{2}B_{5}. Thus f̃_{2} restricted to H̃ ≅ SO(2, 1) satisfies all the conditions of Lemma 5.13, hence there exists B_{6} > 0 so that for all t > 1 and all Δ ∈ C,

∫_{K} α_{2}^{#}(a_{t}kΔ) dm(k) = ∫_{K} f̃_{2}(a_{t}k) dm(k) < B_{6}t.  (5.78)

Now (3.5) follows from (5.78), (5.42) and (5.75).

Proof of Theorem 3.5. We can assume that ϕ is nonnegative. Let A(r) = {x ∈ G/Γ : α(x) > r}. Choose a continuous nonnegative function g_{r} on G/Γ such that g_{r}(x) = 1 if x ∈ A(r + 1), g_{r}(x) = 0 if x ∉ A(r) and 0 ≤ g_{r}(x) ≤ 1 if x ∈ A(r) − A(r + 1). Then

\begin{matrix}∫_{K} ϕ(a_{t}kx)ν(k) dm(k) = \\ = ∫_{K} (ϕg_{r})(a_{t}kx)ν(k) dm(k) + ∫_{K} (ϕ − ϕg_{r})(a_{t}kx)ν(k) dm(k). \end{matrix}  (5.79)

But (letting β = 2 − s), (ϕg_{r})(y) ≤ B_{1}α(y)^{2−β}g_{r}(y) = B_{1}α(y)^{2− \frac{β}{2}} g_{r}(y)α(y)^{− \frac{β}{2}} ≤ B_{1}r^{− \frac{β}{2}} α(y)^{2− \frac{β}{2}} (the last inequality is true because g_{r}(y) = 0 if α(y) ≤ r). Therefore

∫_{K} (ϕg_{r})(a_{t}kx)ν(k) dm(k) ≤ B_{1}r^{− \frac{β}{2}} ∫_{K} α(a_{t}kx)^{2− \frac{β}{2}} ν(k) dm(k).  (5.80)

According to Theorem 3.2 there exists B such that

∫_{K} α(a_{t}kx)^{2− \frac{β}{2}} dm(k) < B

for any t ≥ 0 and uniformly over x ∈ C. Then (5.80) implies that

∫_{K} (ϕg_{r})(a_{t}kx)ν(k) dm(k) ≤ BB_{1}(sup ν)r^{− \frac{β}{2}} .  (5.81)

Since the function ϕ − ϕg_{r} is continuous and has a compact support, Theorem 4.5 implies that for every ϵ > 0 there exists a finite set of points x_{1}, …, x_{ℓ} with Hx_{i} closed for each i so that for every compact subset F of C − ⋃_{i=1}^{ℓ} Hx_{i} there exists t_{0} > 0 such that for every t > t_{0} and every x ∈ F,

|∫_{K} (ϕ − ϕg_{r})(a_{t}kx)ν(k) dm(k) − ∫_{G/Γ} (ϕ − ϕg_{r})(y) dμ(y) ∫_{K} ν(k) dm(k)| < \frac{ϵ}{2}. (5.82)

It is easy to see that (5.79), (5.81) and (5.82) imply (3.6) if r is sufficiently large.

References

[Bea] A. Beardon. The Geometry of Discrete Groups. Springer-Verlag, 1983.

[Bor1] A. Borel. Introduction aux groupes arithmetiques. Publ. de l’Inst. Math. del l’Univ. de strassbourg XV. Hermann, Paris, 1969.

[Bor2] A. Borel. Values of indefinite quadratic forms at integral points and flows on spaces of lattices. Bull. Amer. Math. Soc. (N.S.) 32(1995), 184–204.

[Bre] T. Brennan. . Princeton University undergraduate thesis, 1994.

[BMP] M.R. Brenner, R.V Moody, and J. Patera. Tables of Dominant Weight Multiplicities for Representations of Simple Lie Algebras. M. Dekker, New York, 1985.

[CSD] J.W.S. Cassels and H.P.F. Swinnerton-Dyer. On the product of three homogeneous linear forms and the indefinite ternary quadratic forms. Philos. Trans. Roy. Soc. London Ser. A. 248(1955), 73–96.

[Dan1] S.G. Dani. On invariant measures, minimal sets and a lemma of Margulis. Invent. Math 51(1979), 239–260.

[Dan2] S.G. Dani. Invariant measures and minimal sets of horoshperical flows. Invent. Math 64(1981), 357–385.

[Dan3] S.G. Dani. On orbits of unipotent flows on homogeneous spaces. Ergod. Theor. Dynam. Syst. 4(1984), 25–34.

[Dan4] S.G. Dani. On orbits of unipotent flows on homogenous spaces II. Ergod. Theor. Dynam. Syst 6(1986), 167–182.

[DM1] S.G. Dani and G.A. Margulis. Values of quadratic forms at primitive integral points. Invent. Math. 98(1989), 405–424.

[DM2] S.G. Dani and G.A. Margulis. Orbit closures of generic unipotent flows on homogeneous spaces of SL(3, ℝ). Math. Ann. 286(1990), 101–128.

[DM3] S.G. Dani and G.A. Margulis. Limit distributions of orbits of unipotent flows and values of quadratic forms. Advances in Soviet Math. 16(1993), 91–137.

[EMM] A. Eskin, G. Margulis, and S. Mozes. On a quantitative version of the Oppenheim conjecture. ERA. Amer. Math. Soc. 01(1995), 124–130.

[EM] A. Eskin and C. McMullen. Mixing, counting and equidistribution in Lie groups. Duke Math. J. 71(1993), 143–180.

[Hel] S. Helgason. Groups and Geometric Analysis. Academic Press, New York, 1984.

[How] R. Howe. A notion of rank for unitary representations of the classical groups. In A. Figa Talàmianca, editor, Harmonic analysis and group representaions. CIME, 1980.

[HT] R. Howe and E.C. Tan. Non-abelian harmonic analysis: applications of SL(2, ℝ). Springer-Verlag, New York, 1992.

[KS] A. Katok and R. Spatzier. Differential rigidity of hyperbolic abelian actions. Publ. Math. IHES 79(1994), 131–156.

[KM] D. Kleinbok and G.A. Margulis. Bounded orbits of non-quasiunipotent flows on homogeneous spaces. AMS. Math. Soc. Trans. (2) 171(1996), 141–172.

[Kna] A. W. Knapp. Representation Theory of Semisimple Groups. Princeton University Press, 1986.

[Mar1] G. A. Margulis. On the action of unipotent groups in the space of lattices. In Lie Groups and their representations, (Proc. of Summer School in Group Representations, Bolyai Janos Math. Soc., Akademai Kiado, Budapest, 1971), pages 365–370. Halsted, New York, 1975.

[Mar2] G. A. Margulis. Formes quadratiques ind‘efinies et flots unipotents sur les spaces homog‘enes. C. R. Acad. Sci. Paris Ser. I 304(1987), 247–253.

[Mar3] G. A. Margulis. Discrete Subgroups and Ergodic Theory. In Number theory, trace formulas and discrete subgroups (a symposium in honor of A. Selberg), pages 377–398. Academic Press, Boston, MA, 1989.

[Mar4] G. A. Margulis. Indefinite quadatic forms and unipotent flows on homogeneous spaces. In Dynamical systems and ergodic theory, volume 23, pages 399–409. Banach Center Publ., PWN — Polish Scientific Publ., Warsaw, 1989.

[Mar5] G. A. Margulis. Dynamical and ergodic properties of subgroup actions on homogeneous spaces with applications to number theory. In Proc. of ICM (Kyoto, 1990), pages 193–215. Math. Soc. of Japan and Springer, 1991.

[MT1] G.A. Margulis and G.M. Tomanov. Invariant measures for actions of unipotent groups over local fields on homogeneous spaces. Invent. Math. 116(1994), 347–392.

[MT2] G.A. Margulis and G.M. Tomanov. Measure rigidity for almost linear groups and its applications. J. d’Anal. Math. 69(1996), 25–54.

[NS] A. Nevo and E. Stein. A generalization of Wiener’s pointwise ergodic theorem. Preprint.

[Rat1] M. Ratner. Strict measure rigidity for unipotent subgroups of solvable groups. Invent. Math. 101(1990), 449–482.

[Rat2] M. Ratner. On measure rigidity of unipotent subgroups of semisimple groups. Acta. Math. 165(1990), 229–309.

[Rat3] M. Ratner. On Raghunathan’s measure conjecture. Annals of Math. 134(1991), 545–607.

[Rat4] M. Ratner. Raghunathan’s topological conjecture and distributions of unipotent flows. Duke Math. J. 63(1991), 235–290.

[Rat5] M. Ratner. Raghunathan’s conjectures for Cartesian products of real and p-adic Lie groups. Duke Math. J. 77(1995), 275–382.

[Rat6] M. Ratner. Interactions between Lie groups, ergodic theory and number theory. In Proc. of ICM (Zurich, 1994).

[Rud] W. Rudin. Real and Complex Analysis. McGraw-Hill, 2d edition, 1974.

[Sar] P. Sarnak. Values at integers of binary quadratic forms. In preparation.

[Sch] W. Schmidt. Asymptotic formulae for point lattices of bounded determinant and subspaces of bounded height. Duke Math. J. 35(1968), 327–339.

[Sha1] N. A. Shah. Uniformly distributed orbits of certain flows on homogeneous spaces. Math. Ann. 289(1991), 315–334.

[Sha2] N. A. Shah. Limit distributions of polynomial trajectories on homogeneous spaces. Duke Math. J. 75(1994), 711–732.

[Sha3] N. A. Shah. PhD thesis, Tata Institute for Fundamental Research.

[Spi] M. Spivak. A Comprehensive Introduction to Differential Geometry. Publish or Perish, Inc., 1970.

[Sta1] A.N. Starkov. Solvable homogeneous flows. Math. Sbornik 176(1987), 242– 259. In Russian.

[Sta2] A.N. Starkov. The ergodic decomposition of flows on homogenous spaces of finite volume. Math. Sbornik 180(1989), 1614–1633. In Russian.

Department of Mathematics, University of Chicago, Chicago IL 60637, USA; eskin@math.uchicago.edu

Department of Mathematics, Yale University, New Haven CT, USA; margulis@math.yale.edu

Institute of Mathematics, Hebrew University, Jerusalem 91904, ISRAEL; mozes@math.huji.ac.il