Groups of homeomorphisms of one-manifolds, I: actions of nonlinear groups

Benson Farb^{∗} and John Franks^{†}

June 20, 20001

Abstract

This self-contained paper is part of a series [FF2, FF3] on actions by diffeomorphisms of infinite groups on compact manifolds. The two main results presented here are:

1. Any homomorphism of (almost any) mapping class group or automorphism group of a free group into Diff_{+}^{r}(S^{1}), r ≥ 2 is trivial. For r = 0 Nielsen showed that in many cases nontrivial (even faithful) representations exist. Somewhat weaker results are proven for finite index subgroups.

2. We construct a finitely-presented group of real-analytic diffeomorphisms of R which is not residually finite.

1 Introduction

In this paper we consider infinite groups acting by diffeomorphisms on onedimensional manifolds. For lattices in higher rank semisimple Lie groups such actions are essentially completely understood:

Theorem 1.1 (Ghys [Gh], Burger-Monod [BM]). Let Γ be a lattice in a simple Lie group of R-rank at least two. Then any C^{0}-action of Γ on S^{1} has a finite orbit, and any C^{1}-action of Γ on S^{1} must factor through a finite group.

Theorem 1.1 is the solution in dimension one of Zimmer’s program of classifying actions of higher rank lattices in simple Lie groups on compact manifolds. In [La] (see in particular §8), Labourie describes possible extensions of Zimmer’s program to other “big” groups. A. Navas [Na] has recently proven that any C^{1+α}, α > 1/2 action of a group with Kazhdan’s property T factors through a finite group. In this paper we consider three basic examples of nonlinear groups: mapping class groups of surfaces, (outer) automorphism groups of free groups, and Baumslag-Solitar groups.

∗Supported in part by the NSF and by the Sloan Foundation.

†Supported in part by NSF grant DMS9803346.

Mapping class groups and automorphism groups of free groups. Let Mod(g, k) denote the group of isotopy classes of diffeomorphisms of the genus g surface with k punctures. Unlike the case of lattices (Theorem 1.1), the group Mod(g, 1), g ≥ 1 does have a faithful, C^{0}-action on S^{1} without a global fixed point. This is a classical result of Nielsen. However, imposing a small amout of regularity changes the situation dramatically.

Theorem 1.2 (Mapping class groups). For g ≥ 3 and k = 0, 1, any C^{2} action of Mod(g, k) on S^{1} or on I = [0, 1] is trivial.

The S^{1} case of Theorem 1.2 was announced by E. Ghys several years ago (see §8 of [La]), and for real-analytic actions was proved by Farb-Shalen [FS].

Another class of basic examples of “big groups” are automorphism groups of free groups. Let Aut(F_{n}) (resp. Out(F_{n})) denote the group of automorphisms (resp. outer automorphisms) of the free group F_{n} of rank n. It is known that Aut(F_{n}), n > 2 is not linear [FP].

The techniques we develop to prove the results above allow us to prove the following.

Theorem 1.3 (Automorphism groups of free groups). For n ≥ 6, any homomorphism from Aut(F_{n}) to Diff_{+}^{2}(S^{1}) factors through Z/2Z. Any homomorphism from Aut(F_{n}) to Diff_{+}^{2}(I) is trivial. The analagous results hold for Out(F_{n}).

M. Bridson and K. Vogtmann [BV] have recently proven a stronger result: any homomorphism from Aut(F_{n}) to a group which does not contain a symmetric group must have an image which is at most Z/2Z. However, their proof uses torsion elements in an essential way, hence does not extend to finite index subgroups. On the other hand our techniques have implications for finite index subgroups of Aut(F_{n}) and Mod(g, k); such subgroups are typically torsion free.

Theorem 1.4 (Finite index and other subgroups). Let H be any group of C^{2} diffeomorphisms of I or S^{1} with the property that no nontrivial element of H has an interval of fixed points (e.g. H is a group of real-analytic diffeomorphisms). Then H does not contain any finite index subgroup of:

1. Mod(g, k) for g ≥ 3, k ≥ 0.

2. Aut(F_{n}) or Out(F_{n}) for n ≥ 6.

3. The Torelli group T_{g,k} for g ≥ 3, k ≥ 0.

In [FF3] we construct a C^{1} action of T_{g,k} on I and on S^{1}. Note also that Out(F_{2}) has a free subgroup of finite index, which admits a faithful, C^{ω} action on I and on S^{1}.

Baumslag-Solitar groups. The Baumslag-Solitar groups BS(m, n) are defined by the presentation

BS(m, n) =< a, b : ab^{m}a^{−1} = b^{n} >

When n > m > 1 the group BS(m, n) is not residually finite; in particular it is not a subgroup of any linear group (see, e.g. [LS]). We will give a construction which shows that BS(m, n) is a subgroup of one of the “smallest” infinite-dimensional Lie groups.

Theorem 1.5 (Baumslag-Solitar groups: existence). The group Diff_{+}^{ω}(R) of real-analytic diffeomorphisms of R contains a subgroup isomorphic to BS(m, n) for any n > m ≥ 1. The analogous result holds for Homeo_{+}(S^{1}) and Homeo_{+}(I).

It is not difficult to construct pairs of diffeomorphisms a, b ∈ Diff_{+}^{ω}(R) which satisfy the relation ab^{m}a^{−1} = b^{n}; the difficulty is to prove that (in certain situations) this is the only relation. To do this we use a Schottky type argument.

While Ghys-Sergiescu [GS] showed that Diff^{∞}(S^{1}) contains Thompson’s infinite simple (hence non-residually finite) group T, they also showed that T admits no real-analytic action on S^{1} (see also [FS]); indeed we do not know of any subgroups of Diff^{ω}(S^{1}) which are not residually finite.

The construction in the proof of Theorem 1.5 gives an abundance analytic actions of BS(m, n) on R, and C^{0} actions of BS(m, n) on S^{1} and on I. The loss of regularity in moving from R to S^{1} is no accident; we will show in contrast to Theorem 1.5 that (for typical m, n) there are no C^{2} actions of BS(m, n) on S^{1} or I.

Theorem 1.6 (Baumslag-Solitar groups: non-existence). No subgroup of Diff_{+}^{2}(I) is isomorphic to BS(m, n), n > m > 1. If further m does not divide n, then the same holds for Diff_{+}^{2}(S^{1}).

The hypothesis m > 1 in Theorem 1.6 is also necessary since BS(1, n), n ≥ 1 is a subgroup of PSL(2, R), hence of Diff_{+}^{ω}(S^{1}).

In fact BS(1, n) has many actions on S^{1}, and it is natural to attempt a classification (up to conjugacy) of all of them. As a first step in this direction we observe (Theorem 5.8), as a corollary of result of M. Shub [Sh] on expanding maps, that the standard, projective action of BS(1, n) on S^{1} is locally rigid, i.e. nearby actions are conjugate. An example due to M. Hirsch [H] shows that it is not globally rigid, i.e. there are actions which are not conjugate to the standard action (Theorem 5.9 below). It would be interesting to find a numerical invariant which characterizes the standard action, as Ghys [Gh2] has done for Fuchsian groups.

2 Tools

In this section we recall some properties of diffeomorphisms of one-manifolds which will be used throughout the paper.

2.1 Kopell’s Lemma and Hölder’s Theorem

Our primary tool is the following remarkable result of Nancy Kopell which is Lemma 1 of [K].

Theorem 2.1 (Kopell’s Lemma). Suppose f and g are C^{2}, orientationpreserving diffeomorphisms of an interval [a, b) such that fg = gf. If f has no fixed point in (a, b) and g has a fixed point in (a, b) then g = id.

Another useful result is the following theorem, which is classical.

Theorem 2.2 (Hölder’s Theorem). Suppose a group G of homeomorphisms of R acts freely and effectively on a closed subset of R. Then G is abelian.

2.2 The translation number and mean translation number

If f ∈ Homeo_{+}(S^{1}), i.e, f is an orientation preserving homeomorphism of S^{1}, then there is a countable collection of lifts of f to orientation preserving homeomorphisms of the line. If F is one such lift then it satisfies F T = T F where T (x) = x + 1, and all others are of the form F T^{n}, n ∈ Z. Any orientation preserving homeomorphism of R which commutes with T is a lift of an element of Homeo_{+}(S^{1}). We will denote by Homeo_{Z}(R) the group of homeomorphisms of R which commute with T, or equivalently the group of all lifts of elements of Homeo_{+}(S^{1}).

There are two important and closely related functions from Homeo_{+}(S^{1}) and Homeo_{Z}(R) to S^{1} and R respectively, which we now define.

Definition 2.3. If F ∈ Homeo_{Z}(R) define its translation number τ(F) ∈ R by

τ(F) = lim_{n→∞} \frac{F^{n}(x) − x}{n}

If f ∈ Homeo_{+}(S^{1}) define its rotation number, ρ(f) ∈ S^{1}, by

ρ(f) = τ(F) mod (1),

where F is any lift of f.

We summarize some basic properties of the rotation and translation numbers. Proofs of these and additional properties can be found, for example, in [dMvS]

Proposition 2.4 (Properties of rotation number). If F ∈ Homeo_{Z}(R) the number τ(F) always exists and is independent of the x ∈ R used to define it. It satisfies

τ(F^{n}) = nτ(F)
τ(F T^{n}) = τ(F) + n
τ(F_{0}F F_{0}^{−1}) = τ(F) for any F_{0} ∈ Homeo_{Z}(R).

F has a fixed point if and only if τ(F) = 0.

If f ∈ Homeo_{+}(S^{1}) then ρ(f) is independent of the lift F used to define it. It satisfies

ρ(f^{n}) = nρ(f)
ρ(f_{0}ff_{0}^{−1}) = ρ(f) for any f_{0} ∈ Homeo_{+}(S^{1}).

f has a fixed point if and only if ρ(f) = 0.

Suppose G is a subgroup of Homeo_{+}(S^{1}) and Ḡ is the group of all lifts of elements of G to R. If G preserves a Borel probability measure μ_{0} on S^{1} then this measure may be lifted to a Ḡ invariant measure μ on R which is finite on compact sets and which is preserved by the covering translation T (x) = x + 1. This permits us to define the mean translation number with respect to μ.

Definition 2.5. If F ∈ Homeo_{Z}(R) define its mean translation number τ_{μ}(F) ∈ R by

τ_{μ}(F) = \begin{cases}μ([x, F (x))) & if F (x) > x, \\ 0 & if F (x) = x, \\ −μ([F (x), x)) & if F (x) < x, \end{cases}

where x ∈ R.

We enumerate some of the well known basic properties of the mean translation number which we will use later.

Proposition 2.6. Suppose G is a subgroup of Homeo_{+}(S^{1}) which preserves the Borel probability measure and Ḡ is the group of all lifts of elements of G. If F ∈ Ḡ the mean translation number τ_{μ}(F) is independent of the point x used to define it. Indeed τ_{μ}(F) = τ(F). Moreover the function τ_{μ} : Ḡ → R is a homomorphism (and hence so is τ : Ḡ → R.)

Proof. Consider the function

ν(x, y) = \begin{cases}μ([x, y))) & if y > x, \\ 0 & if y = x, \\ −μ([y, x)) & if y < x. \end{cases}

It has the property that ν(x, y) + ν(y, z) = ν(x, z).

We also note that τ_{μ}(F) = ν(x, F (x). To see this is independent of x note for any y ∈ R, ν(x, F (x)) = ν(x, y) + ν(y, F (y)) + ν(F (y), F (x)) = ν(y, F (y)) + ν(x, y) − ν(F (x), F (y)). But F preserves the measure μ and the orientation of R so ν(x, y) = ν(F (x), F (y)). Hence ν(x, F (x) = ν(y, F (y)).

Since μ is the lift of a probability measure on S^{1} we know that μ([x, x+k)) = k for any k ∈ Z. So if F^{n}(x) ∈ [x + k, x + k + 1) we see that

F^{n}(x) − x − 1 ≤ k ≤ ν(x, F^{n}(x)) ≤ k + 1 ≤ F^{n}(x) − x + 1.

To see that τ_{μ}(F) = τ(F) we note

τ(F) = lim_{n→∞} \frac{F^{n}(x) − x}{n}

= lim_{n→∞} \frac{ν(x, F^{n}(x))}{n}

= lim_{n→∞} \frac{1}{n} ∑_{i=0}^{n−1} ν(F^{i}(x), F^{i+1}(x))

= ν(x, F (x)) = τ_{μ}(F).

⋄

There are two well known corollaries of this result which we record here for later use.

Corollary 2.7. Suppose G is a subgroup of Homeo_{+}(S^{1}) which preserves the Borel probability measure μ_{0} and Ḡ is the group of all lifts of elements of G. Then each of the functions τ_{μ} : Ḡ → R, τ : Ḡ → R and ρ : G → S^{1} is a homomorphism.

Proof. To see that τ_{μ} : Ḡ → R is a homomorphism we suppose f, g ∈ Ḡ and consider ν as defined above. Then τ_{μ}(fg) = ν(x, f(g(x))) = ν(x, g(x)) + ν(g(x), f(g(x))) == τ_{μ}(g) + τ_{μ}(f).

Since τ_{μ} = τ we know that τ is also a homomorphism. There is a natural homomorphism π : Ḡ → G which assigns to g its projection on S^{1}. If p : R → S^{1} = R/Z is the natural projection then for any f, g ∈ Ḡ,

ρ(π(fg)) = p(τ(fg)) = p(τ(f)) + p(τ(g)) = ρ(π(f)) + ρ(π(g)).

Hence ρ : G → S^{1} is a homomorphism. ⋄

Corollary 2.8. If G is an abelian subgroup of Homeo_{+}(S^{1}) and Ḡ is the group of all lifts to R of elements of G, then Ḡ is abelian and both τ : Ḡ → R and ρ : G → S^{1} are homomorphisms.

Proof. Since G is abelian it is amenable and there is a Borel probability measure μ_{0} on S^{1} invariant under G. Let μ be the lift of this measure to R. Then Corollary 2.7 implies that τ : Ḡ → R and ρ : G → S^{1} are homomorphisms.

As above let π : Ḡ → G be the natural projection. The kernel of π is {T^{n}}_{n∈Z}, where T (x) = x + 1, i.e., the lifts of the identity. If f, g ∈ Ḡ then [f, g] is in the kernel of π so [f, g] = T^{k}, for some k ∈ Z. But since ρ̄ is a homomorphism ρ([f, g]) = 0 which implies k = 0. Hence Ḡ is abelian. ⋄

The following easy lemma turns out to be very useful.

Lemma 2.9. Suppose G is a subgroup of Homeo_{+}(S^{1}) which preserves the Borel probability measure μ_{0} and f_{0}, g_{0} ∈ G satisfy ρ(f_{0}) = ρ(g_{0}). Then f_{0}(x) = g_{0}(x) for all x in the support of μ_{0}.

Proof. Let μ be the lift of the measure μ_{0} to R. Pick lifts f and g of f_{0} and g_{0} respectively which satisfy τ_{μ}(f) = τ_{μ}(g). Then for any x ∈ R we have μ([x, f(x))) = τ_{μ}(f) = τ_{μ}(g) = μ([x, g(x))).

Suppose that g(x) > f(x). It then follows that μ([f(x), g(x)) = 0, so for a sufficiently small ϵ > 0, μ([f(x), f(x) + ϵ]) = 0 and μ([g(x) − ϵ, g(x))) = 0. Applying gf^{−1} to [f(x), f(x) + ϵ) we see that μ([g(x), g(x) + ϵ′)) = 0. Hence μ([g(x) − ϵ, g(x) + ϵ′)) = 0 which is not possible if x and hence g(x) is in the support of μ.

We have shown that x ∈ supp(μ) implies f(x) ≤ g(x). The inequality g(x) ≤ f(x) is similar. Projecting back to S^{1} we conclude that f_{0} = g_{0} on supp(μ_{0}). ⋄

One additional well known property we will use is the following.

Proposition 2.10. If g ∈ Homeo_{+}(S^{1}) is an irrational rotation then its centralizer Z(g) in Homeo_{+}(S^{1}) is the group of rigid rotations of S^{1}.

Proof. Let f be an element of Z(g). Then the group generated by f and g is abelian and hence amenable so there it has an invariant Borel probability measure. But Lebesgue measure is the unique Borel probability measure invariant by the irrational rotation g. Since f preserves orientation and Lebesgue measure, it is a rotation. ⋄

3 Mapping Class Groups

In this section we prove Theorem 1.2. Several of the results, in particular those in §3.1, will also be used to prove Theorem 1.3.

3.1 Fully Supported Diffeomorphisms

One of the main techniques of this paper is to use Kopell’s Lemma (Theorem 2.1) to understand actions of commuting diffeomorphisms. The key dichotomy that arises is the behavior of diffeomorphisms with an interval of fixed points versus those which we call fully supported.

If f is a homeomorphism of a manifold M, we will denote by ∂ Fix(f) the frontier of Fix(f), i.e. the set ∂ Fix(f) = Fix(f) \ Int(Fix(f)).

Definition 3.1 (Fully supported homeomorphism). A homeomorphism f of a manifold M is fully supported provided that Int(Fix(f)) = ∅, or equivalently ∂ Fix(f) = Fix(f). A subgroup G of Homeo(M) will be called fully supported provided that every nontrivial element is fully supported.

It is a trivial but useful observation that if M is connected then for every nontrivial homeomorphism f of M, the set Fix(f) ≠ ∅ if and only if ∂ Fix(f) ≠ ∅.

The following lemma indicates a strong consequence of Kopell’s Lemma (Theorem 2.1).

Lemma 3.2 (Commuting diffeomorphisms on I). Suppose f and g are commuting orientation preserving C^{2} diffeomorphisms of I. Then f preserves every component of Fix(g) and vice versa. Moreover, ∂ Fix(f)) ⊂ Fix(g) and and vice versa. In particular if f and g are fully supported then Fix(f) = Fix(g).

Proof. The proof is by contradiction. Assume X is a component of Fix(g) and f(X) ≠ X. Since f and g commute f(Fix(g)) = Fix(g) so f(X) ≠ X implies f(X) ∩ X = ∅. Let x be an element of X and without loss of generality assume f(x) < x. Define

a = lim_{n→∞} f^{n}(x) and b = lim_{n→−∞} f^{n}(x).

Then a and b are fixed under both f and g and f has no fixed points in (a, b). Then Kopell’s Lemma (Theorem 2.1) implies g(y) = y for all y ∈ [a, b], contradicting the hypothesis that X is a component of Fix(g). The observation that ∂ Fix(f) ⊂ Fix(g) follows from the fact that x ∈ ∂ Fix(f) implies that either {x} is a component of Fix(f) or x is the endpoint of an interval which is a component of Fix(f) so, in either case, x ∈ Fix(g). ⋄

There is also a version of Lemma 3.2 for the circle. For g ∈ Homeo_{+}(S^{1}) let Per(g) denote the set of periodic points of g.

Lemma 3.3 (Commuting diffeomorphisms on S^{1}). Suppose f and g are commuting orientation preserving C^{2} diffeomorphisms of S^{1}. Then f preserves every component of Per(g) and vice versa. Moreover, ∂ Per(f) ⊂ Per(g) and vice versa. In particular if neither Per(f) or Per(g) has interior then Per(f) = Per(g).

Proof. First consider the case that Per(g) = ∅. The assertion that f preserves components of Per(g) is then trivial. Also, g is topologically conjugate to an irrational rotation which by Proposition 2.10 implies that f is topologically conjugate to a rotation and hence Per(f) = ∅ or Per(f) = S^{1}. In either case we have the desired result.

Thus we may assume both f and g have periodic points. Since for any circle homeomorphism all periodic points must have the same period, we can let p be the least common multiple of the periods and observe that Per(f) = Fix(f^{p}) and Per(g) = Fix(g^{p}). Since f and g commute and both f^{p} and g^{p} have fixed points, if x ∈ Fix(f^{p}) then y = lim_{n→∞} g^{np}(x) will exist and be a common fixed point for f^{p} and g^{p}. If we split the circle at y we obtain two commuting diffeomorphisms, f^{p} and g^{p}, of an interval to which we may apply Lemma 3.2 and obtain the desired result. ⋄

Lemma 3.4. Let g_{0} ∈ Diff_{+}^{2}(S^{1}) and let Z(g_{0}) denote its centralizer. Then the rotation number ρ : Z(g_{0}) → S^{1} is a homomorphism.

Proof. We first observe the result is easy if g_{0} has no periodic points. In this case g_{0} is conjugate in Homeo_{+}(S^{1}) to an irrational rotation by Denjoy’s Theorem, and the centralizer of an irrational rotation is abelian by Proposition 2.10. So the centralizer of g in Diff_{+}^{1}(S^{1}) is abelian. But then ρ restricted to an abelian subgroup of Homeo_{+}(S^{1}) is a homomorphism by Proposition 2.8.

Thus we may assume g_{0} has a periodic point, say of period p. Then h_{0} = g_{0}^{p} has a fixed point. Let h be a lift of h_{0} to R which has a fixed point. Let G denote the group of all lifts to R of all elements of Z(g_{0}). Note that every element of this group commutes with h because by Corollary 2.8 any lifts to R of commuting homeomorphisms of S^{1} commute.

Let X = ∂ Fix(h) and observe that g(X) = X for all g ∈ G, so G acts on the unbounded closed set X and G acts on X_{0} = ∂ Fix(h_{0}) which is the image of X under the covering projection to S^{1} . We will show that if Fix(g) ∩ X ≠ ∅ then X ⊂ Fix(g). Indeed applying Lemma 3.3 to h_{0} and map g_{1} of S^{1} which g covers, we observe that ∂ Fix(h) ⊂ Fix(g_{1}) which implies X ⊂ ∂ Fix(g) if g has has a fixed point.

It follows that if H = {g ∈ G |g(x) = x for all x ∈ X} is the stabilizer of X then G/H acts freely on X and hence is abelian by Hölder’s theorem. Also G/H acts on X_{0} and hence there is a measure μ_{0} supported on X_{0} and invariant under G/H. Clearly this measure is also invariant under the action of Z(g_{0}) on S^{1}. The measure μ_{0} lifts to a G-invariant measure μ.

It follows from Corollary 2.7 that the translation number τ : G → R and the rotation number ρ : Z(g_{0}) → S^{1} are homomorphisms. ⋄

The commutativity graph of a set S of generators of a group is the graph consisting of one vertex for each s ∈ S and an edge connecting elements of S which commute.

Theorem 3.5 (abelian criterion for I). Let {g_{1}, g_{2}, …, g_{k}} be a set of fully supported elements of Diff_{+}^{2}(I) and let G be the group they generate. Suppose that the commutativity graph of this generating set is connected. Then G is abelian.

Proof. By Lemma 3.2 we may conclude that for each j_{1}, j_{2} we have Fix(g_{j_{1}}) = Fix(g_{j_{2}}). Call this set of fixed points F. Clearly F is the set of global fixed points of G.

Fix a value of i and consider Z(g_{i}). Restricting to any component U of the complement of F we consider the possiblity that there is an h ∈ Z(g_{i}) with a fixed point in U. Kopell’s Lemma (Theorem 2.1), applied to the closure of the open inteval U, tells us that such an h is the identity on U. Thus the restriction of Z(g_{i}) to U is free and hence abelian by Hölder’s Theorem. But U was an arbitrary component of the complement of F and obviously elements of Z(g_{i}) commute on their common fixed set F. So we conclude that Z(g_{i}) is abelian.

We have hence shown that if g_{j} and g_{k} are joined by a path of length two in the commutativity graph, they are joined by a path of length one. A straightforward induction shows that any two generators are joined by a path of length one, i.e. any two commute. ⋄

There is also a version of Theorem 3.5 for the circle.

Corollary 3.6 (Abelian criterion for S^{1}). Let {g_{1}, g_{2}, …, g_{k}} be a set of fully supported elements of Diff_{+}^{2}(S^{1}), each of which has a fixed point, and let G be the group they generate. Suppose that the commutativity graph of these generators is connected. Then G is abelian.

Proof. By Lemma 3.3 we may conclude that for each j, k we have Per(g_{j}) = Per(g_{k}). But since each g_{i} has a fixed point Per(g_{i}) = Fix(g_{i}). Hence there is a common fixed point for all of the generators. Splitting S^{1} at a common fixed point we see G is isomorphic to a subgroup of Diff_{+}^{2}(I) satisfying the hypothesis of Theorem 3.5. It follows that G is abelian. ⋄

Question. Let G be a subgroup of Homeo_{+}(S^{1}). If the commutativity graph for a set of generators of G is connected, does this imply that rotation number is a homomorphism when restricted to G?

We will need to understand relations between fixed sets not just of commuting diffeomorphisms, but also of diffeomorphisms with another basic relation which occurs in mapping class groups as well as in automorphism groups of free groups.

Lemma 3.7 (aba lemma). Suppose a and b are elements of Homeo_{+}(I) or elements of Homeo_{+}(S^{1}) which have fixed points. Suppose also that a and b satisfy the relation a^{n_{1}}b^{m_{3}}a^{n_{2}} = b^{m_{1}}a^{n_{3}}b^{m_{2}} with m_{1} + m_{2} ≠ m_{3}. If J is a nontrivial interval in Fix(a) then either J ⊂ Fix(b) or J ∩ Fix(b) = ∅.

Proof. Suppose J ⊂ Fix(a) and suppose z ∈ J ∩ Fix(b). We will show that this implies that J ⊂ Fix(b).

Let x_{0} ∈ J. Observe that a^{n_{1}}b^{m_{3}}a^{n_{2}} = b^{m_{1}}a^{n_{3}}b^{m_{2}} implies a^{−n_{2}}b^{−m_{3}}a^{−n_{1}} = b^{−m_{2}}a^{−n_{3}}b^{−m_{1}}. Hence we may assume without loss of generality that b^{m_{3}}(x_{0}) is in the subinterval of J with endpoints x_{0} and z (otherwise replace a and b by their inverses). In particular b^{m_{3}}(x_{0}) ∈ J. Note that

b^{m_{3}}(x_{0}) = a^{n_{1}}b^{m_{3}}(x_{0}) = a^{n_{1}}b^{m_{3}}a^{n_{2}}(x_{0}) = b^{m_{1}}a^{n_{3}}b^{m_{2}}(x_{0}) = b^{m_{1}}b^{m_{2}}(x_{0}) = b^{m_{1}+m_{2}}(x_{0})

This implies b^{m_{1}+m_{2}−m_{3}}(x_{0}) = x_{0} and since m_{1} +m_{2} −m_{3} ≠ 0 we can conclude that b(x_{0}) = x_{0}. Since x_{0} ∈ J was arbitrary we see J ⊂ Fix(b). ⋄

3.2 Groups of type MC(n)

We will need to abstract some of the properties of the standard generating set for the mapping class group of a surface in order to apply them in other circumstances.

Definition 3.8 (Groups of type MC(n)). We will say that a group G is of type MC(n) provided it is nonabelian and has a set of generators {a_{i}, b_{i}}_{i=1}^{n} ∪ {c_{j}}_{j=1}^{n−1} with the properties that

1. a_{i}a_{j} = a_{j}a_{i}, b_{i}b_{j} = b_{j}b_{i}, c_{i}c_{j} = c_{j}c_{i}, a_{i}c_{j} = c_{j}a_{i}, for all i, j,

2. a_{i}b_{j} = b_{j}a_{i}, if i ≠ j,

3. b_{i}c_{j} = c_{j}b_{i}, if j ≠ i, i − 1, 4. a_{i}b_{i}a_{i} = b_{i}a_{i}b_{i} for 1 ≤ i ≤ n, and b_{i}c_{j}b_{i} = c_{j}b_{i}c_{j} whenever j = i, i − 1.

A useful consequence of the MC(n) condition is the following.

Lemma 3.9. If G is a group of type MC(n) and {a_{i}, b_{i}}_{i=1}^{n} ∪ {c_{j}}_{j=1}^{n−1} are the generators guaranteed by the definition then any two of these generators are conjugate in G.

Proof. The relation aba = bab implies a = (ab)^{−1}b(ab), so a and b are conjugate. Therefore property (4) of the definition implies a_{i} is conjugate to b_{i} and b_{i} is conjugate to c_{i} and c_{i−1}. This proves the result. ⋄

The paradigm of a group of type MC(n) is the mapping class group Mod(n, 0).

Proposition 3.10 (Mod(n, 0) and Mod(n, 1)). For n > 2 and k = 0, 1 the group Mod(n, k) contains a set of elements

S = {a_{i}, b_{i}}_{i=1}^{n} ∪ {c_{j}}_{j=1}^{n−1}

with the following properties:

1. For k = 0 the set S generates all of Mod(n, 0).

2. The group generated by S is of type MC(n).

3. There is an element of g ∈ Mod(n, k) such that g^{−1}a_{1}g = a_{n}, g^{−1}b_{1}g = b_{n}, g^{−1}b_{2}g = b_{n−1}, and g^{−1}c_{1}g = c_{n−1}.

4. If G_{0} is the subgroup of Mod(n, k) generated by the subset

{a_{i}, b_{i}}_{i=1}^{n−1} ∪ {c_{j}}_{j=1}^{n−2}

then G_{0} is of type MC(n − 1).

Proof. Let Σ denote the surface of genus n with k punctures. Let i(α, β) denote the geometric intersection number of the (isotopy classes of) closed curves α and β on Σ. Let a_{i}, b_{i}, c_{j} with 1 ≤ i ≤ n, 1 ≤ j ≤ n be Dehn twists about essential, simple closed curves α_{i}, β_{i}, γ_{j} with the following properties:

• i(α_{i}, α_{j}) = i(β_{i}, β_{j}) = i(γ_{i}, γ_{j}) = 0 for i ≠ j.

• i(α_{i}, β_{i}) = 1 for each 1 ≤ i ≤ n.

• i(γ_{j}, β_{j}) = 1 = i(γ_{j}, β_{j+1}) for 1 ≤ j ≤ n − 1.

It is known (see, e.g.[Iv]) that there exist such a_{i}, b_{i}, c_{j} which generate Mod(n, 0); we choose such elements. These are also nontrivial in Mod(n, 1). Since Dehn twists about simple closed curves with intersection number zero commute, and since Dehn twists a, b about essential, simple closed curves with intersection number one satisy aba = bab (see, e.g. Lemma 4.1.F of [Iv]), properties (1)-(4) in Definition 3.8 follow. We note that the proofs of these relations do not depend on the location of the puncture, as long as it is chosen off the curves α_{i}, β_{i}, γ_{j}.

To prove item (2) of the lemma, consider the surface of genus g − 2 and with 2 boundary components obtained by cutting Σ along α_{1} and γ_{1}. The loop β_{1} becomes a pair of arcs connecting 2 pairs of boundary components, and β_{2} becomes an arc connecting 2 boundary components. The genus, boundary components, and combinatorics of arcs is the exact same when cutting Σ along α_{n} and γ_{n−1}. Hence by the classification of surfaces it follows that there is a homeomorphism between resulting surfaces, inducing a homeomorphism h : Σ → Σ with h(α_{1}) = α_{n}, h(γ_{1}) = γ_{n−1} and h(β_{i}) = β_{n−i+1}, i = 1, 2. The homotopy class of h gives the required element g.

To prove item (3), let τ be an essential, separating, simple closed curve on Σ such that one of the components Σ′ of Σ − τ has genus one and contains a_{n} and b_{n}. Then there is a homeomorphism h ∈ Homeo_{+}(Σ) taking any element of {a_{i}, b_{i}}_{i=1}^{n−1}∪{c_{j}}_{j=1}^{n−2} to any other element, and which is the identity on Σ′. Then the isotopy class of h, as an element of Mod(n, 0), is the required conjugate, since it lies in the subgroup of Mod(0, n) of diffeomorphisms supported on Σ − Σ′, which is isomorphic to Mod(n − 1, 1) and equals G_{0}. ⋄

3.3 Actions on the interval

In this subsetion we consider actions on the interval I = [0, 1].

Theorem 3.11 (MC(n) actions on I). Any C^{2} action of a group G of type MC(n) for n ≥ 2 on an interval I is abelian.

Proof. We may choose a set of generators {a_{i}, b_{i}}_{i=1}^{n} ∪ {c_{j}}_{j=1}^{n−1} for G with the properties listed in Definition 3.8.

If G has global fixed points other than the endpoints of I we wish to consider the restriction of the action to the closure of a component of the complement of these global fixed points. If all of these restricted actions are abelian then the original action was abelian. Hence it suffices to prove that these restrictions are abelian. Thus we may consider an action of G on a closed interval I_{0} with no interior global fixed points. None of the generators above can act trivially on I_{0} since the fact that they are all conjugate would mean they all act trivially.

We first consider the case that one generator has a nontrivial interval of fixed points in I_{0} and show this leads to a contradiction. Since they are all conjugate, each of the generators has a nontrivial interval of fixed points.

Choose an interval J of fixed points which is maximal among all intervals of fixed points for all of the a_{i}. That is, J is a nontrivial interval of fixed points for one of the a_{i}, which we assume (without loss of generality) is a_{2}, and there is no J′ which properly contains J and which is pointwise fixed by some a_{j}.

Suppose J = [x_{0}, x_{1}]. At least one of x_{0} and x_{1} is not an endpoint of I_{0} since a_{2} is not the identity. Suppose it is x_{0} which is in Int(I_{0}). Then x_{0} ∈ ∂ Fix(a_{2}), so by Lemma 3.2 we know that x_{0} is fixed by all of the generators except possibly b_{2}, since b_{2} is the only generator with which a_{2} does not commute. The point x_{0} cannot be fixed by b_{2} since otherwise it would be an interior point of I_{0} fixed by all the generators, but there are no global fixed points in I_{0} other than the endpoints. The identity a_{2}b_{2}a_{2} = b_{2}a_{2}b_{2} together with Lemma 3.7 then tells us that b_{2}(J) ∩ J = ∅.

Now assume without loss of generality that b_{2}(x_{0}) > x_{0} (otherwise replace all generators by their inverses). Define

y_{0} = lim_{n→−∞} b_{2}^{n}(x_{0}) and y_{1} = lim_{n→∞} b_{2}^{n}(x_{0}).

Since b_{2}(J) ∩ J = ∅ we have b_{2}^{−1}(J) ∩ J = ∅ and hence y_{0} < x_{0} < x_{1} < y_{1}. But x_{0} is a fixed point of a_{1} (as well as the other generators which commute with a_{2}) and a_{1} commutes with b_{2}. Hence y_{0} and y_{1} are fixed by a_{1} (since b_{2} preserves Fix(a_{1})). We can now apply Kopell’s Lemma (Theorem 2.1) to conclude that a_{1} is the identity on [y_{0}, y_{1}] which contradicts the fact that J = [x_{0}, x_{1}] is a maximal interval of fixed points among all the a_{i}.

We have thus contradicted the supposition that one of the generators has a nontrivial interval of fixed points in I_{0}, so we may assume that each of the generators, when restricted to I_{0}, has fixed point set with empty interior. That is, each generator is fully supported on I_{0}.

We now note that given any two of the generators above, there is another generator h with which they commute. Thus we may conclude from Lemma 3.5 that the action of G on I_{0} abelian. ⋄

Proof of Theorem 1.2 for I: By Proposition 3.10, the mapping class group Mod(g, k), g ≥ 2, k = 0, 1 is a group of type MC(g). Since every abelian quotient of Mod(g, k), g ≥ 2 is finite (see, e.g. [Iv]), in fact trivial for g ≥ 3, and since finite groups must act trivially on I, the statement of Theorem 1.2 for I follows.

3.4 Actions on the circle

We are now prepared to prove Theorem 1.2 in the case of the circle S^{1}.

Theorem 1.2 (Circle case). Any C^{2} action of the mapping class group M(g, k) of a surface of genus g ≥ 3 with k = 0, 1 punctures on S^{1} must be trivial.

As mentioned in the introduction, the C^{2} hypothesis is necessary.

Proof. Closed case. We first consider the group G = Mod(g, 0), and choose generators {a_{i}, b_{i}}_{i=1}^{n} ∪ {c_{j}}_{j=1}^{n−1} for G with the properties listed in Definition 3.8 and Proposition 3.10. All of these elements are conjugate by Lemma 3.9, so they all have the same rotation number.

We wish to consider a subgroup G_{0} of type MC(n − 1). We let u = a_{n}^{−1} and for 1 ≤ i ≤ n − 1 we set A_{i} = a_{i}u and B_{i} = b_{i}u. For 1 ≤ i ≤ n − 2 we let C_{i} = c_{i}u. Since u commutes with any element of {a_{i}, b_{i}}_{i=1}^{n−1} ∪ {c_{j}}_{j=1}^{n−2} we have the relations 1. A_{i}A_{j} = A_{j}A_{i}, B_{i}B_{j} = B_{j}B_{i}, C_{i}C_{j} = C_{j}C_{i}, A_{i}C_{j} = C_{j}A_{i}, for all i, j,

2. A_{i}B_{j} = B_{j}A_{i}, if i ≠ j,

3. B_{i}C_{j} = C_{j}B_{i}, if j ≠ i, i + 1,

4. A_{i}B_{i}A_{i} = B_{i}A_{i}B_{i} and B_{i}C_{j}B_{i} = C_{j}B_{i}C_{j} whenever j = i, i − 1.

We now define G_{0} to be the subgroup of G generated by {A_{i}, B_{i}}_{i=1}^{n−1} ∪ {C_{j}}_{j=1}^{n−2}. The fact that u commutes with each of {a_{i}, b_{i}}_{i=1}^{n−1} ∪ {c_{j}}_{j=1}^{n−2} and has the opposite rotation number implies that the rotation number of every element of {A_{i}, B_{i}}_{i=1}^{n−1} ∪ {C_{j}}_{j=1}^{n−2} is 0. Thus each of these elements has a fixed point.

We note that given any two of these generators of G_{0} there is another generator with which they commute. If all of these generators have fixed point sets with empty interior then we may conclude from Lemma 3.2 that any two of them have equal fixed point sets, i.e. that Fix(A_{i}) = Fix(B_{j}) = Fix(C_{k}) for all i, j, k. So in this case we have found a common fixed point for all generators of G_{0}. If we split S^{1} at this point we get an action of G_{0} on an interval I which must be abelian by Theorem 3.11. It follows that all of the original generators {a_{i}, b_{i}}_{i=1}^{n} ∪ {c_{j}}_{j=1}^{n−1} commute with each other except possibly c_{n−1} may not commute with b_{n−1} and b_{n}, and a_{n} and b_{n} may not commute.

Let ψ : G → Diff_{+}^{2}(S^{1}) be the putative action, and consider the element g guaranteed by Proposition 3.10. We have that

[ψ(a_{n}), ψ(b_{n})] = ψ(g)^{−1}[ψ(a_{1}), ψ(b_{1})]ψ(g) = id

and

[ψ(c_{n−1}), ψ(b_{n})] = ψ(g)^{−1}[ψ(c_{1}), ψ(b_{1})]ψ(g) = id

and

[ψ(c_{n−1}), ψ(b_{n−1})] = ψ(g)^{−1}[ψ(c_{1}), ψ(b_{2})]ψ(g) = id

Hence G is abelian, hence trivial since (see [Iv]) abelian quotients of Mod(g, k), g ≥ 3 are trivial.

Thus we are left with the case that one generator of G_{0} has a nontrivial interval of fixed points in S^{1}. Since they are all conjugate, each of the generators of G_{0} has has a nontrivial interval of fixed points. Also we may assume no generator fixes every point of S^{1} since if one did the fact that they are all conjugate would imply they all act trivially.

Choose a maximal interval of fixed points J for any of the subset of generators {A_{i}}. That is, J is a nontrivial interval of fixed points for one of the {A_{i}}, which we assume (without loss of generality) is A_{2}, and there is no J′ which properly contains J and which is pointwise fixed by some A_{j}.

Suppose x_{0} and x_{1} are the endpoints of J. Then x_{0}, x_{1} ∈ ∂ Fix(A_{2}) so by Lemma 3.3, x_{0} and x_{1} are fixed points for all of the generators except B_{2}, since B_{2} is the only one with which A_{2} does not commute. If the point x_{0} ∈ Fix(B_{2}) we have found a common fixed point for all the generators of G_{0} and we can split S^{1} at this point obtaining an action of G_{0} on an interval which implies G_{0} is abelian by Theorem 3.11. So suppose x_{0} ∉ Fix(B_{2}).

Recall that the diffeomorphism B_{2} has rotation number 0, so it fixes some point. Thus the identity A_{2}B_{2}A_{2} = B_{2}A_{2}B_{2}, together with the fact that B_{2} cannot fix x_{0}, implies by Lemma 3.7 that B_{2}(J) ∩ J = ∅ and J ∩ B_{2}^{−1}(J) = ∅. Define

y_{0} = lim_{n→−∞} B_{2}^{n}(x_{0}) and y_{1} = lim_{n→∞} B_{2}^{n}(x_{0}).

We will denote by K the interval in S^{1} with endpoints y_{0} and y_{1} which contains x_{0}. Note that K properly contains J and that B_{2}(K) = K.

Since x_{0} ∈ Fix(A_{1}) and A_{1} commutes with B_{2} we conclude y_{0} and y_{1} are fixed by A_{1}. We can now apply Kopell’s Lemma (Theorem 2.1) to the interval K with diffeomorphisms B_{2} and A_{1} to conclude that A_{1} is the identity on K. But J is a proper subinterval of K which contradicts the fact that J was a maximal interval of fixed points for any one of the A_{i}. Thus in this case too we have arrived at a contradiction.

Punctured case. We now consider the group Mod(g, 1), and choose elements {a_{i}, b_{i}}_{i=1}^{n} ∪ {c_{j}}_{j=1}^{n−1} as above. The argument above shows that the subgroup G of Mod(g, 1) generated by these elements ^{1} acts trivially on S^{1}. We claim that the normal closure of G in Mod(g, 1) is all of Mod(g, 1), from which it follows that Mod(g, 1) acts trivially, finishing the proof.

To prove the claim, let S_{g} denote the closed surface of genus g and recall the exact sequence (see, e.g. [Iv]):

1 → π_{1}(S_{g}) → Mod(g, 1) → Mod(g, 0) → 1

where π_{1}(Sigma_{g}) is generated by finitely many “pushing the point” homeomorphisms p_{τ} around generating loops τ in π_{1}(S_{g}) with basepoint the puncture. Each generating loop τ has intersection number one with exactly one of the loops, say β_{i}, corresponding to one of the twist generators of G. Conjugating b_{i} by p_{τ} gives a twist about a loop β′_{i} which together with β_{i} bounds an annulus containing the puncture. The twist about β_{i} composed with a negative twist about β′_{i} gives the isotopy class of p_{τ}. In this way we see that the normal closure of G in Mod(g, 1) contains G together with each of the generators of the kernel of the above exact sequence, proving the claim.

⋄

4 Aut(F_{n}), Out(F_{n}) and other subgroups

In §4.1 we prove Theorem 1.3. Note that the result of Bridson-Vogtmann mentioned in the introduction implies this theorem since it is easy to see that any finite subgroup of Homeo_{+}(S^{1}) is abelian so their result implies ours. We give our proof here because it is short, straightforward and provides a good illustration of the use of the techniques developed above. While several aspects of the proof are similar to the case of Mod(g, 0), we have not been able to find a single theorem from which both results follow.

1We do not know wether or not G actually equals Mod(g, 1); there seems to be some confusion about this point in the literature.

In §4.2 we prove theorem 1.4, extending the application to finite index subgroups of Mod(g, k) and Aut(F_{n}).

4.1 Actions of Aut(F_{n}) and Out(F_{n})

We begin with a statement of a few standard facts about generators and relations of Aut(F_{n}).

Lemma 4.1 (Generators for Aut(F_{n})). The group Aut(F_{n}) has a subgroup of index two which has a set of generators {A_{ij}, B_{ij}} with i ≠ j, 1 ≤ i ≤ n and 1 ≤ j ≤ n. These generators satisfy the relations

A_{ij}A_{kl} = A_{kl}A_{ij} and B_{ij}B_{kl} = B_{kl}B_{ij} if {i, j} ∩ {k, l} = ∅  (i)
[A_{ij}, A_{jk}] = A_{ik}^{−1} and [A_{ij}, A_{jk}^{−1}] = A_{ik}  (ii)
[B_{ij}, B_{jk}] = B_{ik}^{−1} and [B_{ij}, B_{jk}^{−1}] = B_{ik}  (iii)
A_{ij}A_{ji}^{−1}A_{ij} = A_{ji}^{−1}A_{ij}A_{ji}^{−1} and B_{ij}B_{ji}^{−1}B_{ij} = B_{ji}^{−1}B_{ij}B_{ji}^{−1}  (iv)

Proof. Let {e_{i}}_{i=1}^{n} be the generators of F_{n} and define automorphisms A_{ij} and B_{ij} by

A_{ij}(e_{k}) = \begin{cases}e_{i}e_{j}, & if i = k, \\ e_{k}, & otherwise, and \end{cases}

B_{ij}(e_{k}) = \begin{cases}e_{j}e_{i}, & if i = k, \\ e_{k}, & otherwise. \end{cases}

Then {A_{ij}, B_{ij}} with i ≠ j, generate the index two subgroup of Aut(F_{n}) given by those automorphisms which induce on the abelianization Z^{n} of F_{n} an automorphism of determinant one (see, e.g. [LS]). A straightforward but tedious computation shows that relations (i) – (iv) are satisified. ⋄

We next find some fixed points.

Lemma 4.2. Suppose n > 4 and ϕ : Aut(F_{n}) → Diff_{+}^{2}(S^{1}) is a homomorphism with a_{ij} = ϕ(A_{ij}) and b_{ij} = ϕ(B_{ij}) where A_{ij} and B_{ij} are the generators of Lemma 4.1. Then each of the diffeomorphisms a_{ij} and b_{ij} has a fixed point.

Proof. We fix i, j and show a_{ij} has a fixed point. Since n > 4 there is an a_{kl} with {i, j}∩{k, l} = ∅. Let Z(a_{kl}) denote the centralizer of a_{kl}, so a_{ij} ∈ Z(a_{kl}). Also, since n > 4 there is 1 ≤ q ≤ n which is distinct from i, j, k, l, so a_{iq}, a_{qj} ∈ Z(a_{kl}).

By Lemma 3.4 we know that the rotation number ρ : Z(a_{kl}) → R is a homomorphism so ρ(a_{ij}) = ρ([a_{iq}, a_{qj}]) = 0. This implies that a_{ij} has a fixed point. ⋄

We can now prove the main result of this section Theorem 1.3. For n ≥ 6, any homomorphisms from Aut(F_{n}) or or Out(F_{n}) to Diff_{+}^{2}(I) or Diff_{+}^{2}(S^{1}) factors through Z/2Z.

Proof. Let H be the index two subgroup of Aut(F_{n}) from Lemma 4.1. Let ϕ be a homomorphism H to Diff_{+}^{2}(I) or Diff_{+}^{2}(S^{1}) and let a_{ij} = ϕ(A_{ij}) and b_{ij} = ϕ(B_{ij}) where A_{ij} and B_{ij} are the generators from Lemma 4.1. Then {a_{ij}, b_{ij}} are generators for Image(ϕ). We will show in fact that the subgroup G_{a} generated by {a_{ij}} is trivial, as is the subgroup G_{b} generated by {b_{ij}}. The arguments are identical so we consider only the set {a_{ij}}. Since n ≥ 6, we have by Lemma 4.1 that the commutavity graph for the generators {a_{ij}} of G_{a} is connected. Hence by Corollary 3.6, if these generators are fully supported then G_{a} is abelian. But the relations [a_{ij}, a_{jk}^{−1}] = a_{ik} then imply that G_{a} is trivial.

Hence we may assume at least one of these generators is not fully supported. Let J be an interval which is a maximal component of fixed point sets for any of the a_{i}. More precisely, we choose J so that there is a_{pq} with J a component of Fix(a_{pq}) and so that there is no a_{kl} such that Fix(a_{kl}) properly contains J.

We wish to show that J is fixed pointwise by each a_{ij}. In case {p, q}∩{i, j} = ∅ we note that at least the endpoints of J are fixed by a_{ij} because a_{pq} and a_{ij} commute and the endpoints of J are in ∂ Fix(a_{pq}). So Lemma 3.2 or Lemma 3.3 implies these endpoints are fixed by a_{ij}.

For the general case we first show a_{ij}(J) ∩ J ≠ ∅. In fact there is no a_{ij} with the property that a_{ij}(J) ∩ J = ∅ because if there were then the interval J′ defined to be the smallest interval containing {a_{ij}^{n}(J)}_{n∈Z} is an a_{ij}-invariant interval with no interior fixed points for a_{ij}. (In case G_{a} ⊂ Diff_{+}^{2}(S^{1}) we need the fact that a_{ij} has a fixed point, which follows from Lemma 4.2, in order to know this interval exists.) But since n ≥ 6 there is some a_{kl} which commutes with both a_{ij} and a_{pq}, and it leaves both J and J′ invariant. Applying Kopell’s Lemma (Theorem 2.1) to a_{kl} and a_{ij} we conclude that J′ ⊂ Fix(a_{kl}), which contradicts the maximality of J. Hence we have shown that a_{ij}(J) ∩ J ≠ ∅ for all i, j.

Now by Lemma 4.1 the relation a_{pq}a_{qp}^{−1}a_{pq} = a_{qp}^{−1}a_{pq}a_{qp}^{−1}, holds so we may apply Lemma 3.7 to conclude that J is fixed pointwise by a_{qp}.

We next consider the generator a_{pk}. If x_{0} is an endpoint of J then since a_{qk}(J) ∩ J ≠ ∅ at least one of a_{qk}(x_{0}) and a_{qk}^{−1}(x_{0}) must be in J. Hence the relations [a_{pq}, a_{qk}] = a_{pk}^{−1} and [a_{pq}, a_{qk}^{−1}] = a_{pk} imply that a_{pk}(x_{0}) = x_{0}. This holds for the other endpoint of J as well, so J is invariant under a_{pk} for all k. The same argument shows J is invariant under a_{qk} for all k. But since a_{pq} is the identity on J we conclude from [a_{pq}, a_{qk}] = a_{pk}^{−1} that a_{pk} is the identity on J. Similarly a_{qk} is the identity on J.

Next the relation a_{qk}a_{kq}^{−1}a_{qk} = a_{kq}^{−1}a_{qk}a_{kq}^{−1}, together with Lemma 3.7 implies that a_{kq} is the identity on J. A similar argument gives the same result for a_{kp}. Finally, the relation [a_{ip}, a_{pj}^{−1}] = a_{ij} implies that a_{ij} is the identity on J.

Thus we have shown that any subgroup of Diff_{+}^{2}(I) or Diff_{+}^{2}(S^{1}) which is a homomorphic image of H, the index two subgroup of Aut(F_{n}), has an interval of global fixed points. In the case of Diff_{+}^{2}(S^{1}) we can split at a global fixed point to get a subgroup of Diff_{+}^{2}(I) which is a homomorphic image of H. In the I case we can restrict the action to a subinterval on which the action has no global fixed point. But our result then says that for the restricted action there is an interval of global fixed points, which is a contradiction. We conclude that the subgroup of G_{a} generated by {a_{ij}} is trivial and the same argument applies to the subgroup G_{b} generated by {b_{ij}}. So ϕ(H) is trivial. Since Aut(F_{n})/H has order two any homomorphism ϕ from Aut(F_{n}) to Diff_{+}^{2}(I) or Diff_{+}^{2}(S^{1}) has an image whose order is at most two. The fact that Diff_{+}^{2}(I) has no elements of finite order implies that in this case ϕ is trivial.

Since there is a natural homomorphism from Aut(F_{n}) onto the group Out(F_{n}), the statement of Theorem 1.3 for Out(F_{n}) holds. ⋄

4.2 Finite index and other subgroups

In this section we prove Theorem 1.4. Recall that the Torelli group T_{g,k} is the subgroup of Mod(g, k) consisting of diffeomorphisms of the k-punctured, genus g surface Σ_{g,k} which act trivially on H^{1}(Σ_{g,k}; Z). Of course any finite index subgroup of Mod(g, k) contains a finite index subgroup of T_{g,k}, so we need only prove the thoerem for the latter group; that is, (3) implies (1).

So, let H be any finite index subgroup of T_{g,k}, g ≥ 3, k ≥ 0 or of Aut(F_{n}) or Out(F_{n}) for n ≥ 6. Suppose that H acts faithfully as a group of C^{2} diffeomorphisms of I or S^{1}, so that no nontrivial element of H has an interval of fixed points (i.e. the action is fully supported, in the terminology of §3.1). In each case we claim that H contains infinite order elements a, b, c with the property that, for each r ≥ 2, the elements a^{r}, b^{r} generate an infinite, non-abelian group which commutes with c^{s} for all s.

To prove the claim for T_{g,k} one takes a, b, c to be Dehn twists about homologically trivial, simple closed curves with a and b having positive intersection number and c disjoint from both. For r sufficiently large the elements a^{r}, b^{r}, c^{r} clearly lie in H. As powers of Dehn twists commute if and only if the curves have zero intersection number, the claim is proved for T_{g}, k. A direct calculation using appropriate generators A_{ij} as in Lemma 4.1 gives the same claim for Aut(F_{n}) and Out(F_{n}) for n > 2.

Theorem 1.4 in the case of I now follows immediately from Theorem 3.5 and the fact that the group < a^{r}, b^{r}, c^{r} > has connected commuting graph but is non-abelian. For the case of S^{1}, note that if c^{r} has a fixed point for any r > 0 so do a^{r}, b^{r} by Lemma 3.2, so after taking powers we are done by the same argument, this time applying Corollary 3.6.

If no positive power of c has a fixed point on S^{1}, then c must have irrational rotation number, in which case the centralizer of c in Homeo(S^{1}) is abelian by 2.10. But < a, b > lie in this centralizer and do not commute, a contradiction.

The proof of Theorem 1.4 for Aut(F_{n}) and Out(F_{n}) is similar to the above, so we leave it to the reader.

5 Baumslag-Solitar groups

In this section we study actions of the groups

BS(m, n) =< a, b : ab^{m}a^{−1} = b^{n} >

For n > m > 1 these groups are not residually finite, hence are not linear (see, e.g. [LS]).

5.1 An analytic action of BS(m, n) on R

In this subsection we construct an example of an analytic action of BS(m, n), 1 < m < n on R. It is straightforward to find diffeomorphisms of R which satisfy the Baumslag-Solitar relation.

Proposition 5.1 (Diffeomorphisms satisfying the Baumslag-Solitar relation). Let f_{n} : S^{1} → S^{1} be any degree n covering and let f_{m} : S^{1} → S^{1} be any degree m covering. Let g_{n} and g_{m} be lifts of f_{n} and f_{m} respectively to the universal cover R. Then the group of homeomorphisms of R generated by g = g_{m}g_{n}^{−1} and the covering translation h(x) = x + 1 satisfies the relation gh^{m}g^{−1} = h^{n}.

Proof. We have

g_{n}^{−1}(h^{n}(x)) = g_{n}^{−1}(x + n)
= g_{n}^{−1}(x) + 1
= h(g_{n}^{−1}(x)),

and

g_{m}(h(x)) = g_{m}(x + 1)
= g_{m}(x) + m
= h^{m}(g_{m}(x)).

So

g(h^{n}(x)) = g_{m}(g_{n}^{−1}(h^{n}(x)))
= g_{m}(h(g_{n}^{−1}(x)))
= h^{m}(g_{m}(g_{n}^{−1}(x)))
= h^{m}(g(x)).

⋄

We will now show how to choose the diffeomorphisms constructed in Proposition 5.1 more carefully so that we will be able to use a Schottky type argument to show that the diffeomorphisms will satisfy no other relations. Part of the difficulty will be that the “Schottky sets” will have to have infinitely many components.

We first construct, for each j ≥ 1, a C^{∞} diffeomorphism Θ_{j} of R which is the lift of a map of degree j on S^{1}. Let a = 1/10. We choose a C^{∞} function Θ_{j} : (−a/2, 1 + a/2) → R with the following properties:

Θ_{j}(x) = \frac{a}{2}x for x ∈ (−a/2, a/2),
Θ_{j}(x) = j + \frac{a}{2}(x − 1) for ∈ (a, 1 + a/2),
Θ′_{j}(x) > 0.

Note that for x ∈ (−a/2, a/2) we have Θ_{j}(x + 1) = Θ_{j}(x) + j. It follows that we can extend Θ_{j} to all of R by the rule Θ_{j}(x + k) = Θ_{j}(x) + jk. Thus the extended Θ_{j} is the lift of a map of degree j on S^{1}. This map is in fact a covering map since Θ_{j} is a diffeomorphism.

We shall be particularly interested in the intervals A = [−a, a], A^{s} = [−a, 0], A^{u} = [0, a], and C = [a, 1 − a]. Note that [0, 1] = A^{u} ∪ C ∪ (A^{s} + 1). Also by construction Θ_{j}(A^{s}) ⊂ A^{s} and Θ_{j}(C) ⊂ (A^{s} + j). More importantly we have the following inclusions concerning the union of integer translates of these sets. For a set X ⊂ R and a fixed i_{0}, we will denote by X + i_{0} the set {x + i_{0}| x ∈ X}, by X + Z the set {x + i| x ∈ X, i ∈ Z}, and by X + mZ the set {x + mi| x ∈ X, i ∈ mZ}.

Suppose k is not congruent to 0 modulo j. Then we have

Θ_{j}(A^{s} + Z) ⊂ A^{s} + jZ,
Θ_{j}(C + Z) ⊂ A^{s} + jZ,
Θ_{j}(A^{u} + jZ + k) ⊂ A^{s} + jZ
Θ_{j}^{−1}(C + Z) ⊂ A^{u} + Z.

We now define two diffeomorphisms of R. Let g_{n}(x) = Θ_{n}(x) and let g_{m}(x) = Θ_{m}(x − 1/2) + 1/2. We define B = A + 1/2, B^{s} = A^{s} + 1/2, B^{u} = A^{u} + 1/2, and C_{2} = C + 1/2. Then from the equations above we have

g_{n}(C + Z) ⊂ A^{s} + nZ
g_{n}^{−1}(C + Z) ⊂ A^{u} + Z
g_{n}^{−1}(A + k) ⊂ A^{u} + Z if k ∉ nZ  (1)
g_{m}(C_{2} + Z) ⊂ B^{s} + mZ
g_{m}^{−1}(C_{2} + Z) ⊂ B^{u} + Z  (2)
g_{m}^{−1}(B + k) ⊂ B^{u} + Z if k ∉ mZ  (3)

The diffeomorphisms g_{m} and g_{n} can be approximated by analytic diffeomorphisms which still satisfy the equations above and are still lifts of covering maps on S^{1}. More precisely, the function ϕ(x) = g_{n}(x) − nx is a periodic function on R and may be C^{1} approximated by a periodic analytic function. If we replace g_{n} by ϕ(x) + nx then the new analytic g_{n} will have postive derivative and be a lift of a degree n map on S^{1}. If the approximation is sufficiently close, it will also still satisfy the equations above. In a similar fashion we may perturb g_{m} to be analytic while retaining its properties.

Note that B + Z ⊂ C + Z and A + Z ⊂ C_{2} + Z, so we have the following key properties:

g_{n}(B + Z) ⊂ A^{s} + nZ,  (4)
g_{n}^{−1}(B + Z) ⊂ A^{u} + Z  (5)
g_{m}(A + Z) ⊂ B^{s} + mZ  (6)
g_{m}^{−1}(A + Z) ⊂ B^{u} + Z  (7)

Define g(x) = g_{n}(g_{m}^{−1}(x)) and h(x) = x+1, and let G be the group generated by g and h. Propostion 5.1 implies that gh^{m}g^{−1} = h^{n}. Hence there is a surjective homomorphism Φ : BS(m, n) → G sending b to g and a to h. We need only show that Φ is injective.

Lemma 5.2 (normal forms). In the group BS(m, n) with generators b and a satisfying the relation ba^{m}b^{−1} = a^{n} every nontrivial element can be written in the form

a^{r_{n}}b^{e_{n}}a^{r_{n−1}}b^{e_{n−1}} … a^{r_{1}}b^{e_{1}}a^{r_{0}}

where e_{i} = ±1 and r_{i} ∈ Z have the property that whenever e_{i} = 1 and e_{i−1} = −1 we have r_{i} ∉ mZ, and whenever e_{i} = −1 and e_{i−1} = 1 we have r_{i} ∉ nZ.

While Lemma 5.2 follows immediately from the normal form theorem for HNN extensions, we include a proof here since it is so simple in this case.

Proof. If we put no restrictions on the integers r_{i} and, in particular allow them to be 0, then it is trivially true that any element can be put in the form above. If for any i, e_{i} = 1, e_{i−1} = −1 and r_{i} = mk ∈ mZ then the fact that ba^{mk}b^{−1} = a^{nk} allows us to substitute and obtain another expression in the form above, representing the same element of BS(m, n), but with fewer occurences of the terms b and b^{−1}. Similarly if e_{i} = −1, e_{i−1} = 1 and r_{i} = nk ∈ nZ then b^{−1}a^{nk}b can be replaced with a^{mk} further reducing the occurences of the terms b and b^{−1}.

These substitutions can be repeated at most a finite number of times after which we have the desired form. ⋄

We now show that BS(m, n) is a subgroup of Diff_{+}^{ω}(R). Note that Diff_{+}^{ω}(R) ⊂ Homeo_{+}(I) ⊂ Homeo_{+}(S^{1}), the first inclusion being induced by the one-point compactification of R. Hence the following implies Theorem 1.5.

Proposition 5.3. The homomorphism Φ : BS(m, n) → G defined by Φ(b) = g and Φ(a) = h is an isomorphism.

Proof. By construction Φ is surjective; we now prove injectivity. To this end, consider an arbitrary nontrivial element written in the form of Lemma 5.2. After conjugating we may assume that r_{n} = 0 (replacing r_{0} by r_{0} − r_{n}). So we must show that (Φ applied to) the element

α = g^{e_{n}}h^{r_{n−1}}g^{e_{n−1}} … h^{r_{1}}g^{e_{1}}h^{r_{0}}

acts nontrivially on R. If n = 0, i.e. if α = h^{r_{0}}, then α clearly acts nontrivially if r_{0} ≠ 0, so we may assume n ≥ 1.

Let x be an element of the interior of C ∩ C_{2}. We will prove by induction on n that α(x) ∈ (A^{s} + nZ) ∪ (B^{s} + mZ). This clearly implies α(x) ≠ x.

Let n = 1. Then assuming e_{1} = 1, we have α(x) = g(h^{r_{0}}(x)) = g_{n}(g_{m}^{−1}(h^{r_{0}}(x))). But x ∈ C_{2} + Z implies h^{r_{0}}(x) = x + r_{0} ∈ C_{2} + Z. So equation (2) implies g_{m}^{−1}(x+r_{0}) ∈ B^{u} +Z, and then equation (4) implies g_{n}(g_{m}^{−1}(x+r_{0})) ∈ A^{s} +nZ. Thus e_{1} = 1 implies α(x) ∈ A^{s} + nZ. One shows similarly if e_{1} = −1 then α(x) = g(h^{r_{0}}(x)) ∈ B^{s} + mZ.

Now as induction hypothesis assume that

y = g^{e_{k}}h^{r_{k−1}}g^{e_{k−1}} … h^{r_{1}}g^{e_{1}}h^{r_{0}}(x)

and that either

e_{k} = 1 and y ∈ A^{s} + nZ, or
e_{k} = −1 and y ∈ B^{s} + mZ.

We wish to establish the induction hypothesis for k + 1. There are four cases corresponding to the values of ±1 for each of e_{k} and e_{k+1}. If e_{k} = 1 and e_{k+1} = 1 then y ∈ A^{s} + nZ so h^{r_{i}}(y) = y + r_{i} ∈ A^{s} + Z and equation (7) implies that g_{m}^{−1}(y + r_{i}) ∈ B^{u} + Z. Hence by equation (4), g(h^{r_{i}}(y)) = g_{n}(g_{m}^{−1}(y + r_{i})) ∈ A^{s} + nZ as desired. On the other hand if e_{k} = 1 and e_{k+1} = −1 then r_{k} ∉ nZ. Hence y ∈ A^{s} + nZ implies h^{r_{i}}(y) = y + r_{i} ∈ A^{s} + p with p ∉ nZ. Consequently y′ = g_{n}^{−1}(y + r_{i}) ∈ A^{u} + Z by equation (1). So equation (6) implies g^{−1}(h^{r_{i}}(y)) = g_{m}(y′) ∈ B^{s} + mZ. Thus we have verified the induction hypothesis for k + 1 in the case e_{k} = 1.

If e_{k} = −1 and e_{k+1} = −1 then y ∈ B^{s} + mZ so h^{r_{i}}(y) = y + r_{i} ∈ B^{s} + Z and equation (5) implies that g_{n}^{−1}(y + r_{i}) ∈ A^{u} + Z. Hence by equation (6) g(h^{r_{i}}(y)) = g_{m}(g_{n}^{−1}(y + r_{i})) ∈ B^{s} + mZ as desired. Finally for the case e_{k} = −1 and e_{k+1} = 1 we have r_{k} ∉ mZ. Hence y ∈ B^{s} + mZ implies h^{r_{i}}(y) = y + r_{i} ∈ B^{s} + p with p ∉ mZ. Consequently y′ = g_{m}^{−1}(y + r_{i}) ∈ B^{u} + Z by equation (3). So equation (4) implies g(h^{r_{i}}(y)) = g_{n}(y′) ∈ A^{s} + nZ. Thus we have verified the induction hypothesis for k + 1 in the case e_{k} = −1 also. ⋄

Remark. We showed in the proof of Proposition 5.3 that any element of the form

g^{e_{n}}h^{r_{n−1}}g^{e_{n−1}} … h^{r_{1}}g^{e_{1}}h^{r_{0}}

acts nontrivially on R provided e_{i} = ±1, r_{i} ∈ Z, whenever e_{i} = 1 and e_{i−1} = −1 we have r_{i} ∉ mZ, and whenever e_{i} = −1 and e_{i−1} = 1 we have r_{i} ∉ nZ. As a corollary we obtain the following lemma which we will need later.

Lemma 5.4. In the group BS(m, n) with generators b and a satisfying the relation ba^{m}b^{−1} = a^{n} every element of the form

b^{e_{n}}a^{r_{n−1}}b^{e_{n−1}} … a^{r_{1}}b^{e_{1}}a^{r_{0}},

is nontrivial provided e_{i} = ±1, r_{i} ∈ Z, and whenever e_{i} = 1, e_{i−1} = −1 we have r_{i} ∉ mZ and whenever e_{i} = −1, e_{i−1} = 1 we have r_{i} ∉ nZ.

Of course this lemma can also be obtained from the normal form theorem for HNN extensions.

5.2 General properties of BS(m, n) actions

In this short subsection we note two simple implications of the Baumslag-Solitar relation.

Lemma 5.5. If BS(m, n) acts by homeomorphisms on S^{1} with generators g, h satisfying gh^{m}g^{−1} = h^{n} then h has a periodic point whose period is a divisor of |n − m|.

Proof. Consider the group of all lifts to R of elements of BS(m, n) acting on S^{1}. Choose lifts H and G of h and g respectively and let T (x) = x + 1 denote the covering translation. We have

GH^{m}G^{−1} = H^{n}T^{p}

for some integer p. Each lift of an element of BS(m, n) has a well defined translation number in R and these are topological conjugacy invariants. Hence

mτ(H) = τ(H^{m}) = τ(GH^{m}G^{−1}) = τ(H^{n}T^{p}) = p + nτ(H).

Solving we conclude

τ(H) = \frac{p}{m − n}.

Since H has a rational rotation number h has a point whose period is a divisor of |n − m|. ⋄

Lemma 5.6. Suppose g and h are homeomorphisms of R and satisfy gh^{m}g^{−1} = h^{n}. Then if h is fixed point free, g has a fixed point.

Proof. Assume, without loss of generality, that h(x) = x + 1 and n > m and g(0) ∈ [p, p + 1]. Then

h^{nk}(g(0)) = (g(0) + nk) ∈ [p + nk, p + nk + 1].

But

h^{n}g = gh^{m}, so
h^{nk}g = gh^{mk}, and
h^{nk}(g(0)) = g(h^{mk}(0)) = g(mk), so
g(mk) ∈ [p + nk, p + nk + 1].

For k sufficiently large this implies g(mk) > mk and for k sufficiently negative that g(mk) < mk. The intermediate value theorem implies g has a fixed point. ⋄

5.3 C^{2} actions of BS(m, n)

In contrast to the analytic actions of BS(m, n) on R, we will show that there are no such actions, even C^{2} actions, on either I or S^{1}.

Lemma 5.7. Suppose g and h are orientation-preserving C^{2} diffeomorphisms of I satisfying gh^{m}g^{−1} = h^{n}. Then h and ghg^{−1} commute.

Proof. If x ∈ Fix(h) then g(x) = g(h^{m}(x)) = h^{n}(g(x)) so g(x) is a periodic point for h. But the only periodic points of h are fixed. We conclude that g(Fix(h)) = Fix(h).

Let (a, b) be any component of the complement of Fix(h) in [0, 1]. Since g(Fix(h)) = Fix(h) we have that ghg^{−1}([a, b]) = [a, b].

Let H be the centralizer of h^{n} = gh^{m}g^{−1}. Note that h and ghg^{−1} are both in H. Let f be any element of H. Then if f has a fixed point in (a, b), since it commutes with h^{n} we know by Kopell’s Lemma (Theorem 2.1) that f = id. In other words H acts freely (though perhaps not effectively) on (a, b). Hence the restrictions of any elements of H to [a, b] commute by Hölder’s Theorem. In particular the restrictions of h and ghg^{−1} to [a, b] commute. But (a, b) was an arbitrary component of the complement of Fix(h). Since the restrictions of h and ghg^{−1} to Fix(h) are both the identity we conclude that h and ghg^{−1} commute on all of I. ⋄

As a corollary we have:

Theorem 1.6 for I: If m and n are greater than 1 then no subgroup of Diff_{+}^{2}(I) is isomorphic to BS(m, n).

Proof. If there is such a subgroup, it has generators a and b satisfying a^{n} = ba^{m}b^{−1}. Lemma 5.7 asserts that a and c = bab^{−1} commute. But the commutator [c, a] is ba^{−1}b^{−1}abab^{−1}a and this element is nontrivial by Lemma 5.4. This contradicts the existence of such a subgroup. ⋄

Showing that BS(m, n) has no C^{2} action on the circle is a bit harder.

Theorem 1.6 for S^{1}: If m is not a divisor of n then no subgroup of Diff_{+}^{2}(S^{1}) is isomorphic to BS(m, n).

Proof. Suppose there is such a subgroup, and we have generators g, h satisfying gh^{m}g^{−1} = h^{n}. By Lemma 5.5, h has a periodic point. Let P er(h) be the closed set of periodic points of h (all of which have the same period, say p). If x ∈ Per(h) then g(x) = gh^{mp} = h^{np}g(x) which shows g(Per(h)) = Per(h).

The omega limit set ω(x, g) of a point x under g is equal to a subset of the set of periodic points of g if g has periodic points, and is independent of x if g has no periodic points. Since g is C^{2} we know by Denjoy’s theorem that ω(g, x) is either all of S^{1} or is a subset of Per(g) (see [dMvS]). In the case at hand ω(g, x) cannot be all of S^{1} since x ∈ Per(h) implies ω(x, g) ⊂ Per(h) and this would imply h^{p} = id. Hence there exists x_{0} ∈ Per(g) ∩ Per(h). Then x_{0} ∈ Fix(h^{p}) ∩ Fix(g^{q}) where q is the period of points in Per(g).

The fact that x_{0} has period p for h implies that its rotation number under h (well defined as an element of R/Z) is k/p + Z for some k which is relatively prime to p. Using the fact that ρ(h^{m}) = ρ(gh^{m}g^{−1}), we conclude that

m(k/p) + Z = ρ(h^{m}) = ρ(gh^{m}g^{−1}) = ρ(h^{n}) = n(k/p) + Z

Consequently mk ≡ nk mod p and since k and p are relatively prime m ≡ n mod p. The fact that 1 < m < n and n − m = rp for some r ∈ Z tells us that p ∉ mZ because if it were then n would be a multiple of m. Similarly p ∉ nZ.

Let G be the group generated by h_{0} = h^{p} and g_{0} = g^{q}. Now G has a global fixed point x_{0}, so we can split S^{1} at x_{0} to obtain a C^{2} action of G on I. Note that g_{0}h_{0}^{m^{q}}g_{0}^{−1} = g^{q}h^{pm^{q}}g^{−q} = h^{pm^{q}} = h_{0}^{n^{q}}.

We can apply Lemma 5.7 to conclude that h_{0} and g_{0}h_{0}g_{0}^{−1} commute. Thus h^{p} and c = g^{q}h^{p}g^{−q} commute. But their commutator

[c, h^{p}] = g^{q}h^{−p}g^{−q}h^{−p}g^{q}h^{p}g^{−q}h^{p}.

Since p ∉ mZ and p ∉ nZ this element is nontrivial by Lemma 5.4. This contradicts the existence of such a subgroup. ⋄

5.4 Local rigidity of the standard action of BS(1, n)

We consider the action of BS(1, n) on R given by elements of the form x → n^{k}x + b where k ∈ Z and b ∈ Z[1/n]. It is generated by the two elements g_{0}(x) = nx and h_{0}(x) = x + 1.

The following result is essentially the theorem of Shub from [Sh]. We give the proof since framing his result in our context is nearly as long.

Theorem 5.8. There are neighborhoods of g_{0} and h_{0} in the uniform C^{1} topology such that whenever g and h are chosen from these respective neighborhoods, and when the correspondences g_{0} → g and h_{0} → h induce an isomorphism of BS(1, n) to the group generated by g and h, then the perturbed action is topologically conjugate to the original action.

Proof. Since g is uniformly C^{1} close to g_{0}(x) = nx, its inverse is a contraction of R and has a unique fixed point, which after a change of coordinates we may assume is 0. Since h is uniformly close to h_{0}(x) = x + 1 it is fixed point free and after a further change of co-ordinates we may assume h^{n}(0) = n. Consider the space H of C^{0} maps ϕ : R → R such that h_{0}ϕ = ϕh. and h(0) = 0. Any ϕ ∈ H satisfies ϕ(n) = n for all n ∈ Z. Moreover, such a ϕ is completely determined by its values on the interval [0, 1], since ϕ(x + n) = ϕ(x) + n. Indeed any C^{0} map ϕ : [0, 1] → R is the restriction of some element of H. Clearly, a sequence in H will converge if and only if it converges on [0, 1]. Hence H can be considered a closed subset of the complete metric space of continuous functions on [0, 1] with the C^{0} sup norm.

Consider the map F on H given by G(ϕ) = g_{0}^{−1}ϕg. Then G(ϕ)h = g_{0}^{−1}ϕgh = g_{0}^{−1}ϕh^{n}g = g_{0}^{−1}h_{0}^{n}ϕg = h_{0}g_{0}^{−1}ϕg = h_{0}F (ϕ). It follows F : H → H. Moreover in the C^{0} sup norm this map is easily seen to be a contraction. It follows that there is a unique fixed point ϕ_{0} for F in H.

The map ϕ_{0} : R → R is continuous and satisfies

h_{0}^{n}ϕ_{0} = ϕ_{0}h^{n}, and
g_{0}^{n}ϕ_{0} = ϕ_{0}g^{n},

for all n ∈ Z. The first of these equations implies ϕ_{0} has image which is unbounded above and below, so ϕ_{0} is surjective. This equation also implies that there is a uniform bound on the size of ϕ_{0}^{−1}(x) for x ∈ R. Indeed if M is an integer and M > sup|ϕ_{0}(0)−ϕ_{0}(x)| for x ∈ [0, 1] then y > M +1 or y < −M implies ϕ_{0}(y) ≠ ϕ_{0}(x). So the sets ϕ_{0}^{−1}(z) have diameters with an upper bound independent of z ∈ R. In particular, ϕ_{0} is proper.

We can now see that ϕ_{0} is injective. Since ϕ_{0}(y) = ϕ_{0}(x) implies g_{0}^{n}(ϕ_{0}(y)) = g_{0}^{n}(ϕ_{0}(x)) which in turn implies ϕ_{0}(g^{n}(y)) = ϕ_{0}(g^{n}(x)) and g is is a uniform expansion, it follows that if ϕ_{0} fails to be injective then the sets ϕ_{0}^{−1}(z) do not have a diameter with an upper bound independent of z. This contradiction implies ϕ_{0} must be injective. The fact that ϕ_{0} is proper implies ϕ_{0}^{−1} is continuous. Hence ϕ_{0} is a topological conjugacy from the standard affine action of BS(1, n) on R to the action of the group generated by g and h. ⋄

In contrast to local rigidity of the standard action, Hirsch [H] has found realanalytic actions of BS(1, n) on the line which are not topologically conjugate to the standard action.

Theorem 5.9 (Hirsch [H]). There is an analytic action of BS(1, n) on R which is not topologically conjugate to the standard affine action.

The construction is similar to our construction of an analytic BS(m, n) action on R. The relation ghg^{−1} = h^{n} is satisfied by analytic diffeomorphisms g and h and g has a fixed attracting fixed point. Hence g cannot be topologically conjugate to the affine function g_{0}(x) = nx.

References

[BV] M. Bridson and K. Vogtmann, Quotients of automorphism groups of free groups, preprint.

[BM] M. Burger and N. Monod, Bounded cohomology of lattices in higher rank Lie groups, J. Eur. Math. Soc.1 (1999), no. 2, 199–235.

[dMvS] Welington de Melo and Sebastien van Strien One-dimensional Dynamics Springer Verlag, Berlin, (1991)

[FF2] B. Farb and J. Franks, Group actions on one-manifolds, II: Extensions of Hölder’s Theorem, preprint.

[FF3] B. Farb and J. Franks, Groups of homeomorphisms of onemanifolds, III: Nilpotent subgroups, in preparation.

[FS] B. Farb and P. Shalen, Groups of real-analytic diffeomorphisms of the circle, to appear in Ergodic Theory and Dynam. Syst.

[FP] E. Formanek and C. Procesi, The automorphism group of a free group is not linear, J. Algebra 149 (1992), no. 2, 494–499.

[Gh] E. Ghys, Actions de réseaux sur le cercle, Inventiones Math. 137 (1999), no. 1, 199–231.

[Gh2] E. Ghys, Rigidité différentiable des groupes fuchsiens, Inst. Hautes tudes Sci. Publ. Math., No. 78 (1993), 163–185.

[GS] E. Ghys and V. Sergiescu, Sur un groupe remarquable de difféomorphismes du cercle, Comm. Math. Helv., Vol. 62 (1987), p.185-239.

[H] M. Hirsch, A stable analytic foliation with only exceptional minimal set, in Lecture Notes in Math., Vol. 468. Springer-Verlag, 1975.

[Iv] N. Ivanov, Mapping Class Groups, preprint.

[K] N. Kopell, Commuting Diffeomorphisms in Global Analysis, Proceedings of the Symposium on Pure Mathematics XIV, Amer. Math. Soc., Providence, RI (1970), 165–184.

[dM] W. de Melo, Lectures on One-dimensional Dynamics Intstituto de Matematica Pura e Aplicada do CNPq, Rio de Janeiro, (1989)

[La] F. Labourie, Large groups actions on manifolds, Proc. I.C.M. (Berlin, 1998), Doc. Math. 1998, Extra Vol. II, 371–380.

[LS] R.C. Lyndon and P. Schupp, Combinatorial Group Theory, Springer-Verlag, 1977.

[Na] A. Navas, Actions de groupes de Kazhdan sure le cercle, April 2001 preprint.

[Sh] M. Shub, Expanding Maps in Global Analysis, Proceedings of the Symposium on Pure Mathematics XIV, Amer. Math. Soc., Providence, RI (1970), 273–276.

[Zi] R. Zimmer, Actions of semisimple groups and discrete subgroups, Proc. I.C.M., Berkeley 1986, p.1247-1258.

Benson Farb: Dept. of Mathematics, University of Chicago 5734 University Ave. Chicago, Il 60637 E-mail: farb@math.uchicago.edu

John Franks: Dept. of Mathematics, Northwestern University Evanston, IL 60208 E-mail: john@math.northwestern.edu