Manuscript submitted to Website: http://AIMsciences.

org
AIMS’ Journals
Volume 00, Number 0, Xxxx XXXX pp. 000–000

HIGHER-ORDER MELNIKOV METHOD AND CHAOS FOR

TWO-DEGREE-OF-FREEDOM HAMILTONIAN SYSTEMS WITH
SADDLE-CENTERS

Kazuyuki Yagasaki
Department of Mechanical and Systems Engineering
Gifu University
Gifu 501-1193, Japan

Abstract. We consider two-degree-of-freedom Hamiltonian systems with saddle-

centers, and develop a Melnikov-type technique for detecting creation of trans-
verse homoclinic orbits by higher-order terms. We apply the technique to the
generalized Hénon-Heiles system and give a positive answer to a remaining
question of whether chaotic dynamics occurs for some parameter values al-
though it is known to be nonintegrable in a complex analytical meaning.

1. Introduction. In this paper we consider two-degree-of-freedom Hamiltonian

systems of the form
ẋ = JDx H(x, y), ẏ = JDy H(x, y), (x, y) ∈ R2 × R2 , (1.1)
where the dot represents differentiation with respect to time t, H : R × R → R is
2 2

C r+1 (r ≥ 4) and J is the 2 × 2 symplectic matrix,

( )
0 1
J= .
−1 0
We especially assume that the x-plane is invariant under the flow of (1.1) and
there is a saddle-center at the origin (x, y) = (0, 0) (= O) with a homoclinic orbit
on the x-plane. Here a “saddle-center” is an equilibrium at which the Jacobian
matrix has a pair of positive and negative real eigenvalues and a pair of purely
imaginary eigenvalues. See Sect. 2 for our precise assumption. In this situation, via
the Liapunov center theorem [1, 21], there exist a one-parameter family of periodic
orbits near the saddle-center.
Complicated dynamics of two-degree-of-freedom Hamiltonian systems with saddle-
centers have been studied by several researchers [10, 11, 18, 19, 22, 33]. In the earlier
work of [19, 22] a Shil’nikov-type approach [28, 30] was used for general “analytic”
Hamiltonian systems to show the presence of horseshoes, which also implies the
occurrence of chaos, near the saddle-centers under a degenerate condition. For a
more restricted class of systems with potentials including (1.3) below, the non-
degenerate condition was represented in a convenient form which is independent

2000 Mathematics Subject Classification. Primary: 37J45, 70H07, 70H09; Secondary: 34C37,
34E10, 37J30.
Key words and phrases. Melnikov method, chaos, Smale horseshoe, Hamiltonian system,
saddle-center, Hénon-Heiles system.
Present address: Mathematics Division, Department of Information Engineering, Niigata Uni-
versity, Niigata 950-2181, Japan; E-mail: yagasaki@ie.niigata-u.ac.jp.
1
2 KAZUYUKI YAGASAKI

of the coordinates in [10]. Countable infinities of multi-pulse homoclinic orbits

and periodic orbits in perturbed systems or near the homoclinic loops were also
discussed in [11, 18, 22].
In [33] a Melnikov type technique was developed for detecting the existence of
orbits transversely homoclinic to periodic orbits near the saddle-center and hence
the occurrence of chaos in (1.1).
To apply this technique, we only compute a function called the Melnikov function
and determine whether it has a simple zero or not (see Sect. 3, especially Theo-
rem 3.1, for the details), like the standard Melnikov method [12, 31]. The obtained
condition for chaos is equivalent to the previous one of [19, 22] in analytic Hamil-
tonian systems. This type of approach is often superior to the Shil’nikov-type at
the point that it is extensible to higher-degree-of-freedom systems and higher-order
approximations as in [4, 14, 20, 30]. So the technique of [33] was actually extended
to three or more degrees of freedom systems in [37] for studying homoclinic or
heteroclinic connections between whiskered invariant tori. The presence of such
heteroclinic connections implies the occurrence of Arnold diffusion type motions
(see [37] for the details), a numerical evidence of which was given for a three-degree-
of-freedom system in [35]. These techniques were applied to an infinite-degree-of-
freedom system representing a mathematical model of an undamped, buckled beam
in [34]. Equivalence of the criterion for chaos [10,33] and nonintegrability in a com-
plex analytic meaning [23, 25] was also discussed in [23, 24, 36]1 .
Our special attention is paid to the (generalized) Hénon-Heiles Hamiltonian
1 1
H(x, y) = (x21 + x22 + y12 + y22 ) + cx1 y12 + dx31 (1.2)
2 3
with c/d = 3/4. The original system studied by Hénon and Heiles [15] is the
case of c = 1 and d = −1 in √ (1.2). The Hamiltonian (1.2) has a saddle-center at
(x1 , x2 , y1 , y2 ) = (−1/2c, 0, ± 2 − (d/c)/2c, 0) if c/d > 1/2, and at (−1/d, 0, 0, 0)
if c/d < 1/2. When c/d > 1/2 (resp. c/d < 1/2), it is transformed to
1 1 β1 β2 β3
H(x, y) = (x22 + y22 ) + (−x21 + ω 2 y12 ) + x31 + x1 y12 + y13 (1.3)
2 2 3 2 3
by a change of coordinates
x1 − x10 + µy1 x2 + µy2
x̃1 = √ , x̃2 = √ ,
1+µ 2 1 + µ2
−µ(x1 − x10 ) + y1 −µx2 + y2
ỹ1 = √ , ỹ2 = √
1+µ 2 1 + µ2
(resp. x̃1 = x1 − 1/d, x̃2 = x2 , ỹ1 = y1 , ỹ2 = y2 ), where the tilde represents the old
coordinates and
2c 2(d − c) µ(c + d)
β1 = √ , β2 = √ , β3 = √ ,
1+µ 2 1+µ 2 1 + µ2
√ √ (1.4)
µ 1 + µ2 d
ω = √ , x10 = , µ= 2−
2 2c c
( √ )
2c
resp. β1 = d, β2 = 2c, β3 = 0, ω = 1 − .
d
1 We note that the existence of transverse homoclinic orbits was generally proven to admit no

first integral except constant [2, 3, 7].

 HIGHER-ORDER MELNIKOV METHOD AND CHAOS 3

In [10, 33] it was shown that there exist horseshoes near the saddle-centers and
chaotic dynamics occurs in (1.1) with (1.3) if
β2 l(l + 1)
̸= for any non-negative integer l,
β1 6
which corresponds to
c 1 1 3
̸= 0, , , , 1 (1.5)
d 6 2 4
in (1.2). On the other hand, using Ziglin’s method [40], Ito [16,17] showed that the
Hamiltonian (1.2) is nonintegrable in a complex analytic meaning if
c 1 1
̸= 0, , , 1. (1.6)
d 6 2
It is actually integrable in the three cases except c/d = 1/2 (see, e.g., [9]) and
it was recently proven to be nonintegrable in the complex analytic meaning when
c/d = 1/2 (see [23]). So we have a question of whether transverse homoclinic orbits
exist and chaotic dynamics occurs when c/d = 1/2, 3/4 although the Melnikov
function is identically zero and the Melnikov technique of [33] does not apply but
it is nonintegrable in the complex analytic meaning.
The object of this paper is to extend the idea of [33] to second-order approxima-
tions and develop a technique for detecting the existence of transverse homoclinic
orbits even when the techniques of [10, 33] are not applicable. So we give a pos-
itive answer to the above question on the Hénon-Heiles Hamiltonian (1.2) with
c/d = 3/4. The basic ideas used here are similar to those in [33], which the reader
should consult for the details and proofs of some preliminary but key results. The
outline of this paper is as follows. In Sect. 2 we give our precise assumptions on
(1.1) and follow [33] to describe their immediate consequences on the phase space
structure. We state our main result after giving the main result of [33] in Sect. 3
and provide its proof in Sect. 4. In Sect. 5 we illustrate our theory for the Hamil-
tonian (1.3) with β2 /β1 = 1/3 including the Hénon-Heiles Hamiltonian (1.2) with
c/d = 3/4 as a special case. We remark that for c/d = 1/2 our technique does not
directly apply to (1.2) since the equilibrium with a homoclinic orbit is a saddle-
parabolic type for which the Jacobian matrix has a double zero eigenvalue. An
extension of our result to that case, in which a higher-order approximation is also
required will be reported elsewhere.

2. Assumptions and the phase space. We make the following assumptions on

(1.1).
(A1) Dx H(0, 0) = 0 and Dy H(x, 0) = 0 for any x ∈ R2 .
Assumption (A1) means that the origin O is an equilibrium and the x-plane,
{(x, y) ∈ R2 × R2 | y = 0}, is invariant under the flow of (1.1). The system re-
stricted on the x-plane,
ẋ = JDx H(x, 0), (2.1)
has an equilibrium at x = 0.
(A2) The matrix JD2x H(0, 0) has a pair of real eigenvalues ±λ (λ > 0) so that
the equilibrium x = 0 of (2.1) is a hyperbolic saddle. Moreover, there is a
homoclinic orbit xh (t) to the saddle x = 0. See Fig. 1. Let Γ0 = {xh (t)|t ∈
R} ∪ {0}.
4 KAZUYUKI YAGASAKI

x h( t )

0
x2

x1

Figure 1. Phase portrait on the x-plane.

(A3) The matrix JD2y H(0, 0) has a pair of purely imaginary eigenvalues ± i ω
(̸= 0).
Let
1
pj (x, η) = Dj+1 f (x, 0)(η, . . . , η ), j = 0, 1, . . . ,
(j + 1)! y | {z }
j+1

with f (x, y) = JDx H(x, y), where η ∈ R2 . Note that p0 (x, η) ≡ 0 by assumption
(A1).
(A4) p1 (x, η) ̸≡ 0.
The Hamiltonian (1.3) satisfies assumptions (A1)-(A4) if β1 , β2 ̸= 0. In particular,
the homoclinic orbit xh (t) is given by
( ( ) ( ) ( ))
3 t 3 t t
xh (t) = sech2 ,− sech2 tanh . (2.2)
2β1 2 2β1 2 2

Assumptions (A2) and (A3) mean that the equilibrium O is a saddle-center

and has a homoclinic orbit (x, y) = (xh (t), 0). It follows from the center manifold
theory (e.g., [12,29]) that the saddle-center O has a C r , two-dimensional local center
c
manifold, Wloc (O), which may be non-unique, as well as C r , one-dimensional stable
and unstable manifolds, W s (O) and W u (O), which coincide along the homoclinic
orbit (xh (t), 0). The case in which assumption (A4) does not hold was also studied
in [33].
Now we describe immediate consequences of the above assumptions on the phase
space structure of (1.1). These results were proven in [33] and also play an important
role below.
Using the Liapunov center theorem [1, 21], we obtain the following result (see
Proposition 2.1 of [33]).

Proposition 2.1. There exists a one-parameter family of periodic orbits,

γα, 0 < α ≤ α0 ,
 HIGHER-ORDER MELNIKOV METHOD AND CHAOS 5

that approach O as α → 0 and whose periods approach 2π/ω. Moreover, for α0 > 0
sufficiently small the periodic orbits fill up a C r , two-dimensional, normally hyper-
bolic invariant manifold with boundary,
∪
α0
M = γ α ⊂ Wloc
c
(O),
α=0

which is tangent to the y-plane, {(x, y) ∈ R2 × R2 |x = 0}, at O.

We denote Hα = H(γ α ). Without a loss of generality we can assume that
H0 = H(0, 0) = 0.
Let Uϵ1 ,ϵ2 = {(x, y) ∈ R2 × R2 | |x| < ϵ1 , |y| < ϵ2 } be a neighborhood of the
saddle-center O including the invariant manifold with boundary M in R2 × R2 . A
different neighborhood was used in [33] but both are essentially the same. Denote
by ϕt the flow generated by (1.1). From the invariant manifold theory of Fenichel
[8] (see also [32]) we have the following result (see Proposition 2.3 of [33] and
Proposition 1 of [37]).
Proposition 2.2. For ϵj > 0, j = 1, 2, sufficiently small, there exist C r , three-
s u
dimensional, locally invariant manifolds in Uϵ1 ,ϵ2 , denoting by Wloc (M ) and Wloc (M ),
having the following properties.
s
(i) Wloc (M ) ∩ Wloc u
(M ) = M .
s,u
(ii) Let z (t) = (xs,u (t), y s,u (t)) be a trajectory starting in Wloc
s,u
(M ) at t = 0.
Then as t → +∞ (resp. t → −∞), either
(iia) z s (t) (resp. z u (t)) crosses ∂Uρ,α0 ;

or
(iib) z s (t) → M (resp. z u (t) → M ).
s,u
Moreover, Wloc (M ) can be represented as
s,u
Wloc (M ) = {(x, y) ∈ Uϵ1 ,ϵ2 | x ∈ Γy },
where Γy ⊂ R2 is O(|y|2 )-close to Γ0 in Uϵ1 ,ϵ2 .
s u
We refer to Wloc (M ) and Wloc (M ) as the local stable and unstable manifolds
of M . We also have the following result (see Proposition 2.4 of [33].)
Proposition 2.3. Let zαs,u (t) be orbits on W s,u (M ) such that H(zαs,u (t)) = Hα .
Then for α > 0 sufficiently small there exists a periodic orbit γ α ∈ M such that
dist(zαs (t), γ α ) → 0 (resp. dist(zαu (t), γ α ) → 0) as t → +∞ (resp. t → −∞), where
dist(z, A) = inf z′ ∈A |z − z ′ | for A ⊂ R4 . This implies that zαs (t) ∈ W s (γ α ) and
zαu (t) ∈ W u (γ α ), where W s (γ α ) and W u (γ α ), respectively, denote the stable and
unstable manifolds of γ α in a usual meaning.

3. Main result. Consider the variational equations of (1.1) in the y-direction

about the saddle-center O and homoclinic orbit (x, y) = (xh (t), 0),
η̇ = JD2y H(0, 0)η (3.1)
and
η̇ = JD2y H0 (xh (t), 0)η. (3.2)
We call (3.1) and (3.2) the normal variational equations (NVEs) of (1.1) about
the saddle-center O and the homoclinic orbit (x, y) = (xh (t), 0), respectively. Let
Φ(t) and Ψ(t) be fundamental matrices to (3.1) and (3.2), respectively, such that
6 KAZUYUKI YAGASAKI

Φ(0) = Ψ(0) = id, where id is the 2 × 2 identity matrix. In particular, all elements
of Φ(t) can be written as linear combinations of sin ωt and cos ωt (see [36, 37]). We
can easily show that the limits
B± = lim Φ(−t) Ψ(t) (3.3)
t→±∞

exist and B± are nonsingular matrices (see Lemma 3.1 of [33]). We set B0 =
−1
B+ B− . Let
1
qj (x, η) = Dj+2 H(x, 0)(η, . . . , η ), j = 0, 1, . . . ,
(j + 2)! y | {z }
j+2

where η ∈ R . Note that pj+1 (x, η) = JDx qj (x, η), j = 0, 1, . . .. In particular, we

2

have
1
q0 (x, η) = η T D2y H(x, 0)η, (3.4)
2
where T is the transpose operator.
Let e1 = (1, 0)T ∈ R2 . Define the first-order Melnikov function as
M1 (t0 ) = q0 (0, e1 ) − q0 (0, B0 Φ(t0 )e1 ). (3.5)
The following result was proven in [33].
Theorem 3.1. Suppose that M1 (t0 ) has a simple zero, i.e.,
d
M1 (t̄0 ) = 0, M1 (t̄0 ) ̸= 0
dt0
for some t̄0 ∈ R. Then for α > 0 sufficiently small there exist orbits transversely
homoclinic to γ α and the Hamiltonian system (1.1) has a Smale horseshoe in its
dynamics on the energy surface H = Hα .
Now we assume that M1 (t0 ) ≡ 0. As stated in Sect. 1, this assumption holds for
(1.3) with c/d = 3/4 (see also Sect. 5). Let
∫ ∞
K(v) = [q1 (xh (t), Ψ(t)v) − q1 (0, Φ(t)B+ v)]dt
0
∫ 0
+ [q1 (xh (t), Ψ(t)v) − q1 (0, Φ(t)B− v)]dt. (3.6)
−∞

Define the second-order Melnikov function as

−1
M2 (t0 ) = − Φ(t0 )e1 · D2y H(0, 0)B− JDv K(B− Φ(t0 )e1 )
+ q1 (0, Φ(t0 )e1 ) − q1 (0, B0 Φ(t0 )e1 ), (3.7)
where “·” represents the inner product. We state our main result as follows (see
Sect. 4 for the proof).
Theorem 3.2. Suppose that M1 (t0 ) ≡ 0 and that M2 (t0 ) has a simple zero. Then
the conclusion of Theorem 3.1 still holds.
Remark 3.3. For q1 (x, η) ≡ 0, we can obtain a result similar to Theorem 3.2 by
computing higher-order terms of O(|y|4 ) although more tedious computations are
needed.
 HIGHER-ORDER MELNIKOV METHOD AND CHAOS 7

x uε ( t )
x h( t )

M O d( t 0 )

Uε1,ε2
x h( -t 0 )
x εs ( t )

Figure 2. Orbits on the stable and unstable manifolds.

4. Proof.
Lemma 4.1. For ϵ > 0 sufficiently small there exist orbits
zϵs,u (t; η0 ) = (xs,u s,u
ϵ (t; η0 ), yϵ (t; η0 ))

lying on W s,u (M ) and passing through a point near the homoclinic orbit (x, y) =
(xh (t), 0),
x = xh (0) + O(ϵ2 ), y = ϵη0 , (4.1)
at t = 0 for any η0 (̸= 0) ∈ R2 , such that the following expressions hold with uniform
validity in the indicated time intervals:
xsϵ (t; η0 ) = xh (t) + ϵ2 ξϵs (t; η0 ), yϵs (t; η0 ) = ϵηϵs (t; η0 ) (4.2)
for t ∈ [0, t1 ] with any t1 > 0;
xuϵ (t; η0 ) = xh (t) + ϵ2 ξϵu (t; η0 ), yϵu (t; η0 ) = ϵηϵu (t; η0 ) (4.3)
for t ∈ [−t2 , 0] with any t2 > 0. See Fig. 2. Here (ξϵs,u (t; η0 ), ηϵs,u (t; η0 ))
are solutions
to
ξ˙ = JD2x H(xh (t), 0)ξ + p1 (xh (t), η) + ϵp2 (xh (t), η) + O(ϵ2 ),
(4.4)
η̇ = JD2y H(xh (t), 0)η + ϵJDη q1 (xh (t), η) + O(ϵ2 ),
and satisfy
JDx H(xh (0), 0) · ξϵs,u (0; η0 ) = 0 (4.5)
and yϵs,u (0; η0 ) = ϵη0 . In addition, zϵs,u (t; η0 ) have energy
H(zϵs,u (t; η0 )) = ϵ q0 (0, B± η0 ) + O(ϵ3 ).
2
(4.6)
Proof. The proof is very similar to that of Lemma 3.2 of [33]. We first note that
by Proposition 2.3 there exists a periodic orbit γ α ∈ M for ϵ > 0 sufficiently
small, such that orbits on W s,u (M ) passing through the point given by (4.1) are
contained in W s,u (γ α ). Substituting (4.2) and (4.3) into (1.1) and using Gronwall’s
lemma [12], we easily obtain the first part. From the smoothness of xs,u ϵ (t; η0 ) on ϵ
we can choose ξϵs,u (0; η0 ) such that they satisfy (4.5).
8 KAZUYUKI YAGASAKI

Let
1
H̄(ξ, η, t) = Dx H(xh (t), 0)ξ + D2y H(xh (t), 0)(η, η).
2
Then we have
H(zϵs,u (t; η0 )) = ϵ2 H̄(ξϵs,u (t; η0 ), ηϵs,u (t; η0 ), t) + O(ϵ3 )
since H(xh (t), 0) = 0 and Dy H(x, 0) = 0. Using (4.5), we obtain
H̄(ξϵs,u (t; η0 ), ηϵs,u (t; η0 ), t) = q0 (0, B± η0 ) + O(ϵ).
which yields (4.6).

Let zϵs,u (t; η0 ) = (xs,u s,u

ϵ (t; η0 ), yϵ (t; η0 )) be orbits on W
s,u
(M ) in Lemma 4.1
such that yϵ (0; η0 ) = yϵ (0; η0 ) = ϵη0 . Note that one may have limt→∞ zϵs (t; η0 ),
s u

limt→−∞ zϵu (t; η0 ) ̸∈ M and H(zϵs (t; η0 )) ̸= H(zϵu (t; η0 )). Denote by d(η0 , ϵ) be the
distance between zϵs (t; η0 ) and zϵu (t; η0 ) along the direction (Dx H(xh (0), 0), 0) near
the point (x, y) = (xh (0), η0 ). We have
Dx H(xh (0), 0)
d(η0 , ϵ) = · (xuϵ (0; η0 ) − xsϵ (0; η0 )),
|Dx H(xh (0), 0)|
which can also measure the distance between W s (M ) and W u (M ) (see Fig. 2).
Let
ϵ (t; η0 ) = Dx H(x (t), 0) · ξϵ (t; η0 ).
∆s,u h s,u

From Lemma 4.1 we see that

d(η0 , ϵ) = ϵ2 c0 (∆uϵ (0; η0 ) − ∆sϵ (0; η0 )), (4.7)
where
c0 = |Dx H(xh (0), 0)|−1 ̸= 0.
Employing “Melnikov’s trick” as in the standard Melnikov technique (e.g., Sect. 4.3
of [12]) and noting that Dx H(xh (t), 0) exponentially tends to zero as t → ±∞ and
|ξϵs (t; η0 )| < ∞ (resp. |ξϵu (t; η0 )| < ∞) for t ∈ [0, ∞) (resp. t ∈ (−∞, 0]), we obtain
∫ ∞
∆sϵ (0; η0 ) = − Dx H(xh (t), 0) · [p1 (xh (t), ηϵs (t; η0 ))
0
+ ϵp2 (xh (t), ηϵs (t; η0 ))]dt + O(ϵ2 ),
∫ 0 (4.8)
∆uϵ (0; η0 ) = Dx H(x (t), 0) · [p1 (x
h h
(t), ηϵs (t; η0 ))
−∞
+ ϵp2 (xh (t), ηϵs (t; η0 ))]dt + O(ϵ2 ).
Here we used (4.4) and the fact that t1 and t2 can be chosen by arbitrarily large
values.
On the other hand, we can take t0 ∈ R such that Φ(t0 )e1 = B− η0 /|η0 | since
|B− η0 | = |η0 |. Hence, we estimate the solutions η s,u (t; η0 ) of (4.4) to obtain
−1
η s,u (t; η0 ) = |η0 | Ψ(t)B− Φ(t0 )e1 + O(ϵ).
Since by Lemma 3.4 of [33] one can rewrite
∫ ∞
−1
M1 (t0 ) = Dx H(xh (t), 0) · p1 (xh (t), Ψ(t)B− Φ(t0 )e1 ) dt,
−∞
 HIGHER-ORDER MELNIKOV METHOD AND CHAOS 9

it follows from (4.8) that

∆uϵ (0; η0 ) − ∆sϵ (0; η0 )
∫ ∞
−1
= Dx H(xh (t), 0) · p1 (xh (t), |η0 | Ψ(t)B− Φ(t0 )e1 )dt + O(ϵ)
−∞
= |η0 |2 M1 (t0 ) + O(ϵ) (4.9)
since q0 (0, η) is a quadratic form of η (see also Lemma 3.3 of [33]).
Now we assume that M1 (t0 ) ≡ 0. For any nonzero vector η ∈ R2
T 2 T 2
B+ Dy H(0, 0)B+ = B− Dy H(0, 0)B− , (4.10)
since choosing t0 such that Φ(t0 )e1 = B− η/|η|, we have
1
M1 (t0 ) = [Φ(t0 )e1 · D2y H(0, 0)Φ(t0 )e1 − B0 Φ(t0 )e1 · D2y H(0, 0)B0 Φ(t0 )e1 ]
2
1
= [B− η · D2y H(0, 0)B− η − B+ η · D2y H(0, 0)B+ η] = 0.
2|η|2
Denote by B̂0 the matrix given by (4.10). Letting η = Ψ(t)u in the second equation
of (4.4), we have
u̇ = ϵΨ−1 (t)JDη q1 (xh (t), Ψ(t)u)
up to O(ϵ ). As in the averaging method [12, 27], using the near-identity transfor-
2

mation
u = v + ϵw± (v, t),
we obtain
v̇ = ϵJDv q̄1± (v)
up to O(ϵ2 ), where
∫
1 T −1
JDv q̄1± (v) = lim Ψ (t)JDη q1 (xh (t), Ψ(t)v) dt
T →±∞ T 0
∫
1 T −1
= lim B± Φ(−t)JDη q1 (0, Φ(t)B± v) dt, (4.11)
T →±∞ T 0
∫ t
[ −1 ]
±
w (v, t) = Ψ (t)JDη q1 (xh (t), Ψ(t)v) − JDv q̄1± (v) dt.
0
Since Dη q1 (0, η) is a quadratic form of η and consequently the integrand in the
first equation of (4.11) consists of cubic functions of sin ωt and cos ωt, we have
JDv q̄1± (v) ≡ 0, so that
v̇ = O(ϵ2 ) (4.12)
and ∫ t
w± (v, t) = w(v, t) = Ψ−1 (t)JDη q1 (xh (t), Ψ(t)v)dt. (4.13)
0
Hence, Eq. (4.9) is expressed as
∆uϵ (0; η0 ) − ∆sϵ (0; η0 )
{∫ ∞
[
=ϵ Dx H(xh (t), 0) · Dη p1 (xh (t), Ψ(t)η0 )Ψ(t)w(η0 , t)
−∞
}
]
+ p2 (x (t), Ψ(t)η0 ) dt + O(ϵ2 )
h
(4.14)
10 KAZUYUKI YAGASAKI

since the solution (4.12) with v(0) = η0 is given by

v = η0 + O(ϵ2 )
for t = O(1).
Lemma 4.2. We have
d[ ]
Dη q0 (xh (t), Ψ(t)v) · Ψ(t)w(v, t) + q1 (xh (t), Ψ(t)v)
dt [ ]
= −Dx H(xh (t), 0) · Dη p1 (xh (t), Ψ(t)v)Ψ(t)w(v, t) + p2 (xh (t), Ψ(t)v) . (4.15)
Proof. We compute
d[ ]
Dη q0 (xh (t), Ψ(t)v) · Ψ(t)w(v, t) + q1 (xh (t), Ψ(t)v)
dt
= [Dx Dη q0 (xh (t), Ψ(t)v)ẋh (t)
+ D2η q0 (xh (t), Ψ(t)v)Ψ̇(t)v] · Ψ(t)w(v, t)
[ ]
∂w
+ Dη q0 (x (t), Ψ(t)v) · Ψ̇(t)w(v, t) + Ψ(t)
h
(v, t)
∂t
+ Dx q1 (xh (t), Ψ(t)v) · ẋh (t) + Dη q1 (xh (t), Ψ(t)v) · Ψ̇(t)v.
Substituting
ẋh (t) = JDx H(xh (t), 0), Ψ̇(t) = JD2y H(xh (t), 0)Ψ(t),
Dη q0 (xh (t), η) = D2y H(xh (t), 0)η,
∂w
(v, t) = Ψ−1 (t)JDη q1 (xh (t), Ψ(t)v)
∂t
into the above equation and using the relations pj (x, η) = JDx qj (x, η), j = 1, 2,
and Jη · η ′ = −η · Jη ′ for η, η ′ ∈ R2 , we obtain (4.15).
Lemma 4.3. We have
∫ t
D2y H(0, 0)B± v · J ΦT (t)Dη q1 (0, Φ(t)B± v)dt
0
= −q1 (0, Φ(t)B± v) + q1 (0, B± v). (4.16)
Proof. We compute
d
[q1 (0, Φ(t)B± v)] =Φ̇(t)B± v · Dη q1 (0, Φ(t)B± v)
dt
=JD2y H(0, 0)Φ(t)B± v · Dη q1 (0, Φ(t)B± v).
Since Φ(t) is a solution of (3.1), we have
JD2y H(0, 0)Φ(t) = Φ(t)JD2y H(0, 0),
so that by J T = −J
[JD2y H(0, 0)Φ(t)]T = −D2y H(0, 0)JΦT (t).
Hence,
d
[q1 (0, Φ(t)B± v)] = −D2y H(0, 0)B± v · JΦT (t)Dη q1 (0, Φ(t)B± v)
dt
from which Eq. (4.16) immediately follows by Φ(0) = id.
 HIGHER-ORDER MELNIKOV METHOD AND CHAOS 11

Integrating (4.15) from t = −∞ to ∞ and using (3.3), (4.14) and the relation
(Φ(t)B− )T D2y H(0, 0)Φ(t)B− = (Φ(t)B+ )T D2y H(0, 0)Φ(t)B+ = B̂0 ,
we obtain
∆uϵ (0; η0 ) − ∆sϵ (0; η0 )
{
= ϵ lim [η0 · B̂0 w(η0 , t) + q1 (0, Φ(t)B− η0 )]
t→−∞
}
− lim [η0 · B̂0 w(η0 , t) + q1 (0, Φ(t)B+ η0 )] + O(ϵ2 ). (4.17)
t→∞

T
Since B± are symplectic, we have B± JB± = J (see, e.g., Theorem II.A.2 of [21])
so that
T T 2 T T 2
B̂0 JB± = B± Dy H(0, 0)B± JB± = B± Dy H(0, 0)J.
Hence, by Lemma 4.3,
∫ t
η0 · B̂0 JB±
T
ΦT (t)Dη q1 (0, Φ(t)B± η0 )dt
0
= −q1 (0, Φ(t)B± η0 ) + q1 (0, B± η0 ).
Noting that Ψ(t) is symplectic and using (4.13) and the relation Ψ−1 (t) = −JΨT (t)J
(see, e.g., Theorem II.A.2 of [21]), we obtain
η0 · B̂0 w(η0 , t) + q1 (0, Φ(t)B± η0 )
∫ t
[ T
= η0 · B̂0 J Ψ (t)Dη q1 (xh (t), Ψ(t)η0 )
0
]
− B± T T
Φ (t)Dη q1 (0, Φ(t)B± η0 ) dt + q1 (0, B± η0 ). (4.18)
Using (4.18) in (4.17), we have
∆uϵ (0; η0 ) − ∆sϵ (0; η0 )
= ϵ[−B− η0 · D2y H(0, 0)B− JDv K(η0 ) + q1 (0, B− η0 ) − q1 (0, B+ η0 )] + O(ϵ2 )
(4.19)
since
Dv q1 (0, Ψ(t)v) = ΨT (t)Dη q1 (0, Ψ(t)v).
We are now in a position to give a proof of Theorem 3.2.

Proof of Theorem 3.2. Suppose that the hypotheses of Theorem 3.2 hold. As-
−1
sume that |η0 | = 1 without loss of generality and let η0 = B− Φ(t0 )e1 . Then
equation (4.19) becomes
∆uϵ (0; η0 ) − ∆sϵ (0; η0 ) = ϵM2 (t0 ) + O(ϵ2 ),
so that by (4.7)
d(t0 , ϵ) = ϵ3 c0 M2 (t0 ) + O(ϵ4 ).
Application of the implicit function theorem to
1
d(t0 , ϵ) = 0
ϵ3
shows that W s (M ) and W u (M ) intersect near
−1
(x, y) = (xh (0), ϵB− Φ(t0 )e1 ).
12 KAZUYUKI YAGASAKI

Moreover, using (4.7), we obtain

( u )
∂ ∂xϵ ∂xs
d(t̄0 , ϵ) =c0 Dx H(xh (0), 0) · (0; t̄0 ) − ϵ (0; t̄0 )
∂t0 ∂t0 ∂t0
dM 2
=ϵ3 c0 (t̄0 ) + O(ϵ4 ),
dt0
which means that the intersection between W s (M ) and W u (M ) is transversal on
the energy surface H = ϵ2 q0 (Φ(t0 )e1 ) + O(ϵ3 ). Thus, we complete the proof of
Theorem 3.2. 2

5. An example. In this section we apply our theory to the Hamiltonian (1.3)

with β2 /β1 = 1/3, which corresponds to the Hénon-Heiles Hamiltonian (1.2) with
c/d = 3/4. Eq. (2.1) has a homoclinic orbit given by (2.2). The NVEs (3.1) and
(3.2) become
η̇1 = η2 , η̇2 = −ω 2 η1 (5.1)
and ( ( ))
1 2 t
η̇1 = η2 , η̇2 = − ω + sech
2
η1 , (5.2)
2 2
respectively. The fundamental matrices to (5.1) and (5.2) are given by
  ( )
1 Ψ11 (t) Ψ12 (t)
 cos ωt sin ωt 
Φ(t) = ω and Ψ(t) = ,
−ω sin ωt cos ωt Ψ21 (t) Ψ22 (t)
respectively, where
1 ( 1 )
Ψ11 (t) = 2ω cos ωt − sin ωt tanh t ,
2ω 2
2 ( 1 )
Ψ12 (t) = 2 2ω sin ωt + cos ωt tanh t ,
4ω + 1 2
1 ( 2 1 1 )
Ψ21 (t) = − 4ω sin ωt + 2ω cos ωt tanh t + sin ωt sech2 t ,
4ω 2 2
1 ( 1 1 )
Ψ22 (t) = 2 4ω cos ωt − 2ω sin ωt tanh t + cos ωt sech2 t .
2
4ω + 1 2 2
We remark that the fundamental matrix to (5.2) can be expressed in terms of
elementary functions since M1 (t0 ) ≡ 0, which implies, as shown in [36], that the
normal variational equation (5.2) is integrable in the meaning of the differential
Galois theory [23, 25]. We compute
   
2 4ω 2 − 1 4
 1 ±
4ω 2 + 1   2
 , B0 =  4ω + 1 4ω 2 + 1 
B± =   1
.
4ω 2   4ω 2
4ω 2 − 1 
∓ − 2
2 4ω 2 + 1 4ω + 1 4ω 2 + 1
On the other hand,
1 2 1
q0 (x, η) = [(ω + β2 x1 )η12 + η22 ], q1 (x, η) = β3 η13 .
2 3
We easily see that
M1 (t0 ) = q0 (0, Φ(t0 )e1 ) − q0 (0, B0 Φ(t0 )e1 ) ≡ 0.
 HIGHER-ORDER MELNIKOV METHOD AND CHAOS 13

After a lengthy calculation, we obtain

1
K(v) = β3 [K30 (ω)v13 + K12 (ω)v1 v22 ],
3
where
6ω 2 + 1
K30 (ω) = (4 − 3πω(3 cosh 2πω + 1) csch 3πω),
24ω 4
4 − 6πω(6ω 2 + 1)(cosh 2πω + 1) csch 3πω
K12 (ω) = .
ω 2 (4ω 2 + 1)2
So we compute (3.7) as
M2 (t0 ) =β3 (cos ωt0 + 2ω sin ωt0 )
× {K0 (ω) + K1 (ω)[(4ω 2 − 1) cos 2ωt0 − 4ω sin 2ωt0 ]},
where
4(12ω 2 + 1) − 48ω 4 K30 (ω) + ω 2 (4ω 2 + 1)2 K12 (ω)
K0 (ω) = ,
12(4ω 2 + 1)2
4(12ω 2 − 1) − 48ω 4 K30 (ω) + 3ω 2 (4ω 2 + 1)2 K12 (ω)
K1 (ω) = .
12(4ω 2 + 1)3
If K0 (ω) = K1 (ω) = 0, then
4
K12 (ω) = ,
ω 2 (4ω 2 + 1)2
i.e.,
6πω(6ω 2 + 1)(cosh(2πω) + 1) csch(3πω) = 0,
which never holds for ω > 0. Hence, for any ω > 0, K0 (ω) ̸= 0 or K1 (ω) ̸= 0. Thus,
the second-order Melnikov function M2 (t0 ) has zeros at
( ) ( )
1 1
ω0 t0 = − arctan and π − arctan . (5.3)
2ω 2ω
We easily see that
(4ω 2 − 1) cos 2ωt0 − 4ω sin 2ωt0 = 4ω 2 + 1
at the zeros of (5.3) and
24πω(6ω 2 + 1) sinh(πω)
K0 (ω) + (4ω 2 + 1)K1 (ω) = ̸= 0
2 cosh(2πω) + 1
for ω > 0. Thus, the zeros of (5.3) for M2 (t0 ) are simple. So we prove the following
result.
Theorem 5.1. Let β2 /β1 = 1/3 and β3 ̸= 0. Then for α > 0 sufficiently small
the Hamiltonian system (1.1) with the Hamiltonian (1.3) has orbits transversely
homoclinic to the periodic orbit γ α near the saddle-center O and a Smale horseshoe
in its dynamics on the energy surface H = Hα . In particular, this statement holds
for the Hénon-Heiles Hamiltonian (1.2) with c/d = 3/4.
Figure 3 shows projections of numerically computed periodic orbits near the
saddle-center O on the y-plane for the Hamiltonian (1.3), where the parameter
values were given by (1.4) with c = 0.75 and d = 1. To obtain the periodic orbits in
14 KAZUYUKI YAGASAKI

0.2

0.1

y2 0

-0.1

-0.2
-0.3 -0.2 -0.1 0 0.1 0.2
y1
Figure 3. Projections of numerically computed periodic orbits
near the saddle-center O on the y-plane for the Hamiltonian (1.3),
where the parameter values were given by (1.4) with c = 0.75 and
d = 1. Their energy levels are H = 1 × 10−3 , 2 × 10−3 , 3 × 10−3 ,
4 × 10−3 and 5 × 10−3 from the inside.

0.8

x2

-0.8
-0.3 1.5
x1

Figure 4. Numerically computed stable and unstable manifolds

of a periodic orbit near the saddle-center O for the Hamiltonian
(1.3), where the parameter values were chosen via (1.4) with c =
0.75 and d = 1, and the Hamiltonian energy is H = 3 × 10−3 .

Fig. 3, we considered a boundary value problem for (1.1) with (1.3) under boundary
conditions
x1 (0) = x1 (T ), x2 (0) = x2 (T ), y1 (0) = y1 (T ) = 0,
where T is a period, and numerically continued a solution starting at the origin for
H = 0 using AUTO [5]. Figure 4 shows numerically computed stable and unstable
manifolds of the periodic orbit near the saddle-center with H = 3 × 10−3 on the
Poincaré section Σ = {(x1 , x2 , y1 , y2 ) ∈ R4 | y1 = 0, y2 > 0}. Here a computer
software called Dynamics [26] was used with the assistance of a differential equation
 HIGHER-ORDER MELNIKOV METHOD AND CHAOS 15

solver called DOP853 [13], which is based on the explicit Runge-Kutta method of
order 8 [6] and has a fifth order error estimator with third order correction and
a dense output of order 7. To obtain a point at which a computed trajectory
intersects the Poincaré section, an interval [tk−1 , tk ] of numerical integration such
that y1 (tk−1 ) < 0 and y1 (tk ) ≥ 0 was searched and the method of bisection was
used for the interval with an error of 10−8 . A tolerance of 10−8 was chosen in
our computation. See also [38, 39] for more details on the incorporation of DOP853
in Dynamics, which is included in a package of AUTO and Dynamics drivers called
HomMap. From Fig. 4 we see that the stable and unstable manifolds of the periodic
orbit intersect transversely as predicted by Theorem 5.1.

REFERENCES

[1] R. Abraham and J.E. Marsden, “Foundations of Mechanics,” 2nd ed., Addison-Wesley, Red-
wood City, CA, 1978.
[2] J. Cresson, Hyperbolicity, transversality and analytic first integrals, J. Differential Equations,
196 (2004), 289–300.
[3] R. Cushman, Examples of nonintegrable analytic Hamiltonian vector fields with no small
divisor, Trans. Amer. Math. Soc., 238 (1978), 45–55.
[4] H. Dankowicz, Looking for chaos: An extension and alternative to Melnikov’s method, Int.
J. Bifurcation Chaos, 6 (1996), 485–496.
[5] E.J. Doedel, A.R. Champneys, T.F. Fairgrieve, Y.A. Kuznetsov, B. Sandstede and X. Wang,
AUTO97: Continuation and Bifurcation Software for Ordinary Differential Equations (with
HomCont), Concordia University, Montreal, 1997.
[6] J.R. Dormand and P.J. Prince, Practical Runge-Kutta processes, SIAM J. Sci. Stat. Comput.,
10 (1989), 977–989.
[7] S.A. Dovbysh, Transversal intersection of separatrices and branching of solutions as obstruc-
tions to the existence of an analytic integral in many-dimensional systems. I. Basic result:
separatrices of hyperbolic periodic points. Collect. Math. 50 (1999), 119–197.
[8] N. Fenichel, Persistence and smoothness of invariant manifolds for flow, Ind. Univ. Math.
J., 21 (1971), 193–225.
[9] A. Goriely, “Integrability and Nonintegrability of Dynamical Systems,” World Scientific, New
Jersey, 2001.
[10] C. Grotta-Ragazzo, Nonintegrability of some Hamiltonian systems, scattering and analytic
continuation, Commun. Math. Phys., 166 (1994), 255–277.
[11] C. Grotta-Ragazzo, Irregular dynamics and homoclinic orbits to Hamiltonian saddle centers,
Commun. Pure Appl. Math., 50 (1997), 105–147.
[12] J. Guckenheimer and P.J. Holmes, “Nonlinear Oscillations, Dynamical Systems, and Bifur-
cations of Vector Fields,” Springer-Verlag, New York, 1983.
[13] E. Hairer, S.P. Nørsett and G. Wanner, “Solving Ordinary Differential Equations I,” 2nd ed.,
Springer-Verlag, Berlin, 1993.
[14] G. Haller, “Chaos near Resonances,” Springer-Verlag, New York, 1999.
[15] M. Hénon and C. Heiles, The applicability of the third integral of motion: Some numerical
experiments, Astron. J., 69 (1964), 73–79.
[16] H. Ito, Non-integrability of Hénon-Heiles system and a theorem of Ziglin, Kodai Math. J., 8
(1985), 120–138.
[17] H. Ito, A criterion for non-integrability of Hamiltonian systems with nonhomogeneous po-
tentials, J. Appl. Math. Phys. (ZAMP), 38 (1987), 459–476.
[18] O.Y. Koltsova and L.M. Lerman, Periodic and homoclinic orbits in a two-parameter unfolding
of a Hamiltonian system with a homoclinic orbit to a saddle-center, Int. J. Bifurcation Chaos,
5 (1995), 397–408.
[19] L.M. Lerman, Hamiltonian systems with loops of a separatrix of a saddle-center, Selecta
Math. Sov., 10 (1991), 297–306.
[20] V.K. Melnikov, On the stability of the center for time periodic perturbations, Trans. Moscow
Math. Soc., 12 (1963), 1–56.
[21] K.R. Meyer and G.R. Hall, “Introduction to Hamiltonian Dynamical Systems and the N-Body
Problem,” Springer-Verlag, New York, 1992.
16 KAZUYUKI YAGASAKI

[22] A. Mielke, P.J. Holmes and O. O’Reilly, Cascades of homoclinic orbits to, and chaos near, a
Hamiltonian saddle-center, J. Dyn. Diff. Eqn., 4 (1992), 95–126.
[23] J.J. Morales-Ruiz, “Differential Galois Theory and Non-Integrability of Hamiltonian Sys-
tems,” Birkhäuser, Basel, 1999.
[24] J.J. Morales-Ruiz and J.M. Peris, On a Galoisian approach to the splitting of separatrices,
Ann. Fac. Sci. Toulouse Math., 8 (1999), 125–141. MR1721562 (2002d:37093)
[25] J.J. Morales-Ruiz and J.P. Ramis, Galoisian obstructions to integrability of Hamiltonian
systems I, Methods Appl. Anal. 8 (2001), 33–96.
[26] H.E. Nusse and J.A. Yorke, “Dynamics: Numerical Explorations,” 2nd ed., Springer-Verlag,
New York, 1997.
[27] J.A. Sanders and F. Verhulst, “Averaging Methods in Nonlinear Dynamical Systems,”
Springer-Verlag, New York, 1985.
[28] L.P. Shil’nikov, A contribution to the problem of the structure of an extended neighborhood
of a rough equilibrium state of saddle-focus type, Math. USSR Sbornik, 10 (1970), 91–102.
[29] A. Vanderbauwhede, Centre manifolds, normal forms and elementary bifurcations, in: “Dy-
namics Reported,” Vol. 2 (eds. U. Kirchgraber and H.O. Walther), John Wiley and Sons,
Chichester, (1989), pp. 89–169.
[30] S. Wiggins, “Global Bifurcations and Chaos – Analytical Methods,” Springer-Verlag, New
York, 1988.
[31] S. Wiggins, “Introduction to Applied Nonlinear Dynamical Systems and Chaos,” Springer-
Verlag, New York, 1990.
[32] S. Wiggins, “Normally Hyperbolic Invariant Manifolds in Dynamical Systems,” Springer-
Verlag, New York, 1994.
[33] K. Yagasaki, Horseshoes in two-degree-of-freedom Hamiltonian systems with saddle-centers,
Arch. Rational Mech. Anal., 154 (2000), 275–296.
[34] K. Yagasaki, Homoclinic and heteroclinic behavior in an infinite-degree-of-freedom Hamil-
tonian system: Chaotic free vibrations of an undamped, buckled beam, Phys. Lett. A, 285
(2001), 55–62.
[35] K. Yagasaki, Numerical evidence of fast diffusion in a three-degree-of-freedom Hamiltonian
system with a saddle-center, Phys. Lett. A, 301 (2002), 45–52.
[36] K. Yagasaki, Galoisian obstructions to integrability and Melnikov criteria for chaos in two-
degree-of-freedom Hamiltonian systems with saddle-centers, Nonlinearity, 16 (2003), 2003–
2012.
[37] K. Yagasaki, Homoclinic and heteroclinic orbits to invariant tori in multi-degree-of-freedom
Hamiltonian systems with saddle-centers, Nonlinearity, 18 (2005), 1331–1350.
[38] K. Yagasaki, Numerical analysis for global bifurcations of periodic orbits in autonomous
differential equations, in preparation.
[39] K. Yagasaki, “HomMap: A Package of AUTO and Dynamics Drivers for Homoclinic Bifur-
cation Analysis for Periodic Orbits of Maps and ODEs, Version 2.0,” in preparation.
[40] S.L. Ziglin, Branching of solutions and non-existence of first integrals in Hamiltonian me-
chanics, I, Funct. Anal. Appl., 16 (1982), 181–189.