Junos Troubleshooting in the NOC

LAB GUIDE Revision 1'.W

Education Services Courseware

Corporate and Sales Headquarters
Juniper Networks, Inc.
1133 Innovation Way
Sunnyvale, CA 94089 USA
Phone: 888.JUNIPER (888.586.4737)
or 408.745.2000
Fax: 408.745.2100
www.juniper.net

APAC and EMEA Headquarters

Juniper Networks International B.V.
Boeing Avenue 240
1110 PZ SCHIPHOL-RIJK
Amsterdam, The Netherlands
Phone: 31.0.207.125.700
Fax: 31.0.207.125.701

Copyright 2014
Juniper Networks, Inc. All rights reserved.
Juniper Networks, the Juniper Networks logo, Junos, NetScreen, and ScreenOS
are registered trademarks of Juniper Networks, Inc. in the United States and
other countries. All other trademarks, services marks, registered marks, or
registered services marks are the property of their respective owners. Juniper
Networks assumes no responsibility for any inaccuracies in this document.
Juniper Networks reserves the right to change, modify, transfer, or otherwise
revise this publication without notice.

Printed on recycled paper

EDU-JUN-JTNOC, Revision 1'.W

Junos Troubleshooting in the NOC
12.b

Lab Guide

Worldwide Education Services

1133 Innovation Way

Sunnyvale, CA 94089
USA
408-745-2000
www.juniper.net

Course Number: EDU-JUN-JTNOC

This document is produced by Juniper Networks, Inc.
This document or any part thereof may not be reproduced or transmitted in any form under penalty of law, without the prior written permission of Juniper Networks
Education Services.
Juniper Networks, the Juniper Networks logo, Junos, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United States and other
countries. All other trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.

Junos Troubleshooting in the NOC Lab Guide, Revision 12.b

Copyright © 2014 Juniper Networks, Inc. All rights reserved.
Printed in USA.
Revision History:
Revision 10.a—December 2010
Revision 11.a—June 2011
Revision 12.a—March 2013
Revision 12.b—January 2014
The information in this document is current as of the date listed above.
The information in this document has been carefully verified and is believed to be accurate for software Release 12.2R2.5. Juniper Networks assumes no
responsibilities for any inaccuracies that may appear in this document. In no event will Juniper Networks be liable for direct, indirect, special, exemplary,
incidental, or consequential damages resulting from any defect or omission in this document, even if advised of the possibility of such damages.

Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice.
YEAR 2000 NOTICE
Juniper Networks hardware and software products do not suffer from Year 2000 problems and hence are Year 2000 compliant. The Junos operating system has
no known time-related limitations through the year 2038. However, the NTP application is known to have some difficulty in the year 2036.
SOFTWARE LICENSE
The terms and conditions for using Juniper Networks software are described in the software license provided with the software, or to the extent applicable, in an
agreement executed between you and Juniper Networks, or Juniper Networks agent. By using Juniper Networks software, you indicate that you understand and
agree to be bound by its license terms and conditions. Generally speaking, the software license restricts the manner in which you are permitted to use the Juniper
Networks software, may contain prohibitions against certain uses, and may state conditions under which the license is automatically terminated. You should
consult the software license for further details.
 Contents
Lab 1: The Troubleshooting Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1
Part 1: Troubleshooting Scenario 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
Part 2: Troubleshooting Scenario 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4
Part 3: Troubleshooting Scenario 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6

Lab 2: Identifying Hardware Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1

Part 1: Logging In Using the CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2
Part 2: Verifying Initial Device Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3
Part 3: Identifying Hardware and Key Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7
Part 4: Utilizing Online Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9

Lab 3: Using Monitoring Tools and Establishing a Baseline . . . . . . . . . . . . . . . . . . 3-1

Part 1: Viewing Alarms, Environmental Conditions, and Hardware Inventory . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
Part 2: Establishing a Baseline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-9
Part 3: Locating Online Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17

Lab 4: Monitoring Hardware and Environmental Conditions . . . . . . . . . . . . . . . . . . 4-1

Part 1: Monitoring Memory and Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Part 2: Viewing Boot and System Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7
Part 3: Monitoring the Chassis and Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-13

Lab 5: Control Plane Monitoring and Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . 5-1

Part 1: Examining User Processes and Daemons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Part 2: Generating Core Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-10
Part 3: Configuring Routing Protocols and Routing Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-13
Part 4: Configuring Bridging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-27

Lab 6: Monitoring and Troubleshooting Ethernet Interfaces . . . . . . . . . . . . . . . . . . 6-1

Part 1: Performing Interface Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2
Part 2: Performing Loopback Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-13
Part 3: Configuring Ethernet OAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-19

Lab 7: Isolating and Troubleshooting PFE Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-1

Part 1: Examining the Forwarding Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-2
Part 2: Configuring Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-7
Part 3: Troubleshooting Firewall Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-11

Lab 8: Troubleshooting Routing Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1

Part 1: Modifying the Existing Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2
Part 2: Troubleshooting OSPF Adjacencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-3
Part 3: Troubleshooting BGP Sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-19
Part 4: Troubleshooting Routing Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-34

Lab 9: Monitoring the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-1

Part 1: Modifying the Existing Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-2
Part 2: Configuring and Monitoring SNMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-3
Part 3: Configuring and Monitoring RMON . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-11
Part 4: Configuring and Monitoring Inline JFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-19
Part 5: Configuring and Monitoring Inline Port Mirroring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-23

www.juniper.net Contents • iii

iv • Contents www.juniper.net
Course Overview

This three-day course is designed to provide introductory troubleshooting skills for engineers in a network operations center
(NOC) environment. Key topics within this course include troubleshooting methodology, troubleshooting tools, hardware
monitoring and troubleshooting, interface monitoring and troubleshooting, troubleshooting the data plane and control
plane on devices running the Junos operating system, staging and acceptance methodology, troubleshooting routing
protocols, monitoring the network, and working with JTAC. This course is based on Junos operating system Release
12.2R2.5.
Objectives
After successfully completing this course, you should be able to:
• Reduce the time it takes to identify and isolate the root cause of an issue impacting your network.
• Gain familiarity with Junos products as they pertain to troubleshooting.
• Become familiar with online resources valuable to Junos troubleshooting.
• Gain familiarity with Junos tools used in troubleshooting.
• Identify and isolate hardware issues.
• Troubleshoot problems with the control plane.
• Troubleshoot problems with interfaces and other data plane components.
• Describe the staging and acceptance methodology.
• Troubleshoot routing protocols.
• Describe how to monitor your network with SNMP, RMON, JFlow, and port mirroring.
• Become familiar with JTAC procedures.
Intended Audience
The course content is aimed at operators of devices running the Junos OS in a NOC environment. These operators include
network engineers, administrators, support personnel, and reseller support personnel.
Course Level
Junos Troubleshooting in the NOC is an introductory-level course.
Prerequisites
Students should have basic networking knowledge and an understanding of the Open Systems Interconnection (OSI)
reference model and the TCP/IP protocol suite. Students should also attend the Introduction to the Junos Operating System
(IJOS) course and the Junos Routing Essentials (JRE) course, or have equivalent experience prior to attending this class.

www.juniper.net Course Overview • v

Course Agenda

Day 1
Chapter 1: Course Introduction
Chapter 2: Troubleshooting as a Process
The Troubleshooting Process Lab
Chapter 3: Junos Product Families
Identifying Hardware Components Lab
Chapter 4: Troubleshooting Toolkit
Monitoring Tools and Establishing a Baseline Lab

Day 2
Chapter 5: Hardware and Environmental Conditions
Monitoring Hardware and Environmental Conditions Lab
Chapter 6: Control Plane
Control Plane Monitoring and Troubleshooting Lab
Chapter 7: Data Plane: Interfaces
Monitoring and Troubleshooting Ethernet Interfaces Lab
Chapter 8: Data Plane: Other Components
Isolating and Troubleshooting PFE Issues Lab
Day 3
Chapter 9: Staging and Acceptance Testing
Chapter 10: Troubleshooting Routing Protocols
Troubleshooting Routing Protocols Lab
Chapter 11: High Availability
Chapter 12: Network Monitoring
Monitoring the Network Lab
Chapter 13: JTAC Procedures
Appendix A: Interface Troubleshooting

vi • Course Agenda www.juniper.net

Document Conventions

CLI and GUI Text

Frequently throughout this course, we refer to text that appears in a command-line interface (CLI) or a graphical user
interface (GUI). To make the language of these documents easier to read, we distinguish GUI and CLI text from chapter
text according to the following table.

Style Description Usage Example

Franklin Gothic Normal text. Most of what you read in the Lab Guide
and Student Guide.

Courier New Console text:

commit complete
• Screen captures
• Noncommand-related Exiting configuration mode
syntax
GUI text elements:
Select File > Open, and then click
• Menu names Configuration.conf in the
Filename text box.
• Text field entry

Input Text Versus Output Text

You will also frequently see cases where you must enter input text yourself. Often these instances will be shown in the
context of where you must enter them. We use bold style to distinguish text that is input versus text that is simply
displayed.

Style Description Usage Example

Normal CLI No distinguishing variant. Physical interface:fxp0,

Enabled
Normal GUI
View configuration history by clicking
Configuration > History.

CLI Input Text that you must enter. lab@San_Jose> show route
GUI Input Select File > Save, and type
config.ini in the Filename field.

Defined and Undefined Syntax Variables

Finally, this course distinguishes between regular text and syntax variables, and it also distinguishes between syntax
variables where the value is already assigned (defined variables) and syntax variables where you must assign the value
(undefined variables). Note that these styles can be combined with the input style as well.

Style Description Usage Example

CLI Variable Text where variable value is policy my-peers

already assigned.
GUI Variable
Click my-peers in the dialog.

CLI Undefined Text where the variable’s value Type set policy policy-name.
is the user’s discretion and text
ping 10.0.x.y
where the variable’s value as
GUI Undefined shown in the lab guide might Select File > Save, and type
differ from the value the user filename in the Filename field.
must input.

www.juniper.net Document Conventions • vii

Additional Information

Education Services Offerings

You can obtain information on the latest Education Services offerings, course dates, and class locations from the World
Wide Web by pointing your Web browser to: http://www.juniper.net/training/education/.
About This Publication
The Junos Troubleshooting in the NOC Lab Guide was developed and tested using software Release 12.2R2.5. Previous
and later versions of software might behave differently so you should always consult the documentation and release
notes for the version of code you are running before reporting errors.
This document is written and maintained by the Juniper Networks Education Services development team. Please send
questions and suggestions for improvement to training@juniper.net.
Technical Publications
You can print technical manuals and release notes directly from the Internet in a variety of formats:
• Go to http://www.juniper.net/techpubs/.
• Locate the specific software or hardware release and title you need, and choose the format in which you
want to view or print the document.
Documentation sets and CDs are available through your local Juniper Networks sales office or account representative.
Juniper Networks Support
For technical support, contact Juniper Networks at http://www.juniper.net/customers/support/, or at 1-888-314-JTAC
(within the United States) or 408-745-2121 (from outside the United States).

viii • Additional Information www.juniper.net

 Lab
The Troubleshooting Process

Overview
In this lab, you will focus on the process of troubleshooting.
By completing this lab, you will perform the following tasks:
• Define success for a given problem.
• Create a flowchart to help isolate the root cause of a problem.
• Identify possible ways to achieve success.
• Evaluate the potential impact associated with implementing a solution in a
production environment.

www.juniper.net The Troubleshooting Process • Lab 1–1

.
Junos Troubleshooting in the NOC

Part 1: Troubleshooting Scenario 1

In this lab part, you use the troubleshooting process to create an efficient approach
to help you identify the root cause of a specific problem.
In this scenario, a random card is drawn from a standard 52 card deck. Your
objective is to form an approach that will isolate the selected card. Your instructor or
lab partner will provide you with the true or false outcome of your decision points.
Step 1.1
Define success:
Accurately identify the card drawn from a standard deck of cards.

Step 1.2
Isolate the issue:

Draw a simple flowchart of how you might identify the selected card. Remember
each decision point should have a true or false outcome and should eliminate
potential possibilities regardless of the outcome.
Remember, this is just one possible flowchart. The “face value is odd” decision and
the “face value is greater than” decision could have been modified or reordered
without greatly impacting the number of steps required to identify a solution.

Lab 1–2 • The Troubleshooting Process www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net The Troubleshooting Process • Lab 1–3

Junos Troubleshooting in the NOC

Part 2: Troubleshooting Scenario 2

In this lab part, you use the troubleshooting process to create an efficient approach
to help you identify the root cause of a specific problem.
In this scenario, you have nine balls that appear equal in size, shape, and color.
However, one of the balls is very slightly overweight—Not perceivable to the human
eye or feel. That is OK, a scale is provided for your use. Your objective is to identify
the ball that is heavier than the rest. Your instructor or lab partner will provide you
with the True/False outcome of your decision points.
Step 2.1
Define success:
Identify the heavy ball.

Step 2.2
Isolate the issue:

Draw a simple flowchart of how you might identify the heavy ball. Remember each
decision point should have a True or False outcome and should eliminate potential
possibilities regardless of the outcome.
You can take a couple of different approaches to this problem. One approach
guarantees the outcome in two steps, while the other might take three steps, but
also has the possibility of finding the problem in one. The second approach is
common to many network troubleshooting issues, where a quick test can provide
the necessary isolation, or some limited follow-up might be required.
When troubleshooting, it is not always easy to say if one approach is better than the
other. If the outcome is similar, in a similar number of steps, with similar disruption
to the network, then it is an acceptable approach.

Lab 1–4 • The Troubleshooting Process www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net The Troubleshooting Process • Lab 1–5

Junos Troubleshooting in the NOC

Part 3: Troubleshooting Scenario 3

In this lab part, you use the troubleshooting process to create an efficient approach
to help identify the root cause of a specific problem.
Step 3.1
Define success:

Step 3.2
Isolate the issue:
Identify the elements required to achieve success.

Identify possible methods to help isolate the element preventing success.

What impact could testing for the root cause have in a production environment? How
might you mitigate these risks?

Draw a simple flowchart of how you might narrow down the focus. Remember each
decision point should have a True or False outcome and should bring you closer to
understanding the root cause of a problem.

Lab 1–6 • The Troubleshooting Process www.juniper.net

 Junos Troubleshooting in the NOC
Step 3.3
Identify a solution:
Identify possible ways to achieve success. (Can you think of more than one way?)

Can you think of a short-term solution that would return a production network to
operating status while waiting to implement a permanent long-term solution?

Step 3.4
Implement the solution:
Can you think of any potential impact risks associated with implementing the
identified solution in a production environment? How might you mitigate those
risks?

STOP Tell your instructor that you have completed this lab.

www.juniper.net The Troubleshooting Process • Lab 1–7

Junos Troubleshooting in the NOC

Lab 1–8 • The Troubleshooting Process www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net The Troubleshooting Process • Lab 1–9

Junos Troubleshooting in the NOC

Lab 1–10 • The Troubleshooting Process www.juniper.net

 Lab
Identifying Hardware Components

Overview
In this lab, you will log in to your router, identify the installed hardware, determine which
hardware components are field-replaceable units (FRUs), and research how to replace
faulty hardware components.
By completing this lab, you will perform the following tasks:
• Verify proper operation of network interfaces.
• Identify hardware.
• Determine what is and what is not an FRU.
• Research how to replace a faulty hardware component.

www.juniper.net Identifying Hardware Components • Lab 2–1

.
Junos Troubleshooting in the NOC

Part 1: Logging In Using the CLI

In this lab part, you log in to your router and verify that interfaces are operational
and that the base topology used in subsequent labs is in place.
Note
The instructor will tell you the nature of your
access and will provide you with the
necessary details to access your assigned
device.

Step 1.1
Ensure that you know to which device you have been assigned. Check with your
instructor if you are not certain. Consult the management network diagram to
determine your device’s management address.

Question: What is the management address

assigned to your device?

Answer: The answer varies and depends on your

assigned device. If you are unsure of your
assignment, ask your instructor.

Step 1.2
Access the CLI for your device using either the console, Telnet, or SSH as directed by
your instructor. Refer to the management network diagram for the IP address
associated with your station. The following example uses Telnet to access an
MX Series device using the SecureCRT program:

Lab 2–2 • Identifying Hardware Components www.juniper.net

 Junos Troubleshooting in the NOC
Step 1.3
Log in to your device with the username lab and the password lab123. Note that
both the name and password are case-sensitive.
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1>
Step 1.4
Enter into configuration mode and load the device’s reset configuration by issuing
the load override jtnoc/reset.config command. After the configuration
has been loaded, commit the changes and exit to operational mode before
proceeding to Part 2.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# load override jtnoc/reset.config
load complete

[edit]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>

Part 2: Verifying Initial Device Configuration

In this lab part, you verify the initial configuration of your assigned device.
Step 2.1
Issue the show interfaces terse | no-more command.
lab@mxA-1> show interfaces terse | no-more
Interface Admin Link Proto Local Remote
lc-0/0/0 up up
lc-0/0/0.32769 up up vpls
pfe-0/0/0 up up
pfe-0/0/0.16383 up up inet
inet6
pfh-0/0/0 up up
pfh-0/0/0.16383 up up inet
xe-0/0/0 up down
xe-0/0/1 up down
xe-0/0/2 up down
xe-0/0/3 up down
ge-1/0/0 up up
ge-1/0/0.141 up up inet 172.18.1.2/30

www.juniper.net Identifying Hardware Components • Lab 2–3

Junos Troubleshooting in the NOC
multiservice
ge-1/0/0.32767 up up multiservice
ge-1/0/1 up up
ge-1/0/1.141 up up inet 172.18.2.2/30
multiservice
ge-1/0/1.32767 up up multiservice
ge-1/0/2 up up
ge-1/0/2.141 up up inet 172.18.5.1/30
multiservice
ge-1/0/2.32767 up up multiservice
ge-1/0/3 up up
ge-1/0/3.141 up up inet 192.168.11.1/24
multiservice
ge-1/0/3.32767 up up multiservice
ge-1/0/4 up up
ge-1/0/5 up up
ge-1/0/6 up up
ge-1/0/7 up up
ge-1/0/8 up up
ge-1/0/9 up up
ge-1/1/0 up up
ge-1/1/1 up down
ge-1/1/2 up up
ge-1/1/3 up down
ge-1/1/4 up up
ge-1/1/5 up up
ge-1/1/6 up up
ge-1/1/7 up up
ge-1/1/8 up up
ge-1/1/9 up up
cbp0 up up
demux0 up up
dsc up up
em0 up up
em0.0 up up inet 10.0.0.4/8
128.0.0.1/2
128.0.0.4/2
inet6 fe80::200:ff:fe00:4/64
fec0::a:0:0:4/64
tnp 0x4
em1 up down
fxp0 up up
fxp0.0 up up inet 10.210.15.1/27
gre up up
ipip up up
irb up up
lo0 up up
lo0.0 up up inet 192.168.31.1 --> 0/0
lo0.16384 up up inet 127.0.0.1 --> 0/0
lo0.16385 up up inet
lsi up up
me0 up up
me0.0 up up
mtun up up
pimd up up

Lab 2–4 • Identifying Hardware Components www.juniper.net

 Junos Troubleshooting in the NOC
pime up up
pip0 up up
pp0 up up
tap up up

Question: What are the Admin and Link states for

the interfaces listed in the lab diagram for your
device?

Answer: Interfaces shown as configured and

functioning in the diagram for lab 2 should show
Admin and Link status of up as shown in the
previous output. If other configured interfaces show
a down status, notify your instructor.

Question: What are the IP addresses configured on

the interfaces listed in the lab diagram for your
device?

Answer: The IP addresses should match what is

displayed in the lab diagram for lab 2 for your
assigned device.

Step 2.2
Next, ping the virtual router attached to your device to ensure connectivity exists.
lab@mxA-1> ping local-VR-Address count 5
PING 192.168.11.2 (192.168.11.2): 56 data bytes
64 bytes from 192.168.11.2: icmp_seq=0 ttl=64 time=0.530 ms
64 bytes from 192.168.11.2: icmp_seq=1 ttl=64 time=0.823 ms
64 bytes from 192.168.11.2: icmp_seq=2 ttl=64 time=3.424 ms
64 bytes from 192.168.11.2: icmp_seq=3 ttl=64 time=0.484 ms
64 bytes from 192.168.11.2: icmp_seq=4 ttl=64 time=0.483 ms

--- 192.168.11.2 ping statistics ---

5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.483/1.149/3.424/1.145 ms

www.juniper.net Identifying Hardware Components • Lab 2–5

Junos Troubleshooting in the NOC
Question: Are you able to ping the virtual router
attached to your device?

Answer: As indicated by the output, you should be

able to ping the attached virtual router. Notify your
instructor if you are unable to successfully ping the
virtual router.

Step 2.3
Next, ping the address associated with the remote team’s ge-1/0/2 interface.
lab@mxA-1> ping remote-team-ge-1/0/2-address count 5
PING 172.18.5.2 (172.18.5.2): 56 data bytes
64 bytes from 172.18.5.2: icmp_seq=0 ttl=64 time=1.048 ms
64 bytes from 172.18.5.2: icmp_seq=1 ttl=64 time=12.862 ms
64 bytes from 172.18.5.2: icmp_seq=2 ttl=64 time=0.438 ms
64 bytes from 172.18.5.2: icmp_seq=3 ttl=64 time=0.478 ms
64 bytes from 172.18.5.2: icmp_seq=4 ttl=64 time=0.495 ms

--- 172.18.5.2 ping statistics ---

5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.438/3.064/12.862/4.904 ms

Question: Can you ping the remote team’s router?

Answer: As indicated by the output, you should be

able to ping the remote team’s router. Notify your
instructor if you are unable to successfully ping the
remote team’s router.

Lab 2–6 • Identifying Hardware Components www.juniper.net

 Junos Troubleshooting in the NOC

Part 3: Identifying Hardware and Key Components

In this lab part, you identify hardware and key components in your router.
Step 3.1
Issue the show chassis hardware command.
lab@mxA-1> show chassis hardware | no-more
Hardware inventory:
Item Version Part number Serial number Description
Chassis D4889 MX80
Midplane REV 06 711-031594 YK8973 MX80
PEM 0 Rev 03 740-028288 UG00866 AC Power Entry Module
Routing Engine BUILTIN BUILTIN Routing Engine
TFEB 0 BUILTIN BUILTIN Forwarding Engine
Processor
QXM 0 REV 05 711-028408 YK6600 MPC QXM
FPC 0 BUILTIN BUILTIN MPC BUILTIN
MIC 0 BUILTIN BUILTIN 4x 10GE XFP
PIC 0 BUILTIN BUILTIN 4x 10GE XFP
FPC 1 BUILTIN BUILTIN MPC BUILTIN
MIC 0 REV 22 750-028392 YK7479 3D 20x 1GE(LAN) SFP
PIC 0 BUILTIN BUILTIN 10x 1GE(LAN) SFP
Xcvr 0 REV 02 740-013111 A116746 SFP-T
Xcvr 1 REV 02 740-013111 A114818 SFP-T
Xcvr 2 REV 02 740-013111 A116860 SFP-T
Xcvr 3 REV 02 740-013111 A116971 SFP-T
Xcvr 4 REV 02 740-013111 A116814 SFP-T
Xcvr 5 REV 02 740-013111 A116816 SFP-T
Xcvr 6 REV 02 740-013111 A116260 SFP-T
Xcvr 7 REV 02 740-013111 A115260 SFP-T
Xcvr 8 REV 02 740-013111 A116813 SFP-T
Xcvr 9 REV 02 740-013111 A116719 SFP-T
PIC 1 BUILTIN BUILTIN 10x 1GE(LAN) SFP
Xcvr 0 REV 02 740-013111 A115863 SFP-T
Xcvr 1 REV 02 740-013111 A116368 SFP-T
Xcvr 2 REV 02 740-013111 A071203 SFP-T
Xcvr 3 REV 02 740-013111 A116370 SFP-T
Xcvr 4 REV 02 740-013111 A121193 SFP-T
Xcvr 5 REV 02 740-013111 A121188 SFP-T
Xcvr 6 REV 02 740-013111 A116729 SFP-T
Xcvr 7 REV 02 740-013111 A116891 SFP-T
Xcvr 8 REV 02 740-013111 A114787 SFP-T
Xcvr 9 REV 02 740-013111 A116696 SFP-T
Fan Tray Fan Tray

www.juniper.net Identifying Hardware Components • Lab 2–7

Junos Troubleshooting in the NOC
Question: What is the chassis serial number of your
assigned device?

Answer: As shown in the output, the Chassis field

contains the chassis serial number. The chassis
serial number for the MX Series device in the
previous output is D4889.

Question: Can the MIC located in the MPC identified

as FPC 0 be replaced without replacing the
chassis? Explain your answer.

Answer: No. MIC 0, located in FPC 0, is a built-in

MIC. You cannot replace it unless you replace the
chassis.

Question: Can the MIC located in the MPC identified

as FPC 1 be replaced without replacing the
chassis? Explain your answer.

Answer: Yes. The MIC located in FPC 1, is not built-in

to the chassis. You can replace it without replacing
the chassis.

Question: If the fan tray in the previous output

needs replacement, what information from the
output is required?

Answer: Fan trays do not have serial numbers. To

replace a fan tray only the part description and
chassis type is needed.

Lab 2–8 • Identifying Hardware Components www.juniper.net

 Junos Troubleshooting in the NOC

Part 4: Utilizing Online Resources

In this lab part, you find and use online resources to learn how to identify and
replace FRUs in your router.
Note
As you work through this lab part, you are
encouraged to look beyond the questions
asked in the lab to familiarize yourself with
the available online resources. You are able
to use these resources in future labs and
later in your own network environment.

Step 4.1
Issue the show chassis hardware | no-more command.
lab@mxA-1> show chassis hardware | no-more
Hardware inventory:
Item Version Part number Serial number Description
Chassis E3342 MX80
Midplane REV 07 711-031594 YW2075 MX80
PEM 0 Rev 04 740-028288 UH02222 AC Power Entry Module
Routing Engine BUILTIN BUILTIN Routing Engine
TFEB 0 BUILTIN BUILTIN Forwarding Engine
Processor
QXM 0 REV 05 711-028408 YW3088 MPC QXM
FPC 0 BUILTIN BUILTIN MPC BUILTIN
MIC 0 BUILTIN BUILTIN 4x 10GE XFP
PIC 0 BUILTIN BUILTIN 4x 10GE XFP
FPC 1 BUILTIN BUILTIN MPC BUILTIN
MIC 0 REV 24 750-028392 YX0925 3D 20x 1GE(LAN) SFP
PIC 0 BUILTIN BUILTIN 10x 1GE(LAN) SFP
Xcvr 0 REV 02 740-013111 A495782 SFP-T
Xcvr 1 REV 02 740-013111 A494172 SFP-T
Xcvr 2 REV 02 740-013111 A452847 SFP-T
Xcvr 3 REV 02 740-013111 A495298 SFP-T
Xcvr 4 REV 02 740-013111 A438159 SFP-T
Xcvr 5 REV 02 740-013111 A494218 SFP-T
Xcvr 6 REV 02 740-013111 A464011 SFP-T
Xcvr 7 REV 02 740-013111 A495540 SFP-T
Xcvr 8 REV 02 740-013111 A434868 SFP-T
Xcvr 9 REV 02 740-013111 A433506 SFP-T
PIC 1 BUILTIN BUILTIN 10x 1GE(LAN) SFP
Xcvr 0 REV 02 740-013111 A496017 SFP-T
Xcvr 1 REV 02 740-013111 A464017 SFP-T
Xcvr 2 REV 02 740-013111 A494183 SFP-T
Xcvr 3 REV 02 740-013111 A494235 SFP-T
Xcvr 4 REV 02 740-013111 A494159 SFP-T
Xcvr 5 REV 02 740-013111 A382076 SFP-T
Xcvr 6 REV 02 740-013111 A434336 SFP-T
Xcvr 7 REV 02 740-013111 A494406 SFP-T
Xcvr 8 REV 02 740-013111 A494744 SFP-T
Xcvr 9 REV 02 740-013111 A494074 SFP-T
Fan Tray Fan Tray

www.juniper.net Identifying Hardware Components • Lab 2–9

Junos Troubleshooting in the NOC
Step 4.2
Open a Web browser on your local PC.
From the open Web browser, navigate to http://www.juniper.net/
techpubs/, find the MX80 FRU documentation, and answer the following
questions.

Lab 2–10 • Identifying Hardware Components www.juniper.net

 Junos Troubleshooting in the NOC

Question: What are some of the FRUs found inside

your assigned device?

Answer: In the previous output and online

documentation, the following components are
FRUs: PEM 0, MIC 0 located in FPC 1, all the SFPs
and XFPs, and the fan tray.

Note
If time allows, research information for your
own network devices and familiarize
yourself with the hardware and available
FRUs.

www.juniper.net Identifying Hardware Components • Lab 2–11

Junos Troubleshooting in the NOC
Step 4.3
Return to the open terminal session with your assigned device.
From the open terminal session with your assigned device, log out of your assigned
device.
lab@mxA-1> exit

mxA-1 (ttyu0)

login:

STOP Tell your instructor that you have completed this lab.

Lab 2–12 • Identifying Hardware Components www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Identifying Hardware Components • Lab 2–13

Junos Troubleshooting in the NOC

Lab 2–14 • Identifying Hardware Components www.juniper.net

 Lab
Using Monitoring Tools and Establishing a Baseline

Overview
This lab demonstrates the monitoring and troubleshooting tools of the Junos operating
system.
By completing this lab, you will perform the following tasks:
• Monitor the health of the router and troubleshoot issues.
• Configure and identify baseline components within the router.
• Use online resources to troubleshoot issues.

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–1

.
Junos Troubleshooting in the NOC

Part 1: Viewing Alarms, Environmental Conditions, and Hardware Inventory

In this lab part, you use commands that will allow you to view alarms, environmental
conditions, and hardware Inventory on your router.
Step 1.1
Ensure that you know to which device you have been assigned. Check with your
instructor if you are not certain. Consult the management network diagram to
determine your device’s management address.

Question: What is the management address

assigned to your device?

Answer: The answer varies and depends on your

assigned device. If you are unsure of your
assignment, ask your instructor.

Step 1.2
Access the CLI for your device using either the console, Telnet, or SSH as directed by
your instructor. Refer to the management network diagram for the IP address
associated with your station. The following example uses Telnet to access an
MX Series device using the SecureCRT program:

Step 1.3
Log in to your device with the username lab and a password of lab123. Note that
both the name and password are case-sensitive.

Lab 3–2 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1>
Step 1.4
Enter into configuration mode and load the device’s reset configuration by issuing
the load override jtnoc/lab3-start.config command. After the
configuration has been loaded, commit the changes and exit to operational mode
before moving on to the next step.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# load override jtnoc/lab3-start.config
load complete

[edit]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>

Step 1.5
Issue the show interfaces terse ge* | match inet | except down
command.
lab@mxA-1> show interfaces terse ge* | match inet | except down
ge-1/0/0.141 up up inet 172.18.1.2/30
ge-1/0/1.141 up up inet 172.18.2.2/30
ge-1/0/2.141 up up inet 172.18.5.1/30
ge-1/0/3.141 up up inet 192.168.11.1/24

Question: Which interfaces are present in the

output?

Answer: Interfaces ge-1/0/0, ge-1/0/1, ge-1/0/2

and ge-1/0/3 are present in the output.

Step 1.6
Issue the show chassis craft-interface command.
lab@mxA-1> show chassis craft-interface

Front Panel System LEDs:

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–3

Junos Troubleshooting in the NOC
Routing Engine
------------------
OK *
Fail .

Front Panel Alarm Indicators:

-----------------------------
Red LED .
Yellow LED .
Major relay .
Minor relay .

PS Status:
PS 0 1
---------------
Red . .
Green * .

Fan Tray Status:

FT 0
----------
Red .
Green *

Question: What is the status of the routing engine?

Answer: Although your results might vary, the

routing engine section displays an asterisk in the
OK field. If the asterisk is not in the OK field, please
inform your instructor as this can indicate a
hardware problem with your device.

Question: Are any alarms displayed in this output?

Answer: No. The Front Panel Alarm

Indicators section does not display asterisks
next to any of the alarm indicators are displayed.

Step 1.7
Issue the show pfe statistics error command.
lab@mxA-1> show pfe statistics error

Slot 0

Lab 3–4 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC
HSL2 Errors:
------------
***** No errors on this PFE *****

Question: Are any errors present on this PFE?

Answer: No errors should exist. If errors are present

please inform your instructor.

Step 1.8
Issue the show log messages | match error command and answer the
following questions.

Note
The messages might vary depending on
your device.

lab@mxA-1> show log messages | match error

Aug 31 10:44:16 cfmd[1212]: junos_dfw_session_reply_get:1138 Error "session"
is NULL.
Aug 31 10:47:08 alarmd[1050]: shutting down craftd connection: craftd ipc pipe
read error
Aug 31 10:47:12 l2cp[1084]: junos_dfw_session_connect:984 Error "select"
returns timeout event for session connection.
Aug 31 10:47:15 dfcd[1083]: junos_dfw_session_connect:1005 Error "recv" call
with MSG_PEEK flag returns error status (54).
Aug 31 10:53:11 craftd[1099]: craftd_user_conn_shutdown: socket 13, errno = 0
Aug 31 10:53:11 cfmd[1087]: junos_dfw_trans_purge:1377 Error "session" is not
allocated.
Sep 13 17:36:54 /kernel: em0.0 iff_ifarequest (DELETE) addr: 128.0.0.6 err 22
Sep 13 17:36:54 /kernel: em0.0 iff_ifarequest (DELETE) addr: 10.0.0.6 err 22
Sep 13 17:36:58 l2cp[1145]: junos_dfw_session_connect:1005 Error "recv" call
with MSG_PEEK flag returns error status (61).
Sep 13 17:36:59 dfcd[1144]: junos_dfw_session_connect:1005 Error "recv" call
with MSG_PEEK flag returns error status (61).
Sep 15 22:02:38 mxA-1 login: LOGIN_PAM_AUTHENTICATION_ERROR: PAM
authentication error for user lab
Sep 15 22:08:11 mxA-1 craftd[1048]: craftd_user_conn_shutdown: socket 8, errno
= 0
Sep 15 22:08:12 mxA-1 cfmd[1148]: dfwdlib_full_send:127 Complete send can not
be done. Error Broken pipe
Sep 15 22:08:12 mxA-1 cfmd[1148]: junos_dfw_trans_send:1104 Error in completing
transaction from client 4 with user_context 0
Sep 15 22:16:38 mxA-1 chassisd[1117]: CHASSISD_I2C_READ_ERROR:
pca9548_disable_all_channels: read error for group 0 at address 0xa, offset
-1

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–5

Junos Troubleshooting in the NOC
Question: How does the match error option
change the normal show log messages output?

Answer: The match error option filters the output

and includes lines that contain error in them only.

Note
Although in the previous output, messages
contain the word “error” in them, these
messages are not cause for concern. The
previous output is typically seen during the
boot process when communication
between the routing engine and the
router’s other components are initially
being established.

Step 1.9
Issue the show chassis hardware | no-more command.
lab@mxA-1> show chassis hardware | no-more
Hardware inventory:
Item Version Part number Serial number Description
Chassis D4889 MX80
Midplane REV 06 711-031594 YK8973 MX80
PEM 0 Rev 03 740-028288 UG00866 AC Power Entry Module
Routing Engine BUILTIN BUILTIN Routing Engine
TFEB 0 BUILTIN BUILTIN Forwarding Engine
Processor
QXM 0 REV 05 711-028408 YK6600 MPC QXM
FPC 0 BUILTIN BUILTIN MPC BUILTIN
MIC 0 BUILTIN BUILTIN 4x 10GE XFP
PIC 0 BUILTIN BUILTIN 4x 10GE XFP
FPC 1 BUILTIN BUILTIN MPC BUILTIN
MIC 0 REV 22 750-028392 YK7479 3D 20x 1GE(LAN) SFP
PIC 0 BUILTIN BUILTIN 10x 1GE(LAN) SFP
Xcvr 0 REV 02 740-013111 A116746 SFP-T
Xcvr 1 REV 02 740-013111 A114818 SFP-T
Xcvr 2 REV 02 740-013111 A116860 SFP-T
Xcvr 3 REV 02 740-013111 A116971 SFP-T
Xcvr 4 REV 02 740-013111 A116814 SFP-T
Xcvr 5 REV 02 740-013111 A116816 SFP-T
Xcvr 6 REV 02 740-013111 A116260 SFP-T
Xcvr 7 REV 02 740-013111 A115260 SFP-T
Xcvr 8 REV 02 740-013111 A116813 SFP-T
Xcvr 9 REV 02 740-013111 A116719 SFP-T
PIC 1 BUILTIN BUILTIN 10x 1GE(LAN) SFP
Xcvr 0 REV 02 740-013111 A115863 SFP-T
Xcvr 1 REV 02 740-013111 A116368 SFP-T
Xcvr 2 REV 02 740-013111 A071203 SFP-T

Lab 3–6 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC
Xcvr 3 REV 02 740-013111 A116370 SFP-T
Xcvr 4 REV 02 740-013111 A121193 SFP-T
Xcvr 5 REV 02 740-013111 A121188 SFP-T
Xcvr 6 REV 02 740-013111 A116729 SFP-T
Xcvr 7 REV 02 740-013111 A116891 SFP-T
Xcvr 8 REV 02 740-013111 A114787 SFP-T
Xcvr 9 REV 02 740-013111 A116696 SFP-T
Fan Tray Fan Tray

Question: How many 10-Gigabit Ethernet ports does

your assigned device have?

Answer: Although your results might vary, the

previous output shows that there are four
10-Gigabit Ethernet ports. Alternatively, you can use
the show chassis fpc pic-status
command to gather this information.

Question: How many 1-Gigabit Ethernet ports does

your assigned device have?

Answer: Although your results might vary, the

previous output shows that your device has twenty
1-Gigabit Ethernet ports. Alternatively, you can use
the show chassis fpc pic-status
command to gather this information.

Question: Does your assigned device have

redundant power supplies?

Answer: The previous output shows that your there

is just one power supply, PEM 1. The MX80 is
capable of redundant power supplies but only one
is necessary to power the chassis and all the other
components.

Step 1.10
Issue the show chassis fpc detail command.
lab@mxA-1> show chassis fpc detail
Slot 0 information:
State Online
Temperature 32 (C)
Total CPU DRAM 1024 MB
Total SRAM 403 MB
Total SDRAM 1316 MB

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–7

Junos Troubleshooting in the NOC
Start time 2013-01-31 01:13:22 UTC
Uptime 8 days, 18 hours, 49 minutes, 43 seconds
Slot 1 information:
State Online
Temperature 32 (C)
Total CPU DRAM 1024 MB
Total SRAM 403 MB
Total SDRAM 1316 MB
Start time 2013-01-31 01:13:22 UTC
Uptime 8 days, 18 hours, 49 minutes, 43 seconds

Question: How much DRAM, SRAM, and SDRAM

exist within each FPC?

Answer: 1024 MB of DRAM, 403 MB of SRAM, and

1316 MB of SDRAM exist within each FPC.

Question: What are the current temperatures of

both FPCs?

Answer: Although your results might vary, the

previous output shows that both FPCs are running
at a temperature of 32 degrees Celsius.

Step 1.11
Issue the show chassis tfeb command.
lab@mxA-1> show chassis tfeb
TFEB status:
Slot 0 information:
State Online
Intake temperature 32 degrees C / 89 degrees F
Exhaust temperature 50 degrees C / 122 degrees F
CPU utilization 4 percent
Interrupt utilization 0 percent
Heap utilization 19 percent
Buffer utilization 13 percent
Total CPU DRAM 1024 MB
Start time: 2013-01-31 01:13:22 UTC
Uptime: 8 days, 18 hours, 51 minutes, 16 seconds

Lab 3–8 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC
Question: How long has it been since the last time
the TFEB was rebooted?

Answer: Although your results might vary, the

previous output shows that the TFEB booted 8 days,
18 hours, 51 minutes, and 16 seconds ago.

Question: Does your assigned device have a

redundant data plane hardware configuration?

Answer: No. There is only one PFE, the TFEB, which

means there is no data plane redundancy.

Question: How much DRAM exists on the TFEB?

Answer: There are 1024 MB of DRAM on the TFEB.

Question: What are the current intake and exhaust

temperatures?

Answer: In the previous output, the intake

temperature is 32 degrees Celsius and the exhaust
temperature is 50 degrees Celsius. Your results
might vary, but if they are significantly different than
the previous output, please inform your instructor.

Part 2: Establishing a Baseline

In this lab part, you configure the components SNMP, NTP, and remote system
logging, which are necessary to establish a baseline.
Step 2.1
Issue the show configuration | no-more command and answer the
following questions:
lab@mxA-1> show configuration | no-more
## Last commit: 2013-02-08 18:32:42 UTC by lab
version 12.2R2.5;
system {

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–9

Junos Troubleshooting in the NOC
host-name mxA-1;
root-authentication {
encrypted-password "$1$KI99zGk6$MbYFuBbpLffu9tn2.sI7l1"; ## SECRET-DATA
ssh-dsa "ssh-dss
AAAAB3NzaC1kc3MAAACBAMQrfP2bZyBXJ6PC7XXZ+MzErI8Jl6jah5L4/
O8BsfP2hC7EvRfNoX7MqbrtCX/9gUH9gChVuBCB+ERULMdgRvM5uGhC/
gs4UX+4dBbfBgKYYwgmisM8EoT25m7qI8ybpl2YZvHNznvO8h7kr4kpYuQEpKvgsTdH/
Jle4Uqnjv7DAAAAFQDZaqA6QAgbW3O/
zveaLCIDj6p0dwAAAIB1iL+krWrXiD8NPpY+w4dWXEqaV3bnobzPC4eyxQKBUCOr80Q5YBlWXVBH
x9elwBWZwj0SF4hLKHznExnLerVsMuTMA846RbQmSz62vM6kGM13HFonWeQvWia0TDr78+rOEgWF
2KHBSIxL51lmIDW8Gql9hJfD/Dr/
NKP97w3L0wAAAIEAr3FkWU8XbYytQYEKxsIN9P1UQ1ERXB3G40YwqFO484SlyKyYCfaz+yNsaAJu
2C8UebDIR3GieyNcOAKf3inCG8jQwjLvZskuZwrvlsz/xtcxSoAh9axJcdUfSJYMW/
g+mD26JK1Cliw5rwp2nH9kUrJxeI7IReDp4egNkM4i15o= configurator@server1.he"; ##
SECRET-DATA
}
login {
user lab {
uid 2000;
class super-user;
authentication {
encrypted-password "$1$84J5Maes$cni5Hrazbd/IEHr/50oY30"; ##
SECRET-DATA
}
}
}
services {
ftp;
ssh;
telnet;
}
syslog {
user * {
any emergency;
}
file messages {
any notice;
authorization info;
}
file interactive-commands {
interactive-commands any;
}
}
}
interfaces {
ge-1/0/0 {
vlan-tagging;
unit 141 {
vlan-id 141;
family inet {
address 172.18.1.2/30;
}
}
}
ge-1/0/1 {

Lab 3–10 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC
vlan-tagging;
unit 141 {
vlan-id 141;
family inet {
address 172.18.2.2/30;
}
}
}
ge-1/0/2 {
vlan-tagging;
unit 141 {
vlan-id 141;
family inet {
address 172.18.5.1/30;
}
}
}
ge-1/0/3 {
vlan-tagging;
unit 141 {
vlan-id 141;
family inet {
address 192.168.11.1/24;
}
}
}
fxp0 {
description "MGMT INTERFACE - DO NOT DELETE";
unit 0 {
family inet {
address 10.210.15.1/27;
}
}
}
lo0 {
unit 0 {
family inet {
address 192.168.31.1/32;
}
}
}
}
routing-options {
static {
route 0.0.0.0/0 {
qualified-next-hop 172.18.1.1;
qualified-next-hop 172.18.2.1 {
preference 7;
}
}
}
}

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–11

Junos Troubleshooting in the NOC
Question: Which interfaces are shown in the
output?

Answer: Interfaces are ge-1/0/0, ge-1/0/1,

ge-1/0/2, ge-1/0/3, fxp0, and lo0 are present in
the output.

Question: Does your device automatically

synchronize with an external NTP server?

Answer: No. There is no NTP configuration in the

previous output.

Question: Does your assigned device send syslog

information to a remote syslog server?

Answer: No. The previous output shows that only

local syslogging is present.

Step 2.2
Enter configuration mode and navigate to the [edit system ntp] hierarchy
level and configure an NTP server for your device. Refer to the lab diagram for the
specific address of the NTP server.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit system ntp

[edit system ntp]

lab@mxA-1# set server X.X.X.X

[edit system ntp]

lab@mxA-1#

Lab 3–12 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC
Step 2.3
Navigate to the [edit system] hierarchy level and configure the time zone in
relation to the physical location of your router. Don’t forget that you can use the Tab
key to auto-complete the country and city names. Activate the configuration by
issuing the commit command.
Note
The exact location of your router is not
needed. Any city within the same time zone
will work.

[edit system ntp]

lab@mxA-1# up

[edit system]
lab@mxA-1# set time-zone America/Los_Angeles

[edit system]
lab@mxA-1# commit
commit complete

[edit system]
lab@mxA-1#
Step 2.4
Issue the set date ntp command to synchronize your assigned device with the
NTP server.
[edit system]
lab@mxA-1# run set date ntp
8 Feb 11:47:59 ntpdate[30494]: step time server 10.210.15.24 offset 0.000697
sec
Step 2.5
Issue the run show ntp associations command.

Note
It might take a few minutes to completely
synchronize with the NTP server.

[edit system]
lab@mxA-1# run show ntp associations
remote refid st t when poll reach delay offset jitter
==============================================================================
*10.210.15.24 10.210.8.72 4 - 62 64 7 0.166 0.022 0.014

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–13

Junos Troubleshooting in the NOC
Question: What is the NTP synchronization status?

Answer: The previous output the shows that there is

an asterisk (*) in the far left. This asterisk indicates
that the router was able to synchronize with the NTP
server.

Step 2.6
Issue the run show system uptime command.
[edit system]
lab@mxA-1# run show system uptime
Current time: 2013-02-08 11:53:41 PST
System booted: 2013-01-30 16:23:51 PST (1w1d 19:29 ago)
Protocols started: 2013-02-07 12:14:14 PST (23:39:27 ago)
Last configured: 2013-02-08 11:47:31 PST (00:06:10 ago) by lab
11:53AM up 8 days, 19:30, 1 user, load averages: 0.00, 0.00, 0.00

Question: Is the correct time shown?

Answer: Although your output might vary, the

previous output shows that the correct time. If the
correct time is not shown, please inform your
instructor.

Step 2.7
Navigate to the [edit system syslog] hierarchy level. Configure your device to
send any successful or failed login attempts to the 172.18.5.2 address. Then
commit your configuration and exit configuration mode.
[edit system]
lab@mxA-1# edit syslog

[edit system syslog]

lab@mxA-1# set host 172.18.5.3 authorization any

[edit system syslog]

lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>

Lab 3–14 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC
Step 2.8
Issue the monitor traffic interface ge-1/0/2.VLAN-ID command
where VLAN-ID is the VLAN ID that is associated with your device’s ge-1/0/2
interface.
lab@mxA-1> monitor traffic interface ge-1/0/2.VLAN-ID
verbose output suppressed, use <detail> or <extensive> for full protocol decode
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on ge-1/0/2.141, capture size 96 bytes

...
Step 2.9
Open a new Telnet session to your assigned device’s management address and
attempt to log in using an invalid name and password.

mxA-1 (ttyp0)

login: 123
Password:
Login incorrect
login:
Step 2.10
Return to the original console, Telnet, or SSH session.
From the original session, examine the output from the monitor traffic
interface ge-1/0/2.VLAN-ID command. You can press Ctrl+C key
combination to stop the output.
lab@mxA-1> monitor traffic interface ge-1/0/2.VLAN-ID
verbose output suppressed, use <detail> or <extensive> for full protocol decode
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on ge-1/0/2.141, capture size 96 bytes

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–15

Junos Troubleshooting in the NOC
Reverse lookup for 172.18.5.1 failed (check DNS reachability).
Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.

12:10:43.057934 Out IP truncated-ip - 58 bytes missing! 172.18.5.1.syslog >

172.18.5.3.syslog: SYSLOG auth.notice, length: 80
12:10:43.580435 Out IP truncated-ip - 76 bytes missing! 172.18.5.1.syslog >
172.18.5.3.syslog: SYSLOG auth.error, length: 98
12:10:43.580845 Out IP truncated-ip - 78 bytes missing! 172.18.5.1.syslog >
172.18.5.3.syslog: SYSLOG auth.notice, length: 100
^C

3 packets received by filter

0 packets dropped by kernel

Question: Did the output contain any syslog

messages about the authentication failure?

Answer: The authentication failures should

generate syslog messages that were sent to the
172.18.5.3 address. If you do not see the syslog
messages indicating an authentication failure
please inform your instructor.

Lab 3–16 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC

Part 3: Locating Online Resources

In this lab part, you use a Web browser to locate online resources for
troubleshooting purposes.
Step 3.1
Open a Web browser on your local PC.
From the open Web browser, navigate to http://www.juniper.net/
techpubs and click the link for the MX Series documentation.

Step 3.2
Click the Junos 12.3 link located in the Software Documentation column.

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–17

Junos Troubleshooting in the NOC

Step 3.3
The previous link brought you to the software documentation for M Series,
MX Series, and T Series routers. Take some time to examine the page and use it to
answer the following questions.

Question: Where do you find the information to

troubleshoot a VPLS configuration?

Answer: You can find information on VPLS in the

VPNs section.

Question: Where do you find information to

troubleshoot CoS?

Answer: You can find information on CoS under the

Features, Class of Service section.

Lab 3–18 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC

Step 3.4
Using a Web browser, navigate to http://www.juniper.net/customers/
support/ and click the Browse Knowledge Base link.

Step 3.5
Type the phrase how to troubleshoot BGP documentation in the
question box and click Ask.

Step 3.6
Examine the Results page examine the output and answer the following question.

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–19

Junos Troubleshooting in the NOC

Question: Did the search produce any results

related to troubleshooting BGP?

Answer: As seen in the previous output, the search

produced results related to troubleshooting BGP.
Your results might vary from the output and other
KB articles might be available.

Note
If time allows, search the KB database for
other topics of your interest.

Step 3.7
Using the Web browser, navigate to http://www.juniper.net/techpubs/
software/nog/ and answer the following question.

Lab 3–20 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC

Question: Which Network Operations Guides are

available?

Answer: The Network Operations Guides that are

available are Baseline, Interfaces, MPLS, MPLS
Fast Reroute, MPLS Logs, and Router Hardware.

Step 3.8
Return to the open session with your assigned device.
From the open session, log out of your assigned device.
lab@mxA-1> exit

mxA-1 (ttyu0)

login:

STOP Tell your instructor that you have completed this lab.

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–21

Junos Troubleshooting in the NOC

Lab 3–22 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Using Monitoring Tools and Establishing a Baseline • Lab 3–23

Junos Troubleshooting in the NOC

Lab 3–24 • Using Monitoring Tools and Establishing a Baseline www.juniper.net

 Lab
Monitoring Hardware and Environmental Conditions

Overview
This lab demonstrates how to monitor hardware and environmental conditions. You will
use the command line interface (CLI) to test and view the status of your router’s storage
devices. You will view your router’s boot messages, log messages, and chassisd logs to
determine which hardware events have occurred over time. You will also use various
commands to determine the status of memory and storage, log messages, as well as the
chassis temperatures.
Note that your lab login grants you super-user permissions and that you might have to
become root during this lab. Please be careful, and have fun!
By completing this lab, you will perform the following tasks:
• View and test a router’s storage devices.
• Parse through log messages to view hardware events that have occurred.
• Configure nondefault alarm settings.

www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4–1

.
Junos Troubleshooting in the NOC

Part 1: Monitoring Memory and Storage

In this lab part, you use various commands to view the status of the RE and its
memory and storage devices.
Step 1.1
Ensure that you know to which device you have been assigned. Check with your
instructor if you are not certain. Consult the management network diagram to
determine your device’s management address.

Question: What is the management address

assigned to your device?

Answer: The answer varies and depends on your

assigned device. If you are unsure of your
assignment, ask your instructor.

Step 1.2
Access the CLI for your device using either the console, Telnet, or SSH as directed by
your instructor. Refer to the management network diagram for the IP address
associated with your station. The following example uses Telnet to access an
MX Series device using the SecureCRT program:

Step 1.3
Log in to your device with the username lab and the password lab123. Note that
both the name and password are case-sensitive.

Lab 4–2 • Monitoring Hardware and Environmental Conditions www.juniper.net

 Junos Troubleshooting in the NOC
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1>
Step 1.4
Enter into configuration mode and load the device’s reset configuration by issuing
the load override jtnoc/lab4-start.config command.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# load override jtnoc/lab4-start.config
load complete

[edit]
lab@mxA-1#

Step 1.5
Change the root password to lab123 by issuing the set system
root-authentication plain-text-password command.Then, commit
the changes and exit to operational mode before moving on to the next step.
[edit]
lab@mxA-1# set system root-authentication plain-text-password
New password:
Retype new password:

[edit]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

[edit]
lab@mxA-1>
Step 1.6
Issue the show chassis routing-engine command to view the details of
your router’s RE.
lab@mxA-1> show chassis routing-engine
Routing Engine status:
Temperature 35 degrees C / 95 degrees F
CPU temperature 46 degrees C / 114 degrees F
DRAM 2048 MB
Memory utilization 28 percent
CPU utilization:
User 0 percent
Background 0 percent
Kernel 0 percent

www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4–3

Junos Troubleshooting in the NOC
Interrupt 0 percent
Idle 99 percent
Model RE-MX80
Serial ID S/N YK8973
Start time 2013-01-31 01:10:41 UTC
Uptime 7 days, 23 hours, 52 minutes, 9 seconds
Last reboot reason Router rebooted after a normal shutdown.
Load averages: 1 minute 5 minute 15 minute
0.00 0.02 0.03

Question: What is the current temperature of the

RE?

Answer: The answer might vary by student. In the

example output, the current RE temperature is
35 degrees C or 95 degrees F.

Question: What is the current temperature of the

CPU on the RE?

Answer: The answer might vary by student. In the

example output, the current RE temperature is
46 degrees C or 114 degrees F.

Question: How much DRAM exists on the RE?

Answer: The answer might vary by student. In the

example output, there is 2048 MB of DRAM
installed.

Question: What is the RE CPU’s non-Idle utilization?

Answer: The answer might vary by student. In the

example output, the RE CPU is currently 0% utilized.

Lab 4–4 • Monitoring Hardware and Environmental Conditions www.juniper.net

 Junos Troubleshooting in the NOC
Question: What is the uptime of the RE?

Answer: The answer might vary by student. In the

example output, the RE has been powered up for
7 days, 23 hours, 52 minutes, and 9 seconds.

Step 1.7
Issue the show system storage command to determine the amount of storage
space available on your router.
lab@mxA-1> show system storage
Filesystem Size Used Avail Capacity Mounted on
/dev/da0s1a 885M 125M 688M 15% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/md0 42M 42M 0B 100% /packages/mnt/jbase
/dev/md1 184M 184M 0B 100% /packages/mnt/
jkernel-ppc-12.2R2.5
/dev/md2 33M 33M 0B 100% /packages/mnt/
jpfe-MX80-12.2R2.5
/dev/md3 9.1M 9.1M 0B 100% /packages/mnt/
jdocs-12.2R2.5
/dev/md4 55M 55M 0B 100% /packages/mnt/
jroute-ppc-12.2R2.5
/dev/md5 12M 12M 0B 100% /packages/mnt/
jcrypto-ppc-12.2R2.5
/dev/md6 2.8G 10.0K 2.6G 0% /tmp
/dev/md7 2.8G 4.7M 2.6G 0% /mfs
/dev/da0s1e 98M 268K 90M 0% /config
procfs 4.0K 4.0K 0B 100% /proc
/dev/da1s1f 2.8G 1.0G 1.6G 39% /var

Question: What are the names of the two internal

NAND flash drives that exist on your router?

Answer: In the output of the command, two drives

are listed, da0 and da1.

Question: What is the current available storage

capacity on da0?

Answer: The answer will vary by student. In the

example output, the da0s1a partition has 668 MB
available, whereas the da0s1e partition has
90 MB available.
www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4–5
Junos Troubleshooting in the NOC
Question: Which flash drive is used to store user
home directories?

Answer: User home directories are located at /var/

home. A partition of da1 is mounted on /var so it is
currently being used to store user home directories.

Step 1.8
Enter the shell as the root user by issuing the start shell user root
command. Enter the password of lab123 when prompted.
lab@mxA-1> start shell user root
Password:
root@mxA-1%
Step 1.9
Perform a read-only test to determine the integrity of the da0 internal NAND flash
drive. Use the dd if=/dev/da0 of=/dev/null bs=1m command to perform
the test.

Note
It might take between 5 or 10 minutes for
the test to complete. Be patient.

root@mxA-1% dd if=/dev/da0 of=/dev/null bs=1m

3920+0 records in
3920+0 records out
4110417920 bytes transferred in 367.708375 secs (11178472 bytes/sec)

Question: Did the integrity test complete

successfully?

Answer: Yes. The test should have completed

successfully. If the test results show that errors
occurred, please notify your instructor because your
router might be experiencing a hardware failure.

Lab 4–6 • Monitoring Hardware and Environmental Conditions www.juniper.net

 Junos Troubleshooting in the NOC
Question: How many bytes were read from the da0
flash drive?

Answer: The answer might vary by student. In the

example output, 4110417920 bytes were read.

Step 1.10
Exit from the shell and return to the CLI.
root@mxA-1% exit
exit

Part 2: Viewing Boot and System Logs

In this lab part, you use the boot and system logs to view events that have occurred
on your router over time. You must complete this part of the lab by accessing your
router’s console port.
Step 2.1
Direct your router to reboot in 20 minutes using the request system reboot
in 20 command.
lab@mxA-1> request system reboot in 20
Reboot the system in 20? [yes,no] (no) yes

Shutdown at Fri Feb 8 19:38:29 2013.

[pid 17194]

*** System shutdown message from lab@mxA-1 ***

System going down at 19:38

lab@mxA-1>

Question: At what time will the router reboot?

Answer: The answer will vary by student. In the

example output, the router will reboot at 19:38.

Step 2.2
Clear the schedule reboot using the clear system reboot command.
lab@mxA-1> clear system reboot
reboot requested by lab at Fri Feb 8 19:38:29 2013

www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4–7

Junos Troubleshooting in the NOC
[process id 17194]
Terminating...

lab@mxA-1>
Step 2.3
Direct your router to reboot immediately using the request system reboot
command. From the console, you should be able to watch the entire process. Wait
for the reboot to complete before continuing.
lab@mxA-1> request system reboot
Reboot the system ? [yes,no] (no) yes

Shutdown NOW!
[pid 17210]

*** FINAL System shutdown message from lab@mxA-1 ***

System going down IMMEDIATELY

lab@mxA-1> JWaiting (max 300 seconds) for system process `vnlru' to stop...done
Waiting (max 300 seconds) for system process `vnlru_mem' to stop...done
Waiting (max 300 seconds) for system process `bufdaemon' to stop...done
Waiting (max 300 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining...1 pfe_send_failed(index 0, type 17), err=32
1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 done

syncing disks... All buffers synced.

Uptime: 71d10h59m9s
recorded reboot as normal shutdown
Rebooting...
Step 2.4
Log in to your router and view the boot messages that occurred during the reboot
process by issuing the show system boot-messages | no-more command.
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1> show system boot-messages | no-more
platform_early_bootinit: MX-PPC Series Early Boot Initialization
mxppc_set_re_type: hw.board.type is MX80
mxppc_set_re_type: REtype:78, model:mx80, model:MX80, i2cid:2447
WDOG initialized
Copyright (c) 1996-2012, Juniper Networks, Inc.
All rights reserved.
Copyright (c) 1992-2006 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
JUNOS 12.2R2.5 #0: 2012-11-15 14:13:01 UTC

Lab 4–8 • Monitoring Hardware and Environmental Conditions www.juniper.net

 Junos Troubleshooting in the NOC
builder@faranth.juniper.net:/volume/build/junos/12.2/release/12.2R2.5/
obj-powerpc/junos/bsd/kernels/JUNIPER-PPC/kernel
WARNING: debug.mpsafenet forced to 0 as ipsec requires Giant
Timecounter "decrementer" frequency 66666666 Hz quality 0
cpu0: Freescale e500v2 core revision 3.0
cpu0: HID0 80004000<EMCP,TBEN>
real memory = 2122317824 (2024 MB)
avail memory = 2081423360 (1985 MB)
Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot)
Security policy loaded: JUNOS MAC/pcap (mac_pcap)
ETHERNET SOCKET BRIDGE initialising
Initializing M/T platform properties ..
nexus0: <PPC e500 Nexus device>
ocpbus0: <on-chip peripheral bus> on nexus0
openpic0: <OpenPIC in on-chip peripheral bus> iomem 0xf7f40000-0xf7f600b3 on
ocpbus0
uart0: <16550 or compatible> iomem 0xf7f04500-0xf7f0450f irq 58 on ocpbus0
uart0: console (9600,n,8,1)
uart1: <16550 or compatible> iomem 0xf7f04600-0xf7f0460f irq 58 on ocpbus0
lbc0: <Freescale 8533 Local Bus Controller> iomem
0xf7f05000-0xf7f05fff,0xf8000000-0xffffffff irq 20,21,22,24 on ocpbus0
cfi0: <AMD/Fujitsu - 8MB> iomem 0xff800000-0xffffffff on lbc0
tbbcpld0 iomem 0xff700000-0xff7fffff on lbc0
tbbcpld_attach: 1st IRQ alloc; start:4 end:4 flags:7
tbbcpld_attach: 2st IRQ alloc; start:6 end:6 flags:7
uart2: <16750 or compatible> iomem 0xff600000-0xff600007 on lbc0
i2c0: <MPC85XX OnChip i2c Controller> iomem 0xf7f03000-0xf7f03014 irq 59 on
ocpbus0
ds1672 rtc0: <DS1672 RTC> on i2c0
i2c1: <MPC85XX OnChip i2c Controller> iomem 0xf7f03100-0xf7f03114 irq 59 on
ocpbus0
tsec0: <eTSEC ethernet controller> iomem 0xf7f24000-0xf7f24fff irq 45,46,50 on
ocpbus0
tsec0: hardware MAC address 02:00:00:00:00:0b
miibus0: <MII bus> on tsec0
gentbi0: <Generic ten-bit interface> on miibus0
gentbi0: 1000baseSX-FDX, 1000baseT-FDX, auto
tsec1: <eTSEC ethernet controller> iomem 0xf7f25000-0xf7f25fff irq 51,52,56 on
ocpbus0
tsec1: hardware MAC address 02:00:00:00:00:04
miibus1: <MII bus> on tsec1
gentbi1: <Generic ten-bit interface> on miibus1
gentbi1: 1000baseSX-FDX, 1000baseT-FDX, auto
tsec2: <eTSEC ethernet controller> iomem 0xf7f26000-0xf7f26fff irq 47,48,49 on
ocpbus0
tsec2: hardware MAC address 80:71:1f:c3:03:ff
miibus2: <MII bus> on tsec2
e1000phy0: <Marvell 88E1112 Gigabit PHY> on miibus2
e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX,
auto
tsec3: <eTSEC ethernet controller> iomem 0xf7f27000-0xf7f27fff irq 53,54,55 on
ocpbus0
tsec3: hardware MAC address 02:00:02:00:00:04
miibus3: <MII bus> on tsec3
gentbi2: <Generic ten-bit interface> on miibus3

www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4–9

Junos Troubleshooting in the NOC
gentbi2: 1000baseSX-FDX, 1000baseT-FDX, auto
pcib0: <Freescale MPC8572 PCI Express host controller> iomem
0xf7f09000-0xf7f09fff,0xe8000000-0xebffffff on ocpbus0
pci0: <PCI bus> on pcib0
pcib1: <PCI-PCI bridge> mem 0xe8000000-0xe80fffff at device 0.0 on pci0
pci1: <PCI bus> on pcib1
pcib2: <PCI-PCI bridge> mem 0xe8100000-0xe810ffff irq 0 at device 0.0 on pci1
pci2: <PCI bus> on pcib2
pci2: <serial bus, USB> at device 1.0 (no driver attached)
ehci0: <NEC uPD 72010x USB 2.0 controller> mem 0xe8200000-0xe82000ff irq 21 at
device 1.1 on pci2
usb0: EHCI version 1.0
usb0: <NEC uPD 72010x USB 2.0 controller> on ehci0
usb0: USB revision 2.0
uhub0: NEC EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub0: 3 ports with 3 removable, self powered
umass0: ATP Electronics ATP IG eUSB SSD, rev 2.00/11.00, addr 2
umass0: SCSI over Bulk-Only; quirks = 0x0000
umass0:0:0:-1: Attached to scbus0
umass1: ATP Electronics ATP IG eUSB SSD, rev 2.00/11.00, addr 3
umass1: SCSI over Bulk-Only; quirks = 0x0000
umass1:1:1:-1: Attached to scbus1
Initializing product: 88 ..
Setting up M/T interface operations and attributes
platform_cookie_read not implemented
###PCB Group initialized for udppcbgroup
###PCB Group initialized for tcppcbgroup
da1 at umass-sim1 bus 1 target 0 lun 0
da1: <ATP ATP IG eUSB SSD 1100> Fixed Direct Access SCSI-0 device
da1: 40.000MB/s transfers
da1: 3920MB (8028160 512 byte sectors: 255H 63S/T 499C)
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <ATP ATP IG eUSB SSD 1100> Fixed Direct Access SCSI-0 device
da0: 40.000MB/s transfers
da0: 3920MB (8028160 512 byte sectors: 255H 63S/T 499C)
Trying to mount root from ufs:/dev/da0s1a

Question: The boot messages record the

step-by-step process that the RE goes through to
boot. Did any errors occur during the boot process?

Answer: No errors should occur.

Step 2.5
Use the show system uptime command to determine the router’s current time
and date. The date, as known by your router, will be used to parse log messages in
future steps.
lab@mxA-1> show system uptime
Current time: 2013-02-08 00:37:58 UTC
System booted: 2013-01-31 00:23:51 UTC (1w1d 00:14 ago)

Lab 4–10 • Monitoring Hardware and Environmental Conditions www.juniper.net

 Junos Troubleshooting in the NOC
Protocols started: 2013-02-07 20:14:14 UTC (04:23:44 ago)
Last configured: 2013-02-08 00:24:43 UTC (00:13:15 ago) by lab
12:37AM up 8 days, 14 mins, 1 user, load averages: 0.00, 0.00, 0.00

Question: What is the current time and date?

Answer: The answer will vary by student. In the

example output, the time and date are currently
00:37:58 UTC on Feb 8, 2013.

Step 2.6
View the messages log file using the show log messages to view detailed
information about the PFE during the reboot that just occurred. It might be helpful to
use the match modifier to ensure that only entries from today’s date are shown. For
example, if today’s date is Feb 8th, issue the command show log messages |
match "Feb 8" (you might need to use two spaces between month and day).
Make an attempt to find a log that shows when the reboot was initiated. If you
cannot immediately find the entry, hit Ctrl+c and go to the next step.
lab@mxA-1> show log messages | match "Feb 8"
Feb 8 00:01:06 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+54373
Feb 8 00:03:34 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+55229
Feb 8 00:06:02 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+55506
Feb 8 00:08:30 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+61679
Feb 8 00:10:58 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+62191
Feb 8 00:13:26 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+53976
Feb 8 00:15:54 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+53347
Feb 8 00:18:22 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+49385
Feb 8 00:20:50 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+60403
Feb 8 00:23:18 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+60057
Feb 8 00:25:46 mx80 rpd[1130]: bgp_listen_accept: Connection attempt from
unconfigured neighbor: 172.22.201.2+56149
Step 2.7
You might notice that matching on the date might not narrow the search down
enough because thousands of entries might happen on any one day. Use the
previous command but add a second pipe that matches on reboot. For example, if
today’s date is Dec 7th, issue the command show log messages | match
"Feb 8" | match reboot.

www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4–11

Junos Troubleshooting in the NOC
lab@mxA-1> show log messages | match "Feb 8" | match reboot
Feb 8 19:18:29 mxA-1 mgd[16416]: UI_REBOOT_EVENT: System rebooted by 'lab'
Feb 8 19:18:29 mxA-1 shutdown: reboot requested by lab at Wed Feb 8 19:38:29
2011
Feb 8 19:19:43 mxA-1 mgd[16416]: UI_REBOOT_EVENT: System rebooted by 'lab'
Feb 8 19:19:48 mxA-1 shutdown: reboot by lab:
Feb 8 19:22:40 mxA-1 savecore: Router rebooting after a normal shutdown...
Feb 8 19:22:40 mxA-1 savecore: Router rebooting after a normal shutdown...

Question: Two reboots were requested. At what time

did the first reboot request occur?

Answer: The answer will vary by student. In the

example, the first request was issued at 19:18:29.

Question: Two reboots were requested. At what time

did the second reboot request occur?

Answer: The answer will vary by student. In the

example, the second request was issued at
19:19:43.

Step 2.8
Using the show log messages command, determine the exact time that FPC 1
came back online after the reboot. (Hint: Status of FPCs are tracked by
chassisd.)
lab@mxA-1> show log messages | match "Feb 8" | match chassisd | match fpc
Feb 8 19:19:56 mxA-1 chassisd[1137]: CHASSISD_IFDEV_DETACH_FPC:
ifdev_detach_fpc(0)
Feb 8 19:19:56 mxA-1 chassisd[1137]: CHASSISD_IFDEV_DETACH_FPC:
ifdev_detach_fpc(1)
Feb 8 19:19:57 mxA-1 chassisd[1137]: CHASSISD_IFDEV_DETACH_FPC:
ifdev_detach_fpc(0)
Feb 8 19:19:57 mxA-1 chassisd[1137]: CHASSISD_IFDEV_DETACH_FPC:
ifdev_detach_fpc(1)
Feb 8 19:22:59 mxA-1 chassisd[1121]: CHASSISD_IFDEV_DETACH_FPC:
ifdev_detach_fpc(0)
Feb 8 19:22:59 mxA-1 chassisd[1121]: CHASSISD_IFDEV_DETACH_FPC:
ifdev_detach_fpc(1)
Feb 8 19:23:52 mxA-1 chassisd[1121]: CHASSISD_FRU_EVENT: set_fpc_online:
restarted FPC 0
Feb 8 19:23:52 mxA-1 chassisd[1121]: CHASSISD_FRU_EVENT: set_fpc_online:
restarted FPC 1

Lab 4–12 • Monitoring Hardware and Environmental Conditions www.juniper.net

 Junos Troubleshooting in the NOC
Question: At what time did FPC 1 come back online
after the reboot?

Answer: The answer will vary by student. In the

example, FPC 1 came back online at 19:23:52.

Part 3: Monitoring the Chassis and Environment

In this lab part, you will use the CLI to monitor the status of the chassis and
environment of your router.
Step 3.1
Issue the show chassis temperature-thresholds command to determine
the default threshold settings.
lab@mxA-1> show chassis temperature-thresholds
Fan speed Yellow alarm Red alarm Fire Shutdown
(degrees C) (degrees C) (degrees C) (degrees C)
Item Normal High Normal Bad fan Normal Bad fan Normal
Chassis default 48 54 65 55 75 75 100
Routing Engine 65 70 85 75 95 80 100

Question: At what temperature will the RE cause the

router to increase the speed of the fans in the fan
tray?

Answer: When the RE reaches a temperature of

70 degrees Celsius, the router should increase the
speed of the fans.

Step 3.2
Issue the show chassis environment command to determine the current
temperature of the routing engine in degrees Celsius.
lab@mxA-1> show chassis environment
Class Item Status Measurement
Temp PEM 0 OK
PEM 1 Absent
RE 0 Intake OK 28 degrees C / 82 degrees F
RE 0 Front Exhaust OK 32 degrees C / 89 degrees F
RE 0 Rear Exhaust OK 30 degrees C / 86 degrees F
Routing Engine OK 35 degrees C / 95 degrees F
Routing Engine CPU OK 46 degrees C / 114 degrees F
TFEB 0 QX 0 TSen OK 32 degrees C / 89 degrees F
TFEB 0 QX 0 Chip OK 41 degrees C / 105 degrees F
TFEB 0 LU 0 TSen OK 32 degrees C / 89 degrees F

www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4–13

Junos Troubleshooting in the NOC
TFEB 0 LU 0 Chip OK 46 degrees C / 114 degrees F
TFEB 0 MQ 0 TSen OK 32 degrees C / 89 degrees F
TFEB 0 MQ 0 Chip OK 39 degrees C / 102 degrees F
TFEB 0 TBB PFE TSen OK 29 degrees C / 84 degrees F
TFEB 0 TBB PFE Chip OK 37 degrees C / 98 degrees F
TFEB 0 TFEB PCIE TSen OK 32 degrees C / 89 degrees F
TFEB 0 TFEB PCIE Chip OK 49 degrees C / 120 degrees F
Fans Fan 1 OK Spinning at normal speed
Fan 2 OK Spinning at normal speed
Fan 3 OK Spinning at normal speed
Fan 4 OK Spinning at normal speed
Fan 5 OK Spinning at normal speed

Question: What is the current temperature of the

RE?

Answer: The answer will might by student. In the

example, the RE is currently 35 degrees Celsius.

Question: What is the current speed of the fans?

Answer: The answer might vary by student. In the

example, the fans are spinning at normal speed.

Question: When compared to the temperature

thresholds, are the fan speed appropriate?

Answer: The fans should be spinning at a speed

(normal or high) as per the show chassis
temperature-thresholds command.

Step 3.3
Determine any alarms exist by issuing the show chassis alarms command.
lab@mxA-1> show chassis alarms
No alarms currently active

Lab 4–14 • Monitoring Hardware and Environmental Conditions www.juniper.net

 Junos Troubleshooting in the NOC
Question: Do any active alarms exist?

Answer: The answer might vary by student. In the

example, no active alarms exist.

Step 3.4
Enter configuration mode and change the default chassis alarm settings such that if
any Ethernet ports are in the link-down state, the router will generate a red alarm.
Commit your configuration an exit to operational mode.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# set chassis alarm ethernet link-down red

[edit]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode
Step 3.5
Once again, determine if any alarms exist by issuing the show chassis alarms
command.
lab@mxA-1> show chassis alarms
6 alarms currently active
Alarm time Class Description
2013-02-08 00:49:54 UTC Major ge-1/1/2: Link down
2013-02-08 00:49:54 UTC Major ge-1/1/1: Link down
2013-02-08 00:49:54 UTC Major xe-0/0/3: Link down
2013-02-08 00:49:54 UTC Major xe-0/0/2: Link down
2013-02-08 00:49:54 UTC Major xe-0/0/1: Link down
2013-02-08 00:49:54 UTC Major xe-0/0/0: Link down

Question: How many active alarms are reported by

your router?

Answer: The answer might vary by student. In the

example, 6 active alarms exist.

Step 3.6
Log out of your assigned device.

www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4–15

Junos Troubleshooting in the NOC
lab@mxA-1> exit

mxA-1 (ttyu0)

login:

STOP Tell your instructor that you have completed this lab.

Lab 4–16 • Monitoring Hardware and Environmental Conditions www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Monitoring Hardware and Environmental Conditions • Lab 4–17

Junos Troubleshooting in the NOC

Lab 4–18 • Monitoring Hardware and Environmental Conditions www.juniper.net

 Lab
Control Plane Monitoring and Troubleshooting

Overview
This lab demonstrates how to work with user processes, system daemons, core dump
files, and the routing table. You will also configure and work with OSPF and BGP. You will
then configure basic Layer 2 bridging.
By completing this lab, you will perform the following tasks:
• View what users are logged into the router and forcibly remove unwanted or
hung users.
• View and restart system daemons.
• Generate and retrieve core dump files.
• Configure and monitor OSPF and BGP.
• Configure and monitor Layer 2 bridging.

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–1

.
Junos Troubleshooting in the NOC

Part 1: Examining User Processes and Daemons

In this lab part, you examine users logged in to your assigned device. You will also
examine and restart daemons that are currently running on the router.
Step 1.1
Ensure that you know to which device you have been assigned. Check with your
instructor if you are not certain. Consult the management network diagram to
determine your device’s management address.

Question: What is the management address

assigned to your device?

Answer: The answer varies and depends on your

assigned device. If you are unsure of your
assignment, ask your instructor.

Step 1.2
Access the CLI for your device using either the console, Telnet, or SSH as directed by
your instructor. Refer to the management network diagram for the IP address
associated with your station. The following example uses Telnet to access an
MX Series device using the SecureCRT program:

Step 1.3
Log in to your device with the username lab and the password lab123. Note that
both the name and password are case-sensitive.

Lab 5–2 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1>
Step 1.4
Enter into configuration mode and load the device’s reset configuration by issuing
the load override jtnoc/lab5-start.config command. After the
configuration has been loaded, commit the changes and exit to operational mode
before continuing to the next step.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1#

[edit]
lab@mxA-1# load override jtnoc/lab5-start.config
load complete

[edit]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

[edit]
lab@mxA-1>
Step 1.5
Issue the show system users command and answer the following questions:
lab@mxA-1> show system users
8:08PM up 47 mins, 1 user, load averages: 0.00, 0.03, 0.01
USER TTY FROM LOGIN@ IDLE WHAT
lab p0 10.210.14.30 8:09PM - -cli (cli)

Question: What users are logged in to your assigned

device?

Answer: As shown in the previous output, only the

user, lab, is logged in to your assigned device.

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–3

Junos Troubleshooting in the NOC
Question: From which IP address is the user lab
logged in to your assigned device?

Answer: Although your output might vary, the

previous output shows that user lab is logged in
from the IP address of 10.210.14.30.

Step 1.6
Open a new Telnet session to your assigned device.

Step 1.7
On the new Telnet session, log in as user lab with the password supplied by your
instructor.
mxA-1 (ttyp0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1>
Step 1.8
Return to your original session. On your original session, issue the show system
users command.
lab@mxA-1> show system users
8:12PM up 51 mins, 2 users, load averages: 0.02, 0.02, 0.00
USER TTY FROM LOGIN@ IDLE WHAT
lab p0 10.210.14.30 8:09PM - -cli (cli)
lab p2 10.210.14.30 8:12PM - -cli (cli)

Lab 5–4 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
Question: Which users are present on your
assigned device?

Answer: Two instances of the lab user should be

present.

Step 1.9
Issue the request system logout user lab terminal ? command.
lab@mxA-1> request system logout user lab terminal ?
Possible completions:
<terminal> Terminal user is logged in to

Question: What value should you use to complete

the command to forcibly remove the second
instance of user lab from the router?

Answer: Although your answer might vary, the

previous output shows that the second user has a
TTY value of p2. You can use the value of p2 to
forcibly logout the second instance of the lab user.

Step 1.10
Forcibly remove the second instance of user lab.
lab@mxA-1> request system logout user lab terminal p2
logout-user: done

lab@mxA-1>
Step 1.11
Issue the show system users command and answer the following question:
lab@mxA-1> show system users
8:14PM up 53 mins, 1 user, load averages: 0.38, 0.20, 0.08
USER TTY FROM LOGIN@ IDLE WHAT
lab p0 10.210.14.30 8:09PM - -cli (cli)

Question: How many users are logged in to your

assigned device?

Answer: Only one instance of the lab user is

present on your assigned device.

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–5

Junos Troubleshooting in the NOC
Step 1.12
Issue the show system processes extensive command.
lab@mxA-1> show system processes extensive
last pid: 23568; load averages: 0.00, 0.00, 0.00 up 7+19:47:33 20:10:54
133 processes: 3 running, 103 sleeping, 27 waiting

Mem: 362M Active, 58M Inact, 80M Wired, 144M Cache, 112M Buf, 1345M Free
Swap: 2915M Total, 2915M Free

PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAND
11 root 1 171 52 0K 16K RUN 182.9H 98.05% idle
1296 root 11 97 0 16732K 8404K ucond 86:48 0.00% clksyncd
13 root 1 -20 -139 0K 16K RUN 75:40 0.00% swi7: clock
1281 root 1 96 0 110M 14004K select 18:15 0.00% chassisd
1417 root 1 96 0 57384K 32128K select 15:29 0.00% mib2d
1300 root 1 96 0 11960K 7364K select 11:23 0.00% license-check
1386 root 1 96 0 19168K 11616K select 6:56 0.00% l2ald
1415 root 1 96 0 32072K 12520K select 6:49 0.00% pfed
1403 root 3 20 0 10144K 3028K sigwai 6:33 0.00% jddosd
1299 root 1 96 0 13768K 3324K select 6:06 0.00% shm-rtsdbd
1278 root 1 96 0 2528K 1240K select 4:57 0.00% bslockd
12 root 1 -40 -159 0K 16K WAIT 4:07 0.00% swi2: netisr 0
1418 root 1 96 0 41004K 35564K select 3:53 0.00% snmpd
1413 root 1 96 0 21628K 11812K select 3:53 0.00% smid
1390 root 1 96 0 34652K 12960K select 3:12 0.00% cosd
28 root 1 -68 -187 0K 16K WAIT 3:07 0.00% irq36: tsec1
1291 root 1 96 0 8972K 4996K select 1:52 0.00% ppmd
1282 root 1 96 0 11992K 6980K select 1:45 0.00% alarmd
9 root 1 171 52 0K 16K pgzero 1:27 0.00% pagezero
23 root 1 -52 -171 0K 16K WAIT 1:09 0.00% irq43: i2c0 i2c1
27 root 1 -68 -187 0K 16K WAIT 1:06 0.00% irq35: tsec1
1385 root 1 4 0 61724K 21776K kqread 1:04 0.00% rpd
15 root 1 -16 0 0K 16K - 1:04 0.00% yarrow
1414 root 1 96 0 6172K 2736K select 1:04 0.00% irsd
...

Question: What daemon manages the chassis

components of the router?

Answer: The chassisd daemon manages the

chassis components of the router.

Question: What daemon manages the routing

function?

Answer: The rpd daemon manages the routing

function.

Lab 5–6 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
Step 1.13
Issue the show route command.
lab@mxA-1> show route

inet.0: 15 destinations, 16 routes (15 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

0.0.0.0/0 *[Static/5] 01:00:37

> to 172.18.1.1 via ge-1/0/0.141
[Static/7] 19:53:30
> to 172.18.2.1 via ge-1/0/1.141
10.210.15.0/27 *[Direct/0] 1w0d 19:47:04
> via fxp0.0
10.210.15.1/32 *[Local/0] 1w0d 19:47:04
Local via fxp0.0
172.18.1.0/30 *[Direct/0] 19:53:30
> via ge-1/0/0.141
172.18.1.2/32 *[Local/0] 19:53:30
Local via ge-1/0/0.141
172.18.2.0/30 *[Direct/0] 19:53:30
> via ge-1/0/1.141
172.18.2.2/32 *[Local/0] 19:53:30
Local via ge-1/0/1.141
172.18.5.0/30 *[Direct/0] 01:00:37
> via ge-1/0/2.141
172.18.5.1/32 *[Local/0] 01:00:37
Local via ge-1/0/2.141
172.18.112.1/32 *[Local/0] 01:00:37
Reject
172.18.113.0/30 *[Direct/0] 01:00:34
> via ge-1/1/3.0
172.18.113.1/32 *[Local/0] 01:00:37
Local via ge-1/1/3.0
192.168.11.0/24 *[Direct/0] 16:53:18
> via ge-1/0/3.141
192.168.11.1/32 *[Local/0] 1d 01:12:31
Local via ge-1/0/3.141
192.168.31.1/32 *[Direct/0] 1d 01:12:30
> via lo0.0

Question: What is the amount of time listed for the

oldest route in the routing table?

Answer: Although your output might vary, the

previous output shows that the oldest route has
been in the routing table for 1 week.

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–7

Junos Troubleshooting in the NOC
Question: If for some reason the rpd process were
to restart, what would be the effect on the routing
table?

Answer: The Junos OS removes all routing

information from the routing table. Then, the
Junos OS adds any current routing information
through independent and dynamic routing
protocols.

Step 1.14
Issue the restart routing command.
lab@mxA-1> restart routing
Routing protocols process started, pid 37465

lab@mxA-1>
Step 1.15
Issue the show route command.
lab@mxA-1> show route

inet.0: 15 destinations, 16 routes (15 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

0.0.0.0/0 *[Static/5] 00:00:10

> to 172.18.1.1 via ge-1/0/0.141
[Static/7] 00:00:10
> to 172.18.2.1 via ge-1/0/1.141
10.210.15.0/27 *[Direct/0] 00:00:10
> via fxp0.0
10.210.15.1/32 *[Local/0] 00:00:10
Local via fxp0.0
172.18.1.0/30 *[Direct/0] 00:00:10
> via ge-1/0/0.141
172.18.1.2/32 *[Local/0] 00:00:10
Local via ge-1/0/0.141
172.18.2.0/30 *[Direct/0] 00:00:10
> via ge-1/0/1.141
172.18.2.2/32 *[Local/0] 00:00:10
Local via ge-1/0/1.141
172.18.5.0/30 *[Direct/0] 00:00:10
> via ge-1/0/2.141
172.18.5.1/32 *[Local/0] 00:00:10
Local via ge-1/0/2.141
172.18.112.1/32 *[Local/0] 00:00:10
Reject
172.18.113.0/30 *[Direct/0] 00:00:10
> via ge-1/1/3.0

Lab 5–8 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
172.18.113.1/32 *[Local/0] 00:00:10
Local via ge-1/1/3.0
192.168.11.0/24 *[Direct/0] 00:00:10
> via ge-1/0/3.141
192.168.11.1/32 *[Local/0] 00:00:10
Local via ge-1/0/3.141
192.168.31.1/32 *[Direct/0] 00:00:10
> via lo0.0

Question: What changes are present in the routing

table?

Answer: All the routes in the routing table were

removed and replaced with new routes, which you
can see by viewing the current age of the routes in
the routing table.

Question: What effect would restarting rpd have on

OSPF?

Answer: Restarting rpd causes OSPF to completely

reconverge. All neighbor adjacencies must
re-establish, the router must repopulate the
linkstate database, and calculations must run on all
possible routes to determine the best path.

Question: How do you restart only OSPF without

disrupting other routing protocols running on the
router?

Answer: You can restart OSPF by deactivating the

[edit protocols ospf] hierarchy level,
committing the configuration, activating the [edit
protocols ospf] hierarchy level, and
committing the configuration again.

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–9

Junos Troubleshooting in the NOC

Part 2: Generating Core Files

In this lab part, you generate a core dump file, pull it from the router, and prepare it
to be inspected.
Step 2.1
Issue the show system core-dumps command and answer the following
question:
lab@mxA-1> show system core-dumps
/var/crash/*core*: No such file or directory
/var/tmp/*core*: No such file or directory
/var/crash/kernel.*: No such file or directory
/tftpboot/corefiles/*core*: No such file or directory

Question: Are there any core dump files present on

your assigned device?

Answer: No core dump files are present on your

assigned device.

Step 2.2

Note
Generating a core dump file can possibly
cause adverse effects on your device and it
is not recommended as a troubleshooting
step unless directed by JTAC.

Issue the request system core-dump ? command.

Note
The core-dump part of the previous
command is a hidden command and will
not autocomplete.

lab@mxA-1> request system core-dump ?

Possible completions:
adaptive-services Adaptive services process
audit-process Audit process
cfm Ethernet OAM connectivity-fault-management process
class-of-service Class-of-service daemon
database-replication Database replication process
ddos-service DDOS Protection Service daemon
dhcp-service Dynamic Host Configuration Protocol process
dynamic-flow-capture Dynamic flow capture service
event-processing Event processing process

Lab 5–10 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
firewall Firewall daemon
helper Port-forwarding daemon
hostname-caching-process Hostname caching process
interface-control Interface daemon
jflow Traffic sampling daemon
l2-control Layer 2 control protocols
l2-learning Layer 2 address flooding/learning process
l2tp-service Layer 2 Tunneling Protocol daemon
lacp Link Aggregation Control Protocol
lfm Ethernet OAM link-fault-management process
link-management Link management
management Management daemon
mib-process SNMP MIB-II daemon
mpls-traceroute MPLS Periodic Traceroute process
pic-services-logging PIC services logging daemon
ppp Point-to-point protocol daemon
ppp-service Universal edge Point-to-point protocol daemon
pppoe PPP over Ethernet daemon
redundancy-services Redundancy device process
remote-operations Remote operations daemon
rmopd RMOPD daemon
routing Routing protocol daemon
sampling Traffic sampling daemon
send Secure Neighbor Discovery Protocol process
service-deployment Service deployment system daemon
snmp SNMP daemon
snooping Multicast snooping protocol daemon
subscriber-management Subscriber management process

Question: What option can you use to generate an

rpd core dump file?

Answer: You can use the routing option to

generate an rpd core dump file.

Step 2.3
Generate a core dump file using the rpd process.
lab@mxA-1> request system core-dump routing
Generating core dump for routing process using running method
Step 2.4
Issue the show system core-dumps command to list the core dump files.
lab@mxA-1> show system core-dumps
/var/crash/*core*: No such file or directory
-rw-rw---- 1 root field 37277696 Feb 7 20:20 /var/tmp/rpd.core.0
/var/crash/kernel.*: No such file or directory
/tftpboot/corefiles/*core*: No such file or directory
total 1

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–11

Junos Troubleshooting in the NOC
Question: What is the location of the core dump
file?

Answer: As shown in the previous output, the core

dump file is in the /var/tmp/ directory.

Step 2.5
Next, retrieve the core dump file using FTP. Using the local Windows FTP client, open
an FTP session to your assigned device’s management address.

Step 2.6

Note
Make sure that once the FTP session is
open, you set the transfer mode to binary
and enable hash marks.

Once the FTP session is open, log in using the lab user name and perform an FTP
get to transfer the core dump file to your local machine.
Connected to 10.210.15.1.
220 mxA-1 FTP server (Version 6.00LS) ready.
User (10.210.15.1:(none)): lab
331 Password required for lab.
Password:
230 User lab logged in.
ftp> bin
200 Type set to I.
ftp> hash
Hash mark printing On ftp: (2048 bytes/hash mark) .
ftp> cd /var/tmp/
250 CWD command successful.
ftp> get rpd.core.0
200 PORT command successful.
150 Opening BINARY mode data connection for 'rpd.core.0' (24875008 bytes).
##############################################################################
##############################################################################
...
##############################################################################

Lab 5–12 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
##################################################################
226 Transfer complete.
ftp: 24875008 bytes received in 225.53Seconds 110.30Kbytes/sec.
ftp>

Part 3: Configuring Routing Protocols and Routing Tables

In this lab part, you configure OSPF and BGP. You will also examine the contents of
the routing table.
Step 3.1
Issue the show route command.
lab@mxA-1> show route

inet.0: 15 destinations, 16 routes (15 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

0.0.0.0/0 *[Static/5] 00:09:18

> to 172.18.1.1 via ge-1/0/0.141
[Static/7] 00:09:18
> to 172.18.2.1 via ge-1/0/1.141
10.210.15.0/27 *[Direct/0] 00:09:18
> via fxp0.0
10.210.15.1/32 *[Local/0] 00:09:18
Local via fxp0.0
172.18.1.0/30 *[Direct/0] 00:09:18
> via ge-1/0/0.141
172.18.1.2/32 *[Local/0] 00:09:18
Local via ge-1/0/0.141
172.18.2.0/30 *[Direct/0] 00:09:18
> via ge-1/0/1.141
172.18.2.2/32 *[Local/0] 00:09:18
Local via ge-1/0/1.141
172.18.5.0/30 *[Direct/0] 00:09:18
> via ge-1/0/2.141
172.18.5.1/32 *[Local/0] 00:09:18
Local via ge-1/0/2.141
172.18.112.1/32 *[Local/0] 00:09:18
Reject
172.18.113.0/30 *[Direct/0] 00:09:18
> via ge-1/1/3.0
172.18.113.1/32 *[Local/0] 00:09:18
Local via ge-1/1/3.0
192.168.11.0/24 *[Direct/0] 00:09:18
> via ge-1/0/3.141
192.168.11.1/32 *[Local/0] 00:09:18
Local via ge-1/0/3.141
192.168.31.1/32 *[Direct/0] 00:09:18
> via lo0.0

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–13

Junos Troubleshooting in the NOC
Question: Which protocols populate the routing
table?

Answer: Routes from the protocols static, direct,

and local are populating the routing table.

Question: Which route points to unknown

destinations?

Answer: The default static route points to unknown

destinations.

Question: Which routing table is present in the

output?

Answer: The inet.0 routing table is present in the

output.

Step 3.2
To test connectivity, ping the virtual router attached to the remote team’s device
within your pod.
lab@mxA-1> ping remote-teams-VR
PING 192.168.12.2 (192.168.12.2): 56 data bytes
^C
--- 192.168.12.2 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

Question: What is the result of the ping test?

Answer: The ping test was not successful.

Lab 5–14 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
Step 3.3
Because the ping test failure in the previous step, issue a traceroute to the remote
team’s virtual router.
lab@mxA-1> traceroute remote-teams-VR
traceroute to 192.168.12.2 (192.168.12.2), 30 hops max, 40 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
...
28 * * *
29 * * *
30 * * *

Question: What does the output from the traceroute

reveal?

Answer: The output from the trace route shows that

the first router, which is controlled by ISP A, does
not have the proper routing knowledge.

Step 3.4
Enter configuration mode and navigate to the [edit protocols ospf]
hierarchy level.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit protocols ospf

[edit protocols ospf]

lab@mxA-1#
Step 3.5
Add interfaces ge-1/0/2.VLAN-ID, and lo0 to Area 0 under the OSPF protocol.

Note
The unit associated with interfaces
ge-1/0/2 and ge-1/0/3 is not unit 0. Take
extra care when configuring these
interfaces under OSPF.

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–15

Junos Troubleshooting in the NOC
[edit protocols ospf]
lab@mxA-1# set area 0 interface ge-1/0/2.VLAN-ID

[edit protocols ospf]

lab@mxA-1# set area 0 interface lo0
Step 3.6
Add the ge-1/0/3.141 as a passive interface in Area 0 under the OSPF protocol.
Activate the configuration by issuing the commit command.
[edit protocols ospf]
lab@mxA-1# set area 0 interface ge-1/0/3.141 passive

[edit protocols ospf]

lab@mxA-1# commit
commit complete

STOP Before proceeding, ensure that the remote student team within your
pod is ready to continue on to the next step.
Step 3.7
Issue the run show ospf interface and run show ospf neighbor
commands.

Note
It might take a minute for the OSPF
adjacencies to reach the Full state.

[edit protocols ospf]

lab@mxA-1# run show ospf interface
Interface State Area DR ID BDR ID Nbrs
ge-1/0/2.141 BDR 0.0.0.0 192.168.32.1 192.168.31.1 1
ge-1/0/3.141 DRother 0.0.0.0 0.0.0.0 0.0.0.0 0
lo0.0 DR 0.0.0.0 192.168.31.1 0.0.0.0 0

[edit protocols ospf]

lab@mxA-1# run show ospf neighbor
Address Interface State ID Pri Dead
172.18.5.2 ge-1/0/2.141 Full 192.168.32.1 128 31

Question: How many OSPF adjacencies are in the

Full state?

Answer: One OSPF adjacency is in the Full state.

Lab 5–16 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
Question: Why have no OSPF adjacencies formed
over the ge-1/0/3.141 interface?

Answer: Remember that the ge-1/0/3.141 interface

has the passive option. With the passive option
no OSPF adjacencies can form over the interface.
However, the subnet associated with that interface
is present in the OSPF link-state database.

Step 3.8
Issue the run show route command and answer the following questions:
[edit protocols ospf]
lab@mxA-1# run show route

inet.0: 18 destinations, 19 routes (18 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

0.0.0.0/0 *[Static/5] 01:30:38

> to 172.18.1.1 via ge-1/0/0.141
[Static/7] 01:30:38
> to 172.18.2.1 via ge-1/0/1.141
10.210.15.0/27 *[Direct/0] 01:30:38
> via fxp0.0
10.210.15.1/32 *[Local/0] 01:30:38
Local via fxp0.0
172.18.1.0/30 *[Direct/0] 01:30:38
> via ge-1/0/0.141
172.18.1.2/32 *[Local/0] 01:30:38
Local via ge-1/0/0.141
172.18.2.0/30 *[Direct/0] 01:30:38
> via ge-1/0/1.141
172.18.2.2/32 *[Local/0] 01:30:38
Local via ge-1/0/1.141
172.18.5.0/30 *[Direct/0] 01:30:38
> via ge-1/0/2.141
172.18.5.1/32 *[Local/0] 01:30:38
Local via ge-1/0/2.141
172.18.112.1/32 *[Local/0] 01:30:38
Reject
172.18.113.0/30 *[Direct/0] 01:30:38
> via ge-1/1/3.0
172.18.113.1/32 *[Local/0] 01:30:38
Local via ge-1/1/3.0
192.168.11.0/24 *[Direct/0] 01:30:38
> via ge-1/0/3.141
192.168.11.1/32 *[Local/0] 01:30:38
Local via ge-1/0/3.141
192.168.12.0/24 *[OSPF/10] 00:00:52, metric 2
> to 172.18.5.2 via ge-1/0/2.141

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–17

Junos Troubleshooting in the NOC
192.168.31.1/32 *[Direct/0] 01:30:38
> via lo0.0
192.168.32.1/32 *[OSPF/10] 00:00:52, metric 1
> to 172.18.5.2 via ge-1/0/2.141
224.0.0.5/32 *[OSPF/10] 00:43:34, metric 1
MultiRecv

Question: Which protocols are populating the

routing table?

Answer: Routes from the protocols OSPF, static,

direct, and local are populating the routing table.

Question: Is there a route to the remote team’s

virtual router?

Answer: Yes. There is an OSPF route for the remote

team’s virtual router.

Step 3.9
Next, ping the remote team’s virtual router.
[edit protocols ospf]
lab@mxA-1# run ping remote-teams-VR
PING 192.168.12.2 (192.168.12.2): 56 data bytes
64 bytes from 192.168.12.2: icmp_seq=0 ttl=64 time=0.568 ms
64 bytes from 192.168.12.2: icmp_seq=1 ttl=64 time=0.489 ms
^C
--- 192.168.12.2 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.489/0.528/0.568/0.039 ms

Question: What is the result of the ping test?

Answer: The ping test is successful. Using the OSPF

protocol to exchange routing information, the
Junos OS is able to acquire better routing
information.

Step 3.10
Next, perform a traceroute to the remote team’s virtual router.

Lab 5–18 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
[edit protocols ospf]
lab@mxA-1# run traceroute remote-teams-VR
traceroute to 192.168.12.2 (192.168.12.2), 30 hops max, 40 byte packets
1 172.18.5.2 (172.18.5.2) 0.687 ms 0.438 ms 0.386 ms
2 192.168.12.2 (192.168.12.2) 0.610 ms 0.596 ms 0.576 ms

Question: What is the result of the traceroute test?

Answer: The traceroute reaches the remote team’s

device over the ge-1/0/2 link, and then reaches the
remote team’s virtual router through their ge-1/0/3
link.

Step 3.11
Navigate to the [edit routing-options] hierarchy level.
[edit protocols ospf]
lab@mxA-1# top edit routing-options

[edit routing-options]
lab@mxA-1#
Step 3.12
Configure a local autonomous system of 65412.
[edit routing-options]
lab@mxA-1# set autonomous-system 65412
Step 3.13
Navigate to the [edit protocols bgp] hierarchy level.
[edit routing-options]
lab@mxA-1# top edit protocols bgp

[edit protocols bgp]

lab@mxA-1#

Note
You must use the interface IP addresses
associated with ISP A and ISP B for the BGP
peering sessions. Please refer to the lab
diagram for further details.

Step 3.14
Configure the BGP group ISP_A by issuing the edit group ISP-A command.
[edit protocols bgp]
lab@mxA-1# edit group ISP-A

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–19

Junos Troubleshooting in the NOC
[edit protocols bgp group ISP-A]
lab@mxA-1#
Step 3.15
Configure the neighbor IP address of ISP A’s directly connected interface by issuing
the set neighbor ISP-A-IP command.
[edit protocols bgp group ISP-A]
lab@mxA-1# set neighbor ISP-A-IP
Step 3.16
Configure the peer autonomous system number for ISP A by issuing the
set peer-as 56155 command.
[edit protocols bgp group ISP-A]
lab@mxA-1# set peer-as 56155
Step 3.17
Configure the BGP group ISP-B by issuing the up command and edit group
ISP-B command.
[edit protocols bgp group ISP-B]
lab@mxA-1# up

[edit protocols bgp]

lab@mxA-1# edit group ISP-B

[edit protocols bgp group ISP_B]

lab@mxA-1#
Step 3.18
Configure the neighbor IP address of ISP B’s directly connected interface by issuing
the set neighbor ISP-B-IP command.
[edit protocols bgp group ISP_B]
lab@mxA-1# set neighbor ISP-B-IP
Step 3.19
Configure the peer autonomous system number for ISP B by issuing the
set peer-as 56144 command. Activate the configuration by issuing the
commit command.
[edit protocols bgp group ISP_B]
lab@mxA-1# set peer-as 56144

[edit protocols bgp group ISP-B]

lab@mxA-1# commit
commit complete
Step 3.20
Issue the run show bgp summary command.
[edit protocols bgp group ISP-B]
lab@mxA-1# run show bgp summary

Lab 5–20 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
Groups: 2 Peers: 2 Down peers: 2
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0 0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn
State|#Active/Received/Accepted/Damped...
172.18.1.1 56155 0 6 0 0 1:49 Active
172.18.2.1 56144 0 6 0 0 1:49 Active

lab@mxA-1>

Question: Did the BGP sessions reach the

Established state?

Answer: No. The BGP sessions transitioned into the

Active state but are not able to reach the
Established state.

Step 3.21
To investigate why the BGP sessions cannot reach the Established state,
configure traceoptions under the BGP protocol. Navigate to the [edit
protocols bgp traceoptions] hierarchy level. Once under the BGP trace
options hierarchy level, configure the flag of open and the file to be
bgp-trace.log. Commit the configuration when complete.
[edit protocols bgp group ISP-B]
lab@mxA-1# up 1 edit traceoptions

[edit protocols bgp traceoptions]

lab@mxA-1# set flag open

[edit protocols bgp traceoptions]

lab@mxA-1# set file bgp-trace.log

[edit protocols bgp traceoptions]

lab@mxA-1# commit
commit complete

[edit protocols bgp traceoptions]

lab@mxA-1#
Step 3.22
Issue the run show log bgp-trace.log command.
Note
You might have to wait a minute for the
traceoptions log file to begin filling with
information.

[edit protocols bgp traceoptions]

lab@mxA-1# run show log bgp-trace.log

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–21

Junos Troubleshooting in the NOC
Feb 7 21:53:31 trace_on: Tracing to "/var/log/bgp-trace.log" started
Feb 7 21:54:08.391037 advertising graceful restart receiving-speaker-only
capability to neighbor 172.18.1.1 (External AS 56155)
Feb 7 21:54:08.391217 bgp_send: sending 59 bytes to 172.18.1.1 (External AS
56155)
Feb 7 21:54:08.391259
Feb 7 21:54:08.391259 BGP SEND 172.18.1.2+62244 -> 172.18.1.1+179
Feb 7 21:54:08.391298 BGP SEND message type 1 (Open) length 59
Feb 7 21:54:08.400164
Feb 7 21:54:08.400164 BGP RECV 172.18.1.1+179 -> 172.18.1.2+62244
Feb 7 21:54:08.400206 BGP RECV message type 1 (Open) length 59
Feb 7 21:54:08.400296 bgp_process_open:2824: NOTIFICATION sent to 172.18.1.1
(External AS 56155): code 2 (Open Message Error) subcode 2 (bad peer AS
number), Reason: peer 172.18.1.1 (External AS 56155) claims 65001, 56155
configured
Feb 7 21:54:08.400324 bgp_send: sending 21 bytes to 172.18.1.1 (External AS
56155)
Feb 7 21:54:08.400346
Feb 7 21:54:08.400346 BGP SEND 172.18.1.2+62244 -> 172.18.1.1+179
Feb 7 21:54:08.400380 BGP SEND message type 3 (Notification) length 21
Feb 7 21:54:12.391034 advertising graceful restart receiving-speaker-only
capability to neighbor 172.18.2.1 (External AS 56144)
Feb 7 21:54:12.391122 bgp_send: sending 59 bytes to 172.18.2.1 (External AS
56144)
Feb 7 21:54:12.391148
Feb 7 21:54:12.391148 BGP SEND 172.18.2.2+54532 -> 172.18.2.1+179
Feb 7 21:54:12.391182 BGP SEND message type 1 (Open) length 59
Feb 7 21:54:12.400187
Feb 7 21:54:12.400187 BGP RECV 172.18.2.1+179 -> 172.18.2.2+54532
Feb 7 21:54:12.400230 BGP RECV message type 1 (Open) length 59
Feb 7 21:54:12.400319 bgp_process_open:2824: NOTIFICATION sent to 172.18.2.1
(External AS 56144): code 2 (Open Message Error) subcode 2 (bad peer AS
number), Reason: peer 172.18.2.1 (External AS 56144) claims 65002, 56144
configured
Feb 7 21:54:12.400346 bgp_send: sending 21 bytes to 172.18.2.1 (External AS
56144)
Feb 7 21:54:12.400367
Feb 7 21:54:12.400367 BGP SEND 172.18.2.2+54532 -> 172.18.2.1+179
Feb 7 21:54:12.400401 BGP SEND message type 3 (Notification) length 21

Question: What is the cause of the BGP peering

problems?

Answer: An autonomous system mismatch exists

between your assigned device and the ISP A and
ISP B routers.

Lab 5–22 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
Question: What must you do to resolve the issue?

Answer: You must change the peer-as value to

65001 for ISP A and the peer-as value to 65002
for ISP B.

Step 3.23
Remove the traceoptions configuration that you recently configured and commit
the configuration.
Note
Traceoptions can be processor intensive.
We recommend to only use traceoptions for
troubleshooting purposes and remove the
traceoptions when the troubleshooting is
finished.

[edit protocols bgp traceoptions]

lab@mxA-1# up

[edit protocols bgp]

lab@mxA-1# delete traceoptions

[edit protocols bgp]

lab@mxA-1# commit
commit complete

[edit protocols bgp]

lab@mxA-1#
Step 3.24
Change the peer-as value for the BGP group ISP-A to 65001 by issuing the
set ISP-A peer-as 65001 command.
[edit protocols bgp]
lab@mxA-1# set group ISP-A peer-as 65001
Step 3.25
Change the peer-as value for the BGP group ISP-B to 65002 by issuing the set
group ISP-B peer-as 65002 command. Commit the configuration and exit to
operational mode.
[edit protocols bgp]
lab@mxA-1# set group ISP-B peer-as 65002

[edit protocols bgp]

lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>
www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–23
Junos Troubleshooting in the NOC
Step 3.26
Issue the show bgp summary command.
Note
You might have to wait few minutes for the
BGP session to reach the Established
state.

lab@mxA-1> show bgp summary

Groups: 2 Peers: 2 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0 24 11 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn
State|#Active/Received/Accepted/Damped...
172.18.1.1 65001 4 5 0 0 11 10/
12/12/0 0/0/0/0
172.18.2.1 65002 3 6 0 0 7 1/
12/12/0 0/0/0/0

Question: Did the BGP sessions reach the

Established state?

Answer: Yes. Even though the output does not show

the Established state, it does show that your
assigned device is receiving routes from the ISP
routers.

Step 3.27
Issue the show route | no-more command.
lab@mxA-1> show route | no-more

inet.0: 29 destinations, 43 routes (29 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

0.0.0.0/0 *[Static/5] 02:06:45

> to 172.18.1.1 via ge-1/0/0.141
[Static/7] 02:06:45
> to 172.18.2.1 via ge-1/0/1.141
[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
10.210.15.0/27 *[Direct/0] 02:06:45
> via fxp0.0
10.210.15.1/32 *[Local/0] 02:06:45
Local via fxp0.0

Lab 5–24 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC
172.18.1.0/30 *[Direct/0] 02:06:45
> via ge-1/0/0.141
172.18.1.2/32 *[Local/0] 02:06:45
Local via ge-1/0/0.141
172.18.2.0/30 *[Direct/0] 02:06:45
> via ge-1/0/1.141
172.18.2.2/32 *[Local/0] 02:06:45
Local via ge-1/0/1.141
172.18.5.0/30 *[Direct/0] 02:06:45
> via ge-1/0/2.141
172.18.5.1/32 *[Local/0] 02:06:45
Local via ge-1/0/2.141
172.18.112.1/32 *[Local/0] 02:06:45
Reject
172.18.113.0/30 *[Direct/0] 02:06:45
> via ge-1/1/3.0
172.18.113.1/32 *[Local/0] 02:06:45
Local via ge-1/1/3.0
172.31.16.0/24 *[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
[BGP/170] 00:00:33, localpref 100
AS path: 802 5512 3318 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
172.31.17.0/24 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
172.31.18.0/24 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
172.31.19.0/24 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
172.31.20.0/24 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
172.31.21.0/24 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–25

Junos Troubleshooting in the NOC
192.168.11.0/24 *[Direct/0] 02:06:45
> via ge-1/0/3.141
192.168.11.1/32 *[Local/0] 02:06:45
Local via ge-1/0/3.141
192.168.12.0/24 *[OSPF/10] 00:36:59, metric 2
> to 172.18.5.2 via ge-1/0/2.141
192.168.31.1/32 *[Direct/0] 02:06:45
> via lo0.0
192.168.32.1/32 *[OSPF/10] 00:36:59, metric 1
> to 172.18.5.2 via ge-1/0/2.141
192.168.80.0/28 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
192.168.81.0/28 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
192.168.82.0/29 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
192.168.83.0/27 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
192.168.84.0/27 *[BGP/170] 00:00:33, localpref 100
AS path: 65001 I, validation-state: unverified
> to 172.18.1.1 via ge-1/0/0.141
[BGP/170] 00:04:33, localpref 100
AS path: 65002 I, validation-state: unverified
> to 172.18.2.1 via ge-1/0/1.141
224.0.0.5/32 *[OSPF/10] 01:19:41, metric 1
MultiRecv

Question: Which protocols are populating the

routing table?

Answer: Routes from protocols BGP, OSPF, static,

direct, and local are populating the routing table.

Lab 5–26 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC

Part 4: Configuring Bridging

In this lab part, you examine the ARP table, configure bridging, and examine the
bridge table.
Step 4.1
Issue the show arp command.
lab@mxA-1> show arp
MAC Address Address Name Interface
Flags
5c:5e:ab:00:6f:ff 10.210.14.2 10.210.14.2 fxp0.0
none
5c:5e:ab:00:16:ff 10.210.14.5 10.210.14.5 fxp0.0
none
5c:5e:ab:00:59:ff 10.210.14.6 10.210.14.6 fxp0.0
none
00:10:db:ff:21:50 10.210.14.30 10.210.14.30 fxp0.0
none
02:00:00:00:00:10 128.0.0.50 128.0.0.50 em0.0
none
84:18:88:8e:ca:95 172.18.1.1 172.18.1.1 ge-1/0/0.141
none
84:18:88:8e:ca:96 172.18.2.1 172.18.2.1 ge-1/0/1.141
none
5c:5e:ab:00:6f:62 172.18.5.2 172.18.5.2 ge-1/0/2.141
none
84:18:88:8e:ca:98 192.168.11.2 192.168.11.2 ge-1/0/3.141
none
Total entries: 9

Question: What interfaces are present in the ARP

table?

Answer: Although your output may might, the output

shows that interfaces fxp0.0, em0.0, ge-1/0/0.141,
ge-1/0/1.141, ge-1/0/2.141, and ge-1/0/3.141
are present in the ARP table.

Step 4.2
Use the clear arp interface ge-1/0/3.141 command to remove the ARP
entry associated with the ge-1/0/3.141 interface.
lab@mxA-1> clear arp interface ge-1/0/3.141
192.168.11.2 deleted
Step 4.3
Issue the show arp command.

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–27

Junos Troubleshooting in the NOC
lab@mxA-1> show arp
MAC Address Address Name Interface
Flags
5c:5e:ab:00:6f:ff 10.210.14.2 10.210.14.2 fxp0.0
none
5c:5e:ab:00:16:ff 10.210.14.5 10.210.14.5 fxp0.0
none
5c:5e:ab:00:59:ff 10.210.14.6 10.210.14.6 fxp0.0
none
00:10:db:ff:21:50 10.210.14.30 10.210.14.30 fxp0.0
none
02:00:00:00:00:10 128.0.0.50 128.0.0.50 em0.0
none
84:18:88:8e:ca:95 172.18.1.1 172.18.1.1 ge-1/0/0.141
none
84:18:88:8e:ca:96 172.18.2.1 172.18.2.1 ge-1/0/1.141
none
5c:5e:ab:00:6f:62 172.18.5.2 172.18.5.2 ge-1/0/2.141
none
Total entries: 8

lab@mxA-1>

Question: What is the status of the ARP entry

associated with the ge-1/0/3.141 interface?

Answer: The ARP entry is not present in the ARP

table.

Question: What must you do to replace the ARP

entry that was associated with the
ge-1/0/3.141 interface?

Answer: The ARP entry is replaced automatically

when traffic is sent to or from the virtual router
connected to your device.

Step 4.4
Enter configuration mode and navigate to the [edit interfaces ge-1/0/2]
hierarchy level and issue the delete command to remove any configuration at this
level.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit interfaces ge-1/0/2

Lab 5–28 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC

[edit interfaces ge-1/0/2]

lab@mxA-1# delete
Delete everything under this level? [yes,no] (no) yes

[edit interfaces ge-1/0/2]

lab@mxA-1#
Step 4.5
Configure a physical encapsulation of ethernet-bridge.
[edit interfaces ge-1/0/2]
lab@mxA-1# set encapsulation ethernet-bridge
Step 4.6
Configure family bridge under the logical Unit 0.
[edit interfaces ge-1/0/2]
lab@mxA-1# set unit 0 family bridge
Step 4.7
Navigate to the [edit bridge-domain lab] hierarchy level.
[edit interfaces ge-1/0/2]
lab@mxA-1# top edit bridge-domains lab
Step 4.8
Configure a domain-type of bridge.
[edit bridge-domains lab]
lab@mxA-1# set domain-type bridge
Step 4.9
Configure the ge-1/0/2 interface to participate in this bridge domain.
[edit bridge-domains lab]
lab@mxA-1# set interface ge-1/0/2
Step 4.10
Commit the configuration and exit to operational mode.
[edit bridge-domains lab]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>
Step 4.11
Issue the show bridge domain detail command.
lab@mxA-1> show bridge domain detail

Routing instance: default-switch

Bridge domain: lab State: Active
Bridge VLAN ID: NA

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–29

Junos Troubleshooting in the NOC
Interfaces:
ge-1/0/2.0
Total mac count: 1

Question: How many MAC addresses have been

learned in the bridge domain?

Answer: Although your output might vary, the

previous output shows that only one MAC address
has been learned in the bridge domain.

Question: Which VLAN IDs are associated with the

lab bridge domain?

Answer: The Bridge VLAN ID field displays NA

which means there are no VLAN IDs associated with
the lab bridge domain.

Step 4.12
Log out of your assigned device.
lab@mxA-1> exit

mxA-1 (ttyu0)

login:

STOP Tell your instructor that you have completed this lab.

Lab 5–30 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–31

Junos Troubleshooting in the NOC

Lab 5–32 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Control Plane Monitoring and Troubleshooting • Lab 5–33

Junos Troubleshooting in the NOC

Lab 5–34 • Control Plane Monitoring and Troubleshooting www.juniper.net

 Lab
Monitoring and Troubleshooting Ethernet Interfaces

Overview
This lab demonstrates how to monitor and troubleshoot Ethernet interfaces. You will
examine and troubleshoot an Ethernet interface that is in the physical down state. You will
then set that interface in local loopback mode to test the health of the interface. Next, you
will configure Ethernet OAM and monitor the operation of it.
By completing this lab, you will perform the following tasks:
• Troubleshoot an Ethernet interface.
• Configure and run a local loopback test.
• Configure and monitor Ethernet OAM.

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–1

.
Junos Troubleshooting in the NOC

Part 1: Performing Interface Troubleshooting

In this lab part, you will troubleshoot a link-down issue. You will first discover the
issue, devise a solution, and implement that solution.
Note
Depending on the class, the lab equipment
used might be remote from your physical
location. The instructor will inform you as to
the nature of your access and will provide
you the details needed to access your
assigned device.

Step 1.1
Ensure that you know to which student device you have been assigned. Check with
your instructor if you are not certain. Consult the Management Network Diagram to
determine the management address of your student device.

Question: What is the management address

assigned to your station?

Answer: The answer varies and depends on your

assigned device. If you are unsure of your
assignment, ask your instructor.

Step 1.2
Access the CLI for your device using either the console, Telnet, or SSH as directed by
your instructor. Refer to the management network diagram for the IP address
associated with your station. The following example uses Telnet to access an
MX Series device using the SecureCRT program:

Lab 6–2 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC

Step 1.3
Log in to your device with the username lab and the password lab123. Note that
both the name and password are case-sensitive.
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1>
Step 1.4
Enter into configuration mode and load the device’s reset configuration by issuing
the load override jtnoc/lab6-start.config command. Activate the
configuration by issuing the commit command.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# load override jtnoc/lab6-start.config
load complete

[edit]
lab@mxA-1# commit
commit complete

[edit]
lab@mxA-1#

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–3

Junos Troubleshooting in the NOC
Step 1.5
Issue the run show interfaces terse ge-1/0/9 command.
[edit]
lab@mxA-1# run show interfaces terse ge-1/0/9
Interface Admin Link Proto Local Remote
ge-1/0/9 up down
ge-1/0/9.0 up down inet 10.10.10.1/30
multiservice

Question: What is the status of the ge-1/0/9

interface?

Answer: The ge-1/0/9 interface is in the link down

state.

Question: Which commands can you use to

investigate the problem further?

Answer: You can use the show log messages,

show log chassisd, show interfaces,
show interfaces media, and the show
interfaces extensive commands to
troubleshoot this issue.

Question: Is the monitor traffic interface

command for the ge-1/0/9 interface useful in this
situation? Why?

Answer: No. The command cannot be run on an

interface that is in the link down state.

Step 1.6
Issue the run show log messages | match ge-1/0/9 command.
[edit]
lab@mxA-1# run show log messages | match ge-1/0/9
Jan 31 01:13:43 mxA-1 chassisd[1281]: CHASSISD_IFDEV_CREATE_NOTICE:
create_pics: created interface device for ge-1/0/9
Jan 31 01:13:47 mxA-1 tfeb0 mic_sfp_phy_create: - Create SFP PHY ge-1/0/9
successfully

Lab 6–4 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC
Jan 31 01:13:54 mxA-1 mib2d[1417]: SNMP_TRAP_LINK_DOWN: ifIndex 521,
ifAdminStatus up(1), ifOperStatus down(2), ifName ge-1/0/9
Feb 5 15:29:42 mxA-1 mib2d[1417]: SNMP_TRAP_LINK_DOWN: ifIndex 640,
ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/0/9.0
Feb 5 15:29:50 mxA-1 mib2d[1417]: SNMP_TRAP_LINK_DOWN: ifIndex 521,
ifAdminStatus up(1), ifOperStatus down(2), ifName ge-1/0/9
Feb 6 19:46:50 mxA-1 mib2d[1417]: SNMP_TRAP_LINK_DOWN: ifIndex 640,
ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/0/9.0
Feb 7 01:43:29 mxA-1 mib2d[1417]: SNMP_TRAP_LINK_DOWN: ifIndex 521,
ifAdminStatus up(1), ifOperStatus down(2), ifName ge-1/0/9
Feb 7 01:43:31 mxA-1 alarmd[1282]: Alarm set: ge-1/0/9 color=RED, class=ETHER,
reason=ge-1/0/9: Link down
Feb 7 01:43:31 mxA-1 craftd[1283]: Major alarm set, ge-1/0/9: Link down

Question: Does the previous output contain any

information that is helpful in troubleshooting this
issue?

Answer: Although the previous output does not tell

you why the interface is physically down, it does
lead you to believe that this is not a hardware issue.

Step 1.7
Issue the run show log chassisd | match ge-1/0/9 command.
[edit]
lab@mxA-1# run show log chassisd | match ge-1/0/9
Jan 10 22:57:05 CHASSISD_IFDEV_CREATE_NOTICE: create_pics: created interface
device for ge-1/0/9
Jan 10 22:57:05 ifdev_create entered ge-1/0/9
Jan 10 22:57:05 ge-1/0/9: large delay buffer cleared
Jan 31 01:07:25 ifd ge-1/0/9 marked as gone
Jan 31 01:13:43 Created pic for ge-1/0/9
Jan 31 01:13:43 CHASSISD_IFDEV_CREATE_NOTICE: create_pics: created interface
device for ge-1/0/9
Jan 31 01:13:43 ifdev_create entered ge-1/0/9
Jan 31 01:13:43 ge-1/0/9: large delay buffer cleared

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–5

Junos Troubleshooting in the NOC
Question: Does the previous output contain any
information that is helpful in troubleshooting this
issue?

Answer: Although the previous output does not tell

you why the interface is physically down, it also
leads you to believe that this is not a hardware
issue.

Step 1.8
Issue the run clear interfaces statistics ge-1/0/9 command, wait
a minute, and then issue the run show interfaces ge-1/0/9 extensive
command.

Note
Clearing the interface statistics before
gathering statistical information is a
recommended troubleshooting step. This
step removes any stale information that is
not related to the present issue.

[edit]
lab@mxA-1# run clear interfaces statistics ge-1/0/9

[edit]
lab@mxA-1# run show interfaces ge-1/0/9 extensive
Physical interface: ge-1/0/9, Enabled, Physical link is Down
Interface index: 149, SNMP ifIndex: 521, Generation: 152
Link-level type: Ethernet, MTU: 1514, Speed: 1000mbps, BPDU Error: None,
MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled,
Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online
Device flags : Present Running Down
Interface flags: Hardware-Down SNMP-Traps Internal: 0x0
Link flags : None
CoS queues : 8 supported, 8 maximum usable queues
Hold-times : Up 0 ms, Down 0 ms
Current address: 80:71:1f:c3:03:69, Hardware address: 80:71:1f:c3:03:69
Last flapped : 2013-02-07 01:43:29 UTC (01:19:31 ago)
Statistics last cleared: 2013-02-07 03:02:36 UTC (00:00:24 ago)
Traffic statistics:
Input bytes : 0 0 bps
Output bytes : 0 0 bps
Input packets: 0 0 pps
Output packets: 0 0 pps
IPv6 transit statistics:
Input bytes : 0
Output bytes : 0

Lab 6–6 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC
Input packets: 0
Output packets: 0
Dropped traffic statistics due to STP State:
Input bytes : 0
Output bytes : 0
Input packets: 0
Output packets: 0
Input errors:
Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0,
L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0,
FIFO errors: 0, Resource errors: 0
Output errors:
Carrier transitions: 0, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0,
FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0
Egress queues: 8 supported, 4 in use
Queue counters: Queued packets Transmitted packets Dropped packets
0 best-effort 0 0 0
1 expedited-fo 0 0 0
2 assured-forw 0 0 0
3 network-cont 0 0 0
Queue number: Mapped forwarding classes
0 best-effort
1 expedited-forwarding
2 assured-forwarding
3 network-control
Active alarms : LINK
Active defects : LINK
MAC statistics: Receive Transmit
Total octets 0 0
Total packets 0 0
Unicast packets 0 0
Broadcast packets 0 0
Multicast packets 0 0
CRC/Align errors 0 0
FIFO errors 0 0
MAC control frames 0 0
MAC pause frames 0 0
Oversized frames 0
Jabber frames 0
Fragment frames 0
VLAN tagged frames 0
Code violations 0
Total errors 0 0
Filter statistics:
Input packet count 0
Input packet rejects 0
Input DA rejects 0
Input SA rejects 0
Output packet count 0
Output packet pad count 0
Output packet error count 0
CAM destination filters: 0, CAM source filters: 0
Autonegotiation information:
Negotiation status: Incomplete
Packet Forwarding Engine configuration:

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–7

Junos Troubleshooting in the NOC
Destination slot: 1
CoS information:
Direction : Output
CoS transmit queue Bandwidth Buffer Priority
Limit
% bps % usec
0 best-effort 95 950000000 95 0 low none
3 network-control 5 50000000 5 0 low none
Interface transmit statistics: Disabled

Logical interface ge-1/0/9.0 (Index 331) (SNMP ifIndex 640) (Generation 225)
Flags: Device-Down SNMP-Traps 0x0 Encapsulation: ENET2
Traffic statistics:
Input bytes : 0
Output bytes : 0
Input packets: 0
Output packets: 0
Local statistics:
Input bytes : 0
Output bytes : 0
Input packets: 0
Output packets: 0
Transit statistics:
Input bytes : 0 0 bps
Output bytes : 0 0 bps
Input packets: 0 0 pps
Output packets: 0 0 pps
Protocol inet, MTU: 1500, Generation: 330, Route table: 0
Flags: Sendbcast-pkt-to-re
Addresses, Flags: Dest-route-down Is-Preferred Is-Primary
Destination: 10.10.10.0/30, Local: 10.10.10.1, Broadcast: 10.10.10.3,
Generation: 266
Protocol multiservice, MTU: Unlimited, Generation: 331, Route table: 0
Flags: Is-Primary
Policer: Input: __default_arp_policer__

Question: Are any errors present in this output?

Answer: No errors should be present in this output.

Step 1.9
Issue the run show interface ge-1/0/9 media command.
[edit]
lab@mxA-1# run show interfaces ge-1/0/9 media
Physical interface: ge-1/0/9, Enabled, Physical link is Down
Interface index: 149, SNMP ifIndex: 521
Link-level type: Ethernet, MTU: 1514, Speed: 1000mbps, BPDU Error: None,
MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled,
Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online
Device flags : Present Running Down

Lab 6–8 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC
Interface flags: Hardware-Down SNMP-Traps Internal: 0x0
Link flags : None
CoS queues : 8 supported, 8 maximum usable queues
Current address: 80:71:1f:c3:03:69, Hardware address: 80:71:1f:c3:03:69
Last flapped : 2013-02-07 01:43:29 UTC (01:20:50 ago)
Input rate : 0 bps (0 pps)
Output rate : 0 bps (0 pps)
Active alarms : LINK
Active defects : LINK
MAC statistics:
Input bytes: 0, Input packets: 0, Output bytes: 0, Output packets: 0
Filter statistics:
Filtered packets: 0, Padded packets: 0, Output packet errors: 0
Autonegotiation information:
Negotiation status: Incomplete
Interface transmit statistics: Disabled

Question: Does the previous output contain any

information that is helpful in troubleshooting this
issue?

Answer: The previous output reveals that the

negotiation status for the link is incomplete. This
information is visible in the show interfaces
ge-1/0/9 extensive command, but it was
buried among the large amount of output produced
by that command.

Question: What might cause autonegotiation to fail

and the interface to be declared physically down?

Answer: A link speed mismatch can cause

autonegotiation to fail and the interface to be
declared physically down.

Step 1.10
Navigate to the [edit interfaces ge-1/0/9] hierarchy level.
[edit]
lab@mxA-1# edit interfaces ge-1/0/9

[edit interfaces ge-1/0/9]

lab@mxA-1#

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–9

Junos Troubleshooting in the NOC
Step 1.11
Change the link speed for interface ge-1/0/9 to 100 Mbps. Activate the
configuration by issuing the commit command.
[edit interfaces ge-1/0/9]
lab@mxA-1# set speed 100m

[edit interfaces ge-1/0/9]

lab@mxA-1# commit
commit complete

[edit interfaces ge-1/0/9]

lab@mxA-1#
Step 1.12
Issue the run show interfaces ge-1/0/9 media command.
[edit interfaces ge-1/0/9]
lab@mxA-1# run show interfaces ge-1/0/9 media
Physical interface: ge-1/0/9, Enabled, Physical link is Down
Interface index: 149, SNMP ifIndex: 521
Link-level type: Ethernet, MTU: 1514, Speed: 100mbps, BPDU Error: None,
MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled,
Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online
Device flags : Present Running Down
Interface flags: Hardware-Down SNMP-Traps Internal: 0x0
Link flags : None
CoS queues : 8 supported, 8 maximum usable queues
Current address: 80:71:1f:c3:03:69, Hardware address: 80:71:1f:c3:03:69
Last flapped : 2013-02-07 01:43:29 UTC (01:24:56 ago)
Input rate : 0 bps (0 pps)
Output rate : 0 bps (0 pps)
Active alarms : LINK
Active defects : LINK
MAC statistics:
Input bytes: 0, Input packets: 0, Output bytes: 0, Output packets: 0
Filter statistics:
Filtered packets: 0, Padded packets: 0, Output packet errors: 0
Autonegotiation information:
Negotiation status: Incomplete
Interface transmit statistics: Disabled

Question: Did changing the link speed solve the

autonegotiation issue?

Answer: No. The negotiation status displays

Incomplete.

Lab 6–10 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC
Question: Because setting the link speed to
100 Mbps did not solve the problem, what is
another possible troubleshooting step?

Answer: By default, the link speed is set to 1 Gbps

and the only other link speed option is 10 Mbps.

Step 1.13
Change the link speed for the ge-1/0/9 interface to 10 Mbps and commit the
configuration.
[edit interfaces ge-1/0/9]
lab@mxA-1# set speed 10m

[edit interfaces ge-1/0/9]

lab@mxA-1# commit
commit complete
Step 1.14
Issue the run show interfaces ge-1/0/9 media command.
[edit interfaces ge-1/0/9]
lab@mxA-1# run show interfaces ge-1/0/9 media
Physical interface: ge-1/0/9, Enabled, Physical link is Up
Interface index: 149, SNMP ifIndex: 521
Link-level type: Ethernet, MTU: 1514, Speed: 10m, BPDU Error: None,
MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled,
Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online
Device flags : Present Running
Interface flags: SNMP-Traps Internal: 0x0
Link flags : None
CoS queues : 8 supported, 8 maximum usable queues
Current address: 80:71:1f:c3:03:69, Hardware address: 80:71:1f:c3:03:69
Last flapped : 2013-02-07 03:09:59 UTC (00:00:22 ago)
Input rate : 0 bps (0 pps)
Output rate : 0 bps (0 pps)
Active alarms : None
Active defects : None
MAC statistics:
Input bytes: 0, Input packets: 0, Output bytes: 0, Output packets: 0
Filter statistics:
Filtered packets: 0, Padded packets: 0, Output packet errors: 0
Autonegotiation information:
Negotiation status: Complete
Link partner:
Link mode: Full-duplex, Flow control: Symmetric/Asymmetric,
Remote fault: OK, Link partner Speed: 10 Mbps
Local resolution:
Flow control: Symmetric, Remote fault: Link OK
Interface transmit statistics: Disabled

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–11

Junos Troubleshooting in the NOC
Question: Did changing the link speed to 10 Mbps
solve the autonegotiation problem? What is the
autonegotiation status?

Answer: Yes. The autonegotiation status has

reached the Complete state and the link is
physically up. Also, take note that through
autonegotiation your router was able to detect a
received link speed of 10 Mbps.

Step 1.15
To ensure that the ge-1/0/9 interface is now working correctly, exit to operational
mode and issue the ping 10.10.10.2 detail count 5 command to test
connectivity to your local R1 device.
[edit interfaces ge-1/0/9]
lab@mxA-1# exit configuration-mode
Exiting configuration mode

lab@mxA-1> ping 10.10.10.2 detail count 5

PING 10.10.10.2 (10.10.10.2): 56 data bytes
64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp_seq=0 ttl=64 time=2.221 ms
64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp_seq=1 ttl=64 time=0.856 ms
64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp_seq=2 ttl=64 time=0.850 ms
64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp_seq=3 ttl=64 time=0.803 ms
64 bytes from 10.10.10.2 via ge-1/0/9.0: icmp_seq=4 ttl=64 time=0.865 ms

--- 10.10.10.2 ping statistics ---

5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.803/1.119/2.221/0.551 ms

lab@mxA-1>

Question: What did the ping test reveal?

Answer: The ping test shows that Layers 1 - 3 are

functional across the link.

Lab 6–12 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC
Question: Why is it important to test Layer 3
connectivity on the link?

Answer: Even though a link might show that it is in

the operational state, an intermediate device might
be stopping Layer 3 communication rendering the
link unusable.

Part 2: Performing Loopback Testing

In this lab part, you will configure an interface in local loopback mode to test the
integrity of the ge-1/0/9 interface.
Step 2.1
Enter configuration mode and navigate to the [edit interfaces ge-1/0/9]
hierarchy level. Then, configure the loopback option.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit interfaces ge-1/0/9

[edit interfaces ge-1/0/9]

lab@mxA-1# set gigether-options loopback
Step 2.2
Issue the run show interfaces ge-1/0/9 | match hardware
command to obtain the MAC address associated with the ge-1/0/9 interface.
[edit interfaces ge-1/0/9]
lab@mxA-1# run show interfaces ge-1/0/9 | match hardware
Current address: 80:71:1f:c3:03:69, Hardware address: 80:71:1f:c3:03:69

Question: What is the MAC address associated with

the ge-1/0/9 interface?

Answer: Although your output might vary, the

previous output shows that the MAC address for the
ge-1/0/9 interface is 80:71:1f:c3:03:69.

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–13

Junos Troubleshooting in the NOC
Step 2.3
Configure a static ARP entry for the 10.10.10.2 IP address under the 10.10.10.1/24
prefix, which resolves to the MAC address associated with the ge-1/0/9 interface.
[edit interfaces ge-1/0/9]
lab@mxA-1# set unit 0 family inet address 10.10.10.1/30 arp 10.10.10.2 mac
XX:XX:XX:XX:XX:XX

[edit interfaces ge-1/0/9]

lab@mxA-1# show
speed 10m;
gigether-options {
loopback;
}
unit 0 {
family inet {
address 10.10.10.1/30 {
arp 10.10.10.2 mac 80:71:1f:c3:03:69;
}
}
}
Step 2.4
Commit the configuration and exit to operational mode. Then issue the command
clear interfaces statistics ge-1/0/9 to reset all of the statistics on
the interface.
[edit interfaces ge-1/0/9]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1> clear interfaces statistics ge-1/0/9

lab@mxA-1>
Step 2.5
Issue the ping 10.10.10.2 rapid count 10 command to loop traffic inside
the ge-1/0/9 interface.

Note
The expected behavior is to see a
time-to-live exceeded message.

lab@mxA-1> ping 10.10.10.2 rapid count 10

PING 10.10.10.2 (10.10.10.2): 56 data bytes
36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 4fdd 0 0000 01 01 41b6 10.10.10.1 10.10.10.2
.36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 4fe6 0 0000 01 01 41ad 10.10.10.1 10.10.10.2
.36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
Lab 6–14 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net
 Junos Troubleshooting in the NOC
4 5 00 0054 4ff0 0 0000 01 01 41a3 10.10.10.1 10.10.10.2
.36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 4ff9 0 0000 01 01 419a 10.10.10.1 10.10.10.2
.36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 5003 0 0000 01 01 4190 10.10.10.1 10.10.10.2
.36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 500c 0 0000 01 01 4187 10.10.10.1 10.10.10.2
.36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 5017 0 0000 01 01 417c 10.10.10.1 10.10.10.2
.36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 5022 0 0000 01 01 4171 10.10.10.1 10.10.10.2
.36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 502f 0 0000 01 01 4164 10.10.10.1 10.10.10.2
.36 bytes from 10.10.10.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 5038 0 0000 01 01 415b 10.10.10.1 10.10.10.2
.
--- 10.10.10.2 ping statistics ---
10 packets transmitted, 0 packets received, 100% packet loss

Question: Why are the Time to live

exceeded messages appearing?

Answer: The Time to live exceeded

messages are appearing because ping packets are
being looped inside the interface hardware until the
time-to-live field expires.

Step 2.6
Issue the show interface ge-1/0/9 extensive command.
lab@mxA-1> show interfaces ge-1/0/9 extensive
Physical interface: ge-1/0/9, Enabled, Physical link is Up
Interface index: 149, SNMP ifIndex: 521, Generation: 152
Link-level type: Ethernet, MTU: 1514, Speed: 10m, BPDU Error: None,
MAC-REWRITE Error: None, Loopback: Enabled, Source filtering: Disabled,
Flow control: Enabled, Auto-negotiation: Enabled, Remote fault: Online
Device flags : Present Running Loop-Detected
Interface flags: SNMP-Traps Internal: 0x0
Link flags : None
CoS queues : 8 supported, 8 maximum usable queues
Hold-times : Up 0 ms, Down 0 ms
Current address: 80:71:1f:c3:03:69, Hardware address: 80:71:1f:c3:03:69

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–15

Junos Troubleshooting in the NOC
Last flapped : 2013-02-07 03:09:59 UTC (00:28:06 ago)
Statistics last cleared: 2013-02-07 03:36:14 UTC (00:01:51 ago)
Traffic statistics:
Input bytes : 53760 0 bps
Output bytes : 53960 0 bps
Input packets: 640 0 pps
Output packets: 640 0 pps
IPv6 transit statistics:
Input bytes : 0
Output bytes : 0
Input packets: 0
Output packets: 0
Dropped traffic statistics due to STP State:
Input bytes : 0
Output bytes : 0
Input packets: 0
Output packets: 0
Input errors:
Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0,
L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0,
FIFO errors: 0, Resource errors: 0
Output errors:
Carrier transitions: 0, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0,
FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0
Egress queues: 8 supported, 4 in use
Queue counters: Queued packets Transmitted packets Dropped packets
0 best-effort 640 640 0
1 expedited-fo 0 0 0
2 assured-forw 0 0 0
3 network-cont 0 0 0
Queue number: Mapped forwarding classes
0 best-effort
1 expedited-forwarding
2 assured-forwarding
3 network-control
Active alarms : None
Active defects : None
MAC statistics: Receive Transmit
Total octets 65280 65280
Total packets 640 640
Unicast packets 640 640
Broadcast packets 0 0
Multicast packets 0 0
CRC/Align errors 0 0
FIFO errors 0 0
MAC control frames 0 0
MAC pause frames 0 0
Oversized frames 0
Jabber frames 0
Fragment frames 0
VLAN tagged frames 0
Code violations 0
Total errors 0 0
Filter statistics:
Input packet count 640

Lab 6–16 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC
Input packet rejects 0
Input DA rejects 0
Input SA rejects 0
Output packet count 640
Output packet pad count 0
Output packet error count 0
CAM destination filters: 0, CAM source filters: 0
Autonegotiation information:
Negotiation status: Incomplete
Packet Forwarding Engine configuration:
Destination slot: 1
CoS information:
Direction : Output
CoS transmit queue Bandwidth Buffer Priority
Limit
% bps % usec
0 best-effort 95 9500000 95 0 low none
3 network-control 5 500000 5 0 low none
Interface transmit statistics: Disabled

Logical interface ge-1/0/9.0 (Index 331) (SNMP ifIndex 640) (Generation 225)
Flags: SNMP-Traps 0x0 Encapsulation: ENET2
Traffic statistics:
Input bytes : 53760
Output bytes : 53900
Input packets: 640
Output packets: 640
Local statistics:
Input bytes : 0
Output bytes : 980
Input packets: 0
Output packets: 10
Transit statistics:
Input bytes : 53760 0 bps
Output bytes : 52920 0 bps
Input packets: 640 0 pps
Output packets: 630 0 pps
Protocol inet, MTU: 1500, Generation: 330, Route table: 0
Flags: Sendbcast-pkt-to-re
Addresses, Flags: Is-Preferred Is-Primary
Destination: 10.10.10.0/30, Local: 10.10.10.1, Broadcast: 10.10.10.3,
Generation: 272
Protocol multiservice, MTU: Unlimited, Generation: 331, Route table: 0
Flags: Is-Primary
Policer: Input: __default_arp_policer__

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–17

Junos Troubleshooting in the NOC
Question: Does this output show that this interface
is in local loopback mode?

Answer: Yes, the Device flags field shows the

flag Loop-Detected, which lets you know that
the interface is running in local loopback mode.

Question: How many input and output packets are

recorded on this interface? Is this what you
expected to find?

Answer: There are 640 input and output packets

are on the interface. Even though only 10 ping
packets sent, every one passed through the
interface 64 times, which is the default time-to-live
value for ping packets in the Junos OS.

Question: Are any errors listed in this output?

Answer: No. Under the Input errors and

Output errors sections, none of the counters
have increased.

Question: Is this interface hardware in good

condition? Why?

Answer: Yes. The interface hardware is in good

condition. No errors are present after the local
loopback testing.

Lab 6–18 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC

Part 3: Configuring Ethernet OAM

In this lab part, you will configure Ethernet OAM and monitor its operation.
Step 3.1
Enter configuration mode and navigate to the [edit protocols oam
ethernet link-fault-management] hierarchy level.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit protocols oam ethernet link-fault-management

[edit protocols oam ethernet link-fault-management]

lab@mxA-1#
Step 3.2
Configure the ge-1/0/9 interface to participate in remote loopback mode.
[edit protocols oam ethernet link-fault-management]
lab@mxA-1# set interface ge-1/0/9 remote-loopback
Step 3.3
Configure the ge-1/0/9 interface with the allow-remote-loopback option
under the negotiation-options. Activate the configuration and exit to
operational mode by issuing the commit and-quit command.
[edit protocols oam ethernet link-fault-management]
lab@mxA-1# set interface ge-1/0/9 negotiation-options allow-remote-loopback

[edit protocols oam ethernet link-fault-management]

lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>
Step 3.4
Issue the show oam ethernet link-fault-management command and
answer the following questions:
lab@mxA-1> show oam ethernet link-fault-management
Interface: ge-1/0/9
Status: Running, Discovery state: Send Any
Peer address: 80:71:1f:c3:03:69
Flags:Remote-Stable Remote-State-Valid Local-Stable 0x50
Remote loopback status: Enabled on local port, Enabled on peer port
Remote entity information:
Remote MUX action: forwarding, Remote parser action: loopback
Discovery mode: active, Unidirectional mode: unsupported
Remote loopback mode: supported, Link events: supported
Variable requests: unsupported

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–19

Junos Troubleshooting in the NOC
Question: What is the OAM status of the ge-1/0/9
interface?

Answer: As shown in the previous output, the

Status field shows that OAM is running on the
interface.

Question: What is the peer address? Where is it

being acquired from?

Answer: Although your output might vary, the peer

address in the previous output is
80:71:1f:c3:03:69. The MAC address is acquired
from the ge-1/0/9 interface.

Question: Why is the MAC address of the ge-1/0/9

interface being used as the peer address?

Answer: The ge-1/0/9 interface is detecting its own

MAC address as its peer because it has been
placed in local loopback mode in the previous lab
part.

Step 3.5
Log out of your assigned device.
lab@mxA-1> exit

mxA-1 (ttyu0)

login:

STOP Tell your instructor that you have completed this lab.

Lab 6–20 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–21

Junos Troubleshooting in the NOC

Lab 6–22 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Monitoring and Troubleshooting Ethernet Interfaces • Lab 6–23

Junos Troubleshooting in the NOC

Lab 6–24 • Monitoring and Troubleshooting Ethernet Interfaces www.juniper.net

 Lab
Isolating and Troubleshooting PFE Issues

Overview
This lab demonstrates how to examine and work with the forwarding table, configure and
examine load balancing, and troubleshoot firewall filter issues.
By completing this lab, you will perform the following tasks:
• Examine and remove entries from the forwarding table.
• Configure and verify load balancing.
• Troubleshoot firewall filter issues.

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–1

.
Junos Troubleshooting in the NOC

Part 1: Examining the Forwarding Table

In this lab part, you will examine the forwarding table and its relation to the routing
table. You will also clear entries in the forwarding table.
Note
Depending on the class, the lab equipment
used might be remote from your physical
location. The instructor will inform you as to
the nature of your access and will provide
you the details needed to access your
assigned device.

Step 1.1
Ensure that you know to which student device you have been assigned. Check with
your instructor if you are not certain. Consult the Management Network Diagram to
determine the management address of your student device.

Question: What is the management address

assigned to your station?

Answer: The answer varies and depends on your

assigned device. If you are unsure of your
assignment, ask your instructor.

Step 1.2
Access the CLI for your device using either the console, Telnet, or SSH as directed by
your instructor. Refer to the management network diagram for the IP address
associated with your station. The following example uses Telnet to access an
MX Series device using the SecureCRT program:

Lab 7–2 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC

Step 1.3
Log in to your device with the username lab and the password lab123. Note that
both the name and password are case-sensitive.
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1>
Step 1.4
Enter into configuration mode and load the device’s reset configuration by issuing
the load override jtnoc/lab7-start.config command. After the
configuration has been loaded, commit the changes before moving on to the next
step.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# load override jtnoc/lab7-start.config
load complete

[edit]
lab@mxA-1# commit
commit complete

[edit]
lab@mxA-1#

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–3

Junos Troubleshooting in the NOC
Step 1.5
Navigate to the [edit routing-options] hierarchy level and configure the
static route of 192.168.50.1/32 with a next hop of the IP address of the directly
connected interface of your team’s VR device. Activate the configuration by issuing
the commit command.
[edit]
lab@mxA-1# edit routing-options

[edit routing-options]
lab@mxA-1# set static route 192.168.50.1 next-hop local-VR-IP

[edit routing-options]
lab@mxA-1# commit
commit complete

[edit routing-options]
lab@mxA-1#
Step 1.6
Examine the routing table and the forwarding table for the 192.168.50.1 route by
issuing the run show route 192.168.50.1 and the run show route
forwarding-table destination 192.168.50.1 commands.
[edit routing-options]
lab@mxA-1# run show route 192.168.50.1

inet.0: 30 destinations, 44 routes (30 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

192.168.50.1/32 *[Static/5] 00:00:30

> to 192.168.11.2 via ge-1/0/3.141

[edit routing-options]
lab@mxA-1# run show route forwarding-table destination 192.168.50.1
Routing table: default.inet
Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
192.168.50.1/32 user 0 192.168.11.2 ucst 572 3 ge-1/0/3.141

Routing table: __master.anon__.inet

Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 540 1

Question: What information for this route matches

in both the routing and the forwarding tables?

Answer: The next hop IP address and interface

match in the output obtained from the routing and
forwarding tables.

Lab 7–4 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
Step 1.7
Issue the run clear route forwarding-table 192.168.50.1
local-VR-IP command.
Note
The route part of the clear route
forwarding table command is a
hidden command and will not
autocomplete.

[edit routing-options]
lab@mxA-1# run clear route forwarding-table 192.168.50.1 local-VR-IP
delete 192.168.50.1: gateway 192.168.11.2
Step 1.8
Issue the run show route 192.168.50.1 and the run show route
forwarding-table destination 192.168.50.1 commands.
[edit routing-options]
lab@mxA-1# run show route 192.168.50.1

inet.0: 30 destinations, 44 routes (30 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

192.168.50.1/32 *[Static/5] 00:02:11

> to 192.168.11.2 via ge-1/0/3.141

[edit routing-options]
lab@mxA-1# run show route forwarding-table destination 192.168.50.1
Routing table: default.inet
Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
default user 0 84:18:88:8e:ca:95 ucst 585 14 ge-1/0/0.141
default perm 0 rjct 36 2

Routing table: __master.anon__.inet

Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 540 1

Question: Is there anything that does not match

between the routing and forwarding table for this
route?

Answer: Yes. The next-hop interface is different for

the route.

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–5

Junos Troubleshooting in the NOC
Question: What happens to a packet that is
destined for the 192.168.50.1 IP address?

Answer: Your answer might vary, however, in the

previous output the packet would be routed through
the ge-1/0/0.141 interface.

Question: What can you do to synchronize the route

in the routing and forwarding table?

Answer: You must remove the route from the routing

table and then reapply it to the routing table. This
process also adds the new routing information to
the forwarding table.

Step 1.9
To remove the 192.168.50.1 route from the routing table deactivate the static
route from the configuration, commit the configuration, and activate the static
route. Activate the configuration and exit to operational mode by issuing the commit
and-quit command.
[edit routing-options]
lab@mxA-1# deactivate static route 192.168.50.1/32

[edit routing-options]
lab@mxA-1# commit
commit complete

[edit routing-options]
lab@mxA-1# activate static route 192.168.50.1/32

[edit routing-options]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>
Step 1.10
Issue the show route 192.168.50.1 and the show route
forwarding-table destination 192.168.50.1 commands.
lab@mxA-1> show route 192.168.50.1

inet.0: 30 destinations, 44 routes (30 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

Lab 7–6 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
192.168.50.1/32 *[Static/5] 00:00:09
> to 192.168.11.2 via ge-1/0/3.141

lab@mxA-1> show route forwarding-table destination 192.168.50.1

Routing table: default.inet
Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
192.168.50.1/32 user 0 192.168.11.2 ucst 572 3 ge-1/0/3.141

Routing table: __master.anon__.inet

Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 540 1

Question: Does the forwarding table have the

correct information for the route?

Answer: Yes. The forwarding table has the correct

information for the route, and traffic will be
forwarded out the interface listed in the routing
table.

Part 2: Configuring Load Balancing

In this lab part, you will configure and examine the effects of load balancing. First
you will load a configuration on the router that contains a preconfigured BGP group
that will be receiving routes from a single AS. Please take a few minutes to examine
the lab diagram.
Step 2.1
Enter configuration mode and issue the load override
jtnoc/lab7-p2-start.config command, then active the configuration by
issuing the commit command.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# load override jtnoc/lab7-p2-start.config
load complete

[edit]
lab@mxA-1# commit
commit complete

[edit]
lab@mxA-1#
Step 2.2
Issue the run show bgp summary command.
www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–7
Junos Troubleshooting in the NOC
[edit]
lab@mxA-1# run show bgp summary
Groups: 1 Peers: 2 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0 24 23 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn
State|#Active/Received/Accepted/Damped...
172.18.6.1 65000 5 7 0 0 37 11/
12/12/0 0/0/0/0
172.18.7.1 65000 4 6 0 0 33 12/
12/12/0 0/0/0/0

Question: Is your assigned device receiving

approximately the same amount of routes from both
its BGP peers?

Answer: Yes. The number of routes received from

both peers is almost equal.

Step 2.3
Issue the run show route 172.31.20.0/24 command.
[edit]
lab@mxA-1# run show route 172.31.20.0/24

inet.0: 24 destinations, 36 routes (24 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

172.31.20.0/24 *[BGP/170] 00:01:21, localpref 100, from 172.18.6.1

AS path: 65000 I
to 172.18.6.1 via ge-1/0/0.141
> to 172.18.7.1 via ge-1/0/1.141
[BGP/170] 00:01:17, localpref 100
AS path: 65000 I
> to 172.18.7.1 via ge-1/0/1.141

Lab 7–8 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
Question: Does the previous command alone give
enough information to determine if traffic destined
for the 172.31.20.0/24 network will be load
balanced across the two links?

Answer: No. Although the previous command does

let you know that the routing plane is set up to load
balance to the 172.31.20.0/24 network, you must
examine the forwarding plane to determine if load
balancing will occur.

Step 2.4
Issue the run show route forwarding-table destination
172.31.20.0/24 command.
[edit]
lab@mxA-1# run show route forwarding-table destination 172.31.20.0/24
Routing table: default.inet
Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
172.31.20.0/24 user 0 ulst 1048574 12
172.18.8.1 ucst 548 4 ge-1/0/1.141

Routing table: __master.anon__.inet

Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 527 1

Question: Will load balancing occur for the

172.31.20.0/24 prefix? Why?

Answer: No. In the previous output, only one next

hop and only one interface are listed for the prefix.

Question: What must you do to enable load

balancing?

Answer: You must create a policy that allows load

balancing and then you must apply that policy to
the forwarding table.

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–9

Junos Troubleshooting in the NOC
Step 2.5
Navigate to the [edit policy-options] hierarchy level. Once there, configure
the policy LOAD-BALANCE that has an action of load-balance per-packet.
[edit]
lab@mxA-1# edit policy-options

[edit policy-options]
lab@mxA-1# set policy-statement LOAD-BALANCE then load-balance per-packet
Step 2.6
Navigate to the [edit routing-options] hierarchy level and apply the newly
created policy to the forwarding table as an export policy. Activate the configuration
and exit to operational mode by issuing the commit and-quit command.
[edit policy-options]
lab@mxA-1# top edit routing-options

[edit routing-options]
lab@mxA-1# set forwarding-table export LOAD-BALANCE

[edit routing-options]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>
Step 2.7
Issue the show route forwarding-table destination
172.31.20.0/24 command.
lab@mxA-1> show route forwarding-table destination 172.31.20.0/24
Routing table: default.inet
Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
172.31.20.0/24 user 0 ulst 1048574 12
172.18.6.1 ucst 561 4 ge-1/0/0.141
172.18.7.1 ucst 562 5 ge-1/0/1.141

Routing table: __master.anon__.inet

Internet:
Destination Type RtRef Next hop Type Index NhRef Netif
default perm 0 rjct 538 1

Question: Can load balancing occur for the

172.31.20.0/24 prefix? Why?

Answer: Yes. Two next hops and two interfaces are

listed in the output for the 172.31.20.0/24 prefix.

Lab 7–10 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
Question: If equal load balancing does not occur
when traffic begins to flow over the links to the
172.31.20.0/24 network, what could you do to
equalize the traffic load across both links?

Answer: The hash-key option, located in the

forwarding-options hierarchy level, could be
changed to include Layer 3 and Layer 4 information
in the hashing algorithm.

Part 3: Troubleshooting Firewall Filters

In this lab part, you will load a configuration that has a preconfigured firewall filter.
You will examine the router to determine where the firewall is applied. Then you will
examine the protocols on the router to ensure that they are still functioning properly.
Step 3.1
Enter configuration mode and issue the load override
jtnoc/lab7-p3-start.config command. Activate the configuration by
issuing the commit command.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# load override jtnoc/lab7-p3-start.config
load complete

[edit]
lab@mxA-1# commit
commit complete

[edit]
lab@mxA-1#
Step 3.2
Issue the show firewall command.
[edit]
lab@mxA-1# show firewall
family inet {
filter PROTECT-RE {
term ssh {
from {
protocol tcp;
port ssh;
}
then accept;

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–11

Junos Troubleshooting in the NOC
}
term telnet {
from {
protocol tcp;
port telnet;
}
then accept;
}
term bgp {
from {
protocol tcp;
port bgp;
}
then accept;
}
term icmp-req {
from {
protocol icmp;
icmp-type echo-request;
}
then accept;
}
term icmp-reply {
from {
protocol icmp;
icmp-type echo-reply;
}
then accept;
}
term tracert {
from {
protocol udp;
ttl 1;
}
then {
reject;
}
}
term ospf {
from {
source-address {
224.0.0.5/32;
}
protocol ospf;
}
}
term frag {
from {
is-fragment;
protocol [ tcp udp icmp ];
}
then {
reject;
}
}

Lab 7–12 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
term vrrp {
from {
protocol vrrp;
}
then accept;
}
term discard-all {
then {
log;
discard;
}
}
}
}

Question: Does the firewall filter allow Telnet traffic?

Answer: Yes. The term telnet permits Telnet

traffic.

Question: Does the firewall filter allow FTP traffic?

Answer: No. Any FTP traffic is discarded by the

discard-all term.

Step 3.3
Issue the run show interface filters command.
[edit]
lab@mxA-1# run show interfaces filters
Interface Admin Link Proto Input Filter Output Filter
lc-0/0/0 up up
lc-0/0/0.32769 up up vpls
xe-0/0/0 up down
xe-0/0/1 up down
xe-0/0/2 up down
xe-0/0/3 up down
ge-1/0/0 up up
ge-1/0/0.141 up up inet
multiservice
ge-1/0/0.32767 up up multiservice
ge-1/0/1 up up
ge-1/0/1.141 up up inet
multiservice
ge-1/0/1.32767 up up multiservice
ge-1/0/2 up up
ge-1/0/2.141 up up inet
multiservice
www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–13
Junos Troubleshooting in the NOC
ge-1/0/2.32767 up up multiservice
ge-1/0/3 up down
ge-1/0/3.141 up down inet
multiservice
ge-1/0/3.32767 up down multiservice
ge-1/0/4 up up
ge-1/0/5 up up
ge-1/0/6 up up
ge-1/0/7 up up
ge-1/0/8 up up
ge-1/0/9 up down
ge-1/1/0 up down
ge-1/1/1 up down
ge-1/1/2 up up
ge-1/1/3 up up
ge-1/1/4 up up
ge-1/1/5 up up
ge-1/1/6 up up
ge-1/1/7 up up
ge-1/1/8 up up
ge-1/1/9 up down
cbp0 up up
demux0 up up
dsc up up
em0 up up
em0.0 up up inet
inet6
tnp
em1 up down
fxp0 up up
fxp0.0 up up inet
gre up up
ipip up up
irb up up
lo0 up up
lo0.0 up up inet PROTECT-RE
lo0.16384 up up inet
lo0.16385 up up inet
lsi up up
me0 up up
mtun up up
pimd up up
pime up up
pip0 up up
pp0 up up
tap up up

lab@mxA-1>

Lab 7–14 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
Question: Where is the PROTECT-RE firewall filter
applied?

Answer: The PROTECT-RE firewall filter is applied

to the loopback interface.

Step 3.4
Issue the run show bgp summary and the run show ospf neighbor
commands to check the dynamic routing protocols running on your router.
[edit]
lab@mxA-1# run show bgp summary
Groups: 1 Peers: 2 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0 24 23 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn
State|#Active/Received/Accepted/Damped...
172.18.6.1 65000 154 163 0 0 1:10:06 11/
12/12/0 0/0/0/0
172.18.7.1 65000 153 161 0 0 1:10:02 12/
12/12/0 0/0/0/0

[edit]
lab@mxA-1# run show ospf neighbor

[edit]
lab@mxA-1#

Question: Do any problems exist with either of the

dynamic routing protocols?

Answer: Yes. Although the BGP sessions are up, no

OSPF neighbor adjacencies exist.

Step 3.5
To make sure that OSPF is configured for the ge-1/0/2 interface, issue the
run show ospf interface command.
[edit]
lab@mxA-1# run show ospf interface
Interface State Area DR ID BDR ID Nbrs
ge-1/0/2.141 DR 0.0.0.0 192.168.31.1 0.0.0.0 0
ge-1/0/3.141 Down 0.0.0.0 0.0.0.0 0.0.0.0 0
lo0.0 DR 0.0.0.0 192.168.31.1 0.0.0.0 0

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–15

Junos Troubleshooting in the NOC
Question: Is the ge-1/0/2 interface participating in
OSPF?

Answer: Yes. The ge-1/0/2 interface is participating

in OSPF.

Step 3.6
To examine the traffic being received on the ge-1/0/2 interface, issue the
run monitor traffic interface ge-1/0/2.VLAN-ID command.
[edit]
lab@mxA-1# run monitor traffic interface ge-1/0/2.VLAN-ID
verbose output suppressed, use <detail> or <extensive> for full protocol decode
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on ge-1/0/2.141, capture size 96 bytes

Reverse lookup for 172.18.5.1 failed (check DNS reachability).

Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.

10:34:20.105955 Out IP truncated-ip - 14 bytes missing! 172.18.5.1 > 224.0.0.5:

OSPFv2, Hello, length 44
10:34:29.787916 Out IP truncated-ip - 14 bytes missing! 172.18.5.1 > 224.0.0.5:
OSPFv2, Hello, length 44
...^C
2 packets received by filter
0 packets dropped by kernel

lab@mxA-1>

Question: What OSPF activity do you see in the

output?

Answer: The only OSPF activity is OSPF hello

messages being sent out of the ge-1/0/2 interface.

Lab 7–16 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
Question: What can cause this condition with
OSPF?

Answer: Either your assigned device is discarding

the OSPF hello messages, or the remote team’s
router is not sending them.

Question: Can enabling OSPF traceoptions be

useful in troubleshooting this issue?

Answer: No. Enabling OSPF traceoptions is not

helpful because your router is not receiving OSPF
hello messages from the remote team’s router,
traceoptions would provide no useful data.

Question: What might be blocking the OSPF hello

messages on your router?

Answer: The firewall filter that was recently applied

to the loopback interface can block exception
traffic, such as OSPF hello messages, that is
destined for the routing engine.

Step 3.7
Examine the firewall filter again by issuing the show firewall command.
[edit]
lab@mxA-1# show firewall
family inet {
filter PROTECT-RE {
term ssh {
from {
protocol tcp;
port ssh;
}
then accept;
}
term telnet {
from {
protocol tcp;
port telnet;
}

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–17

Junos Troubleshooting in the NOC
then accept;
}
term bgp {
from {
protocol tcp;
port bgp;
}
then accept;
}
term icmp-req {
from {
protocol icmp;
icmp-type echo-request;
}
then accept;
}
term icmp-reply {
from {
protocol icmp;
icmp-type echo-reply;
}
then accept;
}
term tracert {
from {
protocol udp;
ttl 1;
}
then {
reject;
}
}
term ospf {
from {
source-address {
224.0.0.5/32;
}
protocol ospf;
}
}
term frag {
from {
is-fragment;
protocol [ tcp udp icmp ];
}
then {
reject;
}
}
term discard-all {
then {
discard;
}

Lab 7–18 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
}
}
}

Question: Is there at term that handles OSPF

traffic?

Answer: Yes. The ospf term is configured in the

firewall filter to handle OSPF traffic.

Step 3.8
To determine if the OSPF hello messages are being processed by the ospf term,
navigate to the [edit firewall family inet filter PROTECT-RE
term ospf] hierarchy level and configure the counter ospf-hellos under that
term. Activate the configuration by issuing the commit command.
[edit]
lab@mxA-1# edit firewall family inet filter PROTECT-RE term ospf

[edit firewall family inet filter PROTECT-RE term ospf]

lab@mxA-1# set then count ospf-hellos

[edit firewall family inet filter PROTECT-RE term ospf]

lab@mxA-1# commit
commit complete

[edit firewall family inet filter PROTECT-RE term ospf]

lab@mxA-1#
Step 3.9
To examine the recently configured counter, issue the run show firewall
filter PROTECT-RE command.
[edit firewall family inet filter PROTECT-RE term ospf]
lab@mxA-1# run show firewall filter PROTECT-RE

Filter: PROTECT-RE
Counters:
Name Bytes Packets
ospf-hellos 0 0

Question: Did the counter increment? Why?

Answer: No. The counter did not increment, which

means that the ospf term is not processing any
OSPF traffic.

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–19

Junos Troubleshooting in the NOC
Step 3.10
Examine the configuration of the PROTECT-RE firewall filter again.
[edit firewall family inet filter PROTECT-RE term ospf]
lab@mxA-1# up 2 show
filter PROTECT-RE {
term ssh {
from {
protocol tcp;
port ssh;
}
then accept;
}
term telnet {
from {
protocol tcp;
port telnet;
}
then accept;
}
term bgp {
from {
protocol tcp;
port bgp;
}
then accept;
}
term icmp-req {
from {
protocol icmp;
icmp-type echo-request;
}
then accept;
}
term icmp-reply {
from {
protocol icmp;
icmp-type echo-reply;
}
then accept;
}
term tracert {
from {
protocol udp;
ttl 1;
}
then {
reject;
}
}
term ospf {
from {
source-address {
224.0.0.5/32;
}

Lab 7–20 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
protocol ospf;
}
then count ospf-hellos;
}
term frag {
from {
is-fragment;
protocol [ tcp udp icmp ];
}
then {
reject;
}
}
term vrrp {
from {
protocol vrrp;
}
then accept;
}
term discard-all {
then {
log;
discard;
}
}
}

Question: If the ospf term is not processing OSPF

hello messages, which term is processing these
messages?

Answer: The term discard-all should be

processing the hello messages.

Step 3.11
To examine the traffic being processed by the term discard-all, configure
firewall logging for everything that is processed by this term. Activate the
configuration by issuing the commit command.
[edit firewall family inet filter PROTECT-RE term ospf]
lab@mxA-1# up 1

[edit firewall family inet filter PROTECT-RE]

lab@mxA-1# set term discard-all then log

[edit firewall family inet filter PROTECT-RE]

lab@mxA-1# commit
commit complete

[edit firewall family inet filter PROTECT-RE]

lab@mxA-1#

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–21

Junos Troubleshooting in the NOC
Step 3.12
Examine the traffic being collected by the firewall log by issuing the run show
firewall log command.

Note
Because OSPF hello messages are sent
every few seconds, you might need to issue
the run show firewall log
command a few times to see the hello
messages.

[edit firewall family inet filter PROTECT-RE]

lab@mxA-1# run show firewall log
Log :
Time Filter Action Interface Protocol Src Addr
Dest Addr
11:03:49 pfe D ge-1/0/2.141 OSPF 172.18.5.2
224.0.0.5
11:03:39 pfe D ge-1/0/2.141 OSPF 172.18.5.2
224.0.0.5
11:03:30 pfe D ge-1/0/2.141 OSPF 172.18.5.2
224.0.0.5
11:03:22 pfe D ge-1/0/2.141 OSPF 172.18.5.2
224.0.0.5
11:03:13 pfe D ge-1/0/2.141 OSPF 172.18.5.2
224.0.0.5
11:03:05 pfe D ge-1/0/2.141 OSPF 172.18.5.2
224.0.0.5
11:02:57 pfe D ge-1/0/2.141 OSPF 172.18.5.2
224.0.0.5
11:02:49 pfe D ge-1/0/2.141 OSPF 172.18.5.2
224.0.0.5
...

Question: Do you see any OSPF messages? What

does your answer indicate?

Answer: Yes. OSPF hello messages from the

destination address of 224.0.0.5 on the ge-1/0/2
interface and from the 172.18.5.1 source address
appear, which means that the OSPF hello messages
are being discarded by the term discard-all.

Lab 7–22 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
Step 3.13
To determine why the ospf term is not processing OSPF hello messages, issue the
show term ospf command.
[edit firewall family inet filter PROTECT-RE]
lab@mxA-1# show term ospf
from {
source-address {
224.0.0.5/32;
}
protocol ospf;
}
then count ospf-hellos;

Question: Is anything missing from then stanza of

the ospf term that could cause a problem?

Answer: No. Although you might think that an

accept action is required in the then stanza, the
count action by itself counts and accepts packets
that match the term.

Step 3.14
Examine the match criteria more closely by issuing the show term ospf from
command.
[edit firewall family inet filter PROTECT-RE]
lab@mxA-1# show term ospf from
source-address {
224.0.0.5/32;
}
protocol ospf;

Question: What item in the match criteria is causing

the OSPF traffic to not be evaluated by this term?

Answer: OSPF hello messages are sent to the

multicast address 224.0.0.5 and are sourced from
the interface address of the local router originating
them. You must remove the source-address
option and you must add a
destination-address value of 224.0.0.5.

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–23

Junos Troubleshooting in the NOC
Step 3.15
Remove the source-address option and add a destination address of 224.0.0.5
in the from stanza of the ospf term. Activate the configuration by issuing the
commit command.
[edit firewall family inet filter PROTECT-RE]
lab@mxA-1# delete term ospf from source-address

[edit firewall family inet filter PROTECT-RE]

lab@mxA-1# set term ospf from destination-address 224.0.0.5

[edit firewall family inet filter PROTECT-RE]

lab@mxA-1# show term ospf
from {
destination-address {
224.0.0.5/32;
}
protocol ospf;
}
then count ospf-hellos;

[edit firewall family inet filter PROTECT-RE]

lab@mxA-1# commit
commit complete

STOP Before proceeding, ensure that the remote student team within your
pod is ready to continue on to the next step.
Step 3.16
Issue the run show ospf neighbor command to determine if the OSPF
adjacency reaches the Full state.
[edit firewall family inet filter PROTECT-RE]
lab@mxA-1# run show ospf neighbor
Address Interface State ID Pri Dead
172.18.5.2 ge-1/0/2.141 ExStart 192.168.32.1 128 37

Question: Does the OSPF adjacency reach the

Full state? Why?

Answer: No. The OSPF adjacency does not reach the

Full state. The OSPF adjacency is stuck in the
ExStart state, which means a problem with
exchanging the database descriptor messages
exist.

Lab 7–24 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
Step 3.17
Issue the run show firewall filter PROTECT-RE command to examine
the counter that was originally setup to determine if OSPF traffic is being processed
by the ospf term.
[edit firewall family inet filter PROTECT-RE]
lab@mxA-1# run show firewall filter PROTECT-RE

Filter: PROTECT-RE
Counters:
Name Bytes Packets
ospf-hellos 17104 224

Question: Is the counter incrementing? Why?

Answer: Yes. The counter should be incrementing,

which means that some OSPF traffic is being
processed by the ospf term.

Step 3.18
Issue the run show firewall log command to examine the firewall log to
determine if the traffic is being discarded by the discard-all term.
[edit firewall family inet filter PROTECT-RE]
lab@mxA-1# run show firewall log
Log :
Time Filter Action Interface Protocol Src Addr
Dest Addr
11:11:46 pfe D ge-1/0/2.141 OSPF 172.18.5.2
172.18.5.1
11:11:42 pfe D ge-1/0/2.141 OSPF 172.18.5.2
172.18.5.1
...
11:11:05 pfe D ge-1/0/2.141 OSPF 172.18.5.2
172.18.5.1
---(more)---[abort]

Question: What does the firewall log reveal?

Answer: The firewall log shows OSPF traffic sent

from the remote team’s router, with a source
address of the remote team’s ge-1/0/2 interface
and a destination address of your router’s ge-1/0/2
interface, is being discarded.

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–25

Junos Troubleshooting in the NOC
Question: Which action must you take to allow this
traffic?

Answer: You must configure the ospf term to allow

this traffic.

Step 3.19
Configure the ospf term to match traffic destined to the 172.18.5.0/30 subnet.
Activate the configuration and exit to operational mode by issuing the commit
and-quit command.
[edit firewall family inet filter PROTECT-RE]
lab@mxA-1# set term ospf from destination-address 172.18.5.0/30

[edit firewall family inet filter PROTECT-RE]

lab@mxA-1# show term ospf
from {
destination-address {
224.0.0.5/32;
172.18.5.0/30;
}
protocol ospf;
}
then count ospf-hellos;

[edit firewall family inet filter PROTECT-RE]

lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>

STOP Before proceeding, ensure that the remote student team within your
pod is ready to continue on to the next step.
Step 3.20
Issue the show ospf neighbor command to determine if the OSPF adjacency
reaches the Full state.
lab@mxA-1> show ospf neighbor
Address Interface State ID Pri Dead
172.18.5.1 ge-1/0/2.143 Full 192.168.35.1 128 38

Lab 7–26 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC
Question: Does the OSPF adjacency reach the
Full state?

Answer: Yes. As shown in the previous output, the

OSPF adjacency has now reached the Full state

Step 3.21
Log out of your assigned device.
lab@mxA-1> exit

mxA-1 (ttyu0)

login:

STOP Tell your instructor that you have completed this lab.

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–27

Junos Troubleshooting in the NOC

Lab 7–28 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Isolating and Troubleshooting PFE Issues • Lab 7–29

Junos Troubleshooting in the NOC

Lab 7–30 • Isolating and Troubleshooting PFE Issues www.juniper.net

 Lab
Troubleshooting Routing Protocols

Overview
In this lab, you discover and troubleshoot OSPF adjacency and BGP session problems.
Then, after you fix the problems associated with the OSPF adjacencies and BGP sessions,
a routing loop is introduced into the network. You must troubleshoot and fix the routing
loop that becomes present in the network.
By completing this lab you will perform the following tasks:
• Troubleshoot OSPF adjacency problems.
• Troubleshoot BGP session problems.
• Troubleshoot routing loop problems.

www.juniper.net Troubleshooting Routing Protocols • Lab 8–1

.
Junos Troubleshooting in the NOC

Part 1: Modifying the Existing Configurations

In this lab part, you load and activate a predefined configuration saved on your
assigned device. This predefined configuration will introduce a number of issues in
your network. You will then troubleshoot the issues to restore network functionality.
Step 1.1
Ensure that you know to which device you have been assigned. Check with your
instructor if you are not certain. Consult the management network diagram to
determine your device’s management address.

Question: What is the management address

assigned to your device?

Answer: The answer varies and depends on your

assigned device. If you are unsure of your
assignment, ask your instructor.

Step 1.2
Access the CLI for your device using either the console, Telnet, or SSH as directed by
your instructor. Refer to the management network diagram for the IP address
associated with your station. The following example uses Telnet to access an
MX Series device using the SecureCRT program:

Lab 8–2 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Step 1.3
Log in to your device with the username lab and the password lab123. Note that
both the name and password are case-sensitive. Enter configuration mode and load
the lab8-start.config configuration from the /var/home/lab/jtnoc/
directory. Issue the commit and-quit command to activate your changes and
exit to operational mode.
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# load override jtnoc/lab8-start.config
load complete

[edit]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>

Part 2: Troubleshooting OSPF Adjacencies

In this lab part, you troubleshoot OSPF adjacency issues. First you must discover
what OSPF adjacency issues exist, then you must fix any problems that prevent
OSPF adjacencies from reaching the Full state.
Step 2.1

Note
To simulate a multi-router topology, every
physical MX Series device in the lab has
been divided into many logical systems. To
view the information for a logical system
you must use the logical-system
option followed by the logical system’s
name. For example, if you want to view the
OSPF adjacencies for logical system R2,
you enter the command show ospf
neighbor logical-system R2.

Examine the OSPF adjacency information on R2 by issuing the show ospf

neighbor logical-system R2 command.

www.juniper.net Troubleshooting Routing Protocols • Lab 8–3

Junos Troubleshooting in the NOC
lab@mxA-1> show ospf neighbor logical-system R2

lab@mxA-1>

Question: How many OSPF adjacencies should be

present in the output?

Answer: From examining the Lab 8 diagram, you

can see that R2 should have one OSPF adjacency
with R3.

Question: Where should you start in troubleshooting

the missing OSPF adjacency?

Answer: Verifying the Layers 1 through 3 of the OSI

model by issuing ping tests is a good place to start.

Step 2.2
Attempt to ping R3 from R2 by issuing the ping 10.1.1.2 logical-system
R2 rapid command.
lab@mxA-1> ping 10.1.1.2 logical-system R2 rapid
PING 10.1.1.2 (10.1.1.2): 56 data bytes
!!!!!
--- 10.1.1.2 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.439/0.688/1.660/0.486 ms

Question: What do the results of the ping test

mean?

Answer: The successful ping tests show that Layers

1 through 3 are operational.

Lab 8–4 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Step 2.3
Examine the state of the OSPF interfaces on R2 by issuing the show ospf
interface logical-system R2 command.
lab@mxA-1> show ospf interface logical-system R2
Interface State Area DR ID BDR ID Nbrs
lo0.2 DRother 0.0.0.0 0.0.0.0 0.0.0.0 0
lt-1/0/0.1 DR 0.0.0.0 10.1.100.2 0.0.0.0 0

Question: What is the current state of the lt-1/0/0.1

interface in OSPF?

Answer: The lt-1/0/0.1 currently thinks that it is the

DR, which means it is setup as a broadcast
interface. Note that the interface is participating in
OSPF area 0, as it should be, and it is not detecting
any neighbors on the broadcast segment it is
associated with.

Step 2.4
Troubleshoot the OSPF adjacency problem on R2 further by issuing the monitor
traffic interface lt-1/0/0.1 detail command
lab@mxA-1> monitor traffic interface lt-1/0/0.1 detail
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on lt-1/0/0.1, capture size 1514 bytes

Reverse lookup for 10.1.1.2 failed (check DNS reachability).

Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.

01:26:21.955223 In IP (tos 0xc0, ttlJunos Troubleshooting in the NOC 1, id 3608,

offset 0, flags [none], proto: OSPF (89), length: 76) 10.1.1.2 > 224.0.0.5:
OSPFv2, Hello, length 56 [len 44]
Router-ID 10.1.100.3, Backbone Area, Authentication Type: simple (1)
Simple text password: 12345
Options [External, LLS]
Hello Timer 3s, Dead Timer 20s, Mask 255.255.255.248, Priority 128
Designated Router 10.1.1.2
LLS: checksum: 0xfff6, length: 3
Extended Options (1), length: 4
Options: 0x00000001 [LSDB resync]
01:26:23.803879 Out IP (tos 0xc0, ttl 1, id 3637, offset 0, flags [none],
proto: OSPF (89), length: 76) 10.1.1.1 > 224.0.0.5: OSPFv2, Hello, length 56
[len 44]
Router-ID 10.1.100.2, Backbone Area, Authentication Type: simple (1)
Simple text password: 12345
Options [External, LLS]

www.juniper.net Troubleshooting Routing Protocols • Lab 8–5

Junos Troubleshooting in the NOC
Hello Timer 3s, Dead Timer 20s, Mask 255.255.255.252, Priority 128
Designated Router 10.1.1.1
LLS: checksum: 0xfff6, length: 3
Extended Options (1), length: 4
Options: 0x00000001 [LSDB resync]

Question: What are some of the reasons that can

cause any OSPF adjacency to not reach the Full
state?

Answer: Some of the reasons that can cause OSPF

adjacencies to fail to reach the Full state are
listed here:
Password value mismatch
Password type mismatch
Hello timer mismatch
Dead timer mismatch
Subnet mask mismatch
Interface type mismatch
Area number mismatch
Area type mismatch
MTU mismatch
Duplicate Router IDs

Question: From examining the previous output, why

is the OSPF adjacency failing to reach the Full
state? What must you do to fix the problem?

Answer: The previous output shows that the subnet

mask is mismatched for the link between R2 and
R3. The lt-1/0/0.2 interface on R3 is configured
with a /29 subnet mask. You must configure the
lt-1/0/0.2 interface on R3 with the correct subnet
mask of /30 to resolve the problem.

Lab 8–6 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Step 2.5
Enter configuration mode and navigate to R3’s lt-1/0/0.2 interface hierarchy by
issuing the edit logical-systems R3 interfaces lt-1/0/0.2
command.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit logical-systems R3 interfaces lt-1/0/0.2

[edit logical-systems R3 interfaces lt-1/0/0 unit 2]

lab@mxA-1# show
encapsulation ethernet;
peer-unit 1;
family inet {
mtu 1500;
address 10.1.1.2/29;
}

[edit logical-systems R3 interfaces lt-1/0/0 unit 2]

lab@mxA-1#
Step 2.6
Replace the /29 subnet mask on the lt-1/0/0.2 interface with a /30 subnet mask.
Then, commit the configuration.
[edit logical-systems R3 interfaces lt-1/0/0 unit 2]
lab@mxA-1# replace pattern /29 with /30

[edit logical-systems R3 interfaces lt-1/0/0 unit 2]

lab@mxA-1# show
encapsulation ethernet;
peer-unit 1;
family inet {
mtu 1500;
address 10.1.1.2/30;
}

[edit logical-systems R3 interfaces lt-1/0/0 unit 2]

lab@mxA-1# commit
commit complete
Step 2.7
Examine the R2 to R3 OSPF adjacency by issuing the run show ospf neighbor
logical-system R2
[edit logical-systems R3 interfaces lt-1/0/0 unit 2]
lab@mxA-1# run show ospf neighbor logical-system R2
Address Interface State ID Pri Dead
10.1.1.2 lt-1/0/0.1 Full 10.1.100.3 128 19

www.juniper.net Troubleshooting Routing Protocols • Lab 8–7

Junos Troubleshooting in the NOC
Question: What is the state of the R2 to R3 OSPF
adjacency?

Answer: The R2 to R3 OSPF adjacency should reach

the Full state. If the OSPF adjacency has not
reached the Full state yet, please wait a few
minutes. If after a few minutes the OSPF adjacency
still does not reach the Full state, please inform
your instructor.

Step 2.8
Examine the OSPF adjacency states on R3 by issuing the run show ospf
neighbor logical-system R3 command.
[edit logical-systems R3 interfaces lt-1/0/0 unit 2]
lab@mxA-1# run show ospf neighbor logical-system R3
Address Interface State ID Pri Dead
10.1.1.1 lt-1/0/0.2 Full 10.1.100.2 128 19

Question: Which OSPF adjacencies are not present

in the output?

Answer: The R3 to R5 and R3 to R6 OSPF

adjacencies are not present in the output.

Step 2.9
Test the Layer 3 connectivity by pinging R5 and R6 from R3 by issuing the run
ping 10.1.1.10 logical-system R3 rapid and run ping 10.1.1.6
logical-system R3 rapid commands.
[edit logical-systems R3 interfaces lt-1/0/0 unit 2]
lab@mxA-1# run ping 10.1.1.10 logical-system R3 rapid
PING 10.1.1.10 (10.1.1.10): 56 data bytes
!!!!!
--- 10.1.1.10 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.441/0.489/0.655/0.083 ms

[edit logical-systems R3 interfaces lt-1/0/0 unit 2]

lab@mxA-1# run ping 10.1.1.6 logical-system R3 rapid
PING 10.1.1.6 (10.1.1.6): 56 data bytes
!!!!!
--- 10.1.1.6 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.454/0.699/1.661/0.481 ms

Lab 8–8 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Question: What are the results of the ping tests?

Answer: The successful ping tests show that Layers

1 through 3 are operational.

Step 2.10
Navigate to the [edit logical-systems R3 protocols ospf] hierarchy
level. Configure the OSPF traceoptions to flag OSPF hello packets and OSPF errors in
detail. Then, configure the Junos OS to store the traceoptions in the file
ospf-trace.log. Then, commit the configuration.
[edit logical-systems R3 interfaces lt-1/0/0 unit 2]
lab@mxA-1# up 3 edit protocols ospf

[edit logical-systems R3 protocols ospf]

lab@mxA-1# set traceoptions flag hello detail

[edit logical-systems R3 protocols ospf]

lab@mxA-1# set traceoptions flag error detail

[edit logical-systems R3 protocols ospf]

lab@mxA-1# set traceoptions file ospf-trace.log

[edit logical-systems R3 protocols ospf]

lab@mxA-1# show
traceoptions {
file ospf-trace.log;
flag hello detail;
flag error detail;
}
export ospf-export;
area 0.0.0.0 {
interface lt-1/0/0.2 {
hello-interval 3;
dead-interval 20;
authentication {
simple-password "$9$J2GHqP5Qn9Af5hS"; ## SECRET-DATA
}
}
interface lt-1/0/0.3 {
hello-interval 5;
dead-interval 50;
authentication {
simple-password "$9$N1dYgaZUH.PJZ9A"; ## SECRET-DATA
}
}
interface lt-1/0/0.5 {
interface-type p2p;
hello-interval 10;

www.juniper.net Troubleshooting Routing Protocols • Lab 8–9

Junos Troubleshooting in the NOC
dead-interval 40;
authentication {
simple-password "$9$ZuUk.fTz6Ct5Tcy"; ## SECRET-DATA
}
}
interface lo0.3 {
passive;
}
}

[edit logical-systems R3 protocols ospf]

lab@mxA-1# commit
commit complete

[edit logical-systems R3 protocols ospf]

lab@mxA-1
Step 2.11
Examine the traceoptions file by issuing the run show log
R3/ospf-trace.log | match 10.1.1.10 and the run show log
R3/ospf-trace.log | find 10.1.1.6 commands.

Note
Viewing traceoptions log files is a little bit
different since you are using logical
systems. Notice that the previous
command has you place a R3/ in front of
the traceoptions log file name that you
created a few steps ago. The R3/
designation denotes that the traceoptions
file is stored within the R3 logical system.

[edit logical-systems R3 protocols ospf]

lab@mxA-1# run show log R3/ospf-trace.log | match 10.1.1.10
Jan 14 20:46:07.175178 OSPF packet ignored: authentication failure from
10.1.1.10
Jan 14 20:46:15.623090 OSPF packet ignored: authentication failure from
10.1.1.10
Jan 14 20:46:24.655029 OSPF packet ignored: authentication failure from
10.1.1.10
Jan 14 20:46:33.060970 OSPF packet ignored: authentication failure from
10.1.1.10
Jan 14 20:46:41.358808 OSPF packet ignored: authentication failure from
10.1.1.10
Jan 14 20:46:50.157730 OSPF packet ignored: authentication failure from
10.1.1.10
...

[edit logical-systems R3 protocols ospf]

lab@mxA-1# run show log R3/ospf-trace.log | find 10.1.1.6
Jan 14 21:12:05.188671 OSPF packet ignored: area mismatch (1.0.0.0) from
10.1.1.6 on intf lt-1/0/0.5 area 0.0.0.0

Lab 8–10 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Jan 14 21:12:05.188754 OSPF rcvd Hello 10.1.1.6 -> 224.0.0.5 (lt-1/0/0.5 IFL 346
area 0.0.0.0)
Jan 14 21:12:05.188778 Version 2, length 44, ID 10.1.100.6, area 1.0.0.0
Jan 14 21:12:05.188798 checksum 0x7118, authtype 1
Jan 14 21:12:05.188820 mask 255.255.255.248, hello_ivl 10, opts 0x12, prio 128
Jan 14 21:12:05.188840 dead_ivl 40, DR 10.1.1.6, BDR 0.0.0.0
...

Question: What problems do you see in the

traceoptions output?

Answer: Although your output might vary slightly

from the previous output, you should see an OSPF
authentication failure between R3 and R5. Then,
you should see an area ID mismatch between R3
and R6.

Question: What are the next steps you can take to

resolve the problems seen in the traceoptions
output?

Answer: The next steps you can take, is to set the

authentication on the OSPF adjacency between R3
and R5 to the same password. Then, you can
change the OSPF area ID on R6 to area 0 to solve
the problem between R3 and R6.

Step 2.12
Change the OSPF password that is in use between R3 and R5 by issuing the set
area 0 interface lt-1/0/0.3 authentication simple-password
jtnoc command for R3. The issue the top set logical-systems R5
protocols ospf area 0 interface lt-1/0/0.4 authentication
simple-password jtnoc command for R5.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# set area 0 interface lt-1/0/0.3 authentication simple-password jtnoc

[edit logical-systems R3 protocols ospf]

lab@mxA-1# top set logical-systems R5 protocols ospf area 0 interface lt-1/0/0.4
authentication simple-password jtnoc

www.juniper.net Troubleshooting Routing Protocols • Lab 8–11

Junos Troubleshooting in the NOC
Step 2.13
Change the OSPF area on R6 to OSPF area 0 by issuing the top rename
logical-systems R6 protocols ospf area 1.0.0.0 to area 0
command. Commit the configuration when complete.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# top rename logical-systems R6 protocols ospf area 1.0.0.0 to area 0

[edit logical-systems R3 protocols ospf]

lab@mxA-1# top show logical-systems R6 protocols ospf
export ospf-export;
area 0.0.0.0 {
interface lt-1/0/0.6 {
authentication {
simple-password "$9$axZikmfT3/CPfEc"; ## SECRET-DATA
}
}
interface lt-1/0/0.8 {
authentication {
simple-password "$9$ZsUk.fTz6Ct5Tcy"; ## SECRET-DATA
}
}
interface lo0.6 {
passive;
}
}

[edit logical-systems R3 protocols ospf]

lab@mxA-1# commit
commit complete
Step 2.14
Issue run show ospf neighbor logical-system R3 command to
examine the OSPF adjacency states of R3.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# run show ospf neighbor logical-system R3
Address Interface State ID Pri Dead
10.1.1.1 lt-1/0/0.2 Full 10.1.100.2 128 18

Question: What does the output show?

Answer: The R3 to R5 and R3 to R6 adjacency

states are still absent from the output.

Lab 8–12 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Step 2.15
Clear the OSPF traceoptions log by issuing the run clear log
R3/ospf-trace.log command. Issue the run show log R3/
ospf-trace.log | find "hello 10.1.1.10" and the run show log
R3/ospf-trace.log | find "hello 10.1.1.6" commands.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# run clear log R3/ospf-trace.log

[edit logical-systems R3 protocols ospf]

lab@mxA-1# run show log R3/ospf-trace.log | find "hello 10.1.1.10"
Jan 14 21:44:21.215309 OSPF rcvd Hello 10.1.1.10 -> 224.0.0.5 (lt-1/0/0.3 IFL
345 area 0.0.0.0)
Jan 14 21:44:21.215386 Version 2, length 44, ID 10.1.100.5, area 0.0.0.0
Jan 14 21:44:21.215409 checksum 0x0, authtype 1
Jan 14 21:44:21.215431 mask 255.255.255.252, hello_ivl 10, opts 0x12, prio 128
Jan 14 21:44:21.215452 dead_ivl 40, DR 10.1.1.10, BDR 0.0.0.0
Jan 14 21:44:21.215478 OSPF restart signaling: Received hello with LLS data from
nbr ip=10.1.1.10 id=10.1.100.5.
Jan 14 21:44:21.215503 OSPF packet ignored: hello interval mismatch 10 from
10.1.1.10 on intf lt-1/0/0.3 area 0.0.0.0
...

edit logical-systems R3 protocols ospf]

lab@mxA-1# run show log R3/ospf-trace.log | find "hello 10.1.1.6"
Jan 14 22:21:15.767039 OSPF rcvd Hello 10.1.1.6 -> 224.0.0.5 (lt-1/0/0.5 IFL 346
area 0.0.0.0)
Jan 14 22:21:15.767088 Version 2, length 44, ID 10.1.100.3, area 0.0.0.0
Jan 14 22:21:15.767109 checksum 0x0, authtype 1
Jan 14 22:21:15.767131 mask 255.255.255.252, hello_ivl 10, opts 0x12, prio 128
Jan 14 22:21:15.767152 dead_ivl 40, DR 10.1.1.6, BDR 0.0.0.0
Jan 14 22:21:15.767177 OSPF restart signaling: Received hello with LLS data from
nbr ip=10.1.1.6 id=10.1.100.3.
Jan 14 22:21:15.767201 OSPF packet ignored: our router ID received from 10.1.1.6
on intf lt-1/0/0.5 area 0.0.0.0
...

Question: What appears to be the current problems

with the OSPF adjacencies?

Answer: The problem with the R3 to R5 OSPF

adjacency is a hello interval mismatch. The problem
with the R3 to R6 OSPF adjacency is a duplicate
router ID.

www.juniper.net Troubleshooting Routing Protocols • Lab 8–13

Junos Troubleshooting in the NOC
Question: What are the hello and dead interval
settings that are being received on R3 from R5?

Answer: The current hello and dead interval settings

that R3 is receiving from R5 are 10 and 40,
respectively.

Question: What should the router ID of R6 be?

Answer: The router ID of R6 should be 10.1.100.6.

Step 2.16
Examine the current hello and dead interval settings for R3 by issuing the
run show ospf interface lt-1/0/0.3 detail logical-system R3
| match hello command.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# run show ospf interface lt-1/0/0.3 detail logical-system R3 | match
hello
Hello: 5, Dead: 50, ReXmit: 5, Not Stub

Question: How can you change R3’s configuration

to fix the R3 to R5 OSPF adjacency problem?

Answer: Changing the hello interval to 10 seconds

and the dead interval to 40 seconds fixes the
current R3 to R5 OSPF adjacency problem.

Step 2.17
Make sure your current configuration hierarchy position is
[edit logical-systems R3 protocols ospf], then change the hello
interval to 10 seconds and dead interval to 40 seconds on R3’s lt-1/0/0.3 interface.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# set area 0 interface lt-1/0/0.3 hello-interval 10 dead-interval 40

Lab 8–14 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC

Note
Alternatively, you can simply delete the
hello and dead interval settings for the R3
to R5 OSPF adjacency. The default hello
and dead interval for OSPF are 10 and 40
seconds, respectively.

Step 2.18
Change the router ID for R6 by issuing the top rename logical-systems R6
interfaces lo0.6 family inet address 10.1.100.3/32 to
address 10.1.100.6 command. Then, commit the configuration.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# top rename logical-systems R6 interfaces lo0.6 family inet address
10.1.100.3/32 to address 10.1.100.6

[edit logical-systems R3 protocols ospf]

lab@mxA-1# top show logical-systems R6 interfaces lo0.6
family inet {
address 10.1.100.6/32;
}

[edit logical-systems R3 protocols ospf]

lab@mxA-1# commit
commit complete
Step 2.19
Examine the OSPF adjacency states on R3 by issuing the run show ospf
neighbor logical-system R3 command.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# run show ospf neighbor logical-system R3
Address Interface State ID Pri Dead
10.1.1.1 lt-1/0/0.2 Full 10.1.100.2 128 17
10.1.1.10 lt-1/0/0.3 Full 10.1.100.5 128 35

Question: What are the states of the OSPF

adjacencies on R3?

Answer: Only the R3 to R6 OSPF adjacency is

missing and not in the Full state.

Step 2.20
Issue the run clear log R3/ospf-trace.log to clear old log information.
Then, examine the traceoptions file by issuing the run show log
R3/ospf-trace.log | find "hello 10.1.1.6" command.

www.juniper.net Troubleshooting Routing Protocols • Lab 8–15

Junos Troubleshooting in the NOC
[edit logical-systems R3 protocols ospf]
lab@mxA-1# run clear log R3/ospf-trace.log

[edit logical-systems R3 protocols ospf]

lab@mxA-1# run show log R3/ospf-trace.log | find "hello 10.1.1.6"
Jan 14 23:43:30.585875 OSPF rcvd Hello 10.1.1.6 -> 224.0.0.5 (lt-1/0/0.5 IFL 346
area 0.0.0.0)
Jan 14 23:43:30.585911 Version 2, length 48, ID 10.1.100.6, area 0.0.0.0
Jan 14 23:43:30.585932 checksum 0x0, authtype 1
Jan 14 23:43:30.585959 mask 255.255.255.252, hello_ivl 10, opts 0x12, prio 128
Jan 14 23:43:30.585982 dead_ivl 40, DR 10.1.1.6, BDR 0.0.0.0
Jan 14 23:43:30.586008 OSPF restart signaling: Received hello with LLS data from
nbr ip=10.1.1.6 id=10.1.100.6.
Jan 14 23:43:30.586034 OSPF packet ignored: configuration mismatch from 10.1.1.6
on intf lt-1/0/0.5 area 0.0.0.0

Question: What can you determine from the

traceoptions output?

Answer: The traceoptions output tells you that a

configuration mismatch is present, but it does not
tell you what the configuration mismatch is.

Step 2.21
Examine the OSPF configuration messages that R3 and R6 are exchanging by
issuing the run monitor traffic interface lt-1/0/0.5 detail
command.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# run monitor traffic interface lt-1/0/0.5 detail
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on lt-1/0/0.5, capture size 1514 bytes

Reverse lookup for 10.1.1.6 failed (check DNS reachability).

Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.

23:38:18.302492 In IP (tos 0xc0, ttl 1, id 17508, offset 0, flags [none],

proto: OSPF (89), length: 80) 10.1.1.6 > 224.0.0.5: OSPFv2, Hello, length 60
[len 48]
Router-ID 10.1.100.6, Backbone Area, Authentication Type: simple (1)
Simple text password: 12345
Options [External, LLS]
Hello Timer 10s, Dead Timer 40s, Mask 255.255.255.252, Priority 128
Designated Router 10.1.1.6
Neighbor List:
10.1.100.3
LLS: checksum: 0xfff6, length: 3

Lab 8–16 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Extended Options (1), length: 4
Options: 0x00000001 [LSDB resync]
23:38:22.682130 Out IP (tos 0xc0, ttl 1, id 17602, offset 0, flags [none],
proto: OSPF (89), length: 76) 10.1.1.5 > 224.0.0.5: OSPFv2, Hello, length 56
[len 44]
Router-ID 10.1.100.3, Backbone Area, Authentication Type: simple (1)
Simple text password: 12345
Options [External, LLS]
Hello Timer 10s, Dead Timer 40s, Mask 255.255.255.252, Priority 128
LLS: checksum: 0xfff6, length: 3
Extended Options (1), length: 4
Options: 0x00000001 [LSDB resync]
^C
2 packets received by filter
0 packets dropped by kernel

Question: Which OSPF configuration parameters

between R3 and R6 are mismatched?

Answer: Although it might be difficult to determine

which configuration parameters are mismatched in
the previous output, you might notice that R6 is
sending configuration parameters that contain DR
information, whereas R3 is not. This bit of
information tells you that R6’s lt-1/0/0.6 interface
is configured in the default broadcast mode,
whereas the R3’s lt-1/0/0.5 interface is configured
in point-to-point mode, or point-to-multipoint mode.
These type of configuration parameters are
incompatible.

Step 2.22
Ensure your current configuration hierarchy position is [edit
logical-systems R3 protocols ospf], then place R3’s
lt-1/0/0.5 interface in the default broadcast mode by issuing the delete area 0
interface lt-1/0/0.5 interface-type command. Then, commit the
configuration.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# delete area 0 interface lt-1/0/0.5 interface-type

[edit logical-systems R3 protocols ospf]

lab@mxA-1# show area 0 interface lt-1/0/0.5
hello-interval 10;
dead-interval 40;
authentication {
simple-password "$9$ZuUk.fTz6Ct5Tcy"; ## SECRET-DATA
}

www.juniper.net Troubleshooting Routing Protocols • Lab 8–17

Junos Troubleshooting in the NOC

[edit logical-systems R3 protocols ospf]

lab@mxA-1# commit
commit complete
Step 2.23
Examine the OSPF adjacency states on R3 by issuing the run show ospf
neighbor logical-system R3 command.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# run show ospf neighbor logical-system R3
Address Interface State ID Pri Dead
10.1.1.1 lt-1/0/0.2 Full 10.1.100.2 128 19
10.1.1.10 lt-1/0/0.3 Full 10.1.100.5 128 31
10.1.1.6 lt-1/0/0.5 Full 10.1.100.6 128 36

Question: What are the states of the OSPF

adjacencies on R3?

Answer: All of the necessary OSPF adjacencies are

now in the Full state.

Step 2.24
Now that the OSPF adjacency problems for R2 and R3 have been resolved. Examine
the OSPF adjacency between R5 and R6 by issuing the run show ospf
neighbor logical-system R5 command.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# run show ospf neighbor logical-system R5
Address Interface State ID Pri Dead
10.1.1.9 lt-1/0/0.4 Full 10.1.100.3 128 36
10.1.1.14 lt-1/0/0.7 Full 10.1.100.6 128 31

Question: What are the states of the OSPF

adjacencies on R5?

Answer: All of the necessary OSPF adjacencies are

now in the Full state.

Lab 8–18 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Step 2.25
Make sure your current configuration hierarchy position is [edit
logical-systems R3 protocols ospf], then, remove the traceoptions for
R3. Next, issue the commit and-quit command to activate the configuration
and exit to operational mode.
[edit logical-systems R3 protocols ospf]
lab@mxA-1# delete traceoptions

[edit logical-systems R3 protocols ospf]

lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>

Part 3: Troubleshooting BGP Sessions

In this lab part, you troubleshoot and resolve problems with BGP sessions.
Step 3.1
In your routing domain, R2, R3, and R4 should have full mesh IBGP sessions
between each other. Then, R2 should have an EBGP session with ISP-1, and R3
should have an EBGP session with ISP-2.
Examine the state of the BGP sessions on R2 by issuing the show bgp summary
logical-system R2 command.
lab@mxA-1> show bgp summary logical-system R2
Groups: 2 Peers: 3 Down peers: 2
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn
State|#Active/Received/Accepted/Damped...
10.1.100.3 11111 889 888 0 7 1:13:36 0/
0/0/0 0/0/0/0
10.1.100.4 11111 871 866 0 6 8:00
Active
172.16.17.1 54321 0 3702 0 0 3d 4:24:57
Active

www.juniper.net Troubleshooting Routing Protocols • Lab 8–19

Junos Troubleshooting in the NOC
Question: What are the states of the BGP sessions
on R2?

Answer: The BGP session from R2 to R3 is in the

Established state. The BGP sessions from R2 to
R4 and R2 to ISP-1 are in the Active state.

Question: What does that Active state mean for

the BGP sessions from R2 to R4 and R2 to ISP-1?

Answer: The Active state for these sessions

means that R2 is actively attempting to form a BGP
session with those BGP neighbors. However, the
fact that R2 has not moved from the Active state
to the Established state for these sessions
shows that there is a problem establishing the BGP
sessions.

Question: What address is R4 supposed to be using

to form an IBGP session with R2?

Answer: R4 is supposed to be using the 10.1.100.4

address to form an IBGP session with R2.

Step 3.2
To begin troubleshooting the R2 to R4 IBGP session enter configuration mode. Next,
navigate to the [edit logical-systems R4 protocols bgp] hierarchy
level. Then, configure the BGP traceoptions to flag the open message and place the
information from the traceoptions in the bgp-trace.log file. Commit the
configuration when complete.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit logical-systems R4 protocols bgp

Lab 8–20 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
[edit logical-systems R4 protocols bgp]
lab@mxA-1# set traceoptions file bgp-trace.log

[edit logical-systems R4 protocols bgp]

lab@mxA-1# set traceoptions flag open

[edit logical-systems R4 protocols bgp]

lab@mxA-1# commit
commit complete

[edit logical-systems R4 protocols bgp]

lab@mxA-1#
Step 3.3
Wait a few minutes for the traceoptions file to fill with information, then when a few
minutes pass, issue the run show log R4/bgp-trace.log command.
[edit logical-systems R4 protocols bgp]
lab@mxA-1# run show log R4/bgp-trace.log
Jan 15 03:14:01 trace_on: Tracing to "/var/log/R4/bgp-trace.log" started
Jan 15 03:14:32.750784 advertising graceful restart receiving-speaker-only
capability to neighbor 10.1.100.2 (Internal AS 11111)
Jan 15 03:14:32.750973 bgp_send: sending 59 bytes to 10.1.100.2 (Internal AS
11111)
Jan 15 03:14:32.751001
Jan 15 03:14:32.751001 BGP SEND 172.16.1.6+59694 -> 10.1.100.2+179
Jan 15 03:14:32.751036 BGP SEND message type 1 (Open) length 59
Jan 15 03:14:32.751086 BGP SEND version 4 as 11111 holdtime 90 id 172.16.1.6
parmlen 30
Jan 15 03:14:32.751110 BGP SEND MP capability AFI=1, SAFI=1
Jan 15 03:14:32.751130 BGP SEND Refresh capability, code=128
Jan 15 03:14:32.751149 BGP SEND Refresh capability, code=2
Jan 15 03:14:32.751170 BGP SEND Restart capability, code=64, time=120, flags=
Jan 15 03:14:32.751192 BGP SEND 4 Byte AS-Path capability (65), as_num 11111
Jan 15 03:14:32.752083 bgp_recv: read from peer 10.1.100.2 (Internal AS 11111)
failed: Connection reset by peer

Question: Why is the BGP session between R2 and

R4 failing to establish?

Answer: Although it might be difficult to determine

from the output. You might notice that R4 is sending
BGP messages to R2 from the 172.16.1.6 address.
R2 is configured to receive BGP messages from R4
on the 10.1.100.4 address.

www.juniper.net Troubleshooting Routing Protocols • Lab 8–21

Junos Troubleshooting in the NOC
Step 3.4
Ensure that you are at the [edit logical-systems R4 protocols bgp]
hierarchy level. Then, examine R4’s BGP configuration.
[edit logical-systems R4 protocols bgp]
lab@mxA-1# show
traceoptions {
file bgp-trace.log;
flag open;
}
group IBGP {
type internal;
neighbor 10.1.100.2;
neighbor 10.1.100.3;
}

Question: Which configuration statement must you

add to the IBGP group to tell R4 to source the BGP
messages from the 10.1.100.4 address?

Answer: You must add the local-address

10.1.100.4 statement to the IBGP group to tell
R4 to source BGP messages from the 10.1.100.4
address.

Step 3.5
Configure the IBGP group on R4 to use the 10.1.100.4 address as the source
address to send BGP messages from. Then, commit the configuration.
[edit logical-systems R4 protocols bgp]
lab@mxA-1# set group IBGP local-address 10.1.100.4

[edit logical-systems R4 protocols bgp]

lab@mxA-1# commit
commit complete

Lab 8–22 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Step 3.6
Wait a few minutes for the BGP sessions to attempt to establish again, then issue
the run show bgp summary logical-system R2 command to examine the
R2 to R4 BGP session on R2.
[edit logical-systems R4 protocols bgp]
lab@mxA-1# run show bgp summary logical-system R2
Groups: 2 Peers: 3 Down peers: 2
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn
State|#Active/Received/Accepted/Damped...
10.1.100.3 11111 1034 1035 0 7 2:19:06 0/
0/0/0 0/0/0/0
10.1.100.4 11111 871 866 0 6 1:13:30
Active
172.16.17.1 54321 0 3756 0 0 3d 5:30:27
Active

Question: Which state is the R2 to R4 BGP session

in?

Answer: The R2 to R4 BGP session has remained in

the Active state.

Step 3.7
Examine the R4 to R2 BGP session on R4 by issuing the run show bgp
summary logical-system R4 command.
[edit logical-systems R4 protocols bgp group IBGP]
lab@mxA-1# run show bgp summary logical-system R4
Groups: 1 Peers: 2 Down peers: 2
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn
State|#Active/Received/Accepted/Damped...
10.1.100.2 11111 0 0 0 0 2:03 Idle
10.1.100.3 11111 0 0 0 0 2:03 Idle

www.juniper.net Troubleshooting Routing Protocols • Lab 8–23

Junos Troubleshooting in the NOC
Question: What does the BGP sessions being in the
Idle state mean?

Answer: A BGP session is in the Idle state if the

router cannot begin to initialize the BGP session.
Typically, this state occurs when BGP cannot route
the BGP messages to the peer.

Step 3.8
Examine R4’s BGP traceoptions file by issuing the run show log
R4/bgp-trace.log | last 2 command.
[edit logical-systems R4 protocols bgp group IBGP]
lab@mxA-1# run show log R4/bgp-trace.log | last 2
Jan 15 04:22:13.831815 bgp_peer_init: BGP peer 10.1.100.2 (Internal AS 11111)
local address 10.1.100.4 not found. Leaving peer idled
Jan 15 04:22:13.833662 bgp_peer_init: BGP peer 10.1.100.3 (Internal AS 11111)
local address 10.1.100.4 not found. Leaving peer idled

Question: What can you determine from the

traceoptions file?

Answer: The traceoptions file shows that the

10.1.100.4 address is not found on R4. Because
the address cannot be found, R4 cannot source
BGP packets from that address.

Step 3.9
Examine the loopback interface on R4 by issuing the run show interfaces
terse lo0.4 command.
[edit logical-systems R4 protocols bgp group IBGP]
lab@mxA-1# run show interfaces terse lo0.4
error: interface lo0.4 not found

Lab 8–24 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Question: What can you determine from the
interface output?

Answer: The lo0.4 interface is not configured.

Step 3.10
Navigate to the [edit logical-systems R4 interfaces] hierarchy level.
Next, configure the loopback interface with the set lo0.4 family inet
address 10.1.100.4 command. Then, commit the configuration.
[edit logical-systems R4 protocols bgp group IBGP]
lab@mxA-1# up 2 edit interfaces

[edit logical-systems R4 interfaces]

lab@mxA-1# set lo0.4 family inet address 10.1.100.4

[edit logical-systems R4 interfaces]

lab@mxA-1# commit
commit complete

[edit logical-systems R4 interfaces]

lab@mxA-1#
Step 3.11
Wait a few minutes for the R2 to R4 BGP session to attempt to establish again. Then,
issue the run show bgp summary logical-system R2 command to
examine the R2 to R4 BGP session.
[edit logical-systems R4 interfaces]
lab@mxA-1# run show bgp summary logical-system R2
Groups: 2 Peers: 3 Down peers: 1
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn
State|#Active/Received/Accepted/Damped...
10.1.100.3 11111 1168 1167 0 7 3:18:39 0/
0/0/0 0/0/0/0
10.1.100.4 11111 7 6 0 6 1:58 0/
0/0/0 0/0/0/0
172.16.17.1 54321 0 3804 0 0 3d 6:30:00
Active

www.juniper.net Troubleshooting Routing Protocols • Lab 8–25

Junos Troubleshooting in the NOC
Question: What is the state of the R2 to R4 BGP
session?

Answer: The R2 to R4 BGP session is now in the

Established state.

Step 3.12
Ensure that you are at the [edit logical-systems R4 interfaces]
hierarchy level. Next, remove R4’s BGP traceoptions by issuing the up 1 delete
protocols bgp traceoptions command. Then, commit the configuration.
[edit logical-systems R4 interfaces]
lab@mxA-1# up 1 delete protocols bgp traceoptions

[edit logical-systems R4 interfaces]

lab@mxA-1# commit
commit complete
Step 3.13
Examine the BGP session between R2 and ISP-1 by issuing the run show bgp
neighbor 172.16.17.1 logical-system R2 command.
[edit logical-systems R4 interfaces]
lab@mxA-1# run show bgp neighbor 172.16.17.1 logical-system R2
Peer: 172.16.17.1 AS 54321 Local: 172.16.17.2 AS 11111
Type: External State: Active Flags: <ImportEval>
Last State: Idle Last Event: Start
Last Error: Open Message Error
Export: [ agg-rts ]
Options: <Preference PeerAS Refresh>
Holdtime: 90 Preference: 165
Number of flaps: 0
Error: 'Open Message Error' Sent: 2257 Recv: 0

Question: What is the current state of the BGP

session?

Answer: The BGP session is in the Active state.

Lab 8–26 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Question: What does the error message in the
output tell you?

Answer: The error message of Open Message

Error lets you know that there is a error contained
in the BGP open message.

Step 3.14
Navigate to the [edit logical-systems R2 protocols bgp group
isp-1] hierarchy level. Next, configure traceoptions and flag the BGP open
messages in detail. Store the information in the bgp-isp1-trace.log file.
Finally, commit the configuration.
[edit logical-systems R4 interfaces]
lab@mxA-1# up 2

[edit logical-systems]
lab@mxA-1# edit R2 protocols bgp group isp-1

[edit logical-systems R2 protocols bgp group isp-1]

lab@mxA-1# set traceoptions flag open detail

[edit logical-systems R2 protocols bgp group isp-1]

lab@mxA-1# set traceoptions file bgp-isp1-trace.log

[edit logical-systems R2 protocols bgp group isp-1]

lab@mxA-1# commit
commit complete

[edit logical-systems R2 protocols bgp group isp-1]

lab@mxA-1#
Step 3.15
Wait a few minutes for the traceoptions file to fill with information, then issue the
run show log R2/bgp-isp1-trace.log command.
[edit logical-systems R2 protocols bgp]
lab@mxA-1# run show log R2/bgp-isp1-trace.log
Jan 15 19:20:20 trace_on: Tracing to "/var/log/R2/bgp-isp1-trace.log" started
Jan 15 19:20:30.399094 advertising graceful restart receiving-speaker-only
capability to neighbor 172.16.17.1 (External AS 54321)
Jan 15 19:20:30.399189 bgp_send: sending 59 bytes to 172.16.17.1 (External AS
54321)
Jan 15 19:20:30.399214
Jan 15 19:20:30.399214 BGP SEND 172.16.17.2+49890 -> 172.16.17.1+179
Jan 15 19:20:30.399249 BGP SEND message type 1 (Open) length 59
Jan 15 19:20:30.399310 BGP SEND version 4 as 11111 holdtime 90 id 10.1.100.2
parmlen 30
Jan 15 19:20:30.399334 BGP SEND MP capability AFI=1, SAFI=1

www.juniper.net Troubleshooting Routing Protocols • Lab 8–27

Junos Troubleshooting in the NOC
Jan 15 19:20:30.399353 BGP SEND Refresh capability, code=128
Jan 15 19:20:30.399372 BGP SEND Refresh capability, code=2
Jan 15 19:20:30.399394 BGP SEND Restart capability, code=64, time=120, flags=
Jan 15 19:20:30.399416 BGP SEND 4 Byte AS-Path capability (65), as_num 11111
Jan 15 19:20:30.400172
Jan 15 19:20:30.400172 BGP RECV 172.16.17.1+179 -> 172.16.17.2+49890
Jan 15 19:20:30.400214 BGP RECV message type 1 (Open) length 59
Jan 15 19:20:30.400238 BGP RECV version 4 as 12345 holdtime 90 id 10.10.10.100
parmlen 30
Jan 15 19:20:30.400258 BGP RECV MP capability AFI=1, SAFI=1
Jan 15 19:20:30.400278 BGP RECV Refresh capability, code=128
Jan 15 19:20:30.400297 BGP RECV Refresh capability, code=2
Jan 15 19:20:30.400318 BGP RECV Restart capability, code=64, time=120, flags=
Jan 15 19:20:30.400340 BGP RECV 4 Byte AS-Path capability (65), as_num 12345
Jan 15 19:20:30.400429 bgp_process_open:2824: NOTIFICATION sent to 172.16.17.1
(External AS 54321): code 2 (Open Message Error) subcode 2 (bad peer AS
number), Reason: peer 172.16.17.1 (External AS 54321) claims 12345, 54321
configured
Jan 15 19:20:30.400739 bgp_send: sending 21 bytes to 172.16.17.1 (External AS
54321)
Jan 15 19:20:30.400769
Jan 15 19:20:30.400769 BGP SEND 172.16.17.2+49890 -> 172.16.17.1+179
Jan 15 19:20:30.400803 BGP SEND message type 3 (Notification) length 21
Jan 15 19:20:30.400824 BGP SEND Notification code 2 (Open Message Error) subcode
2 (bad peer AS number)

Question: What is the problem with the R2 to ISP-1

BGP session?

Answer: R2 current has the wrong peer AS number

configured. The traceoptions log show that R2 has
AS 54321 configured, but ISP-2 is claiming that it is
AS 12345.

Step 3.16
Ensure that you are at the [edit logical-systems R2 protocols bgp
group isp-1] hierarchy level. Next, remove the recently configured traceoptions.
Then, change the peer-as value to 12345 for the isp-1 BGP group. Next,
commit the configuration.
[edit logical-systems R2 protocols bgp group isp-1]
lab@mxA-1# delete traceoptions

[edit logical-systems R2 protocols bgp group isp-1]

lab@mxA-1# set peer-as 12345

Lab 8–28 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
[edit logical-systems R2 protocols bgp group isp-1]
lab@mxA-1# commit
commit complete
Step 3.17
Wait a few minutes for the R2 to ISP-1 BGP session to attempt to establish again,
then issue the run show bgp neighbor 172.16.17.1 logical-system
R2 command.
[edit logical-systems R2 protocols bgp group isp-1]
lab@mxA-1# run show bgp neighbor 172.16.17.1 logical-system R2
Peer: 172.16.17.1+179 AS 12345 Local: 172.16.17.2+51971 AS 11111
Type: External State: Established Flags: <Sync>
Last State: OpenConfirm Last Event: RecvKeepAlive
Last Error: None
Export: [ agg-rts ]
Options: <Preference PeerAS Refresh>
Holdtime: 90 Preference: 165
Number of flaps: 0
Peer ID: 10.10.10.100 Local ID: 10.1.100.2 Active Holdtime: 90
Keepalive Interval: 30 Group index: 0 Peer index: 0
BFD: disabled, down
Local Interface: ge-1/0/0.121
NLRI for restart configured on peer: inet-unicast
NLRI advertised by peer: inet-unicast
NLRI for this session: inet-unicast
Peer supports Refresh capability (2)
Stale routes from peer are kept for: 300
Peer does not support Restarter functionality
NLRI that restart is negotiated for: inet-unicast
NLRI of received end-of-rib markers: inet-unicast
NLRI of all end-of-rib markers sent: inet-unicast
Peer supports 4 byte AS extension (peer-as 12345)
Peer does not support Addpath
Table inet.0 Bit: 10001
RIB State: BGP restart is complete
Send state: in sync
Active prefixes: 0
Received prefixes: 5
Accepted prefixes: 5
Suppressed due to damping: 0
Advertised prefixes: 2
Last traffic (seconds): Received 0 Sent 5 Checked 86
Input messages: Total 6 Updates 2 Refreshes 0 Octets 157
Output messages: Total 7 Updates 1 Refreshes 0 Octets 238
Output Queue[0]: 0

www.juniper.net Troubleshooting Routing Protocols • Lab 8–29

Junos Troubleshooting in the NOC
Question: What is the state of the R2 to ISP-1 BGP
session?

Answer: The R2 to ISP-1 BGP session is in the

Established state.

Step 3.18
Examine the R3 to ISP-2 BGP session by issuing the run show bgp neighbor
10.0.1.1 logical-system R3 command.
[edit logical-systems R2 protocols bgp group isp-1]
lab@mxA-1# run show bgp neighbor 10.0.1.1 logical-system R3
Peer: 10.0.1.1 AS 54321 Local: 10.0.1.2 AS 11111
Type: External State: Active Flags: <ImportEval>
Last State: Idle Last Event: Start
Last Error: Cease
Export: [ agg-rts ]
Options: <Preference PeerAS Refresh>
Holdtime: 90 Preference: 170
Number of flaps: 4
Last flap event: RecvNotify
Error: 'Cease' Sent: 2 Recv: 2

Question: What is the current state of the R3 to

ISP-2 BGP session?

Answer: The previous output shows that the R3 to

ISP-2 BGP session is in the Active state.
Depending on when you issue the previous
command, the BGP session might be in the
Connect state.

Question: Which options are present for the R3 to

ISP-2 BGP session?

Answer: The options present for the R3 to ISP-2

BGP session are Preference, PeerAS, and
Refresh.

Lab 8–30 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Step 3.19
Navigate to the [edit logical-systems R3 protocols bgp group
isp-2] hierarchy level.Next, configure traceoptions and flag the BGP open
messages in detail. Store the information in the bgp-isp2-trace.log file.
Finally, commit the configuration.
[edit logical-systems R2 protocols bgp group isp-1]
lab@mxA-1# top edit logical-systems R3 protocols bgp group isp-2

[edit logical-systems R3 protocols bgp group isp-2]

lab@mxA-1# set traceoptions flag open detail

[edit logical-systems R3 protocols bgp group isp-2]

lab@mxA-1# set traceoptions file bgp-isp2-trace.log

[edit logical-systems R3 protocols bgp group isp-2]

lab@mxA-1# commit
commit complete

[edit logical-systems R3 protocols bgp group isp-2]

lab@mxA-1#
Step 3.20
Wait a few minutes for the traceoptions file to fill with information, then issue the
run show log R3/bgp-isp2-trace.log command.
[edit logical-systems R3 protocols bgp group isp-2]
lab@mxA-1# run show log R3/bgp-isp2-trace.log
Jan 15 19:49:42 trace_on: Tracing to "/var/log/R3/bgp-isp2-trace.log" started
Jan 15 19:50:37.306509 bgp_connect_complete: error connecting to 10.0.1.1
(External AS 54321): Socket is not connected
Jan 15 19:53:05.308024 bgp_connect_complete: error connecting to 10.0.1.1
(External AS 54321): Socket is not connected

Question: What can you determine from the

traceoptions log?

Answer: The traceoptions log is not very helpful. It

only contains information about an error when
connecting to ISP-2, but it does not specify what the
error is.

Step 3.21
Ensure that you are at the [edit logical-systems R3 protocols bgp
group isp-2] hierarchy level. Then, remove the recently configured traceoptions
and commit the configuration.
[edit logical-systems R3 protocols bgp group isp-2]
lab@mxA-1# delete traceoptions

www.juniper.net Troubleshooting Routing Protocols • Lab 8–31

Junos Troubleshooting in the NOC

[edit logical-systems R3 protocols bgp group isp-2]

lab@mxA-1# commit
commit complete
Step 3.22
Examine the BGP messages that R3 is sending and receiving by issuing the
run monitor traffic interface ge-1/0/1.VLAN-ID detail
command. (VLAN-ID is the VLAN ID associated with the ge-1/0/1 interface.)
[edit logical-systems R3 protocols bgp group isp-2]
lab@mxA-1# run monitor traffic interface ge-1/0/1.VLAN-ID detail
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on ge-1/0/1.121, capture size 1514 bytes

Reverse lookup for 10.0.1.2 failed (check DNS reachability).

Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.

20:07:00.316625 Out IP (tos 0xc0, ttl 1, id 39056, offset 0, flags [none],

proto: TCP (6), length: 48) 10.0.1.2.50666 > 10.0.1.1.bgp: S
2826701364:2826701364(0) win 16384 <mss 1460,sackOK,eol>
20:07:12.516502 Out IP (tos 0xc0, ttl 1, id 39304, offset 0, flags [none],
proto: TCP (6), length: 48) 10.0.1.2.50666 > 10.0.1.1.bgp: S
2826701364:2826701364(0) win 16384 <mss 1460,sackOK,eol>
20:07:31.089157 In IP (tos 0xc0, ttl 1, id 34841, offset 0, flags [none],
proto: TCP (6), length: 80) 10.0.1.1.55044 > 10.0.1.2.bgp: S
2192098672:2192098672(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp
405089554 0,nop,nop,md5 6331e69a27cceab93c7082a8bdc7ae47>
^C
3 packets received by filter
0 packets dropped by kernel

Question: What can you determine from the output?

Answer: Although it might be difficult to determine

the problem. If you look closely at the BGP
messages that R3 is receiving from ISP-2, you might
notice that MD5 authentication is in use. Whereas
the BGP messages that R3 is sending to ISP-2 are
not using any form of authentication.

Lab 8–32 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Question: Can you determine the password that
ISP-2 is using for the R3 to ISP-2 BGP session?
Why?

Answer: No. The password is encrypted using MD5.

In a real network situation like this, you would have
to contact the administrators of ISP-2 to acquire the
password.

Step 3.23
Ensure you are at the [edit logical-systems R3 protocols bgp group
isp-2] hierarchy level. Then, issue the set authentication-key
jtnoc12345 command. Next, issue the commit and-quit command.
[edit logical-systems R3 protocols bgp group isp-2]
lab@mxA-1# set authentication-key jtnoc12345

[edit logical-systems R3 protocols bgp group isp-2]

lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>
Step 3.24
Wait a few minutes for the R3 to ISP-2 BGP session to attempt to establish again,
then issue the show bgp neighbor 10.0.1.1 logical-system R3
command.
lab@mxA-1> show bgp neighbor 10.0.1.1 logical-system R3
Peer: 10.0.1.1+55243 AS 54321 Local: 10.0.1.2+179 AS 11111
Type: External State: Established Flags: <Sync>
Last State: OpenConfirm Last Event: RecvKeepAlive
Last Error: None
Export: [ agg-rts ]
Options: <Preference AuthKey PeerAS Refresh>
Authentication key is configured
Holdtime: 90 Preference: 170
Number of flaps: 0
Peer ID: 10.0.1.1 Local ID: 10.1.100.3 Active Holdtime: 90
Keepalive Interval: 30 Group index: 0 Peer index: 0
BFD: disabled, down
Local Interface: ge-1/0/1.121
NLRI for restart configured on peer: inet-unicast
NLRI advertised by peer: inet-unicast
NLRI for this session: inet-unicast
Peer supports Refresh capability (2)
Stale routes from peer are kept for: 300
Peer does not support Restarter functionality

www.juniper.net Troubleshooting Routing Protocols • Lab 8–33

Junos Troubleshooting in the NOC
NLRI that restart is negotiated for: inet-unicast
NLRI of received end-of-rib markers: inet-unicast
NLRI of all end-of-rib markers sent: inet-unicast
Peer supports 4 byte AS extension (peer-as 54321)
Peer does not support Addpath
Table inet.0 Bit: 10001
RIB State: BGP restart is complete
Send state: in sync
Active prefixes: 0
Received prefixes: 5
Accepted prefixes: 5
Suppressed due to damping: 0
Advertised prefixes: 2
Last traffic (seconds): Received 16 Sent 1 Checked 12
Input messages: Total 10 Updates 2 Refreshes 0 Octets 273
Output messages: Total 11 Updates 1 Refreshes 0 Octets 314
Output Queue[0]: 0

Question: What is the state of the R3 to ISP-2 BGP

session?

Answer: The R3 to ISP-2 BGP session should be in

Established state.

Part 4: Troubleshooting Routing Loops

In this lab part, you discover and troubleshoot a routing loop between the Host
device and the Server device.
Step 4.1
Preform a traceroute test from the Host device to the Server device by issuing the
traceroute 10.10.10.100 logical-system Host command.
lab@mxA-1> traceroute 10.10.10.100 logical-system Host
traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets
1 10.12.12.1 (10.12.12.1) 0.563 ms 0.362 ms 0.332 ms
2 10.1.1.13 (10.1.1.13) 0.386 ms 2.834 ms 3.134 ms
3 172.16.1.13 (172.16.1.13) 3.207 ms 0.964 ms 0.384 ms
4 172.16.1.5 (172.16.1.5) 0.379 ms 0.385 ms 0.377 ms
5 10.1.1.2 (10.1.1.2) 0.382 ms 0.491 ms 0.389 ms
6 10.1.1.10 (10.1.1.10) 0.417 ms 0.407 ms 0.400 ms
7 172.16.1.13 (172.16.1.13) 0.403 ms 0.410 ms 0.404 ms
8 172.16.1.5 (172.16.1.5) 0.403 ms 0.411 ms 0.393 ms
9 10.1.1.2 (10.1.1.2) 0.403 ms 0.407 ms 0.412 ms
10 10.1.1.10 (10.1.1.10) 0.425 ms 0.475 ms 0.422 ms
11 172.16.1.13 (172.16.1.13) 0.417 ms 0.436 ms 0.420 ms
12 172.16.1.5 (172.16.1.5) 0.414 ms 0.415 ms 0.411 ms
13 10.1.1.2 (10.1.1.2) 0.427 ms 0.440 ms 0.426 ms
14 10.1.1.10 (10.1.1.10) 0.472 ms 0.459 ms 0.439 ms
15 172.16.1.13 (172.16.1.13) 0.445 ms 0.453 ms 0.449 ms

Lab 8–34 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
16 172.16.1.5 (172.16.1.5) 0.537 ms 0.448 ms 0.435 ms
17 10.1.1.2 (10.1.1.2) 0.442 ms 0.451 ms 0.442 ms
18 10.1.1.10 (10.1.1.10) 1.017 ms 0.491 ms 0.463 ms
19 172.16.1.13 (172.16.1.13) 0.466 ms 0.470 ms 0.469 ms
20 172.16.1.5 (172.16.1.5) 0.459 ms 0.470 ms 0.457 ms
21 10.1.1.2 (10.1.1.2) 0.473 ms 0.474 ms 0.460 ms
22 10.1.1.10 (10.1.1.10) 0.486 ms 0.488 ms 0.472 ms
23 172.16.1.13 (172.16.1.13) 0.481 ms 0.487 ms 0.480 ms
24 172.16.1.5 (172.16.1.5) 0.607 ms 0.498 ms 0.487 ms
25 10.1.1.2 (10.1.1.2) 0.482 ms 0.499 ms 0.474 ms
26 10.1.1.10 (10.1.1.10) 0.516 ms 0.518 ms 0.516 ms
27 172.16.1.13 (172.16.1.13) 0.590 ms 0.511 ms 0.515 ms
28 172.16.1.5 (172.16.1.5) 0.505 ms 0.510 ms 0.503 ms
29 10.1.1.2 (10.1.1.2) 0.513 ms 0.519 ms 0.497 ms
30 10.1.1.10 (10.1.1.10) 0.540 ms 0.526 ms 0.512 ms

Question: Which path are the traceroute packets

taking?

Answer: The traceroute path begins with R6, then

R5, then R4, then R2, then R3, and back to R5.
Once it reaches R5 the second time the packets
continue to loop through R4, R2, R4, and R5.

Question: On which routers should you begin your

troubleshooting for the routing loop?

Answer: R2 and R5 are both points of route

redistribution and they are both in the path of the
routing loop. Begin by examining the routing tables
of R2 and R5.

Step 4.2
Examine the routing tables of R2 and R5 by issuing the show route
10.10.10.100 logical-system R2 and show route 10.10.10.100
logical-system R5 commands.
lab@mxA-1> show route 10.10.10.100 logical-system R2

inet.0: 30 destinations, 45 routes (30 active, 0 holddown, 1 hidden)

+ = Active Route, - = Last Active, * = Both

www.juniper.net Troubleshooting Routing Protocols • Lab 8–35

Junos Troubleshooting in the NOC
10.10.0.0/16 *[OSPF/150] 03:29:13, metric 3, tag 0
> to 10.1.1.2 via lt-1/0/0.1
[BGP/165] 03:29:19, localpref 100
AS path: 12345 I, validation-state: unverified
> to 172.16.17.1 via ge-1/0/0.121

lab@mxA-1> show route 10.10.10.100 logical-system R5

inet.0: 29 destinations, 46 routes (29 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

10.10.0.0/16 *[RIP/100] 03:29:34, metric 3, tag 0

> to 172.16.1.13 via lt-1/0/0.13

Question: What can you determine from the

outputs?

Answer: R2 has the 10.10.0.0/16 prefix in it’s

routing table from OSPF and BGP, but the OSPF
version of the route is active. R5 has the
10.10.0.0/16 in it’s routing table from RIP.

Question: Where could R2 be receiving the OSPF

version of the route?

Answer: The 10.10.0.0/16 route is being received

through BGP on R2 from ISP-1. It is reasonable to
assume that R3 might be receiving the same route
from ISP-2 through BGP and might be redistributing
it into OSPF.

Step 4.3
Examine the routing table of R3 by issuing the show route 10.10.10.100
logical-system R3 command.
lab@mxA-1> show route 10.10.10.100 logical-system R3

inet.0: 30 destinations, 38 routes (30 active, 0 holddown, 1 hidden)

+ = Active Route, - = Last Active, * = Both

10.10.0.0/16 *[OSPF/150] 03:45:30, metric 3, tag 0

> to 10.1.1.10 via lt-1/0/0.3

Lab 8–36 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
[BGP/170] 02:10:46, localpref 100
AS path: 54321 I, validation-state: unverified
> to 10.0.1.1 via ge-1/0/1.121

Question: Where is R3 learning the BGP version of

the 10.10.0.0/16 route?

Answer: R3 is learning the BGP version of the route

from ISP-2.

Step 4.4
In an effort to determine which router R3 is learning the OSPF version of the prefix
from, examine the OSPF LSDB on R3 for the LSA that represents the 10.10.0.0/16
prefix. You can examine the LSA by issuing the show ospf database lsa-id
10.10.0.0 logical-system R3 command.
lab@mxA-1> show ospf database lsa-id 10.10.0.0 logical-system R3
OSPF AS SCOPE link state database
Type ID Adv Rtr Seq Age Opt Cksum Len
Extern 10.10.0.0 10.1.100.5 0x80000006 2361 0x22 0xda47 36

Question: Which router is R3 learning the

10.10.0.0/16 prefix from?

Answer: Examining the Adv Rtr field shows that

R3 is learning the prefix from 10.1.100.5, which is
R5.

Step 4.5
Examine R5 for the 10.10.0.0/16 prefix by issuing the show route
10.10.10.100 logical-system R5 command.
lab@mxA-1> show route 10.10.10.100 logical-system R5

inet.0: 29 destinations, 46 routes (29 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

10.10.0.0/16 *[RIP/100] 04:23:46, metric 3, tag 0

> to 172.16.1.13 via lt-1/0/0.13

www.juniper.net Troubleshooting Routing Protocols • Lab 8–37

Junos Troubleshooting in the NOC
Question: Which router is R5 learning the
10.10.0.0/16 prefix from?

Answer: R5 is learning the prefix through RIP from

R4.

Step 4.6
Examine R4 for the 10.10.0.0/16 prefix by issuing the show route
10.10.10.100 logical-system R4 command.
lab@mxA-1> show route 10.10.10.100 logical-system R4

inet.0: 28 destinations, 28 routes (28 active, 0 holddown, 0 hidden)

+ = Active Route, - = Last Active, * = Both

10.10.0.0/16 *[RIP/100] 04:29:03, metric 2, tag 0

> to 172.16.1.5 via lt-1/0/0.12

Question: Which router is R4 learning the

10.10.0.0/16 prefix from?

Answer: R4 is learning the prefix through RIP from

R2.

Step 4.7
Examine the OSPF LSDB on R2 for the LSA that represents the 10.10.0.0/16 prefix
by issuing the show ospf database lsa-id 10.10.0.0
logical-system R2 command.
lab@mxA-1> show ospf database lsa-id 10.10.0.0 logical-system R2
OSPF AS SCOPE link state database
Type ID Adv Rtr Seq Age Opt Cksum Len
Extern 10.10.0.0 10.1.100.5 0x80000006 2296 0x22 0xda47 36

Question: Which router is R2 learning the

10.10.0.0/16 prefix from?

Answer: Examining the Adv Rtr field shows that

R2 is learning the prefix from R5.

Lab 8–38 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Question: The BGP version of the 10.10.0.0/16
route is inactive on R2 and R3, and because the
prefix is inactive it is ineligible to be redistributed
into OSPF. How could the BGP version of the routes
be inactive, but the other versions of the route be
active in RIP and OSPF?

Answer: The route is being redistributed from BGP

into OSPF. Next, it is being redistributed from OSPF
into RIP. Then, the prefix is being redistributed from
RIP back into OSPF. R2 and R3 now believe that
they have two versions of the route that are
independent of each other.

Question: What can you do to resolve the routing

loop problem?

Answer: There are a few steps that you must take to

fix the routing loop problem. However, the first
important step to take is to ensure that the version
of the prefix that first enters your network is the
most preferred version of the prefix. In this
scenario, the first version of the route to enter your
network is the BGP version of the route on R2 and
R3.

Step 4.8
Enter configuration mode and navigate to the [edit logical-systems R2
protocols] hierarchy level. Configure BGP to have a protocol preference value of
99.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit logical-systems R2 protocols

[edit logical-systems R2 protocols]

lab@mxA-1# set bgp preference 99

[edit logical-systems R2 protocols]

lab@mxA-1#

www.juniper.net Troubleshooting Routing Protocols • Lab 8–39

Junos Troubleshooting in the NOC
Step 4.9
Navigate to the [edit logical-systems R3 protocols] hierarchy level
and configure BGP to have a protocol preference value of 149. Then, issue the
commit and-quit command to activate the configuration and exit to operational
mode.
[edit logical-systems R2 protocols]
lab@mxA-1# top edit logical-systems R3 protocols

[edit logical-systems R3 protocols]

lab@mxA-1# set bgp preference 149

[edit logical-systems R3 protocols]

lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>

Question: Why was the preference value for BGP set

to 99 on R2 and 149 on R3?

Answer: R2 is running RIP, which has preference

values of 100, and OSPF, which has a preference
value of 150 for external routes. To avoid any
preference problems between RIP, OSPF, and BGP
on R2, a BGP preference value 99 is advisable. R3
is only running BGP and OSPF which requires you to
set the BGP preference value to 149 or below.

Step 4.10
Preform a traceroute test from the Host device to the Server device by issuing the
traceroute 10.10.10.100 logical-system Host command.
lab@mxA-1> traceroute 10.10.10.100 logical-system Host
traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets
1 10.12.12.1 (10.12.12.1) 0.501 ms 0.387 ms 0.338 ms
2 10.1.1.5 (10.1.1.5) 0.364 ms 0.380 ms 0.368 ms
3 10.10.10.100 (10.10.10.100) 0.573 ms 0.572 ms 0.555 ms

Question: What path did the traceroute take?

Answer: The traceroute went to R6, to R3, and then

arrived at the Server.

Lab 8–40 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC
Question: What other actions can you take to
ensure that the routing loop problem is resolved?

Answer: Traceroute tests from other locations, such

as R4, can help provide further confirmation on the
fix.

Step 4.11
Preform a traceroute test from R4 to the Server device by issuing the traceroute
10.10.10.100 logical-system R4 command.
lab@mxA-1> traceroute 10.10.10.100 logical-system R4
traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets
1 172.16.1.5 (172.16.1.5) 0.495 ms 0.377 ms 0.333 ms
2 10.10.10.100 (10.10.10.100) 0.597 ms 0.566 ms 0.544 ms
Step 4.12
Preform a traceroute test from R1 to the Server device by issuing the traceroute
10.10.10.100 logical-system R1 command.
lab@mxA-1> traceroute 10.10.10.100 logical-system R1
traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets
1 172.16.1.1 (172.16.1.1) 0.556 ms 0.458 ms 0.331 ms
2 10.10.10.100 (10.10.10.100) 0.585 ms 0.551 ms 0.536 ms
Step 4.13
Preform a traceroute test from R5 to the Server device by issuing the traceroute
10.10.10.100 logical-system R5 command.
lab@mxA-1> traceroute 10.10.10.100 logical-system R5
traceroute to 10.10.10.100 (10.10.10.100), 30 hops max, 40 byte packets
1 172.16.1.13 (172.16.1.13) 0.469 ms 0.477 ms 0.359 ms
2 172.16.1.5 (172.16.1.5) 0.383 ms 0.379 ms 0.371 ms
3 10.10.10.100 (10.10.10.100) 0.590 ms 0.568 ms 0.557 ms

Question: Do the traceroute tests show that the

routing loop problem is resolved?

Answer: Yes, the routing loop is fixed.

Step 4.14
Log out of your assigned device using the exit command.

www.juniper.net Troubleshooting Routing Protocols • Lab 8–41

Junos Troubleshooting in the NOC
lab@srxA-1> exit

srxA-1 (ttyu0)

login:

STOP Tell your instructor that you have completed this lab.

Lab 8–42 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Troubleshooting Routing Protocols • Lab 8–43

Junos Troubleshooting in the NOC

Lab 8–44 • Troubleshooting Routing Protocols www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Troubleshooting Routing Protocols • Lab 8–45

Junos Troubleshooting in the NOC

Lab 8–46 • Troubleshooting Routing Protocols www.juniper.net

 Lab
Monitoring the Network

Overview
In this lab, you configure and monitor SNMP, RMON, inline JFlow, and inline port mirroring.
Throughout the lab you will have opportunities to troubleshoot various aspects of the lab.
By completing this lab you will perform the following tasks:
• Configure and Monitor SNMP.
• Configure and Monitor RMON.
• Configure and Monitor inline JFlow.
• Configure and Monitor inline port mirroring.

www.juniper.net Monitoring the Network • Lab 9–1

.
Junos Troubleshooting in the NOC

Part 1: Modifying the Existing Configurations

In this lab part, you load and activate a predefined configuration saved on your
assigned device. This predefined configuration contains minimal configuration for
the main routing system and configuration for various logical systems.
Step 1.1
Ensure that you know to which device you have been assigned. Check with your
instructor if you are not certain. Consult the management network diagram to
determine your device’s management address.

Question: What is the management address

assigned to your device?

Answer: The answer varies and depends on your

assigned device. If you are unsure of your
assignment, ask your instructor.

Step 1.2
Access the CLI for your device using either the console, Telnet, or SSH as directed by
your instructor. Refer to the management network diagram for the IP address
associated with your station. The following example uses Telnet to access an
MX Series device using the SecureCRT program:

Lab 9–2 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
Step 1.3
Log in to your device with the username lab and the password lab123. Note that
both the name and password are case-sensitive. Enter configuration mode and load
the lab9-start.config configuration from the /var/home/lab/jtnoc/
directory. Issue the commit and-quit command to activate your changes and
exit to operational mode.
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# load override jtnoc/lab9-start.config
load complete

[edit]
lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>

Part 2: Configuring and Monitoring SNMP

In this lab part, you configure and monitor SNMP.

Note
To simulate the SNMP server, JFlow
Collector/Analyzer, and Host devices, your
physical MX Series device in the lab has
been divided into three logical systems,
plus the main routing system. To perform
an operation for a logical system you must
use the logical-system option
followed by the logical system’s name. For
example, if you want to ping the SNMP
server from the Host logical system, you
can issue the ping 10.11.11.100
logical-system Host command.

Step 2.1
Examine the MIB database for MIB names associated with the system MIB name
by performing a MIB walk. Perform the MIB walk by issuing the show snmp mib
walk system command.

www.juniper.net Monitoring the Network • Lab 9–3

Junos Troubleshooting in the NOC
lab@mxA-1> show snmp mib walk system
sysDescr.0 = Juniper Networks, Inc. mx80 internet router, kernel JUNOS
12.2R2.5, Build date: 2012-11-15 17:22:13 UTC Copyright (c) 1996-2012 Juniper
Networks, Inc.
sysObjectID.0 = jnxProductNameMX80
sysUpTime.0 = 154627810
sysContact.0
sysName.0 = mxA-1
sysLocation.0
sysServices.0 = 6

Question: Which MIB names are present under the

system MIB name?

Answer: The sysDescr.0, sysObjectID.0,

sysUpTime.0, sysName.0, and
sysServices.0 MIB names are present.

Step 2.2
Examine the last 10 MIB names under the jnxBoxAnatomy MIB name by issuing
the show snmp mib walk jnxBoxAnatomy | last 10 command.
lab@mxA-1> show snmp mib walk jnxBoxAnatomy | last 10
jnxFruPsdAssignment.8.2.3.0 = 0
jnxFruPsdAssignment.8.2.4.0 = 0
jnxFruPsdAssignment.9.1.0.0 = 0
jnxFruPsdAssignment.20.1.1.0 = 0
jnxFruPsdAssignment.20.1.2.0 = 0
jnxFruPsdAssignment.20.2.1.0 = 0
jnxFruPsdAssignment.20.2.2.0 = 0
jnxBoxKernelMemoryUsedPercent.0 = 1
jnxBoxSystemDomainType.0 = 1
jnxBoxPersonality.0 = jnxProductNameMX80

Question: What are the last two MIB names in the

output?

Answer: The last two MIB names in the output are

jnxBoxSystemDomainType.0 and
jnxBoxPersonality.0.

Lab 9–4 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
Question: Can you determine the OID of the
jnxBoxPersonality.0 MIB name from the
previous output? Why?

Answer: No. The OID is not present in the output.

Step 2.3
Examine the XML output of the jnxBoxPersonality.0 MIB name by issuing the
show snmp mib get jnxBoxPersonality.0 | display xml command.
lab@mxA-1> show snmp mib get jnxBoxPersonality.0 | display xml
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/12.2R2/junos">
<snmp-object-information xmlns="http://xml.juniper.net/junos/12.2R2/
junos-snmp">
<snmp-object>
<name>jnxBoxPersonality.0</name>
<object-value-type>OID</object-value-type>
<object-value>jnxProductNameMX80</object-value>
<oid>1.3.6.1.4.1.2636.3.1.18.0</oid>
</snmp-object>
</snmp-object-information>
<cli>
<banner></banner>
</cli>
</rpc-reply>

Question: What is the OID of the

jnxBoxPersonality.0 MIB name?

Answer: The OID of the jnxBoxPersonality.0

MIB name is 1.3.6.1.4.1.2636.3.1.18.0.

Step 2.4
Enter configuration mode and navigate to the [edit snmp] hierarchy level.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit snmp

[edit snmp]
lab@mxA-1#

www.juniper.net Monitoring the Network • Lab 9–5

Junos Troubleshooting in the NOC
Step 2.5
Configure the public community to have read-only access to the MIB.
[edit snmp]
lab@mxA-1# set community public authorization read-only
Step 2.6
Configure the public community to only accept SNMP requests from the SNMP
server (10.11.11.100).
[edit snmp]
lab@mxA-1# set community public clients 10.11.11.100

[edit snmp]
lab@mxA-1# set community public clients 0/0 restrict
Step 2.7
Configure the trap group v2-traps to send SNMPv2c traps to the SNMP server
(10.11.11.100).
[edit snmp]
lab@mxA-1# set trap-group v2-traps targets 10.11.11.100

Question: Is it necessary to configure the SNMP

version of the trap group in this situation? Why?

Answer: It is not necessary to configure the SNMP

version for the trap group because the Junos OS
sends SNMPv1 and SNMPv2c traps by default.

Step 2.8
Configure the Junos OS to send traps to the SNMP server for chassis, startup,
and link events. Activate the configuration and exit to operational mode by issuing
the commit and-quit command.
[edit snmp]
lab@mxA-1# set trap-group v2-traps categories chassis

[edit snmp]
lab@mxA-1# set trap-group v2-traps categories startup

[edit snmp]
lab@mxA-1# set trap-group v2-traps categories link

[edit snmp]
lab@mxA-1# commit and-quit
commit complete

Lab 9–6 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
Exiting configuration mode

lab@mxA-1>
Step 2.9
Open a second Telnet, or SSH, session to your assigned device. You can use this new
session to monitor the interface of the SNMP server logical system.
Refer to the management network diagram for the IP address associated with your
station. The following example uses Telnet to access an MX Series device using the
SecureCRT program:

Step 2.10
Log in to your device with the username lab using a password of lab123. Note
that both the name and password are case-sensitive.
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1>
Step 2.11
Monitor the SNMP server logical system’s interface by issuing the monitor
traffic interface ge-1/1/8 detail command.
lab@mxA-1> monitor traffic interface ge-1/1/8 detail
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on ge-1/1/8, capture size 1514 bytes

www.juniper.net Monitoring the Network • Lab 9–7

Junos Troubleshooting in the NOC
Step 2.12
Return to the original terminal session established with your assigned MX Series
device.
From the original terminal session, issue the request snmp spoof-trap
linkDown command.
lab@mxA-1> request snmp spoof-trap linkDown
Spoof-trap request result: trap sent successfully
Step 2.13
Return to the second terminal session established with your assigned MX Series
device.
From the second terminal session, examine the output and answer the following
questions.
Note
You might notice an ICMP message that the
SNMP server is sending back to your
MX Series device that the SNMP trap port
is not reachable. The SNMP server is just a
logical system on your MX Series device
that does not have the SNMP trap port
open. This unreachable SNMP trap port
message is expected for this lab.

lab@mxA-1> monitor traffic interface ge-1/1/8 detail

Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on ge-1/1/8, capture size 1514 bytes

Reverse lookup for 10.11.11.1 failed (check DNS reachability).

Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.

22:38:10.438325 In IP (tos 0x0, ttl 64, id 45809, offset 0, flags [none],

proto: UDP (17), length: 145) 10.11.11.1.63930 > 10.11.11.100.snmptrap:
|30|73|02|01SNMPv1 |04|08C=v2-traps |a4|64Trap(100) |06|0cE:2636.1.1.1.2.57
|40|0410.210.15.1|02|01 linkDown|02|01 |43|04155414603|30|42
|30|16|06|0einterfaces.ifTable.ifEntry.ifIndex.1073741824=|02|041073741824
|30|13|06|0einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=|02|013
|30|13|06|0einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=|02|017
22:38:10.438363 Out IP (tos 0x0, ttl 255, id 45815, offset 0, flags [DF], proto:
ICMP (1), length: 56) 10.11.11.100 > 10.11.11.1: ICMP 10.11.11.100 udp port
snmptrap unreachable, length 36
IP (tos 0x0, ttl 64, id 45809, offset 0, flags [none], proto: UDP (17),
length: 145) [|ip]

Lab 9–8 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
22:38:10.438444 In IP (tos 0x0, ttl 64, id 45812, offset 0, flags [none],
proto: UDP (17), length: 169) 10.11.11.1.63930 > 10.11.11.100.snmptrap:
|30|81|8a|02|01SNMPv2c |04|08C=v2-traps
|a7|7bV2Trap(123)|02|04|02|01|02|01|30|6d
|30|10|06|08system.sysUpTime.0=|43|04155414603
|30|17|06|0aS:1.1.4.1.0=|06|09S:1.1.5.3
|30|16|06|0einterfaces.ifTable.ifEntry.ifIndex.1073741824=|02|041073741824
|30|13|06|0einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=|02|013
|30|13|06|0einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=|02|017
22:38:10.438462 Out IP (tos 0x0, ttl 255, id 45818, offset 0, flags [DF], proto:
ICMP (1), length: 56) 10.11.11.100 > 10.11.11.1: ICMP 10.11.11.100 udp port
snmptrap unreachable, length 36

Question: Which version of SNMP traps are

present?

Answer: The output shows two link down traps

being sent; an SNMPv1 and an SNMPv2c trap.

Question: What is the SNMP community that is

present in the traps?

Answer: The output shows that the SNMP

community v2-traps is present in the trap
messages.

Question: Why are there not any interfaces present

in the trap message?

Answer: The trap message is simply a spoofed link

down trap message. There is no interface specified
in the spoofed trap.

Question: How can you add interface information to

the spoofed trap?

Answer: You can use the variable-bindings

option with the spoof-trap command to add
interface information to the spoofed trap.

www.juniper.net Monitoring the Network • Lab 9–9

Junos Troubleshooting in the NOC
Step 2.14
Return to the original terminal session established with your assigned MX Series
device.
From the original terminal session, add the ge-1/0/8 interface to the spoofed trap
by issuing the request snmp spoof-trap linkDown
variable-bindings "ifIndex [ge-1/0/8.0] = ge-1/0/8.0"
command.
lab@mxA-1> request snmp spoof-trap linkDown variable-bindings "ifIndex
[ge-1/0/8.0] = ge-1/0/8.0"
Spoof-trap request result: trap sent successfully
Step 2.15
View the SNMP interface index for the ge-1/0/8.0 interface by issuing the show
interfaces ge-1/0/8.0 | match snmp command.
lab@mxA-1> show interfaces ge-1/0/8.0 | match snmp
Logical interface ge-1/0/8.0 (Index 332) (SNMP ifIndex 640)
Flags: SNMP-Traps 0x0 Encapsulation: ENET2

Question: What is the SNMP interface index for the

ge-1/0/8.0 interface?

Answer: Although your results might vary, the

previous output shows that the SNMP interface
index for the ge-1/0/8.0 interface is 640.

Step 2.16
Return to the second terminal session established with your assigned MX Series
device.
From the second terminal session, examine the output for the SNMP interface index
of interface ge-1/0/8.0 that you acquired in the previous step.
02:05:32.966074 In IP (tos 0x0, ttl 64, id 18533, offset 0, flags [none],
proto: UDP (17), length: 140) 10.11.11.1.63930 > 10.11.11.100.snmptrap:
|30|6e|02|01SNMPv1 |04|08C=v2-traps |a4|5fTrap(95) |06|0cE:2636.1.1.1.2.57
|40|0410.210.15.1|02|01 linkDown|02|01 |43|04156658857|30|3d
|30|11|06|0binterfaces.ifTable.ifEntry.ifIndex.640=|02|02640
|30|13|06|0einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=|02|013
|30|13|06|0einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=|02|017
02:05:32.966111 Out IP (tos 0x0, ttl 255, id 18539, offset 0, flags [DF], proto:
ICMP (1), length: 56) 10.11.11.100 > 10.11.11.1: ICMP 10.11.11.100 udp port
snmptrap unreachable, length 36
IP (tos 0x0, ttl 64, id 18533, offset 0, flags [none], proto: UDP (17),
length: 140) [|ip]

Lab 9–10 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
02:05:32.966259 In IP (tos 0x0, ttl 64, id 18536, offset 0, flags [none],
proto: UDP (17), length: 164) 10.11.11.1.63930 > 10.11.11.100.snmptrap:
|30|81|85|02|01SNMPv2c |04|08C=v2-traps
|a7|76V2Trap(118)|02|04|02|01|02|01|30|68
|30|10|06|08system.sysUpTime.0=|43|04156658857
|30|17|06|0aS:1.1.4.1.0=|06|09S:1.1.5.3
|30|11|06|0binterfaces.ifTable.ifEntry.ifIndex.640=|02|02640
|30|13|06|0einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=|02|013
|30|13|06|0einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=|02|017
02:05:32.966279 Out IP (tos 0x0, ttl 255, id 18542, offset 0, flags [DF], proto:
ICMP (1), length: 56) 10.11.11.100 > 10.11.11.1: ICMP 10.11.11.100 udp port
snmptrap unreachable, length 36
IP (tos 0x0, ttl 64, id 18536, offset 0, flags [none], proto: UDP (17),
length: 164) [|ip]

Question: Which SNMP interface index is present in

the output?

Answer: Although your results might vary, the

previous output displays the SNMP interface index
of 640 in the trap message.

Part 3: Configuring and Monitoring RMON

In this lab part, you configure and monitor RMON.

Note
Remember that to simulate the SNMP
server, JFlow Collector/Analyzer, and Host
devices, your physical MX Series device in
the lab has been divided into three logical
systems, plus the main routing system. To
perform an operation for a logical system
you must use the logical-system
option followed by the logical system’s
name. For example, if you want to ping the
SNMP server from the Host logical system,
you can issue the ping 10.11.11.100
logical-system Host command.

Step 3.1
Return to the original terminal session established with your assigned MX Series
device.
From the original terminal session, enter configuration mode and navigate to the
[edit snmp rmon] configuration hierarchy level.
lab@mxA-1> configure
Entering configuration mode

www.juniper.net Monitoring the Network • Lab 9–11

Junos Troubleshooting in the NOC

[edit]
lab@mxA-1# edit snmp rmon

[edit snmp rmon]

lab@mxA-1#
Step 3.2
Examine the SNMP interface index for the ge-1/0/8.0 interface by issuing the
run show interfaces ge-1/0/8.0 | match snmp command.
[edit snmp rmon]
lab@mxA-1# run show interfaces ge-1/0/8.0 | match snmp
Logical interface ge-1/0/8.0 (Index 338) (SNMP ifIndex 642)
Flags: SNMP-Traps 0x0 Encapsulation: ENET2

Question: Which SNMP interface index is present

for the ge-1/0/8.0 interface?

Answer: Although your output might vary, the

previous output shows that the ge-1/0/8.0
interface contains the SNMP interface index of 642.

Step 3.3
Set an RMON alarm, which has an index value of 1, to use the variable
ifHCOut1SecRate.X, where X is the SNMP interface index value that you found
in step 3.2.
[edit snmp rmon]
lab@mxA-1# set alarm 1 variable ifHCOut1SecRate.X
Step 3.4
Set the sample-type for alarm 1 to the absolute-value option. Then,
configure alarm 1 to sample the traffic rate once every 5 seconds.
[edit snmp rmon]
lab@mxA-1# set alarm 1 sample-type absolute-value

[edit snmp rmon]

lab@mxA-1# set alarm 1 interval 5
Step 3.5
Configure alarm 1 with a rising-threshold value of 9000000 and a
failing-threshold value of 2000000.
[edit snmp rmon]
lab@mxA-1# set alarm 1 rising-threshold 9000000

[edit snmp rmon]

lab@mxA-1# set alarm 1 falling-threshold 2000000

Lab 9–12 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
Step 3.6
Configure event 1 to record a log message locally and send a trap message to the
SNMP server when the rising threshold is reached for alarm 1.
[edit snmp rmon]
lab@mxA-1# set event 1 type log-and-trap

[edit snmp rmon]

lab@mxA-1# set alarm 1 rising-event-index 1
Step 3.7
Configure event 2 to only record a log message locally when the falling threshold
is reached for alarm 1. Activate the configuration by issuing the commit
command.
[edit snmp rmon]
lab@mxA-1# set event 2 type log

[edit snmp rmon]

lab@mxA-1# set alarm 1 falling-event-index 2

[edit snmp rmon]

lab@mxA-1# show
alarm 1 {
interval 5;
variable ifHCOut1SecRate.642;
sample-type absolute-value;
rising-threshold 9000000;
falling-threshold 2000000;
rising-event-index 1;
falling-event-index 2;
}
event 1 {
type log-and-trap;
}
event 2 {
type log;
}

[edit snmp rmon]

lab@mxA-1# commit
commit complete
Step 3.8
Examine the current number of sent SNMP traps by issuing the run show snmp
statistics | find output command.
[edit snmp rmon]
lab@mxA-1# run show snmp statistics | find output
Output:
Packets: 321861, Too bigs: 0, No such names: 0,
Bad values: 0, General errors: 0,
Get requests: 0, Get nexts: 0, Set requests: 0,
Get responses: 321839, Traps: 22

www.juniper.net Monitoring the Network • Lab 9–13

Junos Troubleshooting in the NOC
Question: How many traps have been sent?

Answer: Although your output might vary, the

previous output shows that 22 traps have been
sent.

Step 3.9
Return to the second terminal session established with your assigned MX Series
device.
From the second terminal session, terminate the interface monitoring output by
issuing the Ctrl + c key combination. Then, ping the SNMP server from the Host
device by issuing the ping 10.11.11.100 size 1400 rapid count
1000000 logical-system Host command.
IP (tos 0x0, ttl 64, id 31451, offset 0, flags [none], proto: UDP (17), length:
140) [|ip]
02:25:49.505889 In IP (tos 0x0, ttl 64, id 31454, offset 0, flags [none],
proto: UDP (17), length: 164) 10.11.11.1.63930 > 10.11.11.100.snmptrap:
|30|81|85|02|01SNMPv2c |04|08C=v2-traps
|a7|76V2Trap(118)|02|04|02|01|02|01|30|68
|30|10|06|08system.sysUpTime.0=|43|04165420511
|30|17|06|0aS:1.1.4.1.0=|06|09S:1.1.5.3
|30|11|06|0binterfaces.ifTable.ifEntry.ifIndex.642=|02|02642
|30|13|06|0einterfaces.ifTable.ifEntry.ifAdminStatus.1073741824=|02|013
|30|13|06|0einterfaces.ifTable.ifEntry.ifOperStatus.1073741824=|02|017
02:25:49.505909 Out IP (tos 0x0, ttl 255, id 31460, offset 0, flags [DF], proto:
ICMP (1), length: 56) 10.11.11.100 > 10.11.11.1: ICMP 10.11.11.100 udp port
snmptrap unreachable, length 36
IP (tos 0x0, ttl 64, id 31454, offset 0, flags [none], proto: UDP (17),
length: 164) [|ip]
^C
8 packets received by filter
0 packets dropped by kernel

lab@mxA-1> ping 10.11.11.100 size 1400 rapid count 1000000 logical-system Host
PING 10.11.11.100 (10.11.11.100): 1400 data bytes
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Step 3.10
Return to the original terminal session established with your assigned MX Series
device.
From the original terminal session, issue the run show snmp rmon alarms
command.
[edit snmp rmon]
lab@mxA-1# run show snmp rmon alarms

Lab 9–14 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
Alarm
Index Variable description Value State

1 monitor
ifHCOut1SecRate.642 10921728 rising threshold

Question: Which RMON alarm is present?

Answer: The rising threshold alarm is present.

Step 3.11
Examine the RMON events by issuing the run show snmp rmon events
command.
[edit snmp rmon]
lab@mxA-1# run show snmp rmon events

Event
Index Type Last Event
1 log and trap 2013-01-29 20:03:55 UTC

Question: What does the output suggest?

Answer: The output suggests that an event with the

index of 1 was triggered which resulted in a log
being generated and a trap message being sent.

Step 3.12
Examine the logs for the RMON event by issuing the run show log messages
| match SNMPD_RMON_EVENTLOG command.
[edit snmp rmon]
lab@mxA-1# run show log messages | match SNMPD_RMON_EVENTLOG
Jan 29 19:52:06 mxA-1 snmpd[1447]: SNMPD_RMON_EVENTLOG: Event 1 triggered by
Alarm 1, rising threshold (9000000) crossed, (variable: ifHCOut1SecRate.642,
value: 11117112)
Jan 29 19:52:16 mxA-1 snmpd[1447]: SNMPD_RMON_EVENTLOG: Event 2 triggered by
Alarm 1, falling threshold (2000000) crossed, (variable:
ifHCOut1SecRate.642, value: 0)
Jan 29 20:03:56 mxA-1 snmpd[1447]: SNMPD_RMON_EVENTLOG: Event 1 triggered by
Alarm 1, rising threshold (9000000) crossed, (variable: ifHCOut1SecRate.642,
value: 10921728)

www.juniper.net Monitoring the Network • Lab 9–15

Junos Troubleshooting in the NOC
Question: Is the log message for the RMON event
present?

Answer: Yes. The log message for the RMON event

is present.

Step 3.13
Examine the current number of SNMP traps sent by issuing the run show snmp
statistics | find output command.
[edit snmp rmon]
lab@mxA-1# run show snmp statistics | find output
Output:
Packets: 322176, Too bigs: 0, No such names: 0,
Bad values: 0, General errors: 0,
Get requests: 0, Get nexts: 0, Set requests: 0,
Get responses: 322154, Traps: 22

Question: What is the current value of sent SNMP

traps? Did the Junos OS send an SNMP trap in
response to the RMON event?

Answer: Although your output might vary, the

previous output shows that 22 SNMP traps have
been sent. The SNMP traps value has not
incremented from the previous time that you
examined the output. This output shows that traps
are not being sent in response to the RMON event.

Step 3.14
Examine the SNMP traps configuration.
[edit snmp rmon]
lab@mxA-1# up 1 show trap-group v2-traps
categories {
chassis;
link;
startup;
}
targets {
10.11.11.100;
}

Lab 9–16 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
Question: Why is the Junos OS not sending traps in
response to the RMON event?

Answer: The Junos OS is not sending traps in

response to the RMON event because the RMON
alarm category is not defined for the trap group.

Step 3.15
Configure the RMON alarm category for the v2-traps trap group. Activate the
configuration and exit to operational mode by issuing the commit and-quit
command.
[edit snmp rmon]
lab@mxA-1# up 1 set trap-group v2-traps categories rmon-alarm

[edit snmp rmon]

lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>
Step 3.16
Return to the second terminal session established with your assigned MX Series
device.
From the second terminal session, terminate the previous ping operation by issuing
the Ctrl + c key combination. Then, ping the SNMP server from the Host device by
issuing the ping 10.11.11.100 size 1400 rapid count 1000000
logical-system Host command.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!^C
--- 10.11.11.100 ping statistics ---
357077 packets transmitted, 357077 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.799/0.911/39.663/0.690 ms

lab@mxA-1> ping 10.11.11.100 size 1400 rapid count 1000000 logical-system Host
PING 10.11.11.100 (10.11.11.100): 1400 data bytes
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Step 3.17
Return to the original terminal session established with your assigned MX Series
device.
From the original terminal session, issue the show snmp statistics | find
output command.

www.juniper.net Monitoring the Network • Lab 9–17

Junos Troubleshooting in the NOC
lab@mxA-1> show snmp statistics | find output
Output:
Packets: 322383, Too bigs: 0, No such names: 0,
Bad values: 0, General errors: 0,
Get requests: 0, Get nexts: 0, Set requests: 0,
Get responses: 322359, Traps: 24

Question: What is the current value of sent SNMP

traps? Did the Junos OS send an SNMP trap in
response to the RMON event?

Answer: Although your output might vary, the

previous output shows that the Junos OS has sent
24 SNMP traps. This output indicates that the
Junos OS did send an SNMP trap in response to the
RMON event.

Step 3.18
Return to the second terminal session established with your assigned MX Series
device.
From the second terminal session, terminate the output by issuing the Ctrl + c key
combination.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!^C
--- 10.11.11.100 ping statistics ---
357077 packets transmitted, 357077 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.799/0.911/39.663/0.690 ms

lab@mxA-1>
Return to the original terminal session established with your assigned MX Series
device.
From the original terminal session, issue the show snmp rmon alarms
command.
lab@mxA-1> show snmp rmon alarms

Alarm
Index Variable description Value State

1 monitor
ifHCOut1SecRate.642 0 falling threshold

Lab 9–18 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
Question: Which RMON alarm is currently active?

Answer: The falling threshold RMON alarm is

currently active.

Part 4: Configuring and Monitoring Inline JFlow

In this lab part, you configure and monitor inline JFlow.

Note
Remember that to simulate the SNMP
server, JFlow Collector/Analyzer, and Host
devices, your physical MX Series device in
the lab has been divided into three logical
systems, plus the main routing system. To
perform an operation for a logical system
you must use the logical-system
option followed by the logical system’s
name. For example, if you want to ping the
SNMP server from the Host logical system,
you can issue the ping 10.11.11.100
logical-system Host command.

Step 4.1
Enter configuration mode and navigate to the [edit services
flow-monitoring version-ipfix] hierarchy level.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit services flow-monitoring version-ipfix

[edit services flow-monitoring version-ipfix]

lab@mxA-1#
Step 4.2
Configure the my-inline-jflow-t template to use the IPv4 template
configuration.
[edit services flow-monitoring version-ipfix]
lab@mxA-1# set template my-inline-jflow-t ipv4-template
Step 4.3
Navigate to the [edit forwarding-options sampling instance
my-inline-jflow-i] hierarchy level. Configure the my-inline-jflow-i
sampling instance to sample every packet of a flow.

www.juniper.net Monitoring the Network • Lab 9–19

Junos Troubleshooting in the NOC
[edit services flow-monitoring version-ipfix]
lab@mxA-1# top edit forwarding-options sampling instance my-inline-jflow-i

[edit forwarding-options sampling instance my-inline-jflow-i]

lab@mxA-1# set input rate 1

[edit forwarding-options sampling instance my-inline-jflow-i]

lab@mxA-1#
Step 4.4
Configure the my-inline-jflow-i sampling instance’s output parameters to
send IPv4 traffic to the JFlow server (10.10.10.100) using port 2055. Next the
output parameters must use the IPFIX template my-inline-jflow-t. Then,
configure the output parameters to use the source address of 10.10.10.1 for the
inline JFlow process.
[edit forwarding-options sampling instance my-inline-jflow-i]
lab@mxA-1# set family inet output flow-server 10.10.10.100 port 2055

[edit forwarding-options sampling instance my-inline-jflow-i]

lab@mxA-1# set family inet output flow-server 10.10.10.100 version-ipfix
template my-inline-jflow-t

[edit forwarding-options sampling instance my-inline-jflow-i]

lab@mxA-1# set family inet output inline-jflow source-address 10.10.10.1

[edit forwarding-options sampling instance my-inline-jflow-i]

lab@mxA-1# show
##
## Warning: Instance not defined under chassis hierarchy
##
input {
rate 1;
}
family inet {
output {
flow-server 10.10.10.100 {
port 2055;
version-ipfix {
template {
my-inline-jflow-t;
}
}
}
inline-jflow {
source-address 10.10.10.1;
}
}
}
Step 4.5
Navigate to the [edit chassis] hierarchy level. Configure the TFEB to use the
sampling instance my-inline-jflow-i.

Lab 9–20 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
[edit forwarding-options sampling instance my-inline-jflow-i]
lab@mxA-1# top edit chassis

[edit chassis]
lab@mxA-1# set tfeb slot 0 sampling-instance my-inline-jflow-i

[edit chassis]
lab@mxA-1#
Step 4.6
Navigate to the [edit firewall family inet filter
inline-jflow-filter] hierarchy level. Configure a firewall filter term that
samples traffic that is going to the SNMP server (10.11.11.100).
[edit chassis]
lab@mxA-1# top edit firewall family inet filter inline-jflow-filter

[edit firewall family inet filter inline-jflow-filter]

lab@mxA-1# set term 1 from destination-address 10.11.11.100

[edit firewall family inet filter inline-jflow-filter]

lab@mxA-1# set term 1 then sample

[edit firewall family inet filter inline-jflow-filter]

lab@mxA-1# set term 1 then accept

[edit firewall family inet filter inline-jflow-filter]

lab@mxA-1#
Step 4.7
Navigate to the [edit interfaces ge-1/0/7 unit 0] hierarchy level. Apply
the inline-jflow-filter to the ge-1/0/7 interface as an input firewall filter.
Activate the configuration and exit to operational mode by issuing the commit and
quit command.
[edit firewall family inet filter inline-jflow-filter]
lab@mxA-1# top edit interfaces ge-1/0/7.0

[edit interfaces ge-1/0/7 unit 0]

lab@mxA-1# set family inet filter input inline-jflow-filter

[edit interfaces ge-1/0/7 unit 0]

lab@mxA-1# commit and-quit
commit complete
Exiting configuration mode

lab@mxA-1>
Step 4.8
Return to the second terminal session established with your assigned MX Series
device.
From the second terminal session, open an FTP session with the SNMP server from
the Host device by issuing the ftp 10.11.11.100 logical-system Host
command. Login using the username lab and password lab123.

www.juniper.net Monitoring the Network • Lab 9–21

Junos Troubleshooting in the NOC
lab@mxA-1> ftp 10.11.11.100 logical-system Host
Connected to 10.11.11.100.
220 mxA-1 FTP server (Version 6.00LS) ready.
Name (10.11.11.100:lab): lab
331 Password required for lab.
Password:
230 User lab logged in.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>
Step 4.9
Return to the first terminal session established with your assigned MX Series device.
From the first terminal session, issue the show services accounting flow
inline-jflow command.
Note

If you do not see any active flows in the

output, restart the FTP session in Step 4.8.

lab@mxA-1> show services accounting flow inline-jflow

Flow information
TFEB Slot: 0
Flow Packets: 6141, Flow Bytes: 770453
Active Flows: 2, Total Flows: 37
Flows Exported: 27, Flow Packets Exported: 27
Flows Inactive Timed Out: 17, Flows Active Timed Out: 19

Question: How many active flows is inline JFlow

monitoring?

Answer: Although your output might vary, the

previous output shows that the inline JFlow is
monitoring two active flow.

Question: How many flows has inline JFlow

exported?

Answer: Although your answer might vary, the

previous output shows that 27 flows have been
exported by inline JFlow.

Lab 9–22 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC

Part 5: Configuring and Monitoring Inline Port Mirroring

In this lab part, you configure and monitor inline port mirroring.
Note
Remember that to simulate the SNMP
server, JFlow Collector/Analyzer, and Host
devices, your physical MX Series device in
the lab has been divided into three logical
systems, plus the main routing system. To
perform an operation for a logical system
you must use the logical-system
option followed by the logical system’s
name. For example, if you want to ping the
SNMP server from the Host logical system,
you can issue the ping 10.11.11.100
logical-system Host command.

Step 5.1
Enter configuration mode and navigate to the [edit forwarding-options
port-mirroring instance] hierarchy level.
lab@mxA-1> configure
Entering configuration mode

[edit]
lab@mxA-1# edit forwarding-options port-mirroring instance

[edit forwarding-options port-mirroring instance]

lab@mxA-1#
Step 5.2
Configure a port mirroring instance called parent-1 that has input parameters
that port mirrors every packet.
[edit forwarding-options port-mirroring instance]
lab@mxA-1# set parent-1 input rate 1
Step 5.3
Configure a port mirroring instance called child-1 that inherits its input
parameters from the parent-1 port mirroring instance. Then, configure the IPv4
output parameters of the child-1 port mirroring instance to use the ge-1/0/9
interface to send port mirrored packets to the 10.10.10.100 address.
[edit forwarding-options port-mirroring instance]
lab@mxA-1# set child-1 input-parameters-instance parent-1

[edit forwarding-options port-mirroring instance]

lab@mxA-1# set child-1 family inet output interface ge-1/0/9 next-hop
10.10.10.100

www.juniper.net Monitoring the Network • Lab 9–23

Junos Troubleshooting in the NOC
[edit forwarding-options port-mirroring instance]
lab@mxA-1# show
parent-1 {
input {
rate 1;
}
}
child-1 {
input-parameters-instance parent-1;
family inet {
output {
interface ge-1/0/9.0 {
next-hop 10.10.10.100;
}
}
}
}
Step 5.4
Navigate to the [edit chassis] hierarchy level. Bind the parent-1 port
mirroring instance to FPC 1.
[edit forwarding-options port-mirroring instance]
lab@mxA-1# top edit chassis

[edit chassis]
lab@mxA-1# set fpc 1 port-mirror-instance parent-1

[edit chassis]
lab@mxA-1#
Step 5.5
Navigate to the [edit firewall family inet filter
inline-port-mirror] hierarchy level. Configure a firewall filter term that port
mirrors traffic that is going to the SNMP server (10.11.11.100) to the child-1 port
mirroring instance.
[edit chassis]
lab@mxA-1# top edit firewall family inet filter inline-port-mirror

[edit firewall family inet filter inline-port-mirror]

lab@mxA-1# set term 1 from destination-address 10.11.11.100

[edit firewall family inet filter inline-port-mirror]

lab@mxA-1# set term 1 then port-mirror-instance child-1

[edit firewall family inet filter inline-port-mirror]

lab@mxA-1# show
term 1 {
from {
destination-address {
10.11.11.100/32;
}
}
then port-mirror-instance child-1;

Lab 9–24 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
}

[edit firewall family inet filter inline-port-mirror]

lab@mxA-1#
Step 5.6
Navigate to the [edit interfaces ge-1/0/7 unit 0] hierarchy level.
Remove the previous firewall filter that was used to sample packets from the
interface. Then, apply the inline-port-mirror firewall filter as an input filter to
the interface. Finally, activate the configuration by issuing the commit command.
[edit firewall family inet filter inline-port-mirror]
lab@mxA-1# top edit interfaces ge-1/0/7.0

[edit interfaces ge-1/0/7 unit 0]

lab@mxA-1# delete family inet filter

[edit interfaces ge-1/0/7 unit 0]

lab@mxA-1# set family inet filter input inline-port-mirror

[edit interfaces ge-1/0/7 unit 0]

lab@mxA-1# commit
commit complete

[edit interfaces ge-1/0/7 unit 0]

lab@mxA-1#
Step 5.7
Open a third Telnet, or SSH, session to your assigned device. You can use this new
session to monitor the interface of the JFlow-Collector_Analyzer logical system.
Refer to the management network diagram for the IP address associated with your
station. The following example uses Telnet to access an MX Series device using the
SecureCRT program:

www.juniper.net Monitoring the Network • Lab 9–25

Junos Troubleshooting in the NOC

Step 5.8
Log in to your device with the username lab using a password of lab123. Note
that both the name and password are case-sensitive.
mxA-1 (ttyu0)

login: lab
Password:

--- JUNOS 12.2R2.5 built 2013-01-16 04:18:49 UTC

lab@mxA-1>
Step 5.9
From the third terminal session, ping the SNMP server from the Host device by
issuing the ping 10.11.11.100 logical-system Host command.
lab@mxA-1> ping 10.11.11.100 logical-system Host
PING 10.11.11.100 (10.11.11.100): 56 data bytes
64 bytes from 10.11.11.100: icmp_seq=0 ttl=63 time=0.672 ms
64 bytes from 10.11.11.100: icmp_seq=1 ttl=63 time=0.630 ms
64 bytes from 10.11.11.100: icmp_seq=2 ttl=63 time=0.618 ms
Step 5.10
Return to the second terminal session established with your assigned MX Series
device.
From the second terminal session, exit the FTP session and monitor the JFlow
Collector_Analyzer’s interface for the sampled packets by issuing the monitor
traffic interface ge-1/1/9 command.

Lab 9–26 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
...
ftp> bye
221 Goodbye.

lab@mxA-1> monitor traffic interface ge-1/1/9

Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on ge-1/1/9, capture size 1514 bytes

Question: What does the lack of information in the

output reveal?

Answer: The lack of information in the output

reveals that the port mirroring is not working
correctly.

Step 5.11
Return to the first terminal session established with your assigned MX Series device.
From the first terminal session, examine the state of the child-1 port mirroring
instance by issuing the run show forwarding-options port-mirroring
child-1 command.
[edit interfaces ge-1/0/7 unit 0]
lab@mxA-1# run show forwarding-options port-mirroring child-1
Instance Name: child-1
Instance Id: 3
Input parameters:
Rate : 1
Run-length : 0
Maximum-packet-length : 0
Output parameters:
Family State Destination Next-hop
inet down ge-1/0/9.0 10.10.10.100

Question: What is the state of the port mirroring

instance?

Answer: The port mirroring instance is in the down

state.

www.juniper.net Monitoring the Network • Lab 9–27

Junos Troubleshooting in the NOC
Step 5.12
Examine the ARP table for an ARP entry associated with the interface that points
towards the JFlow Collector/Analyzer by issuing the run show arp interface
ge-1/0/9 command.
[edit interfaces ge-1/0/7 unit 0]
lab@mxA-1# run show arp interface ge-1/0/9

[edit interfaces ge-1/0/7 unit 0]

lab@mxA-1#

Question: What does the lack of information in the

output reveal?

Answer: The lack of information in the output

reveals that your assigned device is not learning the
MAC address of the interface associated with the
JFlow Collector/Analyzer.

Question: What must you do to statically add an

entry in the ARP table for the interface associated
with the JFlow Collector/Analyzer?

Answer: You must first learn the MAC address of the

interface that is associated with the JFlow
Collector/Analyzer. Then, you can statically add an
ARP entry to your assigned device’s ge-1/0/9
interface for the interface associated with the JFlow
Collector/Analyzer.

Step 5.13
To find the MAC address of the interface associated with the JFlow Collector/
Analyzer (JFlow-Collector_Analyzer logical system) issue the run show
interfaces ge-1/1/9 | match hardware command.
[edit interfaces ge-1/0/7 unit 0]
lab@mxA-1# run show interfaces ge-1/1/9 | match hardware
Current address: 80:71:1f:c3:03:81, Hardware address: 80:71:1f:c3:03:81

Lab 9–28 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
Question: What is the MAC address of the ge-1/1/9
interface?

Answer: Although your answer might vary, the

previous output displays a MAC address of
80:71:1f:c3:03:81 for the ge-1/1/9 interface.

Step 5.14
Navigate to the [edit interfaces ge-1/0/7 unit 0] hierarchy level.
Configure a static ARP entry for the JFlow Collector/Analyzer (10.10.10.100) using
the MAC address that you acquired in Step 5.13. Then, activate the configuration by
issuing the commit command.
[edit interfaces ge-1/0/7 unit 0]
lab@mxA-1# up 2 edit ge-1/0/9.0

[edit interfaces ge-1/0/9 unit 0]

lab@mxA-1# set family inet address 10.10.10.1/24 arp 10.10.10.100 mac
ge-1/1/9-MAC

[edit interfaces ge-1/0/9 unit 0]

lab@mxA-1# commit
commit complete

[edit interfaces ge-1/0/9 unit 0]

lab@mxA-1#
Step 5.15
Examine the state of the child-1 port forwarding instance by issuing the
run show forwarding-options port-mirroring child-1 command.
[edit interfaces ge-1/0/9 unit 0]
lab@mxA-1# run show forwarding-options port-mirroring child-1
Instance Name: child-1
Instance Id: 3
Input parameters:
Rate : 1
Run-length : 0
Maximum-packet-length : 0
Output parameters:
Family State Destination Next-hop
inet up ge-1/0/9.0 10.10.10.100

www.juniper.net Monitoring the Network • Lab 9–29

Junos Troubleshooting in the NOC
Question: What is the state of the child-1 port
mirroring instance?

Answer: The previous output shows the child-1

port mirroring instance should be in the up state.

Step 5.16
Return to the second terminal session established with your assigned MX Series
device.
From the second terminal session, examine the output in the terminal window. If you
canceled the interface monitoring output, issue the monitor traffic
interface ge-1/1/9 command again. Type the Ctrl + c key combination when
you are finished examining the output. Then, issue the exit command to close the
terminal session.
lab@mxA-1> monitor traffic interface ge-1/1/9
verbose output suppressed, use <detail> or <extensive> for full protocol decode
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on ge-1/1/9, capture size 96 bytes

Reverse lookup for 10.12.12.100 failed (check DNS reachability).

Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.

01:16:46.553813 In IP 10.12.12.100 > 10.11.11.100: ICMP echo request, id 46346,

seq 3294, length 64
01:16:47.555273 In IP 10.12.12.100 > 10.11.11.100: ICMP echo request, id 46346,
seq 3295, length 64
01:16:48.556873 In IP 10.12.12.100 > 10.11.11.100: ICMP echo request, id 46346,
seq 3296, length 64
^C
3 packets received by filter
0 packets dropped by kernel

lab@mxA-1> exit

Lab 9–30 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC
Question: What do you see in the output?

Answer: The JFlow Collector/Analyzer is receiving

port mirrored packets that contain the source
address of 10.12.12.100 (the Host logical system)
and the destination address of 10.11.11.10 (the
SNMP Server logical system).

Step 5.17
Return to the third terminal session established with your assigned MX Series
device.
From the third terminal session, end the ping operation by typing the Ctrl + c key
combination. Then, issue the exit command to close the terminal session.
64 bytes from 10.11.11.100: icmp_seq=3723 ttl=63 time=0.672 ms
64 bytes from 10.11.11.100: icmp_seq=3724 ttl=63 time=0.686 ms
64 bytes from 10.11.11.100: icmp_seq=3725 ttl=63 time=3.643 ms
64 bytes from 10.11.11.100: icmp_seq=3726 ttl=63 time=0.682 ms
64 bytes from 10.11.11.100: icmp_seq=3727 ttl=63 time=5.014 ms
64 bytes from 10.11.11.100: icmp_seq=3728 ttl=63 time=2.118 ms
64 bytes from 10.11.11.100: icmp_seq=3729 ttl=63 time=3.589 ms
64 bytes from 10.11.11.100: icmp_seq=3730 ttl=63 time=0.748 ms
64 bytes from 10.11.11.100: icmp_seq=3731 ttl=63 time=4.118 ms
^C
--- 10.11.11.100 ping statistics ---
3732 packets transmitted, 3732 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.583/1.122/25.294/1.902 ms

lab@mxA-1> exit

Step 5.18
Return to the original terminal session established with your assigned MX Series
device.
From the original terminal session, navigate to the [edit] hierarchy level and issue
the load override jtnoc/reset.config command to load the reset
configuration file. Activate the configuration change and return to operational mode
by using the commit and-quit command.
[edit interfaces ge-1/0/9 unit 0]
lab@mxA-1# top

[edit]
lab@mxA-1# load override jtnoc/reset.config
load complete

www.juniper.net Monitoring the Network • Lab 9–31

Junos Troubleshooting in the NOC
[edit]
lab@mxA-1# commit and-quit
configuration check succeeds
commit complete
Exiting configuration mode

lab@mxA-1>
Step 5.19
Log out of your assigned MX Series device.
lab@mxA-1> exit

mxA-1 (ttyu0)

login:

STOP Tell your instructor that you have completed this lab.

Lab 9–32 • Monitoring the Network www.juniper.net

 Junos Troubleshooting in the NOC

www.juniper.net Monitoring the Network • Lab 9–33

Junos Troubleshooting in the NOC

Lab 9–34 • Monitoring the Network www.juniper.net