XMC 8.5 Technical Reference Guide
XMC 8.5 Technical Reference Guide
1 of 32
Monitor ExtremeControl Engine Memory Use
2 of 32
Monitor ExtremeControl Engine Memory Use
As you monitor an engine's physical and virtual memory, keep in mind that it is common
for Linux-based systems (such as the ExtremeControl engine) to show high memory
utilization. Once a process consumes memory, the memory remains allocated to the
process under the assumption it may be required in the future. If a different process calls
for that memory, and it is not in use, it is made available.
It is also important to monitor swap memory statistics for your ExtremeControl engines.
When an engine starts using swap memory, it indicates a potential issue, and more
active monitoring of the engine may be required. Running commands such as the "top"
command (see Linux "top" Command section under NAC Troubleshooting) provides
more accurate and up-to-date information on whether swap memory is actively being
used, and which processes are consuming the highest memory and CPU.
Use the Network tab to access the Host Resources report for an ExtremeControl engine.
Right-click on an ExtremeControl engine and select View Device Details > System >
Host Resources, as shown here.
Accessing the Host Resources Report
A Host Resources report for the ExtremeControl engine is displayed in a new tab, as
shown below.
3 of 32
View ExtremeControl Engine Historical Data
4 of 32
View ExtremeControl Engine Historical Data
The NAC History report is displayed in a new tab, as shown below. Look at the
NAC Appliance Summary report for engine load data.
NAC History Report
5 of 32
Monitor ExtremeControl Critical Events
6 of 32
Monitor ExtremeControl End-System Health
In Extreme Management Center, select the Control tab. Click on System to view the
Appliance Load report.
7 of 32
Create Alerts with ExtremeControl Notifications
8 of 32
Extreme Management Center Custom Reports
TIP: CPU usage can be monitored more closely in real-time using diagnostic tools such as the
Linux "top" command.
7/2020
8.5 Revision -00
PN: 9036785-00
Contents Subject to Change Without Notice
9 of 32
Configure RADIUS Clients to Monitor
ExtremeControl Engines in Extreme
Management Center (Legacy)
This Help topic tells you how to configure RADIUS monitoring tools to monitor
ExtremeControl engine performance and availability.
Use the following steps to create a list of RADIUS monitoring clients and configure a
special authentication mapping for your AAA configuration used to authenticate the
clients.
If you have multiple engine groups, you can use the same tools to monitor different
engine groups, but each engine group is configured separately.
1. Select the All Appliances group or an individual engine group in the NAC Manager
left-panel tree.
2. In the right-panel Configuration tab, click on the Edit button in the RADIUS Monitor
Clients field.
10 of 32
3. The Configure RADIUS Monitor Clients window opens.
4. Use this window to create a list of the monitoring tools (clients) used, and specify
the shared secret to be used for all of them.
a. Click the button. Enter the IP address for the first client and click OK.
Repeat for each client that you want to add.
b. Enter the Server Shared Secret used. This is a string of characters used to
encrypt and decrypt communications between the RADIUS Monitor clients
and the engines. This string must match the shared secret configured on the
client. Without the shared secret, the engines and clients will be unable to
communicate. The shared secret must be at least 6 characters long; 16
characters is recommended. Dashes are allowed in the string, but spaces are
not.
c. Re-enter the shared secret to verify it.
d. Click OK.
5. Use the NAC Manager toolbar button to open the NAC Configuration window
or use the Edit button in the Configuration tab.
11 of 32
6. Select the AAA configuration in the left panel.
7. In the right-panel mapping table, click the button to add a new mapping. (You
must be using an advanced AAA Configuration in order to see the mapping table. If
you are not, right-click on the AAA Configuration and select Make Advanced.)
8. The Add User to Authentication Mapping window opens.
12 of 32
will be used for all client authentications.
c. Click OK.
9. The new mapping will be listed in the mapping table. You can use the arrows to
adjust the position of the new mapping in the table. In the screen below you can
see that the RADIUS Monitor rule has been moved to the first row in the table
because it is more granular. Click Save to save your changes.
10. Click the Enforce toolbar button to enforce the new configuration to your engine
groups.
Any authentication request coming from an IP address that matches the list of RADIUS
monitor clients will be authenticated using the password you provided in the AAA
mapping. In these cases, the username does not matter. The password configured will
not be able to be used for authentication from any other part of the network. The
ExtremeControl engine responds back with a basic accept to any RADIUS monitor
client’s RADIUS request.
13 of 32
ExtremeControl Performance Tuning in
Extreme Management Center
The following sections provide detailed information on how to use specific
ExtremeControl tools and features to monitor and improve ExtremeControl performance.
l Monitoring Active End-Systems
l Tuning Data Persistence
l Tuning ExtremeControl Capacity
l Using ExtremeControl Distributed Cache
14 of 32
Monitoring Active End-Systems
NOTE: Any authentications previously performed by the unavailable primary engine remain
authenticated until the session is removed or times out. At that point, subsequent
authentication requests are sent to the backup engine. Whether authentication requests
automatically revert to the primary engine once it is deemed available is a function of
individual switch RADIUS operation.
To view capacity information for all ExtremeControl engines in one place, select the All
NAC Appliances folder in the tree and click on the NAC Appliances tab. The
authenticated user counts and engine capacity are displayed under the Capacity
column.
NAC Appliances Tab - Capacity
Engine load reporting is also available within Extreme Management Center. The NAC
Appliance Load report provides a summary of end-system usage for each
15 of 32
Tuning Data Persistence
16 of 32
Tuning Data Persistence
Age End-Systems
Retaining large amounts of stale end-system data can lead to Extreme Management
Center client performance issues as well as server performance degradation in larger
networks or on Extreme Management Center servers that may not have optimal
hardware. Reducing unnecessary stale data in the database leads to improved
performance, smaller (and faster) backup files, as well as reduced disk utilization on the
server. (Performance differences vary between individual ExtremeControl deployments.)
By default, stale end-systems are aged out after 90 days of inactivity. In high volume
networks with frequent short-term users (for example, an environment with a lot of
visitors or contractors), it might be appropriate to change the number of days to a lower
amount. Aging stale end-systems removes inactive and potentially one-time end-
systems from the database and NAC Manager tables, making it easier to monitor and
locate active end-systems on the network.
The option to remove associated MAC locks and occurrences in groups is disabled by
default. For networks with a large volume of short-term authentications, as well as users
who connect to the network infrequently but on a recurring basis, this ensures these end
users retain any assigned end-system group membership and are authorized against the
proper ExtremeControl rule the next time they authenticate to the network, should their
end-system age out. If this is not a concern, you may consider selecting this option to
remove group membership. Excessively large end-system groups can have an impact
on both the server and engine. This varies by deployment, but generally, networks
containing end-system groups with 30,000 to 35,000 end-systems should have this
option selected to ensure stale data is properly handled.
By default, end-system registration data associated with stale end-systems is also
removed when an end-system ages out. Even though registrations have an independent
expiration timer and removal option, this removes registrations associated with stale
end-systems prior to the defined registration expiration, maintaining active end-system
registrations in the database and keeping end-system and registration information in
sync.
17 of 32
Tuning Data Persistence
that are redundant or show no change leaves more space to retain those events that do
indicate active changes, maintaining end-system event efficiency.
However, networks with fewer end-systems, or those not utilizing ExtremeControl
features that create additional events such as registration and assessment, could
choose to enable the option to persist non-critical events, since they can display events
maintained in memory for a longer period than those in a more dynamic environment.
End-system events are stored in log files on the Extreme Management Center server.
These logs are available in the <install directory>/Extreme_
Networks/NetSight/appdata/logs directory and are identified by the filename convention:
nacESE.date_version.log (for example, nacESE.2012_12_31_01.log, nacESE.2012_
12_31_02.log). Events are continuously saved in the nacESE files with each individual
file growing to about 5 MB before it is archived and a new log file is started for that day.
Each day, when the Data Persistence check runs, it removes all log files that are older
than the number of days specified (90 days by default). The length of time to retain the
log files depends on your security policy (how long records need to be kept), system
hardware limitations (disk availability), and the overall amount and type of activity
logged.
The number of end-systems and activity on the network directly impacts the number of
nacESE event log files generated on a daily basis. Monitoring the number of files
generated for a period of time provides a baseline of the amount of space being
consumed by these events and helps determine whether additional action may be
required to manage them.
18 of 32
Tuning ExtremeControl Capacity
If insufficient resources is not the actual problem, the allocation of additional resources
may ultimately have little or no effect on performance. Because of this, it is important to
first verify that the Extreme Management Center server is installed on a system with
appropriate resources in terms of both hardware and role (a dedicated management
server versus one performing multiple roles) and that the resources are commensurate
with the size of the ExtremeControl deployment.
Insufficient server hardware could appear as an Extreme Management Center
performance issue, where in reality, the server hardware resources are not adequate for
the deployment.
19 of 32
Using ExtremeControl Distributed Cache
To adjust ExtremeControl Capacity, access the NAC Manager options. From the NAC
Manager menu bar, select Tools > Options to open the Options window. Expand the
NAC Manager Options folder and select Advanced Settings.
Advanced Settings Options - NAC Capacity
NOTE: The Distributed end-system cache functionality must be enabled in environments using the
ExtremeControl DNS Proxy to redirect clients to the captive portal.
When this feature is enabled, end-system information similar to that in the end-systems
table is stored in memory on the Extreme Management Center server and
ExtremeControl engine. Each cache contains the same up-to-date information, allowing
20 of 32
Using ExtremeControl Distributed Cache
the engine to perform lookups for end-system information in its local memory cache
instead of having to query the server for updated information. Any changes to end-
system information are propagated from each to the Extreme Management Center
server, which then replicates updates to each ExtremeControl engine so all have a
synchronized copy of real time end-system information.
Implementation of this feature is not recommended unless there is sufficient network
bandwidth available to handle the additional overhead in communicating updates, as
well as a fast connection between the Extreme Management Center server and the
ExtremeControl engine. Additional consideration should be taken prior to implementing
this functionality on engines that reside in a location where the data path traverses a
WAN link.
Select the Enable Distributed End-System Cache option on the Administration >
Options > Access Control > Advanced tab. Enabling this option requires an enforce of
the engine.
21 of 32
NAC Manager and
ExtremeControl Troubleshooting in
Extreme Management Center
The following sections provide information on tools used when troubleshooting
NAC Manager and ExtremeControl engine issues.
l NAC Manager Event Logging
l ExtremeControl Engine Real-time Status
l End-System Troubleshooting
l NAC Manager Events – This tab displays error and informational messages about
NAC Manager system operations, including configuration changes and enforce
operations.
Use this tab when trying to locate forensic information such as when and who
made changes to the ExtremeControl configuration, and when and for how long
communication with an ExtremeControl engine was lost. This event log also
captures NAC Manager functional and security-related warnings that the system
issues when auditing its own configuration, as well as events tied to data
persistence checks, including which end-systems were removed and when.
Important system notification messages are also logged here, including when new
22 of 32
ExtremeControl Engine Real-time Status
agent-less assessment updates are available and when certain system default
credentials should be changed.
l End-Systems Activity – This tab provides information on all the end-systems that
have attempted to connect to the network. It displays all end-system activity since
the client was launched.
l NAC Appliance Events – This tab provides information on ExtremeControl engine
system events including RADIUS configuration success or failure, completed
reauthentications, and management logins (such as Telnet or SSH configured for
external authentication). The event log displays engine activity since the NAC
Manager client was launched and like NAC Manager Events, is an excellent
source for historical information when performing a forensic investigation of a
recent event.
l Audit Events – This tab provides information on ExtremeControl Registration
events such as when a device or user is added during the registration process, or
an end-system is added, removed, or updated via the registration administration
web page. It displays all registration activity since the client was launched.
23 of 32
ExtremeControl Engine Real-time Status
In addition to the information in the table, you can launch two FlexViews with CPU,
memory, and disk utilization information from the right-click menu off one or more engine
in the NAC Appliances tab.
Launch the CPU Utilization View (Host Processor Load FlexView).
Host Processor Load FlexView
Launch the Memory and Diskspace Utilization View (Host Storage FlexView).
Host Storage FlexView
24 of 32
ExtremeControl Engine Real-time Status
The Home web page provides resource details such as current CPU and memory
usage. Status details provide a Current and Maximum counter for many critical functions.
Excessive authentication requests or failures are easily identified, including when the
Max Reached value occurred. This helps to identify the severity of a current problem or
match information with prior events when performing a forensic review.
NOTES: Memory usage is normally close to 100% to allow for better performance.
25 of 32
ExtremeControl Engine Real-time Status
For more information, see the ExtremeControl Engine Administration Web Page section
of the ExtremeControl Deployment Guide, which is in the NAC Manager user guide.
At the top, current and historical information is displayed on a per-switch basis. This
provides insight into problems such as a single switch flooding the network with
authentication requests, as well as comparative data that can be used to spot
abnormalities such as a switch with a limited number of active end-systems showing an
excessive number of authentications over the last month.
The Switch Configuration section is an overview of all switches assigned to the
ExtremeControl engine, the RADIUS response attributes they are configured for, and the
SNMP credential the ExtremeControl engine is using to communicate with the switch.
This information can be used to identify whether the ExtremeControl engine is using the
current SNMP credentials to contact the switch. This can be confirmed under the Switch
Dynamic Information where SNMP Contact will show as Contact Lost.
26 of 32
ExtremeControl Engine Real-time Status
More critical information here, although perhaps more useful for support technicians, are
the various workers assigned to each switch. These are dictated through the switch
discovery process and detail how the ExtremeControl engine performs various functions
such as using RFC 3576 or Toggle Link for reauthentication of an end-system. The
SNMP Contact is from the perspective of the ExtremeControl engine to the switch, which
may be different than from Extreme Management Center Console to the switch.
Engine Administration Web Page
27 of 32
End-System Troubleshooting
Administration Tab
ExtremeControl Status
The NAC Status option (previously available from the NAC Appliances tab) has been
updated and replaced by the Extreme Management Center Show Support functionality
described in the Extreme Management Center Troubleshooting section of the Extreme
Management Center Technical Reference.
The nacstatus command is still available from the ExtremeControl engine CLI and can
be executed to provide detailed data regarding the ExtremeControl engine. However,
the Show Support function is the recommended data collection vehicle, as it provides a
comprehensive look into both the operation of the server as well as all active
ExtremeControl engines.
End-System Troubleshooting
Use the following tools to monitor and trouble-shoot end-system issues in NAC
Manager.
28 of 32
End-System Troubleshooting
29 of 32
End-System Troubleshooting
Launch WebView
Expand the Diagnostics folder and select End System Diagnostics. Enable diagnostics
for both MAC and IP address.
Targeting diagnostics for a specific end-system enables a majority of the debug
diagnostics available on a global level, but only for the specific end-system. Therefore,
diagnostics can be enabled for an extended period of time without the concern of
generating the excessive log files that are possible when global diagnostics are
enabled.
The log data is saved to the same location as the global diagnostics, in the
/var/log/tag.log file of the ExtremeControl engine. A log entry is made in the tag.log
helping to locate the portion of the log from which to start a review.
2013-09-13 14:51:20,783 INFO [ESD] Enabling verbose diagnostics for MAC: 00-18-8B-
D6-E6-0C
2013-09-13 14:51:38,195 INFO [ESD] Enabling verbose diagnostics for IP: 10.20.87.100
30 of 32
End-System Troubleshooting
31 of 32
End-System Troubleshooting
32 of 32