Panorama™ Administrator's Guide: Manage Log Collection
Panorama™ Administrator's Guide: Manage Log Collection
Panorama™
Administrator’s
Guide
Version 7.1
Contact Information
Corporate Headquarters:
Palo Alto Networks
4401 Great America Parkway
Santa Clara, CA 95054
www.paloaltonetworks.com/company/contact‐us
About this Guide
This guide describes how to set up and use Panorama™ for centralized management; it is intended for administrators
who want the basic framework to quickly set up the Panorama virtual appliance or the M‐Series appliance for
centralized administration of Palo Alto Networks firewalls.
If you have an M‐Series appliance, this guide takes over after you finish rack mounting your M‐Series appliance.
For more information, refer to the following sources:
For information on how to configure other components in the Palo Alto Networks Next‐Generation Security
Platform, go to the Technical Documentation portal: https://www.paloaltonetworks.com/documentation or
search the documentation.
For access to the knowledge base, complete documentation set, discussion forums, and videos, refer to
https://live.paloaltonetworks.com.
For contacting support, for information on support programs, to manage your account or devices, or to open a
support case, refer to https://www.paloaltonetworks.com/support/tabs/overview.html.
For the most current PAN‐OS and Panorama 7.1 release notes, go to
https://www.paloaltonetworks.com/documentation/71/pan‐os/pan‐os‐release‐notes.html.
To provide feedback on the documentation, please write to us at: documentation@paloaltonetworks.com.
Palo Alto Networks, Inc.
www.paloaltonetworks.com
© 2016 Palo Alto Networks, Inc. Palo Alto Networks is a registered trademark of Palo Alto Networks. A list of our trademarks can be found
at http://www.paloaltonetworks.com/company/trademarks.html. All other marks mentioned herein may be trademarks of their
respective companies.
Revision Date: August 15, 2016
2 • Panorama 7.1 Administrator’s Guide © Palo Alto Networks, Inc.
Manage Log Collection
All Palo Alto Networks next‐generation firewalls can generate logs that provide an audit trail of firewall
activities. For Centralized Logging and Reporting, you must forward the logs generated on the firewalls to
Panorama. You can then configure Panorama to aggregate the logs and forward them to remote logging
destinations. If you forward logs to a Panorama virtual appliance, you don’t need to perform any additional
tasks to enable logging. If you will forward logs to an M‐Series appliance in Panorama mode or Log Collector
mode, you must add the Log Collectors as managed collectors and assign them to Collector Groups to access,
manage, and update the Log Collectors using Panorama. To determine which deployment best suits your
needs, see Plan a Log Collection Deployment.
To manage the System and Config logs that Panorama generates locally, see Monitor Panorama.
Configure a Managed Collector
Manage Collector Groups
Configure Log Forwarding to Panorama
Verify Log Forwarding to Panorama
Modify Log Forwarding and Buffering Defaults.
Configure Log Forwarding from Panorama to External Destinations
Log Collection Deployments
Configure a Managed Collector
To enable the Panorama management server (Panorama virtual appliance or M‐Series appliance in Panorama
mode) to manage a Log Collector, you must add it as a managed collector. The M‐Series appliance in
Panorama mode has a predefined (default) local Log Collector. However, switching from Panorama Mode to
Log Collector Mode would remove the local Log Collector and would require you to re‐configure the
appliance as a Dedicated Log Collector (M‐Series appliance in Log Collector mode). When the Panorama
management server has a high availability (HA) configuration, each HA peer can have a local Log Collector.
Dedicated Log Collectors don’t support HA.
We recommend that you install the same Applications update on Panorama as on managed
Collectors and firewalls. For details, see Panorama, Log Collector, and Firewall Version
Compatibility.
We recommend retaining a local Log Collector and local Collector Group on the M‐Series
appliance in Panorama mode, regardless of whether it manages Dedicated Log Collectors.
Configure a Managed Collector
Configure a Managed Collector (Continued)
Configure a Managed Collector (Continued)
Configure a Managed Collector (Continued)
Manage Collector Groups
A Collector Group is one or more Log Collectors that operate as a single logical unit for collecting firewall
logs. You can configure a Collector Group with multiple Log Collectors to ensure log redundancy or to
accommodate logging rates that exceed the capacity of a single Log Collector (see Panorama Platforms). To
understand the risks and recommended mitigations, see Caveats for a Collector Group with Multiple Log
Collectors.
The M‐Series appliance in Panorama mode (Panorama management server) has a predefined (default) local
Collector Group that contains a predefined local Log Collector. However, switching to Log Collector mode
would remove the local Log Collector and Collector Group; you would have to Set up the M‐Series Appliance
as a Log Collector, add it as a managed collector to the Panorama management server, and configure a
Collector Group to contain the managed collector.
If you delete a Collector Group, you will lose logs.
We recommend retaining a local Log Collector and local Collector Group on the M‐Series
appliance in Panorama mode, regardless of whether it manages Dedicated Log Collectors.
Configure a Collector Group
Move a Log Collector to a Different Collector Group
Remove a Firewall from a Collector Group
Configure a Collector Group
After you configure Log Collectors and firewalls, you must assign them to a Collector Group so that the
firewalls can send logs to the Log Collectors. A Collector Group with multiple Log Collectors has the
following requirements:
All the Log Collectors in any particular Collector Group must be the same platform: all M‐100 appliances
or all M‐500 appliances.
Log redundancy is available only if each Log Collector has the same number of logging disks. To add disks
to a Log Collector, see Increase Storage on the M‐Series Appliance.
Configure a Collector Group
2. In the General tab, enter a Name for the Collector Group if
you are adding one. You cannot rename an existing Collector
Group.
3. Enter the Minimum Retention Period in days (1‐2,000) for
which the Collector Group will retain firewall logs.
4. (Optional) Enable log redundancy across collectors to ensure
that no logs are lost if any one Log Collector in the Collector
Group becomes unavailable. Each log will have two copies and
each copy will reside on a different Log Collector. Redundancy
is available only if each Log Collector has the same number of
logging disks.
Enabling redundancy creates more logs and therefore requires
more storage capacity. When a Collector Group runs out of
space, it deletes older logs. Redundancy also doubles the log
processing traffic in a Collector Group, which reduces its
maximum logging rate by half, as each Log Collector must
distribute a copy of each log it receives.
If you add multiple Log Collectors to a single Collector
group, enabling redundancy is a best practice.
Configure a Collector Group (Continued)
Configure a Collector Group (Continued)
Move a Log Collector to a Different Collector Group
When you Plan a Log Collection Deployment, you assign Log Collectors to a Collector Group based on the
logging rate and log storage requirements of that Collector Group. If the rates and required storage increase
in a Collector Group, the best practice is to Increase Storage on the M‐Series Appliance or Configure a
Collector Group with additional Log Collectors. However, in some deployments, it might be more economical
to move Log Collectors between Collector Groups.
The log data on a Log Collector becomes inaccessible after you remove it from a Collector Group.
Also, you must perform a factory reset on the Log Collector before adding it to another Collector
Group; a factory reset removes all configuration settings and logs.
When a Log Collector is local to an M‐Series appliance in Panorama mode, move it only if the
M‐Series appliance is the passive peer in a high availability (HA) configuration. HA
synchronization will restore the configurations that the factory reset removes. Never move a Log
Collector when it’s local to an M‐Series appliance that is the active HA peer.
All the Log Collectors in any particular Collector Group must be the same platform: all M‐100
appliances or all M‐500 appliances.
Log redundancy is available only if each Log Collector has the same number of logging disks. To
add disks to a Log Collector, see Increase Storage on the M‐Series Appliance.
Move a Log Collector to Different Collector Group
Move a Log Collector to Different Collector Group (Continued)
Remove a Firewall from a Collector Group
In a distributed log collection deployment, where you have Dedicated Log Collectors, if you need a firewall
to send logs to Panorama instead of sending logs to the Collector Group, you must remove the firewall from
the Collector group.
When you remove the firewall from the Collector Group and commit the change, the firewall will
automatically send logs to Panorama instead of sending them to a Log Collector.
To temporarily remove the log forwarding preference list on the firewall, you can delete it using
the CLI on the firewall. You must however, remove the assigned firewalls in the Collector Group
configuration on Panorama. Otherwise, the next time you commit changes to the Collector
Group, the firewall will be reconfigured to send logs to the assigned Log Collector.
Remove a Firewall from a Collector Group
Configure Log Forwarding to Panorama
By default, firewalls store all log files locally. To aggregate logs on Panorama, you must configure the
firewalls to forward logs to Panorama.
To forward firewall logs directly to external services (for example, a syslog server) and also to
Panorama, see Configure Log Forwarding.
For details about all the log collection deployments that Panorama supports, see Log Forwarding
Options.
The PA‐7000 Series firewall can’t forward logs to Panorama, only to external services. However,
when you monitor logs or generate reports for a device group that includes a PA‐7000 Series
firewall, Panorama queries the firewall in real‐time to display its log data.
If Panorama will manage firewalls running software versions earlier than PAN‐OS 7.0, specify a
WildFire server from which Panorama can gather analysis information for WildFire samples that
those firewalls submit. Panorama uses the information to complete WildFire Submissions logs
that are missing field values introduced in PAN‐OS 7.0. Firewalls running earlier releases won’t
populate those fields. To specify the server, select Panorama > Setup > WildFire, edit the
General Settings, and enter the WildFire Private Cloud name. The default is
wildfire-public-cloud, which is the WildFire cloud hosted in the United States.
Configure Log Forwarding to Panorama
Configure Log Forwarding to Panorama (Continued)
Configure Log Forwarding to Panorama (Continued)
Verify Log Forwarding to Panorama
After you Configure Log Forwarding to Panorama, test that your configuration succeeded.
Verify Log Forwarding to Panorama
Step 1 Access the firewall CLI.
Step 2 If you configured Log Collectors, verify that each firewall has a log forwarding preference list.
> show log-collector preference-list
If the Collector Group has only one Log Collector, the output will look something like this:
Log collector Preference List
Serial Number: 003001000024
IP Address:10.2.133.48
Step 3 Verify that each firewall is forwarding logs.
> show logging-status device <firewall-serial-number>
For successful forwarding, the output indicates that the log forwarding agent is active. For a Panorama virtual
appliance, the agent is “Panorama.” For an M‐Series appliance, the agent is a “Log Collector.”
Step 4 View the average logging rate. The displayed rate will be the average logs/second for the last five minutes.
• If Log Collectors receive the logs, access the Panorama web interface, select Panorama > Managed
Collectors and click the Statistics link in the far‐right column.
• If a Panorama virtual appliance receives the logs, access the Panorama CLI and run the following command:
debug log-collector log-collection-stats show incoming-logs
This command also works on an M‐Series appliance.
Modify Log Forwarding and Buffering Defaults
You can define the log forwarding mode that the firewalls use to send logs to Panorama and, when
configured in a high availability (HA) configuration, specify which Panorama peer can receive logs. To access
these options, select Panorama > Setup > Management, edit the Logging and Reporting Settings, and select the
Log Export and Reporting tab.
Define the log forwarding mode on the firewall: The firewalls can forward logs to Panorama (pertains to
both the M‐Series appliance and the Panorama virtual appliance) in either Buffered Log Forwarding mode
or in the Live Mode Log Forwarding mode.
Logging Options Description
Define log forwarding preference on a Panorama virtual appliance that is in a high availability (HA)
configuration:
– When logging to a virtual disk, enable logging to the local disk on the active‐primary Panorama peer
only. By default, both Panorama peers in the HA configuration receive logs.
– When logging to an NFS, enable the firewalls to send only newly generated logs to a secondary
Panorama peer, which is promoted to primary, after a failover.
Configure Log Forwarding from Panorama to External
Destinations
Panorama enables you to forward logs to external servers, including syslog, email, and SNMP trap servers.
Forwarding firewall logs from Panorama reduces the load on the firewalls and provides a reliable and
streamlined approach to forwarding logs to remote destinations. You can also forward logs that Panorama
and its managed collectors generate.
To forward firewall logs directly to external services and also to Panorama, see Configure Log
Forwarding.
For details about all the log collection deployments that Panorama supports, see Log Forwarding
Options.
On a Panorama virtual appliance running Panorama 5.1 or earlier releases, you can use Secure
Copy (SCP) commands from the CLI to export the entire log database to an SCP server and import
it to another Panorama virtual appliance. A Panorama virtual appliance running Panorama 6.0 or
later releases, and M‐Series appliances running any release, do not support these options because
the log database on those platforms is too large for an export or import to be practical.
Configure Log Forwarding from Panorama to External Destinations
Configure Log Forwarding from Panorama to External Destinations (Continued)
Log Collection Deployments
The following topics describe how to configure log collection in the most typical deployments.
The deployments in these topics all describe Panorama in a high availability (HA) configuration.
Palo Alto Networks recommends HA because it enables automatic recovery (in case of server
failure) of components that are not saved as part of configuration backups. In HA deployments,
the Panorama management server only supports an active/passive configuration.
Plan a Log Collection Deployment
Deploy Panorama with Dedicated Log Collectors
Deploy Panorama with Default Log Collectors
Deploy Panorama Virtual Appliances with Local Log Collection
Plan a Log Collection Deployment
Panorama and Log Collector Platforms
Collector Groups with Single or Multiple Log Collectors
Log Forwarding Options
Panorama and Log Collector Platforms
Decide which Panorama Platforms to use for the Panorama management server and Log Collectors based
on the geographic distribution of managed firewalls, logging rate, and log retention requirements.
If you initially implement log collection using the default Log Collectors but later require more
storage or higher logging rates than these support, you can switch to a deployment with
Dedicated Log Collectors (M‐Series appliances in Log Collector mode). You can also implement a
hybrid deployment that includes both default and Dedicated Log Collectors. However, if you
initially implement log collection using Dedicated Log Collectors, you will lose logs if you later
switch to a deployment that involves only the default Log Collectors because of the reduced
storage capacity.
If you deploy firewalls remotely, consider the bandwidth requirement for the connection between the
firewalls and Panorama, in addition to whether Panorama supports the required logging rate. Deploying
Dedicated Log Collectors close to the firewalls can increase the bandwidth for log forwarding.
The following table summarizes your choice of Log Collector when considering the rate at which it receives
firewall logs.
Logging Rate Log Collector
Up to 10,000 Depends on the Panorama management server:
logs/second • Virtual appliance—Panorama collects logs without any Log Collector. Panorama running on
VMware vCloud Air or ESXi 5.5 and later versions can support a virtual disk of up to 8TB.
Earlier versions of the ESXi server support a virtual disk of up to 2TB. You can add an NFS
datastore for more than 8TB of storage.
• M‐Series appliance—Local predefined (default) Log Collector. Each M‐100 appliance can
store up to 4TB of log data; each M‐500 appliance can store up to 8TB of log data.
Up to 30,000 M‐100 appliance in Log Collector Mode. Each M‐100 appliance can process up to 30,000
logs/second logs/second and store up to 4TB of log data.
Up to 60,000 M‐500 appliance in Log Collector Mode. Each M‐500 appliance can process up to 60,000
logs/second logs/second and store up to 8TB of log data.
Collector Groups with Single or Multiple Log Collectors
You can configure a Collector Group with multiple Log Collectors to ensure log redundancy, increase the log
retention period, or accommodate logging rates that exceed the capacity of a single Log Collector (see
Panorama Platforms for capacity information). To understand the requirements, risks and recommended
mitigations, see Caveats for a Collector Group with Multiple Log Collectors.
All the Log Collectors in any particular Collector Group must be the same platform: all M‐500 appliances or all
M‐100 appliances. Log redundancy is available only if each Log Collector has the same number of logging disks.
Log Forwarding Options
By default, each firewall stores its log files locally. To use Panorama for centralized log monitoring and report
generation, you must Configure Log Forwarding to Panorama. You can also Configure Log Forwarding from
Panorama to External Destinations for archiving, notification, or analysis. When forwarding from Panorama,
you can include the System and Config logs that Panorama and its Log Collectors generate. External services
include syslog servers, email servers, or SNMP trap servers. The firewall, Panorama virtual appliance, or
M‐Series appliance that forwards the logs to external services converts the logs to the appropriate format
(syslog message, email notification, or SNMP trap).
Palo Alto Networks firewalls and Panorama support the following log forwarding options:
Forward logs from firewalls to Panorama and from Panorama to external services—This configuration is
best for deployments in which the connections between firewalls and external services have insufficient
bandwidth to sustain the logging rate, which is often the case when the connections are remote. This
configuration improves firewall performance by offloading some processing to Panorama.
You can configure each Collector Group to forward logs to different destinations.
Figure: Log Forwarding to Panorama and then to External Services
Forward logs from firewalls to Panorama and to external services in parallel—In this configuration, both
Panorama and the external services are endpoints of separate log forwarding flows; the firewalls don’t
rely on Panorama to forward logs to external services. This configuration is best for deployments in which
the connections between firewalls and external services have sufficient bandwidth to sustain the logging
rate, which is often the case when the connections are local.
Figure: Log Forwarding to External Services and Panorama in Parallel
Forward logs from firewalls directly to external services and also from Panorama to external services—
This configuration is a hybrid of the previous two and is best for deployments that require sending syslog
messages to multiple Security Information and Event Management (SIEM) solutions, each with its own
message format (for example, Splunk and ArcSight). This duplicate forwarding doesn’t apply to SNMP
traps or email notifications.
Deploy Panorama with Dedicated Log Collectors
The following figures illustrate Panorama in a Distributed Log Collection Deployment. In these examples, the
Panorama management server comprises two M‐Series appliances in Panorama mode that are deployed in
an active/passive high availability (HA) configuration. Alternatively, you can use an HA pair of Panorama
virtual appliances. The firewalls send logs to Dedicated Log Collectors (M‐Series appliances in Log Collector
mode). This is the recommended configuration if the firewalls generate over 10,000 logs/second. (For details
on deployment options, see Plan a Log Collection Deployment.)
If you will assign more than one Log Collector to a Collector Group, see Caveats for a Collector Group with
Multiple Log Collectors to understand the requirements, risks, and recommended mitigations.
Figure: Single Dedicated Log Collector Per Collector Group
Figure: Multiple Dedicated Log Collectors Per Collector Group
Perform the following steps to deploy Panorama with Dedicated Log Collectors. Skip any steps you have
already performed (for example, the initial setup).
Deploy Panorama with Dedicated Log Collectors
Deploy Panorama with Dedicated Log Collectors (Continued)
Deploy Panorama with Dedicated Log Collectors (Continued)
Deploy Panorama with Dedicated Log Collectors (Continued)
Deploy Panorama with Dedicated Log Collectors (Continued)
Deploy Panorama with Default Log Collectors
The following figures illustrate Panorama in a centralized log collection deployment. In these examples, the
Panorama management server comprises two M‐Series appliances in Panorama mode that are deployed in
an active/passive high availability (HA) configuration. The firewalls send logs to the predefined (default) local
Log Collector on each Panorama M‐Series appliance. This is the recommended deployment if the firewalls
generate up to 10,000 logs/second. (For details on deployment options, see Plan a Log Collection
Deployment.)
If you will assign more than one Log Collector to a Collector Group, see Caveats for a Collector Group with
Multiple Log Collectors to understand the requirements, risks, and recommended mitigations.
After implementing this deployment, if the logging rate increases beyond 10,000 logs per second, Palo Alto
Networks recommends that you add Dedicated Log Collectors (M‐Series appliances in Log Collector mode) as
described in Deploy Panorama with Dedicated Log Collectors. Such an expansion might require reassigning
firewalls from the default Log Collectors to Dedicated Log Collectors.
Figure: Single Default Log Collector Per Collector Group
Figure: Multiple Default Log Collectors Per Collector Group
Perform the following steps to deploy Panorama with default Log Collectors. Skip any steps you have already
performed (for example, the initial setup).
Deploy Panorama with Default Log Collectors
Deploy Panorama with Default Log Collectors (Continued)
Deploy Panorama with Default Log Collectors (Continued)
Deploy Panorama with Default Log Collectors (Continued)
Deploy Panorama with Default Log Collectors (Continued)
Deploy Panorama with Default Log Collectors (Continued)
Deploy Panorama Virtual Appliances with Local Log Collection
The following figure illustrates Panorama in a centralized log collection deployment. In this example, the
Panorama management server comprises two Panorama virtual appliances that are deployed in an
active/passive high availability (HA) configuration. This configuration suits firewall management within a
VMware virtual infrastructure in which Panorama processes up to 10,000 logs/second. (For details on
deployment options, see Plan a Log Collection Deployment.) The firewalls send logs to the Panorama
management server (to its virtual disk or Network File System [NFS] datastore). By default, the active and
passive peers both receive logs, though you can Modify Log Forwarding and Buffering Defaults so that only
the active peer does. By default, the Panorama virtual appliance uses approximately 11GB on its internal disk
partition for log storage, though you can Expand Log Storage Capacity on the Panorama Virtual Appliance if
necessary.
If the logging rate increases beyond 10,000 logs per second, it is recommended that you add
Dedicated Log Collectors (M‐Series appliances in Log Collector mode). Deploy Panorama with
Dedicated Log Collectors describes a deployment with Dedicated Log Collectors managed by
Panorama virtual appliances or by M‐Series appliances in Panorama mode.
Figure: Panorama Virtual Appliances with Local Log Collection
Perform the following steps to deploy Panorama virtual appliances with local log collection. Skip any steps
you have already performed (for example, the initial setup).
Deploy Panorama Virtual Appliances with Local Log Collection