White Paper

The Big Security Data Challenge

Make SIEM work for you
 Big Data is not only a challenge for customer-facing organizations—but for security
teams as well. Over the past decade, the demand for stronger security has driven the
collection and analysis of increasingly larger amounts of event and security contextual
data. Security Information and Event Management (SIEM) has long been the core
tool that security teams have depended on to manage and process this information.
However, as security data volume has grown, relational and time-indexed databases
that support SIEM are struggling under the event and analytics load. Legacy SIEM
systems have raised doubts about the potential success of SIEM implementations due
to their slow performance, inability to manage data effectively, and the extremely high
costs associated with scaling. This paper addresses the Big Security Data challenge
and highlights the key criteria organizations need to consider for processing security
information in light of today’s dynamic threat landscape.

Big Security Data

Why security data has become a Big Data problem is obvious for anyone who has tried to manage
a legacy SIEM, particularly when you look at the definition of Big Data. Big Data consists of data sets
that grow so large that they become awkward to work with using existing database management tools.
Challenges include capture, storage, search, sharing, analytics, and visualization.

With this in mind, it’s easy to see that IT and IT security have repeatedly wrestled with Big Data challenges. In
fact, SIEM itself was invented to address a fundamental lack of data processing capabilities. In the early 2000s,
the amount of security information and the level of accuracy of this security data exceeded the capability
of existing technologies, and the lack of centralized visibility developed a strong need for automated data
analysis. Enter the early SIEM tools, which were designed to handle firewall, vulnerability assessment, and
intrusion detection systems (IDS) data with the primary purpose of reducing false positives from IDS plus
the ability to investigate logs. These early SIEM vendors leveraged existing database management tools
and provided specialized analytics on top of event data to enable organizations to eliminate a large number
of IDS false positives.

While SIEM initially was adopted by security-conscious industries—such as large financial services and
government—broad adoption did not take off as a viable market until the mid-2000s, when Sarbanes Oxley
audit became a reality. Overnight, event management was a core component of the “control framework” in
Sarbanes Oxley section 404, and internal and external auditors were requiring it. Sarbanes Oxley was quickly
followed by PCI DSS for retail organizations and card processors, another major regulation that required log
review to pass audit and the automation that SIEM promised to provide. And then the regulatory explosion
began. The SIEM market exploded along with it—into a billion dollar market.

Compliance not only increased SIEM adoption but also led to a flood of additional security instrumenta-
tion and increased logging levels. This simultaneously increased the flood of data SIEM now had
to manage and further stretched analytic capabilities. Legacy SIEM systems had always struggled to
manage any increases in volume and correlation of security data. This dramatic growth in data and
correlation requirements further revealed the inherent scale and analytic limitations that these SIEM
solutions faced.

Fast forward to 2012. The demands on SIEM systems continue to intensify. Devastating data breaches
at organizations that had passed purportedly stringent compliance-based security audits have pushed
IT security to move from “check-the-box” compliance to comprehensive security programs that include
perimeter, insider, data, and system security. In response to these increased security controls, innovative
and persistent attackers have evolved the sophistication level of their attack methods—creating a need
for SIEM to detect low-and-slow attacks, rapidly detect anomalies in event flow, and gain contextual
information about data, applications, and databases.

2 The Big Security Data Challenge

These increasing demands on SIEM have stretched legacy SIEM solutions to their limit. These legacy SIEM
systems were built on databases and architectures with inherent limitations in their ability to handle large
volumes of events, historical data, and extensions of relational data. In addition, the analytic capabilities
of legacy SIEM systems are insufficient. Many organizations turn off important, but non-essential analytic
capabilities and spend hours waiting for a single report. These challenges have led to the question: “Does
SIEM work?” Given advancements in SIEM today, that question needs to shift to: “Does my SIEM solution
meet my current demands and will it scale, in both capacity and analytics, to meet evolving demands?”

Solving today’s Big Security Data challenge requires evolving from the traditional relational databases
and time-based flat file systems that older SIEMs leverage as their core analytic capability. Traditional
relational databases strain under the stress of simultaneous high-speed insertion rates combined with the
added burdens continuous real-time correlation and historical reporting. Time-based flat file systems fall
under the pressure of complex queries and, due to their limited indexing, can only offer basic correlation
capabilities.

Organizations looking to be successful with SIEM—whether they are first-time adopters or replacing
legacy SIEM—need to carefully evaluate the back-end capacity and analytics of SIEM solutions under
consideration to understand how intelligent the front-end will be for their needs today and tomorrow.
Below are some statistics from the Gartner report, “Information Security Is Becoming a Big Data Analytics
Problem, 23 March”:
• The amount of data analyzed by enterprise information security organizations will double every year
through 2016
• By 2016, 40 percent of enterprises will actively analyze at least 10 terabytes of data for information
security intelligence, up from less than 3 percent in 2011

Let’s look at some core capabilities of an ideal SIEM, why these capabilities are important, and how to
evaluate them in light of the Big Security Data problem organizations face today and will continue to
face in the future.

Relational data extensibility

Because the volumes of event data have grown exponentially and attacks have become more sophisticated,
it is critical to enrich event data with relational data about the source, asset, user, and data-intelligent
situational awareness. In addition, real-time correlation of this information with event flows needs to
be accommodated in the database architecture. If the database architecture can’t handle these millions
of relational data points, organizations will quickly hit a brick wall in expanding the intelligence of their
SIEM systems. Extensibility of features such as watch, asset, and user lists should be carefully evaluated, in
combination with the analytic capability to leverage this information intelligently. While many SIEMs have
these features, few can support multiple and expansive lists due to database side table limitations. Also,
to avoid analytic performance degradation, many SIEMs will simply provide a look-up of this information,
on request of the user, rather than correlate and present it in real time. A strong SIEM will use this informa-
tion to intelligently create an accurate, real-time picture of risk.

Dynamic analysis
Requirements for obtaining true situational awareness today goes far beyond simple event flow analysis,
which can tell you the frequency of connections and if there is a change. Today’s SIEM requires dynamic
situational that identifies changes in user behavior and dynamically adjusts risk based on source reputation
and asset risk, as well as the data, applications, and database activity that relates to it. Dynamic analysis is
a critical component of low-and-slow attack detection, and Big Security Data SIEM architectures need to
accommodate that.

The Big Security Data Challenge 3

 Historical analysis
Another key aspect of attack detection and efficient incident response is the ability to analyze historical
event data. With attack methods today, it is essential for an SIEM to be able to access years’ worth of
data to quickly pinpoint patterns and anomalies, while maintaining real-time analysis without performance
degradation. It also needs to be able to integrate easily with storage systems and efficiently store event data
to avoid extensive storage instrumentation and costs, offering an architecture that supports simultaneous
heavy use of real-time and historical functions.

Event surges
Most organizations with SIEM solutions in place will experience event surges—times when event data
grows beyond peak expected limits. When an event surge occurs, it is critical that analysts be able to
determine whether the increased volume is due to an active attack. SIEMs built for Big Security Data
are not only able to handle these surges, but also factor in these surges in their licensing schemes.
SIEMs that do not understand this problem will drop events or lock out analysts from the console when
the events per second (EPS) limits are exceeded—preventing security teams from accessing their primary
means of situational awareness when it matters most.

Summary
Automating security monitoring has proven essential in today’s threat environment, and to succeed,
today’s SIEM must have the right database back-end and must offer security intelligence that leverages
contextual data.

Learn More
For more information, visit www.mcafee.com/SIEM.

About McAfee
McAfee, a wholly owned subsidiary of Intel Corporation (NASDAQ:INTC), is the world’s largest dedicated
security technology company. McAfee delivers proactive and proven solutions and services that help secure
systems, networks, and mobile devices around the world, allowing users to safely connect to the Internet,
browse, and shop the web more securely. Backed by its unrivaled global threat intelligence, McAfee
creates innovative products that empower home users, businesses, the public sector, and service providers
by enabling them to prove compliance with regulations, protect data, prevent disruptions, identify vulner-
abilities, and continuously monitor and improve their security. McAfee is relentlessly focused on constantly
finding new ways to keep our customers safe. http://www.mcafee.com

2821 Mission College Boulevard McAfee and the McAfee logo are registered trademarks or trademarks of McAfee, Inc. or its subsidiaries in the United States and other
Santa Clara, CA 95054 countries. Other marks and brands may be claimed as the property of others. The product plans, specifications and descriptions herein are
888 847 8766 provided for information only and subject to change without notice, and are provided without warranty of any kind, express or implied.
www.mcafee.com Copyright © 2012 McAfee, Inc.
48000wp_siem-big-security_0912_fnl_ASD