Skip to main content

Next-Generation SIEMs

Next-Generation SIEMs

Intelligence, Speed, Simplicity, Automation


Major network and security trends, including exponentially increasing network traffic, cloud architectures, complex attack surfaces, and advanced adversaries, have created new challenges for security operations in adapting to this changing threat landscape.

Next-Generation SIEMs
Increased Traffic, Hybrid Cloud Architectures, and Sophisticated Adversaries Are Overwhelming SOCs

The Security Operations Center is faced with alert overload, often paralyzed with too much information to filter, prioritize, and act upon. The end result is an inability to find the true risk amongst thousands of alerts every day with the largest of organizations easily facing millions of alerts per day.

Incident responders are challenged at finding relevant information or seeing the relationships amongst disparate indicators of compromise that are buried within logs, causing delays in assessing, understanding, and mitigating incidents.

Threat hunting teams struggle with where to start, where to look, and how to analyze a massive amount of data. Log search tools and SIEMs do little to assist in finding new and unknown risk and require the user to have a strong understanding of low-level log data in order to know what to search for.

Traditional measures such as log management, event management, and SIEMs have failed to adapt to the changing threat landscape. These dated technologies are drastically less effective with the huge volumes of data collected from the network, servers, and applications, leading to false positives, false negatives and difficulty in tuning. Most customers continue to struggle to find a way to reduce the noise down to a small number of relevant alerts.

When advanced adversaries attack thinly-stretched security teams using sophisticated methods, it makes it difficult if not impossible for security professionals to protect their organizations in a proactive manner.

A new breed of tools and approaches are needed that have: cloud-enablement, new security intelligence, faster detection and response, scalability in performance and cost, and simplicity in deployment and use.


Today's IT and Security Operations teams are faced with numerous challenges: exponentially increasing network traffic, complex cloud architectures, growing attack surfaces, sophisticated adversaries, and costly and complicated security solutions that require a lot of expertise and time to maintain and use. Further, security teams don't know what they don't know, making the finding of new threats too much of a haphazard exercise. With a scarcity of skilled security professionals, existing teams are overtaxed, making it near-impossible to be proactive in planning security measures.

Each year has brought exponentially increasing volumes of network traffic, web content, messaging (email, chat, IM, VoIP), network traffic (TCP/IP), and application traffic (web services, web applications) to not only on-premise infrastructure, but also growing cloud infrastructures. The amount of traffic or events at the network level has made it difficult or impossible to assess, comprehend, analyze, and react to cyber adversaries, let alone take proactive defensive measures.

This has resulted in several adverse impacts:

  • Higher number of false positives leading to wild goose chases
  • Higher chance for false negatives passing by undetected
  • Difficulty in focusing on true risk amidst an overwhelming number of alerts

Production infrastructures now include a hybrid of on-premise and cloud deployments, but cannot be treated the same as on-premise architectures. Their technology stacks include sophisticated javascript UIs, backend messaging, multi-layered identity/authorization systems, distributed data stores, and elastic application servers and load-balancers. These opaque cloud architectures and complicated web applications have increased the attack surface, made it more complex to understand, and much harder to secure. Security teams have less control and visibility into these highly dynamic environments with elastic application and networking instances, and understanding what is going on in an organization's cloud architecture can be extremely difficult, resulting in missed alerts for availability and performance problems, and undiscovered or misunderstood security incidents.

Adversaries continue to grow more sophisticated, not only in their use of exploits that include polymorphic malware, ransomware, and Zeus bot strains, but also in their organizational capabilities around executing espionage and fraud. The authorities' hunt for the Zeus bot author is just one example of both the technical and organizational sophistication of today's cyber criminals, which in the case of Zeus, has resulted in financial fraud of banks of conservatively 70 to 80 million dollars. (1) The Bangladesh bank heist of $81M via the SWIFT network and possible insider assistance is another example of the evolving sophistication of cyber criminals.(2) Not only is the financial impact larger than ever before, the technical sophistication has also increased.

These attacks showed technical sophistication with redundant command-and-control bot networks, use of DoS traffic (knowing that high volumes of data overwhelm today's networks, tools and SoCs) to obfuscate the true attacks. Additionally, there is even business model sophistication in the renting of the Zeus botnets to other criminal groups for other purposes.

And although there are many high-profile breaches publicized almost on a weekly basis, there are many more that go undetected. As of 2016, the time to detect breaches remains anywhere from 5 months to over a year.(3)

Today's security approaches continue to fall short in adapting to these challenges. Existing tools and approaches lack the intelligence and analytics to identify and respond to true risk amidst the rising flood of expected network traffic, not to mention the distributed denial of service attacks that can be easily created in today's botnets-for-rent world.

SIEMs and log search tools have not adapted to evolving cloud architectures. With their dynamic, API-driven behavior and highly elastic runtime environments, a log-based, event-based view of cloud architectures can result in so much confusing noise, that it can completely obscure any real problems around availability, performance, or compromise.

The rigid frameworks of applying signatures and rules to voluminous data, whether it be packet captures or terabytes of log data is not working for either prevention or detection. Storms of red, priority one alerts continue to plague the SOC. Decreasing false positives often increases false negatives and vice-versa. And one can only create rules for known bad, so new attacks slip through.

Rules-based correlation approaches are also ineffective and unscalable as the rules have obtuse and arcane syntax, many ordering dependencies, and require exorbitant amounts of people time to define, tune, and maintain the logic. This results in security policies that are unclear and unmanageable by the very security professionals who are trying to secure the organization.

Next-Generation SIEMs
Rules-Based Correlation obfuscates, confuses, misses the unknown bad, and requires heavy maintenance.

Read more:

(1) "Inside the Hunt for Russia's Most Notorious Hacker," Graff, Wired, 3/21/17.

(2) "Hackers' $81 Million Sneak Attack on World Banking," Corkery, New York Times, 4/30/16. 

(3) "Breach Detection Time Improves, Destructive Attacks Rise: FireEye," Lennon, Security Week, 2/25/16. 

Popular posts from this blog

Data Exfiltration from AWS S3 Buckets

You will have no doubt heard by now about the recent Booz Allen Hamilton breach that took place on Amazon Web Services – in short, a shocking collection of 60,000 government sensitive files were left on a public S3 bucket (file storage in Amazon Web Services) for all to see. We are all probably too overwhelmed to care, given all the recent breaches we have been hearing about in the news. But with this breach it was different, it involved a trusted and appointed contractor whose job it was to follow security policies, put in place to avoid such incidents. So was this incident accidental or malicious? More, later about the tools we can use to tell the difference between the two. First, lets recap what happened. The Incident According to Gizmodo , the 28GB of data that was leaked not only contained sensitive information on recent government projects, but at least a half dozen unencrypted passwords belonging to government contractors with Top Secret Clearance – meaning anyone who got

Sift Joins Netskope, the Cloud Security Leader

Four years ago, we started Sift with the mission of simplifying security operations and incident response for the public cloud. In that time, we have assembled a fantastic team, created an innovative cloud detection and response solution, and have worked with many market-leading customers. I’m delighted to share that we’ve taken yet another step forward — as announced today, Sift is now officially part of Netskope. You can read more about this on Netskope CEO Sanjay Beri’s  blog  or in the official  announcement  on the Netskope website. For our customers, investors, partners, and team, this is an exciting new chapter. Let me tell you why we’re so excited.  Since the beginning, Netskope has had an unmatched vision for the cloud security market. Having started in 2012, they initially focused on SaaS security and quickly followed that with IaaS security capabilities. Six years later, they are now more than 500 employees strong and used by a quarter of the Fortune 100. They are a l

Sift Security vs. Elastic Search and Elastic Graph

We are often asked, “What is the difference between Sift Security and Elastic Graph ?” This is a great question that typically comes from folks who are already familiar with Elasticsearch [0] and Elastic Graph [1]. The answer boils down to the following: Elastic Graph is a tool for visualizing arbitrary aggregate search results. Elasticsearch is a Restful search that distributed, and has analytics engine that solves a number of use cases such as mapping from Python to ES REST endpoints. Sift Security uses a graph database to simplify and accelerate specific security use cases. In this blog post, we describe the advantages of each of these approaches, and conclude with a discussion of when to use each. Advantages of Sift Security vs ElasticSearch and Elastic Graph Query speed Sift Security builds a property graph to represent security log events at ingestion time.  We do this work at ingestion time for one reason:  to speed up common investigative queries.  When investi