Skip to main content

Sift Security vs. Elastic Search and Elastic Graph

Sift Security vs. Elastic Search and Elastic Graph


We are often asked, “What is the difference between Sift Security and Elastic Graph?” This is a great question that typically comes from folks who are already familiar with Elasticsearch [0] and Elastic Graph [1]. The answer boils down to the following: Elastic Graph is a tool for visualizing arbitrary aggregate search results. Elasticsearch is a Restful search that distributed, and has analytics engine that solves a number of use cases such as mapping from Python to ES REST endpoints. Sift Security uses a graph database to simplify and accelerate specific security use cases. In this blog post, we describe the advantages of each of these approaches, and conclude with a discussion of when to use each.

Advantages of Sift Security vs ElasticSearch and Elastic Graph

Query speed

Sift Security builds a property graph to represent security log events at ingestion time.  We do this work at ingestion time for one reason:  to speed up common investigative queries.  When investigating alerts and incidents, analysts ask questions like:
  • Did this user/host/IP address generate any other alerts?
  • Was there any lateral movement from this host?
  • What users were logged in to this host when this alert triggered?
  • Were there any other entities affected by this attack? 
These investigative queries can all be represented as graph traversals.  The graph traversals are constant time operations because the work to represent them in the graph was done at ingestion time.   This enables analysts to rapidly explore many investigative paths, following log chains of events over multiple hops.  These queries would require multiple aggregation queries using search or many joins using a relational database.

Ease to work with security logs

Our cybersecurity experts have already done the hard work of defining the important entities and relationships for common security logs.  This means that analysts don’t have to worry about data formats, key names, or the information conveyed in individual log lines.  Our graph model defines the possible queries and surfaces this information to the user in an interactive visualization.  The graph model presents enough information to the user for them to understand what happened, without getting bogged down in details.
The following screenshot illustrates this behavior.  We have clicked on “administrator” in the graph.  The right panel shows all the relationships we could add to the graph on the right side.  Relationships are categorized, color coded, and have descriptive names.  Clicking any of the blue relationship names will add the associated edges to the graph.

Sift Security vs. Elastic Search and Elastic Graph 

Graph analytics

Query speed isn’t just important for the analyst, it is also important for the analytics. Having a graph database enables Sift Security to quickly and efficiently find complex, multi-stage threats by automatically identifying threat clusters and other anomalous structures in the graph.  These analytics can also be performed in the context of an investigation.  Clicking an entity like Catheryn  and choosing “Find Risks” in the graph will search the graph for any alerts nearby to Catheryn in the graph, even if they don’t explicitly involve Catherine.  For example, this would also find alerts coming from hosts Catheryn has logged onto and IP addresses Catheryn has used.
Sift Security vs. Elastic Search and Elastic Graph

No limits on result set size

Whereas Elastic uses fixed-size aggregations  to populate its graph visualization, Sift Security enables queries that return all of the possible results, such as “What processes did this user run within this time window?”. This is helpful for traversals that require a set intersection, such as  “What processes did both of these users execute?”.  Sift Security excels at displaying fine-grained, detailed, targeted information.

Advantages of Elastic Graph

Flexibility

Elastic Graph has no pre-defined, fixed structure to the graph. Any two aggregatable fields in a document can be visualized as entities and relationships. This means that a user must specify the fields they want to visualize from a list of all the indexed fields.

Focus on significance

Using significance measures helps investigations over large datasets.  It is most useful when the goal is to find the elements from a set that best characterize an entity.  For example, “What are the most significant processes executed on this host?” could help quickly identify what a host is used for.  Fixed-size result sets enable users to quickly see and understand query results.

Identifying bulk structures in unstructured data

Due to its fixed size queries, Elastic Graph excels as displaying bulk structure surrounding the results of multiple simultaneous significance queries.  This approach is especially useful for tasks such as identifying similar texts based on which significant terms they have in common.

Elasticsearch get your answers instantly

Elasticsearch can perform structured, unstructured, geo and metric searches.

When to use Sift Security vs. Elastic Graph

Sift Security vs. Elastic Search and Elastic Graph

Elastic Graph’s strength is in its flexibility.  If you are a power Elasticsearch user who relies on aggregate queries to find meaning in your structured or unstructured data, Elastic Graph will help visualize and identify structure in those results. If you need the flexibility of visualizing arbitrary fields or unstructured data, and the answers to your questions are best defined by aggregate searches, then Elastic Graph is the right choice.  For example, Elastic Graph can help you explore Shakespeare’s works or product sales data.

Sift Security is built with specific use cases in mind:  security operations and incident response, both on premise and in the cloud. Our graph model, architecture, and analytics help to identify the most serious threats  and enable analysts to rapidly investigate and resolve incidents. If you are looking to streamline your security ops, incident response, threat hunting, or cloud-ops practices,  Sift Security is the right choice.

To get started with the documentation for Elasticsearch, Kibana, Logstash please visit this Elasticsearch tutorial
To learn more about the power of graph databases, see our whitepaper “Why Graph?”.  This tutorial, similar to elastic search tutorials, can help you to understand how Sift Security can benefit your organization, email us at contact@siftsecurity.com.

[0] Elasticsearch.  https://www.elastic.co/products/elasticsearch
[1] Elastic Graph.  https://www.elastic.co/products/x-pack/graph
[2] “Why Graph”,  Sift Security.  https://siftsecurity.com/papers/Sift-Security-Why-Graph.html 

Popular posts from this blog

Sift Joins Netskope, the Cloud Security Leader

Four years ago, we started Sift with the mission of simplifying security operations and incident response for the public cloud. In that time, we have assembled a fantastic team, created an innovative cloud detection and response solution, and have worked with many market-leading customers. I’m delighted to share that we’ve taken yet another step forward — as announced today, Sift is now officially part of Netskope. You can read more about this on Netskope CEO Sanjay Beri’s  blog  or in the official  announcement  on the Netskope website. For our customers, investors, partners, and team, this is an exciting new chapter. Let me tell you why we’re so excited.  Since the beginning, Netskope has had an unmatched vision for the cloud security market. Having started in 2012, they initially focused on SaaS security and quickly followed that with IaaS security capabilities. Six years later, they are now more than 500 employees strong and used by a quarter of the Fortune 100. They are a l

Data Exfiltration from AWS S3 Buckets

You will have no doubt heard by now about the recent Booz Allen Hamilton breach that took place on Amazon Web Services – in short, a shocking collection of 60,000 government sensitive files were left on a public S3 bucket (file storage in Amazon Web Services) for all to see. We are all probably too overwhelmed to care, given all the recent breaches we have been hearing about in the news. But with this breach it was different, it involved a trusted and appointed contractor whose job it was to follow security policies, put in place to avoid such incidents. So was this incident accidental or malicious? More, later about the tools we can use to tell the difference between the two. First, lets recap what happened. The Incident According to Gizmodo , the 28GB of data that was leaked not only contained sensitive information on recent government projects, but at least a half dozen unencrypted passwords belonging to government contractors with Top Secret Clearance – meaning anyone who got

How Sift Security's Analytics Engine Detects Insider Threats

Intro We work with a lot of organizations that are worried about insider threats. Their employees require access to sensitive customer data or other proprietary information. They are worried that a careless or disgruntled employee may expose that information to outsiders. Moreover, they are worried that they might not notice it if it happened. Insider threat detection is one of the main use cases of User and Entity Behavioral Analytics (UEBA). UEBA is the practice of modeling normal user and entity behavior in order to identify anomalies indicative of a cyber threat. This post describes how Sift Security’s detection and analytics engine can be used for insider threat detection. Dataset For this post, we use the CERT insider threat tools datasets [1]. These are synthetic datasets from CERT that include background data and malicious attackers. Included are authentication, email, removable storage, and web browsing data. This post focuses on the first scenario in the r6 datasets,