Skip to main content

Sift Security vs. Elastic Search and Elastic Graph

Sift Security vs. Elastic Search and Elastic Graph


We are often asked, “What is the difference between Sift Security and Elastic Graph?” This is a great question that typically comes from folks who are already familiar with Elasticsearch [0] and Elastic Graph [1]. The answer boils down to the following: Elastic Graph is a tool for visualizing arbitrary aggregate search results. Elasticsearch is a Restful search that distributed, and has analytics engine that solves a number of use cases such as mapping from Python to ES REST endpoints. Sift Security uses a graph database to simplify and accelerate specific security use cases. In this blog post, we describe the advantages of each of these approaches, and conclude with a discussion of when to use each.

Advantages of Sift Security vs ElasticSearch and Elastic Graph

Query speed

Sift Security builds a property graph to represent security log events at ingestion time.  We do this work at ingestion time for one reason:  to speed up common investigative queries.  When investigating alerts and incidents, analysts ask questions like:
  • Did this user/host/IP address generate any other alerts?
  • Was there any lateral movement from this host?
  • What users were logged in to this host when this alert triggered?
  • Were there any other entities affected by this attack? 
These investigative queries can all be represented as graph traversals.  The graph traversals are constant time operations because the work to represent them in the graph was done at ingestion time.   This enables analysts to rapidly explore many investigative paths, following log chains of events over multiple hops.  These queries would require multiple aggregation queries using search or many joins using a relational database.

Ease to work with security logs

Our cybersecurity experts have already done the hard work of defining the important entities and relationships for common security logs.  This means that analysts don’t have to worry about data formats, key names, or the information conveyed in individual log lines.  Our graph model defines the possible queries and surfaces this information to the user in an interactive visualization.  The graph model presents enough information to the user for them to understand what happened, without getting bogged down in details.
The following screenshot illustrates this behavior.  We have clicked on “administrator” in the graph.  The right panel shows all the relationships we could add to the graph on the right side.  Relationships are categorized, color coded, and have descriptive names.  Clicking any of the blue relationship names will add the associated edges to the graph.

Sift Security vs. Elastic Search and Elastic Graph 

Graph analytics

Query speed isn’t just important for the analyst, it is also important for the analytics. Having a graph database enables Sift Security to quickly and efficiently find complex, multi-stage threats by automatically identifying threat clusters and other anomalous structures in the graph.  These analytics can also be performed in the context of an investigation.  Clicking an entity like Catheryn  and choosing “Find Risks” in the graph will search the graph for any alerts nearby to Catheryn in the graph, even if they don’t explicitly involve Catherine.  For example, this would also find alerts coming from hosts Catheryn has logged onto and IP addresses Catheryn has used.
Sift Security vs. Elastic Search and Elastic Graph

No limits on result set size

Whereas Elastic uses fixed-size aggregations  to populate its graph visualization, Sift Security enables queries that return all of the possible results, such as “What processes did this user run within this time window?”. This is helpful for traversals that require a set intersection, such as  “What processes did both of these users execute?”.  Sift Security excels at displaying fine-grained, detailed, targeted information.

Advantages of Elastic Graph

Flexibility

Elastic Graph has no pre-defined, fixed structure to the graph. Any two aggregatable fields in a document can be visualized as entities and relationships. This means that a user must specify the fields they want to visualize from a list of all the indexed fields.

Focus on significance

Using significance measures helps investigations over large datasets.  It is most useful when the goal is to find the elements from a set that best characterize an entity.  For example, “What are the most significant processes executed on this host?” could help quickly identify what a host is used for.  Fixed-size result sets enable users to quickly see and understand query results.

Identifying bulk structures in unstructured data

Due to its fixed size queries, Elastic Graph excels as displaying bulk structure surrounding the results of multiple simultaneous significance queries.  This approach is especially useful for tasks such as identifying similar texts based on which significant terms they have in common.

Elasticsearch get your answers instantly

Elasticsearch can perform structured, unstructured, geo and metric searches.

When to use Sift Security vs. Elastic Graph

Sift Security vs. Elastic Search and Elastic Graph

Elastic Graph’s strength is in its flexibility.  If you are a power Elasticsearch user who relies on aggregate queries to find meaning in your structured or unstructured data, Elastic Graph will help visualize and identify structure in those results. If you need the flexibility of visualizing arbitrary fields or unstructured data, and the answers to your questions are best defined by aggregate searches, then Elastic Graph is the right choice.  For example, Elastic Graph can help you explore Shakespeare’s works or product sales data.

Sift Security is built with specific use cases in mind:  security operations and incident response, both on premise and in the cloud. Our graph model, architecture, and analytics help to identify the most serious threats  and enable analysts to rapidly investigate and resolve incidents. If you are looking to streamline your security ops, incident response, threat hunting, or cloud-ops practices,  Sift Security is the right choice.

To get started with the documentation for Elasticsearch, Kibana, Logstash please visit this Elasticsearch tutorial
To learn more about the power of graph databases, see our whitepaper “Why Graph?”.  This tutorial, similar to elastic search tutorials, can help you to understand how Sift Security can benefit your organization, email us at contact@siftsecurity.com.

[0] Elasticsearch.  https://www.elastic.co/products/elasticsearch
[1] Elastic Graph.  https://www.elastic.co/products/x-pack/graph
[2] “Why Graph”,  Sift Security.  https://siftsecurity.com/papers/Sift-Security-Why-Graph.html 

Comments

Popular posts from this blog

Sift Security Tools Release for AWS Monitoring - CloudHunter

We are excited to release CloudHunter, a web service similar to AWS CloudTrail that allows customers to visually explore and investigate their AWS cloud infrastructure.  At Sift, we felt this integration would be important for 2 main reasons:
Investigating events happening in AWS directly from Amazon is painful, unless you know exactly what event you're looking for.There are not many solutions that allow customers to follow chains of events spanning across the on-premises network and AWS on a single screen. At Netflix, we spent a lot of time creating custom tools to address security concerns in our AWS infrastructure because we needed to supplement the AWS logs, and created visualizations based on that data.  The amazing suite of open source tools from Netflix are the solutions they used to resolve their own pain points.  Hosting microservices in the cloud with continuous integration and continuous deployment can be extremely efficient and robust.  However, tracking events, espec…

Sift Security and WannaCry

😢 The WannaCry ransomware attack has left security teams around the world scrambling to make sure they are protected and to assess whether they have been victimized. To protect themselves, organizations need to have visibility into which systems are vulnerable and be able to rapidly roll out patches.  To understand whether they have been targeted, they need visibility into the channels over which the ransomware is distributed.  To understand whether they have been infected, they need visibility into the endpoints.
Over the past weekend, I was bombarded with questions from current customers, potential customers, former colleagues, friends, and family.  Am I vulnerable? How do I protect myself? How do I know if I’ve been hit? What do I do if I’ve been hit? What can you do to help me?
This post focuses primarily on the last question. What can we at Sift Security do to help an organization respond to a massive ransomware attack? I break this down into four categories, visibility, analyt…

Next-Generation SIEMs

Intelligence, Speed, Simplicity, AutomationOverviewMajor network and security trends, including exponentially increasing network traffic, cloud architectures, complex attack surfaces, and advanced adversaries, have created new challenges for security operations in adapting to this changing threat landscape.
Increased Traffic, Hybrid Cloud Architectures, and Sophisticated Adversaries Are Overwhelming SOCs
The Security Operations Center is faced with alert overload, often paralyzed with too much information to filter, prioritize, and act upon. The end result is an inability to find the true risk amongst thousands of alerts every day with the largest of organizations easily facing millions of alerts per day.Incident responders are challenged at finding relevant information or seeing the relationships amongst disparate indicators of compromise that are buried within logs, causing delays in assessing, understanding, and mitigating incidents.T…