Skip to main content

Sift Security vs. Elastic Graph



We are often asked, “What is the difference between Sift Security and Elastic Graph?” This is a great question that typically comes from folks who are already familiar with Elasticsearch [0] and Elastic Graph [1]. The answer boils down to the following: Elastic Graph is a tool for visualizing arbitrary aggregate search results. Sift Security uses a graph database to simplify and accelerate specific security use cases. In this blog post, we describe the advantages of each of these approaches, and conclude with a discussion of when to use each.

Advantages of Sift Security

Query speed

Sift Security builds a property graph to represent security log events at ingestion time.  We do this work at ingestion time for one reason:  to speed up common investigative queries.  When investigating alerts and incidents, analysts ask questions like:
  • Did this user/host/IP address generate any other alerts?
  • Was there any lateral movement from this host?
  • What users were logged in to this host when this alert triggered?
  • Were there any other entities affected by this attack? 
These investigative queries can all be represented as graph traversals.  The graph traversals are constant time operations because the work to represent them in the graph was done at ingestion time.   This enables analysts to rapidly explore many investigative paths, following log chains of events over multiple hops.  These queries would require multiple aggregation queries using search or many joins using a relational database.

Ease of use

Our cybersecurity experts have already done the hard work of defining the important entities and relationships for common security logs.  This means that analysts don’t have to worry about data formats, key names, or the information conveyed in individual log lines.  Our graph model defines the possible queries and surfaces this information to the user in an interactive visualization.  The graph model presents enough information to the user for them to understand what happened, without getting bogged down in details.
The following screenshot illustrates this behavior.  We have clicked on “administrator” in the graph.  The right panel shows all the relationships we could add to the graph on the right side.  Relationships are categorized, color coded, and have descriptive names.  Clicking any of the blue relationship names will add the associated edges to the graph.

 

Graph analytics

Query speed isn’t just important for the analyst, it is also important for the analytics. Having a graph database enables Sift Security to quickly and efficiently find complex, multi-stage threats by automatically identifying threat clusters and other anomalous structures in the graph.  These analytics can also be performed in the context of an investigation.  Clicking an entity like Catheryn  and choosing “Find Risks” in the graph will search the graph for any alerts nearby to Catheryn in the graph, even if they don’t explicitly involve Catherine.  For example, this would also find alerts coming from hosts Catheryn has logged onto and IP addresses Catheryn has used.

No limits on result set size

Whereas Elastic uses fixed-size aggregations  to populate its graph visualization, Sift Security enables queries that return all of the possible results, such as “What processes did this user run within this time window?”. This is helpful for traversals that require a set intersection, such as  “What processes did both of these users execute?”.  Sift Security excels at displaying fine-grained, detailed, targeted information.

Advantages of Elastic Graph

Flexibility

Elastic Graph has no pre-defined, fixed structure to the graph. Any two aggregatable fields in a document can be visualized as entities and relationships. This means that a user must specify the fields they want to visualize from a list of all the indexed fields.

Focus on significance

Using significance measures helps investigations over large datasets.  It is most useful when the goal is to find the elements from a set that best characterize an entity.  For example, “What are the most significant processes executed on this host?” could help quickly identify what a host is used for.  Fixed-size result sets enable users to quickly see and understand query results.

Identifying bulk structures in unstructured data

Due to its fixed size queries, Elastic Graph excels as displaying bulk structure surrounding the results of multiple simultaneous significance queries.  This approach is especially useful for tasks such as identifying similar texts based on which significant terms they have in common.

When to use Sift Security vs. Elastic Graph


Elastic Graph’s strength is in its flexibility.  If you are a power Elasticsearch user who relies on aggregate queries to find meaning in your structured or unstructured data, Elastic Graph will help visualize and identify structure in those results. If you need the flexibility of visualizing arbitrary fields or unstructured data, and the answers to your questions are best defined by aggregate searches, then Elastic Graph is the right choice.  For example, Elastic Graph can help you explore Shakespeare’s works or product sales data.

Sift Security is built with specific use cases in mind:  security operations and incident response, both on premise and in the cloud. Our graph model, architecture, and analytics help to identify the most serious threats  and enable analysts to rapidly investigate and resolve incidents. If you are looking to streamline your security ops, incident response, threat hunting, or cloud-ops practices,  Sift Security is the right choice.

To learn more about the power of graph databases, see our whitepaper “Why Graph?”.  To learn more about how Sift Security can benefit your organization, email us at contact@siftsecurity.com.

[0] Elasticsearch.  https://www.elastic.co/products/elasticsearch
[1] Elastic Graph.  https://www.elastic.co/products/x-pack/graph
[2] “Why Graph”,  Sift Security.  https://siftsecurity.com/papers/Sift-Security-Why-Graph.html 

Comments

Popular posts from this blog

Cloud Hunter Release

I just wanted to take some time to post some details on our recent release of Cloud Hunter, which allows customers to visually explore and investigate their AWS cloud infrastructure.  At Sift, we felt this integration would be important for 2 main reasons:Investigating events happening in AWS directly from Amazon is painful, unless you know exactly what event you're looking for.There are not many solutions that allow customers to follow chains of events spanning across the on-premises network and AWS on a single screen.At Netflix, we spent a lot of time creating custom tools to address security concerns in our AWS infrastructure because we needed to supplement the AWS logs, and created visualizations based on that data.  The amazing suite of open source tools from Netflix are the solutions they used to resolve their own pain points.  Hosting microservices in the cloud with continuous integration and continuous deployment can be extremely efficient and robust.  However, tracking ev…

Sift Security Launch!

We are excited to announce the launch of Sift Security's Threat Hunting and Incident Response Platform.Our team has been working for more than two years to design the next generation of security operations technology. Our mission is to make it easier and faster for security operations teams to get their jobs done.We take care of a few major headaches, helping you get to the real work.Gathering data
With out-of-the-box support for a dozen data sources and a simple tool for incorporating more, we get all the data you need for an investigation in one place, ready for when you need it.Integrating data
Our relational graph structure enables you to track entities across data types, quickly following a chain of events from the network to endpoints and applications.Evaluating data
Machine learning algorithms developed at Stanford and MIT help bring the most important events to your attention, so you can focus on what’s strange or new.Our team enjoyed making the platform, which builds on some…

Applying Machine Learning to Cybersecurity

In a recent article on the OPM hack, the author describes a pretty typical security situation for a large enterprise:The Office of Personnel Management repels 10 million attempted digital intrusions per month—mostly the kinds of port scans and phishing attacks that plague every large-scale Internet presence—so it wasn’t too abnormal to discover that something had gotten lucky and slipped through the agency’s defenses.Enormous pressure at scale from criminals makes automated systems essential for security. While humans can inspect packages coming into the building, only a computer can work quickly enough to inspect packets. Firewalls are the prototypical example: you allow certain traffic through according to a set of rules based on the source and destination IPs and the ports and protocols being used.In recent years, there's been a lot of buzz about machine learning in cybersecurity--wouldn't it be great if your automated system could learn and adapt, stop threats you don’t ev…