Skip to main content

Sift Security white paper about big data and anomaly detection

Sift Security white paper about big data and anomaly detection

Big Data and Data Science Security White Paper

We are excited to announce our first white paper, describing the unique strengths of Sift Security's data science technology. There are a lot of overblown promises about big data in every industry, including cybersecurity, so we wanted to give you some concrete information about what we do and why it works.

Read the paper to learn about:

  • The relational graph that transforms reams of system logs and netflow data into meaningful connections between entities like users, hosts, and processes. We put the flood of flashing alerts from antivirus and firewall products into proper context, where serious threats stand out like bright clusters.
  • The anomaly detection pipeline, based on recent mathematics research at Stanford and MIT. It's scalable, of course, and works out of the box to start finding suspicious patterns of behavior. While many anomaly detection products require a period of perfect clean data to learn "normal" behavior, our product can operate on real data collected from an enterprise, even one already under attack.
  • The feedback loop running on top of our other algorithms that helps the product get more accurate with use. Over time, Sift Security customizes its estimates of risk to the specifics of a company’s environment.

Of course we can't give away all the details in a blog post! To learn more, get your own copy of the white paper: email contact@siftsec.com.

Popular posts from this blog

Sift Security Tools Release for AWS Monitoring - CloudHunter

We are excited to release CloudHunter, a web service similar to AWS CloudTrail that allows customers to visually explore and investigate their AWS cloud infrastructure.  At Sift, we felt this integration would be important for 2 main reasons:
Investigating events happening in AWS directly from Amazon is painful, unless you know exactly what event you're looking for.There are not many solutions that allow customers to follow chains of events spanning across the on-premises network and AWS on a single screen. At Netflix, we spent a lot of time creating custom tools to address security concerns in our AWS infrastructure because we needed to supplement the AWS logs, and created visualizations based on that data.  The amazing suite of open source tools from Netflix are the solutions they used to resolve their own pain points.  Hosting microservices in the cloud with continuous integration and continuous deployment can be extremely efficient and robust.  However, tracking events, espec…

Data Exfiltration from AWS S3 Buckets

You will have no doubt heard by now about the recent Booz Allen Hamilton breach that took place on Amazon Web Services – in short, a shocking collection of 60,000 government sensitive files were left on a public S3 bucket (file storage in Amazon Web Services) for all to see. We are all probably too overwhelmed to care, given all the recent breaches we have been hearing about in the news. But with this breach it was different, it involved a trusted and appointed contractor whose job it was to follow security policies, put in place to avoid such incidents. So was this incident accidental or malicious? More, later about the tools we can use to tell the difference between the two. First, lets recap what happened.The IncidentAccording to Gizmodo, the 28GB of data that was leaked not only contained sensitive information on recent government projects, but at least a half dozen unencrypted passwords belonging to government contractors with Top Secret Clearance – meaning anyone who got their h…

Using the Security Event Graph to Drive Alert Prioritization

One of the biggest differentiators at Sift Security is our security event graph: We map security events into a graph database. We then analyze the graph structure to prioritize alerts. Specifically, we look for clusters of interrelated alerts, score the clusters, and surface the clusters to the analyst. The analyst can then investigate each cluster in order, quickly assessing the threat and resolving the alerts in bulk.  
Our algorithms do the important work of sifting through isolated alerts and separating the false alarms and low priority alerts from high priority security incidents. We identify the high priority incidents by analyzing how alerts are related to each other. Key to this approach is our security event graph. This graph is stored in a graph database, a relationship-centric database that enables rapid execution of complex queries that would be very expensive to make in a traditional RDBMS.  The graph structure enables us to rapidly traverse relationships and find interr…