Skip to main content

How Sift Security's Analytics Engine Detects Insider Threats


We work with a lot of organizations that are worried about insider threats. Their employees require access to sensitive customer data or other proprietary information. They are worried that a careless or disgruntled employee may expose that information to outsiders. Moreover, they are worried that they might not notice it if it happened.
Insider threat detection is one of the main use cases of User and Entity Behavioral Analytics (UEBA). UEBA is the practice of modeling normal user and entity behavior in order to identify anomalies indicative of a cyber threat. This post describes how Sift Security’s detection and analytics engine can be used for insider threat detection.


For this post, we use the CERT insider threat tools datasets [1]. These are synthetic datasets from CERT that include background data and malicious attackers. Included are authentication, email, removable storage, and web browsing data. This post focuses on the first scenario in the r6 datasets, detecting a disgruntled user who is sharing sensitive internal documents publicly using wikileaks. The data we analyzed for this study covered 12 months and included 123 million events.


There are three different types of activities surrounding the suspicious user that are detected out-of-the-box by Sift Security.
These are:
  • Unusual after hours activity
  • Unusual usage patterns of removable storage devices
  • Multiple visits to WikiLeaks
These three activities are detected using two different mechanisms. The first two are detected using our analytics engine, and the third is detected using our detection rules engine. Our analytics engine uses nonparametric matrix factorization techniques to identify unusual patterns in behavior. Here, the unusual patterns are rare after hours activity and rare removable storage device usage. The visits to wikileaks are detected by our detection rules engine, which is alerting on access to blacklisted websites. The uploads to wikileaks occur on two days, August 19 and August 24. The goal is to be able to detect the insider threat before it comes to fruition and the information is compromised. Here, the goal is to detect the insider before the files are uploaded to wikileaks.


For the 12 months of data in consideration, there were 153 alerts for unusual patterns of removable device usage, 12 of which were true positive detections for the insider threat. For after hours activity, there were 1,224 total alerts, 5 of which were for the insider threat. In isolation, both of these indicators are not particularly useful, each having false positive rates in excess of 90%.
This is, however, where our alert prioritization algorithms come into play. To avoid giving the analyst a long list of isolated alerts to investigate, our prioritization algorithms examine the data with a sliding time window, looking for clusters of alerts surrounding a particular entity. Using this approach, the 1,377 total alerts are distilled down to a succinct summary:
  • The removable device usage alerts were raised for 22 distinct users
  • The after hours activity alerts were raised for 471 distinct users
  • There was exactly one user who raised alerts for both, the user ACM2278 (the insider threat)

Most importantly, the alarm for ACM2278 is first raised on 13 August, 6 days before he started uploading documents to wikileaks. Sift Security’s graph can be used to drive the investigation as shown below. The three red edges represent the three alerts -- abnormal device usage, after hours activity, accessing wikileaks. The other edges show the results of a follow up investigation to see what files ACM2278 was accessing (green) and sending via email (grey) during the same time period. Each of these are aggregate nodes the user can click on to see the details or expand to show all the nodes within.
How Sift Security's Analytics Engine Detects Insider Threats


Sift Security’s analytics engine can automatically detect abnormal entity behavior out of the box. In this scenario, it detected after hours activity and spikes in removable storage device usage. Our prioritization algorithms identified the situations in which both types of alerts were being raised for a particular user that caused that user to stand out among all other users. This enabled us to detect the potential insider threat 6 days before the data exfiltration began.


[1] CERT Insider Threat Tools:

Popular posts from this blog

Sift Joins Netskope, the Cloud Security Leader

Four years ago, we started Sift with the mission of simplifying security operations and incident response for the public cloud. In that time, we have assembled a fantastic team, created an innovative cloud detection and response solution, and have worked with many market-leading customers. I’m delighted to share that we’ve taken yet another step forward — as announced today, Sift is now officially part of Netskope. You can read more about this on Netskope CEO Sanjay Beri’s  blog  or in the official  announcement  on the Netskope website. For our customers, investors, partners, and team, this is an exciting new chapter. Let me tell you why we’re so excited.  Since the beginning, Netskope has had an unmatched vision for the cloud security market. Having started in 2012, they initially focused on SaaS security and quickly followed that with IaaS security capabilities. Six years later, they are now more than 500 employees strong and used by a quarter of the Fortune 100. They are a l

Data Exfiltration from AWS S3 Buckets

You will have no doubt heard by now about the recent Booz Allen Hamilton breach that took place on Amazon Web Services – in short, a shocking collection of 60,000 government sensitive files were left on a public S3 bucket (file storage in Amazon Web Services) for all to see. We are all probably too overwhelmed to care, given all the recent breaches we have been hearing about in the news. But with this breach it was different, it involved a trusted and appointed contractor whose job it was to follow security policies, put in place to avoid such incidents. So was this incident accidental or malicious? More, later about the tools we can use to tell the difference between the two. First, lets recap what happened. The Incident According to Gizmodo , the 28GB of data that was leaked not only contained sensitive information on recent government projects, but at least a half dozen unencrypted passwords belonging to government contractors with Top Secret Clearance – meaning anyone who got