Skip to main content

How Sift Security's Analytics Engine Detects Insider Threats


We work with a lot of organizations that are worried about insider threats. Their employees require access to sensitive customer data or other proprietary information. They are worried that a careless or disgruntled employee may expose that information to outsiders. Moreover, they are worried that they might not notice it if it happened.
Insider threat detection is one of the main use cases of User and Entity Behavioral Analytics (UEBA). UEBA is the practice of modeling normal user and entity behavior in order to identify anomalies indicative of a cyber threat. This post describes how Sift Security’s detection and analytics engine can be used for insider threat detection.


For this post, we use the CERT insider threat tools datasets [1]. These are synthetic datasets from CERT that include background data and malicious attackers. Included are authentication, email, removable storage, and web browsing data. This post focuses on the first scenario in the r6 datasets, detecting a disgruntled user who is sharing sensitive internal documents publicly using wikileaks. The data we analyzed for this study covered 12 months and included 123 million events.


There are three different types of activities surrounding the suspicious user that are detected out-of-the-box by Sift Security.
These are:
  • Unusual after hours activity
  • Unusual usage patterns of removable storage devices
  • Multiple visits to WikiLeaks
These three activities are detected using two different mechanisms. The first two are detected using our analytics engine, and the third is detected using our detection rules engine. Our analytics engine uses nonparametric matrix factorization techniques to identify unusual patterns in behavior. Here, the unusual patterns are rare after hours activity and rare removable storage device usage. The visits to wikileaks are detected by our detection rules engine, which is alerting on access to blacklisted websites. The uploads to wikileaks occur on two days, August 19 and August 24. The goal is to be able to detect the insider threat before it comes to fruition and the information is compromised. Here, the goal is to detect the insider before the files are uploaded to wikileaks.


For the 12 months of data in consideration, there were 153 alerts for unusual patterns of removable device usage, 12 of which were true positive detections for the insider threat. For after hours activity, there were 1,224 total alerts, 5 of which were for the insider threat. In isolation, both of these indicators are not particularly useful, each having false positive rates in excess of 90%.
This is, however, where our alert prioritization algorithms come into play. To avoid giving the analyst a long list of isolated alerts to investigate, our prioritization algorithms examine the data with a sliding time window, looking for clusters of alerts surrounding a particular entity. Using this approach, the 1,377 total alerts are distilled down to a succinct summary:
  • The removable device usage alerts were raised for 22 distinct users
  • The after hours activity alerts were raised for 471 distinct users
  • There was exactly one user who raised alerts for both, the user ACM2278 (the insider threat)

Most importantly, the alarm for ACM2278 is first raised on 13 August, 6 days before he started uploading documents to wikileaks. Sift Security’s graph can be used to drive the investigation as shown below. The three red edges represent the three alerts -- abnormal device usage, after hours activity, accessing wikileaks. The other edges show the results of a follow up investigation to see what files ACM2278 was accessing (green) and sending via email (grey) during the same time period. Each of these are aggregate nodes the user can click on to see the details or expand to show all the nodes within.
How Sift Security's Analytics Engine Detects Insider Threats


Sift Security’s analytics engine can automatically detect abnormal entity behavior out of the box. In this scenario, it detected after hours activity and spikes in removable storage device usage. Our prioritization algorithms identified the situations in which both types of alerts were being raised for a particular user that caused that user to stand out among all other users. This enabled us to detect the potential insider threat 6 days before the data exfiltration began.


[1] CERT Insider Threat Tools:

Popular posts from this blog

Sift Security Tools Release for AWS Monitoring - CloudHunter

We are excited to release CloudHunter, a web service similar to AWS CloudTrail that allows customers to visually explore and investigate their AWS cloud infrastructure.  At Sift, we felt this integration would be important for 2 main reasons:
Investigating events happening in AWS directly from Amazon is painful, unless you know exactly what event you're looking for.There are not many solutions that allow customers to follow chains of events spanning across the on-premises network and AWS on a single screen. At Netflix, we spent a lot of time creating custom tools to address security concerns in our AWS infrastructure because we needed to supplement the AWS logs, and created visualizations based on that data.  The amazing suite of open source tools from Netflix are the solutions they used to resolve their own pain points.  Hosting microservices in the cloud with continuous integration and continuous deployment can be extremely efficient and robust.  However, tracking events, espec…

Sift Security and WannaCry

😢 The WannaCry ransomware attack has left security teams around the world scrambling to make sure they are protected and to assess whether they have been victimized. To protect themselves, organizations need to have visibility into which systems are vulnerable and be able to rapidly roll out patches.  To understand whether they have been targeted, they need visibility into the channels over which the ransomware is distributed.  To understand whether they have been infected, they need visibility into the endpoints.
Over the past weekend, I was bombarded with questions from current customers, potential customers, former colleagues, friends, and family.  Am I vulnerable? How do I protect myself? How do I know if I’ve been hit? What do I do if I’ve been hit? What can you do to help me?
This post focuses primarily on the last question. What can we at Sift Security do to help an organization respond to a massive ransomware attack? I break this down into four categories, visibility, analyt…

Next-Generation SIEMs

Intelligence, Speed, Simplicity, AutomationOverviewMajor network and security trends, including exponentially increasing network traffic, cloud architectures, complex attack surfaces, and advanced adversaries, have created new challenges for security operations in adapting to this changing threat landscape.
Increased Traffic, Hybrid Cloud Architectures, and Sophisticated Adversaries Are Overwhelming SOCs
The Security Operations Center is faced with alert overload, often paralyzed with too much information to filter, prioritize, and act upon. The end result is an inability to find the true risk amongst thousands of alerts every day with the largest of organizations easily facing millions of alerts per day.Incident responders are challenged at finding relevant information or seeing the relationships amongst disparate indicators of compromise that are buried within logs, causing delays in assessing, understanding, and mitigating incidents.T…