Authors: Securonix Labs
Current State of Data and Threats
Today, we see organizations face extraordinary challenges related to the safety of their information. With a majority of it stored and transferred in digital form, there is an important need to secure this data. Different types of stored data include personal credentials, bank account information, credit card transactions and intellectual property. The integrity of this digital information, however, is constantly challenged by attacks aimed at stealing, exposing or manipulating it. This affects all industries, from healthcare to finance to retail, and failing to protect this data can cause an organization great financial and reputational harm.
With the rise of cyber-criminal organizations and expansion of covert cyber operations by nation states, we see the volume and complexity of attacks increasing. The threat actors that seek to attack an organization have gotten more dangerous and use far more sophisticated techniques than previously encountered. External threats generally seek to penetrate an organization’s defenses set up at the perimeter. With a rise in device usage, the range of possible endpoints and the attack surface have also increased significantly. Organizations expending more and more resources to strengthen their perimeter security to protect themselves against these threats.
While we have seen an increase in external attacks, the rise of internal threats from individuals within an organization and its network perimeter has been an important consideration for an organization to undertake. Insiders are trusted by their employers and have access to critical systems and confidential information, and the risk posed by an insider attack is often more harmful compared to an external one.
Furthermore, once external attackers breach the perimeter and infiltrate the internal network they pose as legitimate insiders by either hijacking credentials or creating new accounts. The ability to effectively detect a malicious insider can enable an organization to also stop outsiders when perimeter defenses fail. To combat both internal and external threats, organizations need a comprehensive security analytics solution that complements their perimeter and network security.
Traditional SIEM and its Shortcomings
Security Information and Event Management (SIEM) has been a long-standing system to help organizations with their security needs. Gartner defines SIEM as a “technology that supports threat detection and security incident response through the real-time collection and historical analysis of security events from a wide variety of events and contextual data sources. It also supports compliance reporting and incident investigation through analysis of historical data from these sources.”
Traditionally, SIEM focused on collecting logs from multiple security data sources and employed correlation rules created by security experts to be executed in real time. It also looked at a diverse set of use cases to focus on threat detection and alert management. The analytical capabilities of typical SIEM were limited to the correlation of information at a network level related to a system or an application. It did not understand human activity and how to adapt to changes in their behavior.
With the rise of insider threats, SIEM operators found it extremely difficult to use the system to detect malicious user behavior. As behavior is dynamic, rules manually defined by security experts are not adaptable. For example, when dealing with the volume of data transmitted by a specific account, traditional SIEM would typically define a threshold through a rule to determine suspicious activity. However, this approach has little chance to succeed, because the threshold of normal activity varies depending on the type of user (e.g. marketing vs. call center), as well as the timing of the transmissions (single large transmission vs. multiple small ones spread over several days), and a static rule cannot account for multiple complex scenarios. Increasing the threshold to accommodate high-volume users would leave malicious activity with smaller volumes undetected, and decreasing it to catch them all would lead to a large number of false positives, overwhelming the analysts with an excessive volume of alerts.
Complex threat scenarios posed another challenge for traditional SIEM. For example, in trying to detect the Advanced Persistent Threat (APT), the analyst has to consider a multitude of threat indicators: logins from an unusual source or to the rarely accessed system, increased volume of failed logins, abnormal data consumption, unusual volume of data egress, timing of the requests, and so on. While each of these activities in itself might be insignificant to raise an alert, their occurrence across linked accounts and/or systems is a strong indication of potential APT. The system should be able to dynamically quantify the risk associated with different events and amplify it along the kill chain of probable attack for proper detection.
The data sources required to detect malicious user behavior extend beyond ones related to network security alone. Business transactions, email and chat, Identity and Access Management, and even HR data must be correlated for identity and combined with the traditional security logs to enable effective insider threat detection. Additional user metadata is extremely useful in understanding peer groups within an organization, such as customer service representative, marketing, server engineers, etc., that have different job functions and, therefore, would exhibit different normal behavior. Comparing user behavior with that of his peers allows detecting suspicious activity when it deviates from the peer group, as well as reduce false positives when it’s within the group’s range.
Traditional rule-based systems cannot scale to the increased number of threats, volumes of transactions, and diversity of data sources; nor can it handle the complexity of advanced attacks and dynamic nature of user behavior. Manual maintenance of exploding number of rules and prioritization of the ever increasing alert volume can overwhelm even the most capable team of analysts. Adaptive, automated analytics is needed to address these challenges, to focus analyst’s’ attention on the riskiest events, to capture his thought process and to learn from it to eventually respond to threats on the computer, not human time.
Gartner defines Security Analytics as “advanced analysis of some data to achieve a useful security outcome”, where “advanced analysis” refers to any method better than a simple rule and elementary statistics. Security Analytics lends itself to more efficient and effective ways to detect threats by enabling better context as well as minimizing false positives and reducing the number of alerts. Furthermore, it prioritizes the alerts based on risk to ensure that the riskiest events are addressed first and the average time till detection is reduced. Learning from the analyst’s feedback, it automates responses to most common events, allowing the analyst to focus on new and emerging threats.
Securonix pioneered the use of User Behavior Analytics (UBA) in security by introducing in 2009 a signature-less solution for detecting anomalous user behavior, identifying access outliers, and correlating user identity across multiple systems. Such user-centric solution led to a significant qualitative jump in the evolution of Security Analytics, as it was now able to address insider threats alongside with network and endpoint security. Securonix UBA delivered on the promise to uncover true threat indicators in the vast amount of security data, addressing a wide range of insider threat concerns: from risky access to privileged account abuse to data exfiltration.
By developing a security analytics platform that can process a wide variety of data, correlate identity across multiple systems, utilize peer group analysis and cross-domain correlation to reduce false positives, and apply risk amplification to prioritize alerts for the analyst review, Securonix brought intelligence into the morass of security event analysis, and firmly placed user as the most critical endpoint in the security ecosystem.
The next logical step was to apply the same behavioral analytics techniques that proved so successful in user-centric cases to any other entity attributable to security events, such as computer system, application, or even a document, expanding dimensionality of Security Analytics and allowing correlation of anomalous events across different entity types – critical functionality for detecting lateral movement. Gartner acknowledges this expanded functionality in 2015 as User and Entity Behavior Analytics (UEBA): “UEBA successfully detects malicious and abusive activity that otherwise goes unnoticed, and effectively consolidates and prioritizes security alerts sent from other systems.” The expanded use cases included cyber-threat detection, network and data security, and cloud and application security.
UEBA offers the ability to correlate events originating at any type of entity and to create normal behavior profiles that adapt to dynamic behavior while allowing anomaly detection in real time. Behavioral indicators can be combined with direct threat indicators to risk-score specific threats, and multiple threats can be aggregated into the attack kill chain to amplify risk and facilitate earlier detection of the attack. Prioritized alerts are presented to the analyst, who can utilize powerful link analysis tool acting on context-enriched data for investigation. Analyst’s feedback is captured together with the inceptive threat indicators to train machine learning models to encode analyst’s reasoning and automated resolution of similar cases or to provide a recommendation for such resolution.
SNYPR: NextGen Security Analytics Platform
Securonix’s SNYPR is a next-generation security analytics platform that offers the best of all worlds in one modular package:
- An open data model and self-describing data formats that easily integrate with other tools
- Log management for compliance and reporting
- Threat hunting and data exploration
- SIEM capabilities for data aggregation and near-real-time event correlation
- Market-leading and the most mature UEBA solution for advanced analytics
It is built on top of a modern big data stack that includes Hadoop, Impala, Spark, Kafka, and other proven components, and provides a massively scalable, fault-tolerant open data platform that can analyze large quantities of diverse data, as well as support reliable, long-term data retention. With an open data model and a self-describing data format (Parquet) that utilizes efficient compression and encoding, it is possible to avoid data duplication, while lowering data storage costs and maximizing the query performance.
SNYPR offers a smarter, faster and more economical solution compared to a traditional SIEM. By performing advanced analytics over large volumes of data in real time, it can provide actionable security intelligence to an organization. By super- enriching raw events in real time with contextual information related to identity, asset, network, geolocation and threat intelligence, SNYPR guarantees the point-in-time accuracy of the context and facilitates further correlation and link analysis based on the enriched content. Using a combination of context enrichment, threat modeling, and machine learning, SNYPR can predict, detect and contain advanced threats in real time.
Other chapters in this series:
Introduction – Data Science: A Comprehensive Look
Ch 1 – SIEM 2.0: Why do you need security analytics?
Ch 2 – Data Science: Statistics vs. Machine Learning
Ch 3 – Unsupervised Learning: Combining Security and Data Science
Ch 4 – Supervised Learning: Capturing The Thought Process Of An Analyst