Big Data Security Analytics

Hadoop driving big data projects as scalability and open source nature prove popular

While the world of big data has a variety of tools and applications to enable users to process and analyse their data, much of the attention has focused on the Hadoop platform and solutions based on it.

One of the most exciting applications of big data is security analytics. By using big data security analytics organizations can correlate both internal and external data to create a bigger picture showing pending threats. Big data is ideal for fraud detection and identifying threats before they happen. And since big data tends to pool information in a central location for easy access and analysis, it makes enterprises more susceptible to cyber-attacks so organizations can use big data analytics to gain better intelligence about big data security threats.


Five Metrics for Big Data Security Analytics

Big data security is ushering in a new era of intelligence-driven security capable of predicting and preventing sophisticated, high-stakes security threats. And the data sources used for big data security analytics are the same sources that IT managers have been using for some time. The difference is that the detection tools are now integrated and the analytics engines reveal patterns that make it easier to spot fraudulent activity or security issues.

The other thing that’s different is the size of the datasets. Big data security analytics use big data, which means there are massive data archives to wade through to detect potential hacker attacks. Tools to sift through these datasets were custom-built in the past, but now there are more off-the-shelf security products for big data using existing enterprise data sources:

  1. Network and Host Traffic – The security analytics assess anomalies in data traffic to and from servers and clusters, looking for things such as encryption or suspicious destinations. The analytics draw from data sources such as Security Information and Event Management (SIEM), network monitoring, or application monitoring.
  2. Web Transactions – Is there suspicious activity being observed in high-value applications or sensitive assets? Analytics will use authentication data, transaction monitoring, application logs, SQL server logs, and network session data to identify fraudulent activity.
  3. Infrastructure Changes – Has the server been manipulated? Has there been a recent configuration change? What about policy compliance? To check for infrastructure changes security analytics will use data from IT assets; look at governance, risk management, and compliance (GRC); and configuration management systems.
  4. Information – What types of data is the system storing, transmitting, or processing? Is it regulated data? Is it high-value IP? To assess the security risk from information, big data security analytics draw from GRC, data classification, and data loss prevention (DLP)
  5. Identity management – What users are logged in? Are their privileges current or have they been escalated? When did they last log in? What assets did they access? To track user activity analytics will draw from authentication data, server logs, asset management, SIEM, and network monitoring.