This vendor-written piece has been edited by Executive Networks Media to eliminate product promotion, but readers should note it will likely favour the submitter's approach.
Big Data Security Analytics has long been heralded by Gartner and Forrester analysts as the future of security threat detection and the very pinnacle of Security Operations Centre (SOC) capabilities. This came into focus when Verizon's 2015 data breach investigations report was (in)famously cited at the RSA Conference 2015 by Amit Yoran, President of RSA. Specifically, he quoted just a single number from the entire report: 1% - the amount of advanced cyber threats detected by Security Incident and Event Management (SIEM) systems.
Unsurprisingly, this startling statistic was pounced upon by many vendors of network probe-based security analysis products, along with the battle cry of "logs are dead". Yet, this is both disingenuous and confuses three completely different, but pivotal aspects of security threat detection: data scope, data sources and detection methodology.
Breaking down security threat detection
Data scope describes the amount of the IT estate covered by the data set being considered. This is a fundamental and vital factor, because security threats can't be detected in areas of the IT estate that are not visible. It is relatively trivial to centrally place a solution that ingests logs providing full coverage of an entire IT environment. However, with even the smallest enterprises having multiple sites and external access points from their networks, a probe-based approach requires a large number of probes or substantial additional specialist network infrastructure to provide equivalent coverage in even the simplest environment. With nearly all vendors pricing probes individually, enterprises can expect to pay a hefty price premium to even come close to the coverage of a central log-based solution.
The discussion around data sources is no less important to understand. Theoretically, a network probe can see all communications that flow past its sensors. In practise, encrypted traffic simply can't be inspected in most cases and where it can, it normally causes even larger security and performance issues. Substantial computational overhead is required to reconstruct and track the many different network 'conversations' between different devices on the average network. Multiple disparate protocols are needed on even the simplest network meaning that nearly all probe-based solutions limit their inspection to just a tiny handful of unencrypted protocols.
This severely erodes the supposed value proposition of using a network type probe. So, why not simply collect all the transaction logs (as compared to the pre-determined trigger 'events' consumed by SIEMs), including those for the encrypted data flows from the participants themselves? There are entire protocol suites designed to do exactly this built into every router, proxy, desktop and server operating system.
Sign up for Computerworld eNewsletters.