If you work in IT networking or security, you’re all too familiar with how difficult it can be to effectively manage and analyze large volumes of network data. If so, you aren’t alone – many organizations face significant challenges when it comes to effectively managing the collection and storage of their network and security event telemetry in an efficient and scalable manner and then applying security-focused analytics at scale to detect more threats. Unfortunately, as networks continue to grow in both size and complexity this problem will only continue to get worse over time. At its core, this is a big data problem. Simply put, the growth of today’s networks has led us into a new paradigm which the network analytics tools of the past were never built to handle the explosion of network data that exists today. This problem especially pronounced for large enterprises and service providers with massive network footprints and exceptionally high flow per second volumes, as they are now faced with challenges related to ingestion bandwidth, query performance, long-term data retention, and data resiliency.
Listed below are some of the most common network telemetry collection, storage, and analysis challenges that organizations are facing today:
For many large organizations, the challenges listed above have impeded their ability to perform routine network data management and threat detection efforts. These organizations need a solution that can provide scalable network telemetry collection and storage, highly responsive query times, and reliable data resiliency as core capabilities.
The Secure Network Analytics Data Store has been created with large enterprises and service providers in mind and specifically designed to solve the unique network telemetry management challenges outlined above through an improved database architecture design to enable new ways of managing data more efficiently.
The Data Store, introduced in version 7.3.0, sits between the Secure Network Analytics Manager and Flow Collectors. Flow Collectors ingest, de-duplicate and stitch together network flow data, then send it to the Data Store cluster. Flow data is then distributed across a resilient database cluster, which is comprised of a minimum of three Data Node appliances. This new database architecture offers scalable storage, increased flow rate ingestion capacity, improved resiliency versus the traditional model and higher-performance queries.
The illustration above depicts the components and architecture of a Secure Network Analytic deployment with a Data Store. Similarly, to the current Secure Network Analytics deployment model, Flow Collectors still ingest and process enterprise telemetry such as NetFlow. However, unlike the traditional model, the processed telemetry is not stored locally on each Flow Collector. Instead telemetry is written and stored across each Data Node within the Data Store. This new design allows for ingest and data storage functions to be performed independently from one another enabling the following benefits:
And the above list still is not exhaustive – the Data Store also provides additional follow-on benefits. Take the Data Store’s long-term storage capabilities for instance – with 1-2 years’ worth of data at your fingertips this capability not only enables you to perform investigations on larger data sets, but also make it easier to fulfill regulatory and compliance requirements should you get audited. Moreover, it also reduces both costs and complexity by eliminating the need to purchase expensive and non-integrated third-party storage solutions for data retention purposes.
The Data Store transforms weaknesses to strengths by flipping the
‘big data problems’ that burden many organizations on their heads
In summary, modern networks remain on track to continue expanding – both in terms of size and complexity – without end. Because of this, it has never been more critical that security practitioners adopt tools capable of scaling to the challenges of growing network environments. The Data Store stands in a class of its own as the only solution on the market that was specifically built, not only to enable practitioners to effectively manage, analyze, and retain today’s endlessly increasing volumes of network data, but that is also capable of allowing them to leverage this data to their advantage.
Check out the Secure Network Analytics Data Store Solution Overview, or contact your local Cisco Account Representative to learn more.