Scaling Suricata in the Enterprise

Background

As we have previously written, for all Suricata’s capabilities, building out an enterprise-scale deployment of Suricata with mostly open source tools can be a challenge.

In this third entry in the series of blog posts, we review another one of the five ways to improve the scalability of Suricata in an enterprise deployment - Tuning Suricata Sensors.

Note: for a more complete review of all five improvements, please check out the white paper, Scaling Suricata for Enterprise Deployment. You can download your copy here >>

Tuning the Suricata Sensors

Modern versions of Suricata have deployed successfully on 100Gps links with off-the-shelf hardware outfitted with a specialized capture network interface card (NIC). This was shown as part of a public demonstration in 2017 using Suricata 4.0 dev edition with Napatech NT200A02 NIC . Your specific requirements may vary across the enterprise. As mentioned above, Suricata sensors often require different hardware and software configurations based on their network performance requirements and where they are deployed.

The performance of your Suricata deployment depends on a number of factors, including traffic volume for each sensor, the specifics of the underlying hardware, the nature of your detection, file extraction, and protocol monitoring needs and your requirements for data retention time.

The goal of any tuning exercise is to create a few baseline packages/configurations for each of your scenarios to use a solid starting point for each deployment.

Here are the macro variables to consider when tuning your sensors:

The scale of the ruleset and/or threat intelligence you are using - for example, will you be using 100% of a commercially available ruleset with 50,000 signatures? How extensive is your internally-developed threat intelligence? Are you attempting to match a list of 1 million domains and 270,000 URLs?
The type of traffic mix and volume the sensor will see on the network - while not actually something you can control directly, it is important to understand that the traffic mix running on a monitored network segment has an impact on performance. This should be considered when determining the sensor placement.
The specific type of hardware or virtual machine Suricata is running on - the OS/kernel version, the processor, number of cores, memory, disk space, network interface card (NIC) are all important factors. Obviously if the sensor is deployed in the cloud and/or in a virtual machine, your choices will have an impact on performance and subsequently your optimized configuration.
The version of Suricata you are running - with each new release of Suricata, new features are added. While usually new releases bring some performance improvements, sometimes the new features impact your performance in your particular environment.

Begin your tuning exercise by locking down 3 out of the 4 factors listed above and adjusting the fourth until you maximize your performance in that setting. For example, start with a known version of Suricata, the threat detection and intelligence you will likely want to deploy, and a particular network traffic type/mix. Then experiment with several hardware or container configurations to see if you achieve the performance targets you are looking for.

Once you have baselined this set up, you can move it to a different network segment and revalidate the performance. If you find that this new installation has substantially lower traffic volume for example, you may wish to downsize the hardware and retest to see if you can save some money on all your low traffic network segments.

Keep in mind that any time one of the variables is changed, you will want to revalidate or retest that particular tuning.

After you’ve created one or more deployment packages, you may wish to move to an advanced set of adjustments of the secondary variables that have the potential to impact your performance. Here is a list of those features that may have performance implications in your environment:

Kernel/NIC driver version
Rules/match/drop lists adjustments
eBPF filters adjustments
Protocol detection and logging changes
Multi-tenancy
Per interface configs

Ideally, the tuning above can be performed remotely using a central management system as discussed above with the ability to save ‘golden’ configurations for each scenario you wish to deploy in.

The efforts required to tune the deployment are very different if you have chosen to go with pure open source (e.g. SELKS or other Suricata stack) or if you are using a commercial system such as Stamus Network Detection and Response (Stamus NDR) from Stamus Networks.

If you have chosen the open source route, there are a number of very valuable resources available to help. These include online guides such as https://github.com/pevma/SEPTun and https://github.com/pevma/SEPTun-Mark-II and workshops sponsored by the OISF such as those listed here; https://suricata-ids.org/tag/training/.

If you wish to minimize the time your team and you dedicate to tuning and optimizing your deployment, you may opt for a commercially available solution for which the Suricata installation is pre-optimized for the hardware it is installed on and fully supported. An example of this pre-optimized hardware can be purchased from Stamus Networks. Learn more about the Stamus Networks Appliances here.