Suricata 4.0 is out and this switch from 3.x to 4.x is not marketing driven because the changes are really important. This post is not exhaustive on changes. It is Stamus Networks' take on some of the important changes that have been introduced in this version.
Rust addition
This is the big step forward on the technology side. Suricata is written in C language. This gives performances and a good control over memory. But it goes with a series of well known problems. I name here buffer overflows, use after free, ...
And the worse is that Suricata is parsing traffic content which is a kind of vice supercharged user input. If one should not trust user input, guess how careful we should be with network traffic. At Suricon 2016, Pierre Chifflier did present a proof of concept implementation of protocol parsers in Rust. The idea is to use the property of Rust that has been designed to avoid complete class of attacks on memory handling. But there is more in the approach as the implementation is using Nom which is a Rust parser combinator framework. It allows you to write protocol parser easily and in a reusable way. Thus the addition of Rust is two things at the same time: more security and easier code. Which means a lot of new protocols should be added in the near future.
Suricata 4.0 Rust support comes with NFS, DNS and NTP. NTP support is implemented via an external crate (read library): ntp-parser.
As mentioned before, the code uses Nom and the syntax is very different from traditional code. For instance, here is the code of ntp-parser parsing NTP extension:
named!(pub parse_ntp_extension,
do_parse!(
ty: be_u16
>> len: be_u16 // len includes the padding
>> data: take!(len)
>> (
NtpExtension{
field_type:ty,
length:len,
value:data,
}
))
);
This define a parsing function that read the stream of data. The code says, take 16 bits, store them as unsigned integer in ty. Then store the next 16 bits as unsigned integer in len. Then store in data a chunck of data of length len. And with that build a NTP extension structure. If the writing is concise and efficient, the best thing with Nom is under the hood. Nom is taking care of detecting the invalidities. For instance we could have a chunck of data of length 50, and len being set to 1000 (remember Heartbleed ?). Nom will see that there is not enough data available in the chunck and return it wants more data.
Better alerts
As you may know, the preferred output of Suricata is the EVE JSON format. It is flexible, easy to extend and easy to read by human and tools. Suricata 4.0 is introducing some major changes here:
- 'vars' extraction mechanism
- The new target keyword
- HTTP bodies logging
HTTP body output
Suricata is able to uncompressed HTTP body on the fly and match on the uncompressed content. This means that if you get the payload of the stream triggering the alert in your event, you will just see compression noise and won't be able to analyze why the alert was triggered. Suricata is now able to include the HTTP bodies in the alert. The analyst can then directly see from the event the content that did trigger the alert.
The following event shows how payload_printable is completely compression noise and the http_response_body_printable is readable:
Target keyword
The new target keyword is a fix on a very old problem. It is not possible to know in an alert event which side of the source or destination is the target of the attack. This is a problem as it is not possible to automate things due to that lack of information. The target keyword allow the rules writer to specify which is side is the target. Doing so automated analysis and better visualization can be made.
Usage is simple, signature has to contain the target keyword with value dest_ip or src_ip. For example, in a simple scan alert we have:
alert tcp $EXTERNAL_NET any -> $HOME_NET 3306 (msg:"ET POLICY Suspicious inbound to mySQL port 3306"; flow:to_server; flags:S; threshold: type limit, count 5, seconds 60, track by_src; reference:url,doc.emergingthreats.net/2010937; classtype:bad-unknown; target: dest_ip; sid:2010937; rev:2;)
If target is present in a signature, the alert is added an alert.source and alert.target field:
For example, on a visualization where node are IP address and links are alerts between the two, we can get an idea of the possible compromised path. With the target addition, we can switch from a non oriented graph:
To an oriented graph that show which paths were really possible:
If you know French, you can learn more about this subject with Eric Leblond's talk at SSTIC 2017.
Vars extraction
This is one of the most expected feature of Suricata 4.0. This has been described by Victor Julien in an extensive blog post. The concept is to be able to define in signature data to extract and store them in a key value form. There is a lot of possible usage ranging from application version extraction to getting exfiltred data. For example, let's consider there is a domain we are interested in. One interesting information is the list of email addresses where mail are sent to. To do so we can use the following signature:
alert smtp any any -> any any (msg:"Mail to stamus"; content:"rcpt to|3A|"; nocase; content:"stamus-networks.com"; within: 200; fast_pattern; pcre:"/^RCPT TOx3as*<([w-.]+@stamus-networks.com)>/ism pkt:email"; flow:established,to_server; sid:1; rev:1;)
The magix here is the groupe in the regular expression ([\w-\.]+@stamus-networks.com) that is save in a packet var named email by the pkt:email in the regular expression definition.
Using that signature we get this kind of alerts:
The key point here is the vars sub object:
"vars": {
"pktvars": [
{
"email": "eleblond@stamus-networks.com"
}
]
},
We have an extraction of the data and this can be easily search by tool like Elasticsearch or Splunk.
Conclusion
Suricata 4.0 is really an important milestone for the project. Introduction of Rust is opening a really interesting path. The alerts improvement may change the way signatures are written and it will help to provide really accurate information to the analysts.
Suricata 4.0 is already available in SELKS and it will be available in Stamus Probe by the end of August. To conclude on a personal note, we, Stamus Networks, are really happy to have contributed to this release with features such as via HTTP body logging and target keyword.
