This is a follow-up to our first blog on hunting using the publicly available Newly Registered Domains (NRD) threat intel lists. In this blog, we will explore a hands-on example of infection traffic and review one of three possible detection and investigation techniques that can unearth malicious behavior.
It is important to note that NRDs are not necessarily bad by default, and an NRD alone is not enough of an IoC/match to determine whether or not it is malicious. We need to combine context with the detection itself to make that determination. But what information can help us come to our conclusion? This blog post should help give you a baseline of what to look for.
This blog post is one of five blogs introducing Open NRD and sharing various ways it can be used with Suricata for threat hunting and investigation. To skip to the other blog posts in the series, click on one of the following links:
- Introducing Open NRD: Newly Registered Domain Threat Intel Feeds for Suricata
- Threat Hunting with Suricata and Newly-Registered Domain Threat Intel (Open NRD) Part 1
- Threat Hunting with Suricata and Newly-Registered Domain Threat Intel (Open NRD) Part 3
- Threat Hunting with Suricata and Newly-Registered Domain Threat Intel (Open NRD) Part 4
What tools are needed?
Before beginning, there are some tools you need to ensure you can follow along:
- OpenNRD threat intel feeds - free subscription for any Suricata user
- Suricata - the article assumes Suricata 6.0.14+ or 7.0.0+ installed (with the Open NRD feeds)
- DarkGate malware infection trace PCAP - thanks to Malware traffic analysis - https://www.malware-traffic-analysis.net/2023/10/12/index.html
Before we look into the actual hunting example, we must understand three major Suricata hunting concepts. They are as follows:
- 1. The wealth of data generated by Suricata (especially beyond alerts)
- 2. How correlation is performed with Flow ID
- 3. How flow labeling with Flowbits can impact context
Suricata-generated data - more than just alerts
One of the many powerful features of Suricata is that it can create protocol and transaction logs even in the absence of alerts. These logs include flow records, anomalies, alerts, protocol transactions, and file transaction logs, plus file extraction and packet capture (PCAP).
Here is a full list and details of what those logs and transactions look like.
You also might be interested in the many new things available in the recently released Suricata 7.
Suricata data correlation with Flow ID
Suricata produces all relevant network security monitoring logs: protocol, flow, file transaction, and anomaly logs, including the ones related to an alert - but also independent of alerts. In the regular JSON logs that Suricata generates (eve.json), you will find something called “flow_id” that correlates the network protocol data and evidence that Suricata has logged - to an alert event and that alert’s metadata.
The ability to correlate any existing evidence/logs to an alert (“flow_id”) was introduced in 2014 by Suricata lead developer, Victor Julien
https://github.com/OISF/suricata/commit/f1185d051c210ca0daacdddbe865a51af24f4ea3
Flow labeling with Flowbits can provide context
Suricata can match or highlight on specific simple events, occurrences, anomalies, patterns, or IoCs and also much more complex detection logic either via signature or a lua script – in flows/sessions or cross flows/sessions.
Regardless of what type of match is used we can insert a piece of metadata in all protocol transactions generated by Suricata. This way, multiple different labeling of varying flows can occur that can later be used for detection logic SIEM/DB queries, alerting, and/or automation.
In other words, by using NRD matching all the Suricata protocol records to and from NRD communication that are found in either DNS Query (“event_type”:”dns”), HTTP Hostname (“event_type”:”http”), and TLS SNI (“event_type”:”tls”) will have the flowbit/marker piece of metadata inserted in them as well as any and all corresponding “event_type”:”fileinfo”, “event_type”:”flow”, “event_type”:”anomaly” event logs from those sessions/flows.
Here is an example of a flow in Suricata JSON output:
The above picture has a flow or protocol record labeled (with “stamus.nrd.entropy”) as a Newly Registered Domain communication with high entropy from Stamus’s NRD daily updated lists.
There could even be multiple matches and labeling:
…
{
"flowbits": [
"FBMproto_0",
"stamus.nrd",
"ET.http.binary"
]
}
….
The above record has a flow or protocol record labeled (with “stamus.nrd”) as a Newly Registered Domain communication from Stamus’s NRD daily updated lists.
The visualization above is present in the SN-FLOW dashboard (based on Suricata “event_type”:”flow” log data) as part of free and Open Source
Dashboards that are available in GitHub or as part of SELKS
https://github.com/StamusNetworks/suricata-analytics/tree/main/kibana/7
Detection method one - Executable downloaded from Newly Registered Domain
Now that we have those 3 major points clarified, we can move on with our detection logic.
First, let's clarify what we mean by “detection” here. In this context, detection is simply marking and/or highlighting NRDs in the communication traffic. Then based on our hunting hypothesis we combine that knowledge with the protocol and alert information that Suricata produces. In some cases there could be no alerts, and that’s ok too.
One point that is important to keep in mind is that after the hunt is successful we should check if traces of such traffic or transactions happened in the past. For example, if we find any TLS certificates, file hashes, or domains, we could use Suricata protocol data to check if these occurred previously regardless of alerting.
We are basing this hunt on the hypothesis / idea that clear text executables from a newly registered domain is something worth investigating.
For this hands-on use case review, we run the PCAP through Suricata (via use of SELKS) and then we review the actual data we have available. If you need to install SELKS you can consult the online documentation here. However, it is not a requirement as you can also simply run the PCAP through any Suricata deployment as long as you have the OpenNRD rules installed.
First we read in the PCAP in SELKS:
scripts/readpcap.sh -ac /home/pevma/Downloads/2023-10-12-DarkGate-infection-traffic.pcap
Alert-based hunting with Scirius
Once we have read the PCAP in SELKS, we can then head to the Scirius hunting interface (right upper corner widget switcher, click on Hunting) and look at the alerts and all other generated events.
We can see a few alerts as well as DNS and HTTP based NRD communication combined with a clear text executable download via HTTP:
If we select the “ET POLICY PE EXE or DLL Windows file download HTTP” alert, by way of Suricata’s defaultly available flow_id correlation we can see (related events tab - fileinfo transaction, http log, and other alerts for example) that this is an executable downloaded from a NRD:
The domain, according to Virus total, was registered only 9 days ago (at the time the original check was completed).
Now let’s see if we can create a filter or a query based on the findings above.
We can create a simple Scirius Hunting query as follows (from the filter drop down, select ES query):
metadata.flowbits:*stamus.nrd* AND alert.signature:*exe* AND http.status:200
Now we can easily zoom in on alerts that have executable downloads in clear text from a NRD.
Alert log example
Here is an example of an executable download alert (“ET POLICY PE EXE or DLL Windows file download HTTP”) that has the flowbit / flow label enrichment of a newly registered domain as well - “stamus.nrd”
NOTE: The response body, response body printable, and payload fields below as part of the alert have been clipped out on purpose as they are too big.
{
"timestamp": "2023-10-12T22:09:10.979074+0200",
"flow_id": 1689769056868824,
"pcap_cnt": 55,
"event_type": "alert",
"src_ip": "104.21.91.46",
"src_port": 80,
"dest_ip": "10.10.10.113",
"dest_port": 64642,
"proto": "TCP",
"pkt_src": "wire/pcap",
"metadata": {
"flowbits": [
"stamus.nrd",
"ET.http.binary"
]
},
"ether": {
"src_mac": "00:0c:30:5f:c4:a3",
"dest_mac": "08:1f:71:62:50:7d"
},
"tx_id": 0,
"alert": {
"action": "allowed",
"gid": 1,
"signature_id": 2018959,
"rev": 4,
"signature": "ET POLICY PE EXE or DLL Windows file download HTTP",
"category": "Potential Corporate Privacy Violation",
"severity": 1,
"metadata": {
"created_at": [
"2014_08_19"
],
"former_category": [
"POLICY"
],
"updated_at": [
"2017_02_01"
]
}
},
"http": {
"hostname": "hgfdytrywq.com",
"url": "/a",
"http_user_agent": "Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) WindowsPowerShell/5.1.19041.3031",
"http_content_type": "application/octet-stream",
"http_method": "GET",
"protocol": "HTTP/1.1",
"status": 200,
"length": 41560,
"http_response_body_printable": "MZ......................@...............................................!..L.!This program cannot be run in DOS mode.\r\r\n$........sD.R.*.R.*.R.*..C..P.*.....S.*._@..a.*._@....*._@..g.*.[j..[.*.[j..w.
*.R.+.r.*.......*.....S.*._@..S.*.R...P.*.....S.*.RichR.*.........................PE..L....q.Z..........\"...............................@...........................\r.......\r...@...@.......@.........................|.......P............
....
[clipped out]
.....
....",
"http_response_body": "TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGAEAAA4fug4AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4gRE9TIG1vZGUuDQ0KJAAAAAAAAAAWc0SQUhIqw1ISKsNSEirDFEPLw1ASKsPMsu3DUxI
qw19A9cNhEirDX0DKw+MSKsNfQMvDZxIqw1tqqcNbEirDW2q5w3cSKsNSEivDchAqw+eMwMMCEirD54z1w1MSKsNfQPHDUxIqw1ISvcNQEirD54z0w1MSKsNSaWNoUhIqwwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFBFAABMAQUAv3GqWgAAAAAAAAAA4AAiAQsBDAAA6AgAANgEAAAAAAD6fwIAABAAAAAACQAAAEAAA
BAAAAACAAAFAAEAAAAAAAUAAQAAAAAAAPANAAAEAACQ+A0AAgBAgAAAQAAAEAAAAABAAAAQAAAAAAAAEAAAAAAAAAAAAAAAzNALAHwBAAAAkAwAUNcAAAAAAAAAAAAAAIYNAKgcAAAAcA0ArHEAAJA7CQAcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIFsKAEAAAAAAAAAAAAAAAAAACQCECAAAAAAAAAAAAAAAAAA
....
[clipped out]
.....
...."
},
"files": [
{
"filename": "Autoit3.exe",
"magic": "PE32 executable (GUI) Intel 80386, for MS Windows, 5 sections",
"gaps": false,
"state": "UNKNOWN",
"stored": false,
"storing": true,
"size": 41560,
"tx_id": 0
}
],
"app_proto": "http",
"direction": "to_client",
"flow": {
"pkts_toserver": 19,
"pkts_toclient": 34,
"bytes_toserver": 1300,
"bytes_toclient": 45430,
"start": "2023-10-12T22:09:10.262358+0200",
"src_ip": "10.10.10.113",
"dest_ip": "104.21.91.46",
"src_port": 64642,
"dest_port": 80
},
"payload": "SFRUUC8xLjEgMjAwIE9LDQpEYXRlOiBUaHUsIDEyIE9jdCAyMDIzIDIwOjA5OjExIEdNVA0KQ29udGVudC1UeXBlOiBhcHBsaWNhdGlvbi9vY3RldC1zdHJlYW0NCkNvbnRlbnQtTGVuZ3RoOiA4OTM2MDgNCkNvbm5lY3Rpb246IGtlZXAtYWxpdmUNCkNvbnRlbnQtRGlzcG9zaXRpb246IGF0dGFj
aG1lbnQ7IGZpbGVuYW1lPSJBdXRvaXQzLmV4ZSINCkNGLUNhY2hlLVN0YXR1czogRFlOQU1JQw0KUmVwb3J0LVRvOiB7ImVuZHBvaW50cyI6W3sidXJsIjoiaHR0cHM6XC9cL2EubmVsLmNsb3VkZmxhcmUuY29tXC9yZXBvcnRcL3YzP3M9eDFua1FuUVJuWFRycmF6JTJCcE5COFVGY2FtVjVzNTZZMyUyQlJEd2QlMk
....
[clipped out]
.....
....",
"stream": 1,
"packet": "CB9xYlB9AAwwX8SjCABFAAWIjEVAADAG4WxoFVsuCgoKcQBQ/IJiHoBg3wnxhFAQAAggjgAA/+gCTwEAiwONj3j7//+LQAQDyOiFTwEAiwOLQAQDx4O4ePv//wB1zv8VVAhJAItP3IXJD4WOgQMAi0/QhckPhZeBAwAz24lf2ItPwIXJD4VVAQAAjU+giV/I6P9tAACNj3D+///oL1ABAI2PVP7//+g9n
QAAjY9k/f//6FdWAQCNj1T9///o020AAI2PRP3//+jSnv//jY80/f//6JBaAQCNjyT9///oS1wBAI2PFP3//+hAXAEAjY8E/f//6DVcAQCLj/z8//+FyQ+FH4EDAI2P6Pz//+h7ZP//i4/g/P//hckPhRqBAwCLj9T8//+FyQ+FIIEDAI2PwPz//+hcbQAAjY+o/P//6AOJAACLj5z8//+FyQ+FEIEDAImfpPz//4uPkPz
//4XJD4UQgQMAjY9M/P//iZ+Y/P//6B5tAACNjzz8///oE20AAI2PLPz//+gIbQAAjY/8+///6JhTAQBfXluL5V3Di4c4/f//iwSwiwCFwHQj/zD/NaxyTAD/FfQGSQCLhzj9//+LBLCLCIXJdAZR6N9dAQBGO7c8/f//D4P0/f//67+LcQRR6LJaAQCLzoX2D4SY/v//6+vMzMzMzMzMzMzMVYvsg+T4gey0AAAAU4tdE
FaLdQxXixOD7BCJTCQci0YEx0QkLAMAAACLBJCL1IsID794Col8JHiLAYkCi0EEiUIEi0EIiUIIi0EMi0wkHIlCDIHBNAEAAP8A6MGGAACLTCQMiUQkSIXAD4QcgAMAixOLgUgBAABCi3YEiYQkhAAAADPAiUQkcIlEJHjHRCR8AQAAAIlEJESJRCRAiUQkXIlEJGCJRCRMiUQkZIsElsdEJDSMCUkAx0QkOAAAAADHRCQ
8AAAAAGaDeAhHx0QkWBgsSQCJEw+Fwn8DAEIz/4kTiVQkKIsLiwSOD79ACIP4R30HjUEBiQPr64PoRw+EEQoAAEgPhaF/AwCF/w+Fu38DAIvBiUQkEECJA4tEJEiLQBA7BaCCTAAPj+wJAACFwA+O5AkAAMHgBAMF3IJMAItIBIlEJCSJTCRQiwFmg3gIAHUJgzgrD4R8fwMAvwMAAACLRCRIM9uJfCQUiVwkLIlcJCA5W
BQPj/gFAACLTCQQil0IhNsPhTCAAwCLdCQMi3wkSItEJCw7RxgPjJ6DAwA7RxQPj5WDAwA70Q+FjYMDAItHEMdEJGgAAAAAiUQkbIXAD4jIgAMAjUQkaLmAgkwAUOjmiP//i3QkaIX2D4W6gAMAg38UAQ+N5wIAAItcJAyLRCQsx4QkiAAAAAAAAADHhCSQAAAAAAAAAImDSAEAAIsDx4QklAAAAAEAAADGhCSYAAAAAMe
EJJwAAAAAAAAAi3gEA/vGhCSgAAAAAMeEJKgAAAAAAAAAx4QksAAAAAAAAADHhCS0AAAAAQAAAIB/DQDGhCS4AAAAAA+FAoIDAGpA6E5dAQCL8IPEBIX2D4ThCAAAjYQkiAAAAMdGCAAAAABQi87opZv//42EJKgAAADGRhAAjU4gx0YUAAAAAMZGGABQx0EIAAAAAOh/m///xkYwAItHCIlGOIl3CItEJEiNs1wBAAD/R
wSLu/QAAACLTCQMi0AQix5AUOhfCQAAOx4PgpOBAwCLXCQMiwOJu/QAAACLSASLRBkIA8uAeQ0AD4VVCAAAgHgQAItFGA+E3gYAAIB5DQDGAAEPhWqBAwCLeQiLdRQ793QZg34MBA+EYYEDAIvO6FOa//9Xi87o65r//4sDi3AEi0QeCAPzgH4NAA+FCwgAAIB4GAAPhU2BAwCAfg0Ai0YID4X8BwAAgHgwAA+FTYEDAIs
Di3AEA/OAfg0AD4U=",
"packet_info": {
"linktype": 1
}
}
Elasticsearch/Kibana
The same result can be achieved with a similar query can also be used in Elasticsearch Suricata alert data:
event_type:alert AND metadata.flowbits:*stamus.nrd* AND alert.signature:*exe* AND http.status:200
SIEM/DB Query based
Elasticsearch/Kibana - HTTP
In this example we will use purely HTTP protocol transaction logging from Suricata.
For a regular Elasticsearch query – based only on Suricata HTTP logs (not alerts) – we can use the following (based on “event_type”:”http” logs in Suricata):
event_type:http AND metadata.flowbits:*stamus.nrd* AND metadata.flowbits:*http.binary* AND http.status:200
Which basically says:
- Give me all HTTP transactions
- That have Executables transferred to or from NRD
- with HTTP status code 200
File Transactions
We can do similar for the Fileinfo transactions (based on “event_type”:”fileinfo” logs in Suricata):
event_type:fileinfo AND metadata.flowbits:*stamus.nrd* AND fileinfo.magic:*executable*
Which basically says:
- Give me all file transactions
- That have Executables transferred to or from Newly Registered Domain
That way , in this PCAP review case we go down from over 7600 Suricata file transaction logs…..
…to 1 that needs our attention:
The actual file transaction log contains all the relevant filename/filemagic/sha256,md5 sums and so on:
{
"timestamp": "2023-10-12T22:09:13.481788+0200",
"flow_id": 1689769056868824,
"pcap_cnt": 1026,
"event_type": "fileinfo",
"src_ip": "104.21.91.46",
"src_port": 80,
"dest_ip": "10.10.10.113",
"dest_port": 64642,
"proto": "TCP",
"pkt_src": "wire/pcap",
"metadata": {
"flowbits": [
"stamus.nrd",
"ET.http.binary"
]
},
"ether": {
"src_mac": "00:0c:30:5f:c4:a3",
"dest_mac": "08:1f:71:62:50:7d"
},
"http": {
"hostname": "hgfdytrywq.com",
"url": "/a",
"http_user_agent": "Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) WindowsPowerShell/5.1.19041.3031",
"http_content_type": "application/octet-stream",
"http_method": "GET",
"protocol": "HTTP/1.1",
"status": 200,
"length": 893608
},
"app_proto": "http",
"fileinfo": {
"filename": "Autoit3.exe",
"magic": "PE32 executable (GUI) Intel 80386, for MS Windows, 5 sections",
"gaps": false,
"state": "CLOSED",
"md5": "c56b5f0201a3b3de53e561fe76912bfd",
"sha1": "2a4062e10a5de813f5688221dbeb3f3ff33eb417",
"sha256": "237d1bca6e056df5bb16a1216a434634109478f882d3b1d58344c801d184f95d",
"stored": true,
"file_id": 1,
"size": 893608,
"tx_id": 0
}
}
In the case above the file was also stored / extracted to disk - "stored": true - which can be useful for further file analysis during incident response.
Conclusion
Some people might still consider Suricata a “legacy” intrusion detection system (IDS), but the data produced by Surcata speaks for itself. It is not only a highly capable IDS but has evolved into an impressive tool for gathering NSM data and full protocol, file transaction, flow, and anomaly logs along with file extraction and PCAP logging. And as we’ve shown at Stamus Networks, Suricata is in fact a powerful foundation on which to build a world-class network detection and response (NDR) system.
We hope this article helps to dispel some of these misconceptions and can help users see the many ways to use and optimize Suricata beyond the basic alerts and signatures.
Additional resources
Further reading on this topic can be found in our free book, “Suricata for Analysts.”” the world’s first practical guide to threat detection and hunting using Suricata.
To read part 3 of this series, please visit this link.
Please make sure you check out our free and Open Source contributions to Suricata on our Stamus Labs page.
If you wish to be notified when we publish similar content, please subscribe to the Stamus Networks blog.
Finally, to receive near-real time updates, follow us on Twitter, LinkedIn, and Facebook, or join our Discord.
