Here at Stamus Networks, we are strongly committed to open-source and believe that ease of use has a major impact on the success of free software. It is with this desire to make open-source more user-friendly that we have undertaken an architecture update for SELKS, a turnkey Suricata-based IDS/IPS/NSM ecosystem, in order to benefit from the practicality provided by Docker containers.
With the new architecture, we turned the SELKS OS into a much more portable and platform-agnostic solution. This has a very big importance for seamless integrations into existing environments and established processes.
Requirements and benefits
The previous design of SELKS was monolithic, with all components installed on a single OS. Everything was only intended to run on Debian Buster, and only shipped as a standalone distribution to be run either on a virtual machine or a dedicated box. This made the install quite heavy, requiring to run the whole Debian installer. Updates were also hard to manage, because we could not just build a new version of the SELKS image but had to provide the user with a way to update it from inside the OS.
Moreover, because the update process was based on standard packages upgrade process apt upgrade, the system could end up in a whole variety of configurations. Therefore, it was not easy to reproduce reported bugs as it required installing a new VM with the exact same versions of the different components. Needless to say, this was a lengthy process and did not allow efficient bug tracking.
With the new docker-based architecture, SELKS can now be installed on virtually any linux system, without requiring a heavy installation process. The ease of installation also allows a simpler bug-tracking process. It is faster and easier for us to deploy a new SELKS machine with specific versions of each component. Indeed, apart from Scirius image that is built and stored in Github, all the docker images are pulled from dockerhub, including official images: ElasticSearch, Kibana, Logstash and Nginx, and images maintained by Jason Ish: Suricata and Evebox. Therefore, all it takes to use a different version of a component is to pull the container tagged with the version number.
Design of the new SELKS architecture
After many months of multiple meetings, QA tests, ample cups of coffee, and 198 commits, we are proud to release this new implementation of SELKS that can be deployed on an existing OS that has docker and docker-compose installed.
A setup script checks if all software requirements are met, and can install docker or docker-compose if they are missing.
git clone https://github.com/StamusNetworks/SELKS.git
cd SELKS/docker/
./easy-setup.sh
docker-compose up -d
Portainer CE, an open-source web-ui that allows you to easily manage Docker and Kubernetes environments, is offered as an option for users new to docker management.
The setup script also generates SSL certificates used by nginx for https access to Scirius, and the Django private key used internally. It creates a .env file containing environment variables needed by the containers; the containers are then created from a compose file. Most data are stored in docker’s “named volumes,” except for the config files that are provided to the containers and therefore have to be bind-mounted from the local filesystem.
The containers communicate with each other using a dedicated virtual bridge network (managed by docker) that allows them to resolve each-other's hostnames and to access each other's services without exposing ports to the host, providing a better isolation from the outside.
However, Suricata does not use this network as it uses the host network in order to connect directly to the network interface. The only other network opening on the outside is Nginx, exposing HTTP ports on the host.
For complete installation and update documentation for SELKS on Docker, please checkout the documents on GitHub here: https://github.com/StamusNetworks/SELKS/blob/master/docker/README.md
Challenges we overcame
During the reworking of SELKS, we had a few issues that had to be handled.
- The most important of them was the size of the Docker Scirius image which was 1.5GB due to the required dependencies for some compilation (NodeJS, webpack, GCC, libc-dev, etc.). We had to leverage the Multistage dockerfile functionality of Docker to handle that, and shrink it down to around 400 MB.
- We also had to rethink the way old data are deleted from Elasticsearch. Rather than triggering Elasticsearch Curator from a cronjob, which would have been against docker best practices, we used the built-in ILM feature that allowed us to define an auto removal policy for ES indices.
- Finally, we also had to tweak the behavior of the Suricata image, overwriting the entry-point so that we could add custom configuration files.
We undertook a thorough testing of all major components, dashboards and visualizations throughout the whole migration process to ensure at each point that everything was fully operational and was behaving properly and predictably.
So, with this new architecture, we turned the SELKS OS into a much more portable and agnostic solution, and we are extremely happy with the results. But do not just take my word for it; we encourage you to test it out yourself.
