In the SANS SEC503 Intrusion Detection in Depth class, we teach you quite a lot to get you started with Zeek Network Security Monitoring. One of the things we cannot do because of time, is walk you through the installation, upgrading, etc., of Zeek. In this post, we help you to install Zeek 3.1.4, the current version as of this writing on Ubuntu 20.04. The hope is with the knowledge we provide to you in the SEC 503 class as well as what you get here, you are in a better position to maximize your Zeek deployment.
First, let us confirm my version of Ubuntu. As seen below, I am running Ubuntu 20.04 LTS.
securitynik@securitynik-zeek:~$ lsb_release --all No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04 LTS Release: 20.04 Codename: focal
Next, let us install Zeek's dependencies. As we are using Ubuntu, we need the following dependencies:
cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev
Let's install them by using:
securitynik@securitynik-zeek:~$sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev
Let's now install an optional dependency. Specifically, we will install libmaxminddb (for geolocating IP addresses). To make the most of this, you will need to register with MaxMind before you can download the GeoIP dataset. Once registered and logged in, download the GeoLite2 City database.
Once downloaded, you may need to copy the file to the remote machine running Zeek. In my example, I downloaded the file on a Windows system, then used SCP to transfer it to the device running Zeek.
If you don't wish to use geo-data, feel free to skip this step.
securitynik@securitynik-zeek:~$ sudo apt-get install libmaxminddb-dev
Copy the GeoIP data file from Windows 10 system to the system running Zeek, using the native SCP client in Windows 10.
C:\Users\securitynik\Downloads>scp GeoLite2-City_20200609.tar.gz nik@192.168.0.4:~/ nik@192.168.0.4's password: GeoLite2-City_20200609.tar gz 100% 29MB 10.4MB/s 00:02
Once the file is on the system running Zeek, we then extract the files:
securitynik@securitynik-zeek:~$ tar --extract --verbose --gunzip --file GeoLite2-City_20200609.tar.gz GeoLite2-City_20200609/ GeoLite2-City_20200609/README.txt GeoLite2-City_20200609/COPYRIGHT.txt GeoLite2-City_20200609/GeoLite2-City.mmdb GeoLite2-City_20200609/LICENSE.txt
Next copy the GeoLite2-City_20200609/GeoLite2-City.mmdb into the /usr/share/GeoIP folder. In my case I had to create this folder.
securitynik@securitynik-zeek:~$ sudo mkdir --parents /usr/share/GeoIP securitynik@securitynik-zeek:~$ sudo cp GeoLite2-City_20200609/GeoLite2-City.mmdb /usr/share/GeoIP/
Once the dependencies have been installed successfully, we then transition to the Zeek installation. For our install, we use the latest version at the time of this writing from Zeek's GitHub page.
securitynik@securitynik-zeek:~$ git clone --recursive https://github.com/zeek/zeek
Upon completion of the cloning, we then change directory (cd) into the zeek directory. Once we changed into the zeek directory, we then specify "--with-geoip" argument for the configure script so as to customize our installation. In this example, we are pointing it to the previously created folder which holds or geo-data file.
securitynik@securitynik-zeek:~/zeek$ cd zeek/ securitynik@securitynik-zeek:~/zeek$ ./configure --with-geoip=/usr/share/GeoIP
If everything went ok, you may see something similar to:
.... ====================| Zeek Build Summary |==================== Build type: RelWithDebInfo Build dir: /home/nik/zeek/build Install prefix: /usr/local/zeek Zeek Script Path: /usr/local/zeek/share/zeek Debug mode: false Unit tests: CC: /usr/bin/cc CFLAGS: -Wall -Wno-unused -O2 -g -DNDEBUG CXX: /usr/bin/c++ CXXFLAGS: -Wall -Wno-unused -Wno-register -Werror=vla -std=c++17 -O2 -g -DNDEBUG CPP: /usr/bin/c++ ZeekControl: true Aux. Tools: true libmaxminddb: true Kerberos: false gperftools found: false tcmalloc: false debugging: false jemalloc: false Fuzz Targets: Fuzz Engine: ================================================================ -- Configuring done -- Generating done -- Build files have been written to: /home/nik/zeek/build
Next up, make and make install Zeek. Grab a quick coffee, beer or Guinness. Basically whatever you drink as depending on your system resources, this may take a while to complete.
securitynik@securitynik-zeek:~/zeek$ sudo make securitynik@securitynik-zeek:~/zeek$ sudo make install
Now that the install has completed, test to see if zeek can be executed.
securitynik@securitynik-zeek:~$which zeek securitynik@securitynik-zeek:~$
Looks like zeek is not in the path. Let's add a symbolic link and verify Zeek is now in the path
securitynik@securitynik-zeek:~$ sudo ln --symbolic /usr/local/zeek/bin/zeek /usr/bin/zeek securitynik@securitynik-zeek:~$ which zeek /usr/bin/zeek
Looks good! Similarly, let's setup a symbolic link for zeekctl
securitynik@securitynik-zeek:~$ sudo ln --symbolic /usr/local/zeek/bin/zeekctl /usr/bin/zeekctl
securitynik@securitynik-zeek:~$ which zeekctl /usr/bin/zeekctl
While the above is cool, we need to make this a bit more permanent. More importantly, there are other files in the /usr/local/zeek/bin/ directory such as zeek-cut that we more than likely need to use. Thus adding all of these symbolic links may not be the best use of our time. Let's address this by first modifying the path.
securitynik@securitynik-zeek:~$ export PATH=$PATH:/usr/local/zeek/bin securitynik@securitynik-zeek:~$ echo $PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/loal/zeek/bin:/usr/local/zeek/bin
This now allows us to access the other binaries in the /usr/local/zeek/bin/ directory such as zeek-cut
Let's now make it permanent.
securitynik@securitynik-zeek:~$ echo "export PATH=$PATH:/usr/local/zeek/bin" >> .bashrc securitynik@securitynik-zeek:~$ tail --lines 1 .bashrc export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/loal/zeek/bin:/usr/local/zeek/bin:/usr/local/zeek/bin
At this point you should be able to execute Zeek with its default configuration in standalone mode. However, for us, let move away from the defaults.
First edit the networks.cfg using your favourite editor (vi, nano, gedit, mousepad, etc.), to ensure it reflects your protected network(s). Here is what mine looks like.
securitynik@securitynik-zeek:/usr/local/zeek/etc$ cat networks.cfg # List of local networks in CIDR notation, optionally followed by a # descriptive tag. # For example, "10.0.0.0/8" or "fe80::/64" are valid prefixes. #10.0.0.0/8 Private IP space #172.16.0.0/12 Private IP space 192.168.0.0/24 SecurityNik Protected Network
For demonstration purposes, I commented out 10.0.0.0/8 and 172.16.0.0/12 networks and left the 192.168.0.0/16. I also changed the description to suit my needs. You should also add any other networks you have to this file.
Once that is completed, you should then configure the node.cfg file. Here is what mine looks like. I commented out the standalone section to simulate a distributed deployment. Everything is on one host but the host has two interfaces. One of those is the loopback and the other enp0s25. Note, you don't really need to do it the way I did. I am only doing it this way to demonstrate how you can use different workers on different systems and or interfaces.
#[zeek] #type=standalone #host=securitynik #interface=enp0s25 ## Below is an example clustered configuration. If you use this, ## remove the [zeek] node above. #[logger-1] #type=logger #host=localhost # [securitynik-zeek-manager] type=manager host=192.168.0.4 # [securitynik-zeek-proxy] type=proxy host=192.168.0.4 # [securitynik-zeek-worker-enp0s25] type=worker host=192.168.0.4 interface=enp0s25 # [securitynik-zeek-worker-lo] type=worker host=localhost interface=lo
Once you have modified your node.cfg to reflect your configuration needs, your next step is to make any necessary changes to your zeekctl.cfg. At a minimum in this file you may want to change the recipient email address. Here is a snapshot of my file:
securitynik@securitynik-zeek:/usr/local/zeek/etc$ cat zeekctl.cfg | more ## Global ZeekControl configuration file. ############################################### # Mail Options # Recipient address for all emails sent out by Zeek and ZeekControl. MailTo = admin@securitynik.local ......
At this point, we should be good to go. Once you modify your files, you should deploy your changes. Let's do that.
securitynik@securitynik-zeek:/usr/local/zeek/etc$ sudo zeekctl deploy checking configurations ... installing ... removing old policies in /usr/local/zeek/spool/installed-scripts-do-not-touch/site ... removing old policies in /usr/local/zeek/spool/installed-scripts-do-not-touch/auto ... creating policy directories ... installing site policies ... generating cluster-layout.zeek ... generating local-networks.zeek ... generating zeekctl-config.zeek ... generating zeekctl-config.sh ... stopping ... stopping workers ... stopping proxy ... stopping manager ... starting ... starting manager ... starting proxy ... starting workers ...
Looks like everything started properly. Let's confirm this:
securitynik@securitynik-zeek:/usr/local/zeek/etc$ sudo zeekctl status Name Type Host Status Pid Started securitynik-zeek-manager manager 192.168.0.4 running 207505 13 Jun 14:13:33 securitynik-zeek-proxy proxy 192.168.0.4 running 207554 13 Jun 14:13:35 securitynik-zeek-worker-enp0s25 worker 192.168.0.4 running 207624 13 Jun 14:13:37 securitynik-zeek-worker-lo worker localhost running 207621 13 Jun 14:13:37
Looks like we are good to go. To further confirm, let's see if any logs are being created.
securitynik@securitynik-zeek:/usr/local/zeek/etc$ ls ../logs/current broker.log conn.log files.log loaded_scripts.log reporter.log stats.log stdout.log weird.log cluster.log dns.log http.log packet_filter.log ssl.log stderr.log syslog.log x509.log
Nice. Looks like we are making progress.
Finally, you may wish to have Zeek start as a service via Systemd. Let's make that happen.
Create a file name zeek.zervice and verify its existence as follow.
securitynik@securitynik-zeek:~$ sudo touch /etc/systemd/system/zeek.service securitynik@securitynik-zeek:~$ ls /etc/systemd/system/zeek.service* /etc/systemd/system/zeek.service
Next edit the file and add the following contents:
securitynik@securitynik-zeek:~$ sudo vi /etc/systemd/system/zeek.service securitynik@securitynik-zeek:~$ cat /etc/systemd/system/zeek.service Description=Zeek Network Security Monitoring Wants=network.target After=syslog.target network-online.target [Service] Type=forking ExecStart=zeekctl deploy Restart=on-failure RestartSec=10 KillMode=process [Install] WantedBy=multi-user.target
Next reload systemd
securitynik@securitynik-zeek:~$ sudo systemctl daemon-reload
Enable the zeek service (zeek.zervice)
securitynik@securitynik-zeek:~$ sudo systemctl enable --now zeek.service Created symlink /etc/systemd/system/multi-user.target.wants/zeek.service → /etc/systemd/system/zeek.service.
Start the service
securitynik@securitynik-zeek:~$ sudo systemctl start zeek.service securitynik@securitynik-zeek:~$
Looks like the Zeek service started without errors. Let's confirm the service is now running.
securitynik@securitynik-zeek:~$ systemctl status zeek.service ● zeek.service Loaded: loaded (/etc/systemd/system/zeek.service; enabled; vendor preset: enabled) Active: active (running) since Sat 2020-06-13 14:42:21 EDT; 1min 26s ago Process: 215455 ExecStart=/usr/bin/zeekctl start (code=exited, status=0/SUCCESS) Main PID: 215455 (code=exited, status=0/SUCCESS) Tasks: 50 (limit: 2225) Memory: 293.0M CGroup: /system.slice/zeek.service ├─215481 /usr/bin/bash /usr/local/zeek/share/zeekctl/scripts/run-zeek -1 -U .status -p zeekctl -p zeekctl-> ├─215487 /usr/local/zeek/bin/zeek -U .status -p zeekctl -p zeekctl-live -p local -p securitynik-zeek-manager lo> ├─215533 /usr/bin/bash /usr/local/zeek/share/zeekctl/scripts/run-zeek -1 -U .status -p zeekctl -p zeekctl-> ├─215539 /usr/local/zeek/bin/zeek -U .status -p zeekctl -p zeekctl-live -p local -p securitynik-zeek-proxy loca> ├─215592 /usr/bin/bash /usr/local/zeek/share/zeekctl/scripts/run-zeek -1 -i enp0s25 -U .status -p zeekctl > ├─215600 /usr/bin/bash /usr/local/zeek/share/zeekctl/scripts/run-zeek -1 -i lo -U .status -p zeekctl -p ze> ├─215605 /usr/local/zeek/bin/zeek -i enp0s25 -U .status -p zeekctl -p zeekctl-live -p local -p securitynik-zeek> └─215609 /usr/local/zeek/bin/zeek -i lo -U .status -p zeekctl -p zeekctl-live -p local -p securitynik-zeek-work> Jun 13 14:42:14 securitynik systemd[1]: Started zeek.service. Jun 13 14:42:21 securitynik zeekctl[215455]: starting manager ... Jun 13 14:42:21 securitynik zeekctl[215455]: starting proxy ... Jun 13 14:42:21 securitynik zeekctl[215455]: starting workers ... Jun 13 14:42:21 securitynik systemd[1]: zeek.service: Succeeded.
Above we can see zeek.serviceSuceeded. However, if you look at the Active above, we see Inactive dead. However, I was able to confirm Zeek is running as expected even after stopping Zeek with zeekctl stop. I then restarted the system and run sudo zeekctl status along with systemctl status zeek.service and everything is working as expected.
At this point, I will close off this post. Have fun, stay safe and I look forward to seeing you in one of the upcoming SANS SEC503 Intrusion Detection in Depth class, where we teach you how to maximize your security monitoring with Zeek.
Reference:
Thank you! Just what I needed.... Lab VMs didn't come with my class for some reason, so I'm recreating the environment ...
ReplyDeleteJim,
DeleteGlad you found it helpful. Let me know if you have any questions.
This is great!!!!!
ReplyDeleteThank you very much
You are welcome Sam!
DeleteAmazing tutorial. thank you very much for making our lives easier. Can't thank you enough. your hardwork is helping novice like me.
ReplyDeleteHey Anonymous, that is the whole idea. If I was able to help someone else learn this content then I've done at least one good for today :-)
DeleteLet me know if there is anything else I can help with.
Hello I am a novice zeek/linux user.
ReplyDeleteWhen i run the following command, I continue to get the following error. Can you please help me ?
securitynik@securitynik-zeek:~$sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev
ERROR RECEIVED
-- Found SWIG: /usr/bin/swig4.0 (found version "4.0.1")
-- Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Development) (found version "3.8")
-- Could NOT find PythonDev (missing: PYTHON_INCLUDE_DIR)
CMake Error at auxil/zeekctl/auxil/pysubnettree/CMakeLists.txt:29 (message):
Could not find prerequisite package 'PythonDev'
CMake Error at auxil/zeekctl/auxil/pysubnettree/CMakeLists.txt:31 (message):
Configuration aborted due to missing prerequisites
-- Configuring incomplete, errors occurred!
See also "/home/wajid/bro/build/CMakeFiles/CMakeOutput.log".
------------------I ran following commands but I was not able to get the above issue resolved
george@ubuntu:~/bro$ sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'python-dev-is-python2' instead of 'python-dev'
bison is already the newest version (2:3.5.1+dfsg-1).
cmake is already the newest version (3.16.3-1ubuntu1).
flex is already the newest version (2.6.4-6.2).
g++ is already the newest version (4:9.3.0-1ubuntu2).
gcc is already the newest version (4:9.3.0-1ubuntu2).
libpcap-dev is already the newest version (1.9.1-3).
make is already the newest version (4.2.1-1.2).
python-dev-is-python2 is already the newest version (2.7.17-4).
swig is already the newest version (4.0.1-5build1).
libssl-dev is already the newest version (1.1.1f-1ubuntu2.1).
zlib1g-dev is already the newest version (1:1.2.11.dfsg-2ubuntu1.2).
The following package was automatically installed and is no longer required:
libfprint-2-tod1
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
george@ubuntu:~/bro$ sudo apt-get install python-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'python-dev-is-python2' instead of 'python-dev'
python-dev-is-python2 is already the newest version (2.7.17-4).
The following package was automatically installed and is no longer required:
libfprint-2-tod1
Use 'sudo apt autoremove' to remove it.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
See also "/home/wajid/bro/build/CMakeFiles/CMakeError.log".
Also, before re-running configure, consider cleaning the cache (removing the build directory) via `make distclean`
Hello George,
DeleteFor starters, it seems you may have to install some Python3 and some dependencies based on this:
Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Development) (found version "3.8")
-- Could NOT find PythonDev (missing: PYTHON_INCLUDE_DIR)
CMake Error at auxil/zeekctl/auxil/pysubnettree/CMakeLists.txt:29 (message):
Could not find prerequisite package 'PythonDev'
I recommend you resolve those and start from there.
Nik I do not think you answered his question, I used your instructions months ago and everything worked great. Now jan 2021 I follow the same directions, and get the Python-dev not found error also. so it appears the github/zeek/zeek changed or ubuntu 20.04 changed. Either way your awesome instructions no longer work because something else changed. I think his question was how do we deal with the dependancy issue. whne I try to install python 3 it sayes it is installed already, but same issue remains, any help greatly appreciated.
DeleteHmmm! I can try to see if I can recreate this issue but I'm wondering if something changed with Zeek. I don't want to believe it is a change in Ubuntu 20.04. I can't promise I will look at this soon as I have some things I have to take care of that is going to take some time. However, if anyone finds the solution before I update this post, I would definitely appreciate you posting your findings.
DeleteI had the same problem. After installing the dependencies in the zeek/INSTALL file of the current version, everything worked fine! This file calls for python3 and python3-dev instead of pyhon-dev.
DeleteMuch appreciate the update Anonymous.
DeleteRunning the following command solves the problem:
Deletesudo apt install python-dev-is-python3
sudo apt autoremove
Hi, Do we need to install Zeek as a standalone instance or we can install this on an existing ELK instance? And would it be compatible with Ubuntu 18.04.3? Thank You!
ReplyDeleteHello Anonymous,
DeleteZeek and Elastic are installed on the same device in this post. However, you can install Zeek on it's own device and install Filebeat on that device to forward the logs to your remote Elastic instance. As for Ubuntu 18.04.3, I have not tried it but I don't expect the install to be much different.
If you do install it on Ubuntu 18.x let me know how it goes.
Thanks
Hi, thanks for the tutorial. I've managed to make the service "running" by adding Type=forking in the Service stanza as explained here:
ReplyDeletehttps://unix.stackexchange.com/questions/137028/why-is-my-systemd-unit-loaded-but-inactive-dead
Hmmm! Thanks! Very interesting!! Not sure how that slipped through. I updated the post and the reference.
DeleteVery much appreciated!
I just completed SEC503. This filled a gap. Much appreciated. It was one of the best classes I've taken. Explained a lot of things, I only assumed in the past.
ReplyDeleteAwesome Darren!
Delete