Monday, October 5, 2020

Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04

In this the sixth post in this series, we are looking at installing and securing Filebeat. The first post, we installed Elasticsearc. In the second post we installed Kibana while in the third post we provided basic security to Elastic and Kibana. In the fourth post, we installed, configured and secured Metricbeat. In the fifth post, we installed, configured and secured Auditbeat

In this post, we use the apt package manager to install Filebeat similarly to how we installed Auditbeat. Note you could have also download the .deb file similarly to what was done with Metricbeat install

root@securitynik-monitoring:~# apt-get install filebeat

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  filebeat
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 31.4 MB of archives.
After this operation, 112 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/7.x/apt stable/main amd64 filebeat amd64 7.9.2 [31.4 MB]
Fetched 31.4 MB in 2s (18.6 MB/s)
Selecting previously unselected package filebeat.
(Reading database ... 176095 files and directories currently installed.)
Preparing to unpack .../filebeat_7.9.2_amd64.deb ...
Unpacking filebeat (7.9.2) ...
Setting up filebeat (7.9.2) ...
Processing triggers for systemd (245.4-4ubuntu3.2) ...

With Filebeat installed, let's configure it. As always, I make a copy of the original file before editing it.

root@securitynik-monitoring:~# cd /etc/filebeat/
root@securitynik-monitoring:/etc/filebeat# cp filebeat.yml filebeat.yml.ORIGINAL

I then modified the entries to reflect below:

root@securitynik-monitoring:~# cat /etc/filebeat/filebeat.yml | grep --perl-regexp "^\s+host|^\s+username|^\s+password|^\s+protocol"
  host: "https://10.0.0.1:5601"
  hosts: ["10.0.0.1:9200"]
  protocol: "https"
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"

Finally, I copied the last eight lines from the Metricbeat configuration file into the file "filebeat.yml" file.

root@securitynik-monitoring:~# tail --lines 8 /etc/metricbeat/metricbeat.yml >> /etc/filebeat/filebeat.yml
root@securitynik-monitoring:~# tail --lines 8 /etc/filebeat/filebeat.yml
# SSL Configuration enabled by Nik
ssl.enabled: true
output.elasticsearch.hosts: ["https://10.0.0.1:9200"]
output.elasticsearch.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

setup.kibana.ssl.enabled: true
setup.kibana.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

Enable the "system" module

root@securitynik-monitoring:~# filebeat modules enable system
Enabled system

Load the Kibana dashboards

root@securitynik-monitoring:~# filebeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
Setting up ML using setup --machine-learning is going to be removed in 8.0.0. Please use the ML app instead.

See more: https://www.elastic.co/guide/en/elastic-stack-overview/current/xpack-ml.html
Loaded machine learning job configurations
Loaded Ingest pipelines

Cross our fingers and start Filebeat.

root@securitynik-monitoring:~# systemctl enable --now filebeat.service
Synchronizing state of filebeat.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable filebeat
Created symlink /etc/systemd/system/multi-user.target.wants/filebeat.service → /lib/systemd/system/filebeat.service.


root@securitynik-monitoring:~# systemctl status filebeat.service
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
     Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2020-08-14 12:23:42 EDT; 23s ago
       Docs: https://www.elastic.co/products/beats/filebeat
   Main PID: 33756 (filebeat)
      Tasks: 14 (limit: 4563)
     Memory: 64.6M
     CGroup: /system.slice/filebeat.service
             └─33756 /usr/share/filebeat/bin/filebeat -environment systemd -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.d>

Looking to see if data is being received by Filebeat

With data now being received by Filebeat, let's now enable some modules we may need. Looking to see what modules exist.

root@securitynik-monitoring:~# filebeat modules list
Enabled:
system


Disabled:
activemq
apache
auditd
............

Enabling the ones I believe I will need later. Additionally, in this post, we installed Zeek on Ubuntu. By enabling Zeek below, we can now ingest these logs into Elastic via Filebeat.

root@securitynik-monitoring:~# filebeat modules enable  auditd elasticsearch iptables kibana netflow zeek
Enabled auditd
Enabled elasticsearch
Enabled iptables
Enabled kibana
Enabled netflow
Enabled zeek

From the blog on Zeek, the Zeek logs are stored in "/usr/local/zeek/etc/zeekctl.cfg".

root@securitynik-monitoring:~# cat /usr/local/zeek/etc/zeekctl.cfg  | grep LogDir
# Expiration interval for archived log files in LogDir.  Files older than this
LogDir = /usr/local/zeek/logs

With the above in mind, I then created a "bro" directory under "/var/logs/" and made a symbolic link to the currently configured Zeek log directory.

root@securitynik-monitoring:~# sudo mkdir /var/log/bro
root@securitynik-monitoring:~# sudo ln --symbolic /usr/local/zeek/logs/current /var/log/bro/

Next Zeek is configured to store its logs in JSON format

root@securitynik-monitoring:~# echo "@load policy/tuning/json-logs.zeek" >> /usr/local/zeek/share/zeek/site/local.zeek

Note at this point you may have to restart Zeek and or Filebeat. 

Below we see that Zeek's data is being received successfully.


Let's now look at other data received by Filebeat.


With that out of the way, let's move on to installing, configuring and securing Packetbeat.

Posts in this series:

Security On The Cheap - Beginning Elastic Stack - Installing Elastic 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Installing Kibana 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Providing Basic Security to Elastic and Kibana 7.9 communication on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Auditbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04
Beginning Elastic - Installing, Configuring and Providing Basic Security to Packetbeat
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Winlogbeat

References:

https://www.elastic.co/beats/filebeat
https://www.elastic.co/guide/en/beats/filebeat/7.8/setup-repositories.html#_apt
https://www.ericooi.com/zeekurity-zen-part-iii-how-to-send-zeek-logs-to-splunk/
https://www.elastic.co/blog/collecting-and-analyzing-zeek-data-with-elastic-security

No comments:

Post a Comment