Monday, October 5, 2020

Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04

Now that we have provided some basic security to Elasticsearch and Kibana communications, time to get some logs into the system.

Let's start this fourth post off with installing Metricbeat. Metricbeat can be downloaded directly from Elastic web site site. However, I prefer to use the instructions which can be found within my Kibana install.

Metricbeat

From the "Add Data" page, I then selected "Elasticsearch metrics". As we are running on Ubuntu, the instruction below is from the "Deb" tab. 

First download Metricbeat.

root@securitynik-monitoring:~$ cd /tmp/
root@securitynik-monitoring:~# curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.9.2-amd64.deb
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 37.2M  100 37.2M    0     0  23.4M      0  0:00:01  0:00:01 --:--:-- 23.4M

root@securitynik-monitoring:/tmp$ ls metricbeat-7.9.2-amd64.deb
metricbeat-7.9.2-amd64.deb

Next we install Metricbeat.

securitynik@securitynik-monitoring:/tmp$ sudo dpkg --install metricbeat-7.9.2-amd64.deb
Selecting previously unselected package metricbeat.
(Reading database ... 175727 files and directories currently installed.)
Preparing to unpack metricbeat-7.9.2-amd64.deb ...
Unpacking metricbeat (7.9.2) ...
Setting up metricbeat (7.9.2) ...
Processing triggers for systemd (245.4-4ubuntu3.2) ...

With Metricbeat installed, time to configure it.

securitynik@securitynik-monitoring:/tmp$ cd /etc/metricbeat/
securitynik@securitynik-monitoring:/etc/metricbeat$ sudo cp metricbeat.yml metricbeat.yml.ORIGINAL

Here are the changes I made

root@securitynik-monitoring:~# cat /etc/metricbeat/metricbeat.yml | grep --perl-regexp "^\s+host|^\s+protocol|^\s+username|^\s+password|^\s+protocol"
        host: "https://10.0.0.1:5601"
  hosts: ["https://10.0.0.1:9200"]
  protocol: "https"
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"

With that out of the way, we need to ensure Metricbeat is configured for SSL. As a result, the following lines were also added to the "metricbeat.yml" file.

root@securitynik-monitoring:~# tail --lines 8 /etc/metricbeat/metricbeat.yml
# SSL Configuration enabled by Nik
ssl.enabled: true
output.elasticsearch.hosts: ["https://10.0.0.1:9200"]
output.elasticsearch.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

setup.kibana.ssl.enabled: true
setup.kibana.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

Once those were completed, I then enabled the Elasticsearch module

root@securitynik-monitoring:~# metricbeat modules enable elasticsearch
Enabled elasticsearch

Configuring the metricbeat Kibana module, I modified the "/etc/metricbeat/modules.d/elasticsearch.yml" to look as follow:


root@securitynik-monitoring:~# vi /etc/metricbeat/modules.d/elasticsearch.yml
root@securitynik-monitoring:~# cat /etc/metricbeat/modules.d/elasticsearch.yml
# Module: elasticsearch
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.9/metricbeat-module-elasticsearch.html

- module: elasticsearch
  #metricsets:
  #  - node
  #  - node_stats
  period: 10s
  hosts: ["https://10.0.0.1:9200"]
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"
  ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

  Then the Kibana configuration.

  root@securitynik-monitoring:~# cat /etc/metricbeat/modules.d/kibana.yml
# Module: kibana
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.8/metricbeat-module-kibana.html

- module: kibana
  #metricsets:
  #  - status
  period: 10s
  hosts: ["https://10.0.0.1:5601"]
  #basepath: ""
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"

Run the Metricbeat setup command to load the Kibana dashboards.

root@securitynik-monitoring:/etc/metricbeat# metricbeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

Now let's pray, cross our hands, legs, hair, etc. and start Metricbeat.

root@securitynik-monitoring:/etc/metricbeat# systemctl enable --now metricbeat.service
Synchronizing state of metricbeat.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable metricbeat
Created symlink /etc/systemd/system/multi-user.target.wants/metricbeat.service → /lib/systemd/system/metricbeat.service.

root@securitynik-monitoring:/etc/metricbeat# systemctl status metricbeat.service
● metricbeat.service - Metricbeat is a lightweight shipper for metrics.
     Loaded: loaded (/lib/systemd/system/metricbeat.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2020-08-14 11:22:33 EDT; 8s ago
       Docs: https://www.elastic.co/products/beats/metricbeat
   Main PID: 33050 (metricbeat)
      Tasks: 11 (limit: 4563)
     Memory: 19.9M

     CGroup: /system.slice/metricbeat.service
             └─33050 /usr/share/metricbeat/bin/metricbeat -environment systemd -c /etc/metricbeat/metricbeat.yml -path.home /usr/share/metricbeat -path.config /etc/metric>

Aug 14 11:22:38 securitynik-monitoring metricbeat[33050]: 2020-08-14T11:22:38.169-0400        INFO        [index-management.ilm]        ilm/std.go:139        do not generate ilm p>
Aug 14 11:22:38 securitynik-monitoring metricbeat[33050]: 2020-08-14T11:22:38.169-0400 INFO [index-management] idxmgmt/std.go:274 ILM policy success

Above suggests we are good to go. Let's confirm. 

Metricbeat receiving data

With Kibana letting us know that it is successfully retrieving data, let's enable some modules we know we will need.

To see a list of modules available, we execute.

root@securitynik-monitoring:/etc/metricbeat# metricbeat modules list

Enabled:
elasticsearch
system


Disabled:
activemq
aerospike
apache
....

We can see we have two modules enabled and a number of others disabled. If we go through the returned list, we may find one or more that we are interested in. As I am here, I will enable. 

root@securitynik-monitoring:/etc/metricbeat# metricbeat modules enable beat kibana linux 

Enabled beat
Enabled kibana
Enabled linux

At this point, you now have to configure the various modules as I did above for Kibana.

Next we setup up the Kibana dashboards.

root@securitynik-monitoring:/etc/metricbeat# metricbeat setup --dashboards
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

With all of that out of the way, here is what the data looks like :

Metricbeat


Ok then. With this out of the way, now it is time to move on to Filebeat.

Posts in this series:

Security On The Cheap - Beginning Elastic Stack - Installing Elastic 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Installing Kibana 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Providing Basic Security to Elastic and Kibana 7.9 communication on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Auditbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04
Beginning Elastic - Installing, Configuring and Providing Basic Security to Packetbeat
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Winlogbeat

See you in the next post.

References:
https://www.elastic.co/beats/
https://www.elastic.co/pdf/introduction-to-logging-with-the-elk-stack.pdf
https://github.com/elastic/examples/tree/master/Reference/Beats
https://www.elastic.co/guide/en/beats/filebeat/current/configuration-ssl.html#key


No comments:

Post a Comment