Monday, October 5, 2020

Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Winlogbeat - Elastic Stack 7.9 on Ubuntu 20.04

In this the eight and final post within this series, we install, configure and provide basic security for Winlogbeat. 

The first post, we installed Elasticsearch. In the second post we installed Kibana. This was followed by the third post where we provided basic security to Elastic and Kibana. In the fourth post, we installed, configured and secured Metricbeat and the fifth post, we installed, configured and secured Auditbeat. The sixth post, we installed, configured and provided basic security to Filebeat. In the seventh post, we installed, configured and provided basic security to Packetbeat.

Similar to the previous posts, we can start from "Add Data"

Winlogbeat Add Data

From the login page, I download the WINDOWS ZIP 64-BIT file. 

C:\Users\SecurityNik>certutil -f -URLCache "https://artifacts.elastic.co/downloads/beats/winlogbeat/winlogbeat-7.9.1-windows-x86_64.zip" winlogbeat-7.9.2.zip
****  Online  ****
CertUtil: -URLCache command completed successfully.

Verify the file has been downloaded successfully.

C:\Users\SecurityNik>dir winlogbeat-7.9.2.zip
 Volume in drive C is OS
 Volume Serial Number is D436-4013

 Directory of C:\Users\SecurityNik

2020-09-11  07:47 PM        19,156,840 winlogbeat-7.9.2.zip
               1 File(s)     19,156,840 bytes
               0 Dir(s)  31,707,971,584 bytes free

Expand the "winlogbeat-7.9.0" file into the current directory and verify the files within the archive.

PS C:\Users\SecurityNik> Expand-Archive -LiteralPath .\winlogbeat-7.9.2.zip -DestinationPath .

PS C:\Users\SecurityNik> dir .\winlogbeat-7.9.2-windows-x86_64\


    Directory: C:\Users\SecurityNik\winlogbeat-7.9.2-windows-x86_64                                                                                                                                                                                                                                      Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----        2020-09-11   8:14 PM                kibana
d-----        2020-09-11   8:14 PM                module
-a----        2020-09-01   8:12 PM             41 .build_hash.txt
-a----        2020-09-01   8:10 PM         194542 fields.yml
-a----        2020-09-01   8:12 PM            897 install-service-winlogbeat.ps1
-a----        2020-09-01   6:50 PM          13675 LICENSE.txt
-a----        2020-09-01   6:51 PM        8440372 NOTICE.txt
-a----        2020-09-01   8:12 PM            832 README.md
-a----        2020-09-01   8:12 PM            254 uninstall-service-winlogbeat.ps1
-a----        2020-09-01   8:12 PM       62381056 winlogbeat.exe
-a----        2020-09-01   8:10 PM          54147 winlogbeat.reference.yml
-a----        2020-09-01   8:10 PM           8778 winlogbeat.yml

With those files extracted and since we are also providing some basic security, we need to copy the "SecurityNik-CA.pem" Certification Authority (CA) certificate to this system in the same folder with the other configuration files. Since this is not mutual authentication where the server also has to authenticate the client, we should be good to go here. If we look at the Winlogbeat directory again, we see the certificate and Sysmon.exe. I added Sysmon to get that extra level of logging which Sysmon provides. Basically, if you are monitoring your Windows environment and not using Sysmon, then I guess you are special.

PS C:\Users\SecurityNik> dir .\winlogbeat-7.9.2-windows-x86_64\


    Directory: C:\Users\SecurityNik\winlogbeat-7.9.2-windows-x86_64


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
...
-a----        2020-08-14   3:29 PM           1200 SecurityNik-CA.pem
-a----        2020-08-14   2:17 PM        4282224 Sysmon.exe
...

Let's first install Sysmon on the host.

C:\winlogbeat-7.9.2-windows-x86_64>sysmon -accepteula -i


System Monitor v11.11 - System activity monitor
Copyright (C) 2014-2020 Mark Russinovich and Thomas Garnier
Sysinternals - www.sysinternals.com

Sysmon installed.
SysmonDrv installed.
Starting SysmonDrv.
SysmonDrv started.
Starting Sysmon..
Sysmon started.

With Sysmon now installed, we next modify Winlogbeat's configuration file. Below represents the change in my environment

PS C:\Users\SecurityNik> type C:\winlogbeat-7.9.2\winlogbeat.yml | more
....
#================== Kibana =========================
host: "https://10.0.0.1:5601"


# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["https://10.0.0.1:9200"]

  # Protocol - either `http` (default) or `https`.
  protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"


# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
      # Added by Nik
      geo:
        name: home-ON
        continent_name: North America
        geo.country_name: Canada
        country_iso_code: CA
        region_name: Ontario
        region_iso_code: Ontario
        city_name: GTA


# SSL Configuration enabled by Nik
ssl.enabled: true
output.elasticsearch.hosts: ["https://10.0.0.1:9200"]
output.elasticsearch.ssl.certificate_authorities: ["C:\\winlogbeat-7.9.2\\SecurityNik-CA.pem"]

setup.kibana.ssl.enabled: true
setup.kibana.ssl.certificate_authorities: ["C:\\winlogbeat-7.9.2\\SecurityNik-CA.pem"]

Rename and copy the winlogbeat folder to a simpler name and copy it to the root of the c: drive.

C:\Users\securitynik>xcopy /S /I /E .\winlogbeat-7.9.2 c:\winlogbeat-7.9.2\
.\winlogbeat-7.9.2\.build_hash.txt
.\winlogbeat-7.9.2\fields.yml
.\winlogbeat-7.9.2\install-service-winlogbeat.ps1
.\winlogbeat-7.9.2\LICENSE.txt
.\winlogbeat-7.9.2\NOTICE.txt
.\winlogbeat-7.9.2\README.md
.\winlogbeat-7.9.2\SecurityNik-CA.pem
.\winlogbeat-7.9.2\Sysmon.exe
.\winlogbeat-7.9.2\uninstall-service-winlogbeat.ps1
.\winlogbeat-7.9.2\winlogbeat.exe
.\winlogbeat-7.9.2\winlogbeat.reference.yml
.\winlogbeat-7.9.2\winlogbeat.yml
.\winlogbeat-7.9.2\kibana\7\dashboard\01c54730-fee6-11e9-8405-516218e3d268.json
.\winlogbeat-7.9.2\kibana\7\dashboard\71f720f0-ff18-11e9-8405-516218e3d268.json
.\winlogbeat-7.9.2\kibana\7\dashboard\8223bed0-b9e9-11e9-b6a2-c9b4015c4baf.json
.\winlogbeat-7.9.2\kibana\7\dashboard\bb858830-f412-11e9-8405-516218e3d268.json
.\winlogbeat-7.9.2\kibana\7\dashboard\Powershell-Overview-Dashboard.json
.\winlogbeat-7.9.2\kibana\7\dashboard\Winlogbeat-overview.json
.\winlogbeat-7.9.2\module\powershell\config\winlogbeat-powershell.js
.\winlogbeat-7.9.2\module\security\config\winlogbeat-security.js
.\winlogbeat-7.9.2\module\sysmon\config\winlogbeat-sysmon.js
21 File(s) copied

With the files now copied, let's test our configuration.

C:\winlogbeat-7.9.2>winlogbeat.exe test config
Config OK

Configuration looks "OK". 

Now to test that everything else is good to go.

C:\winlogbeat-7.9.2>winlogbeat.exe test config
Config OKC:\winlogbeat-7.9.1>winlogbeat test output --e
2020-09-11T18:52:42.260-0700    INFO    instance/beat.go:640    Home path: [C:\winlogbeat-7.9.2] Config path: [C:\winlogbeat-7.9.1] Data path: [C:\winlogbeat-7.9.2\data] Logs path: [C:\winlogbeat-7.9.2\logs]
2020-09-11T18:52:42.267-0700    INFO    instance/beat.go:648    Beat ID: 6d3822d8-a900-4c46-b040-ae41659b2745
2020-09-11T18:52:42.284-0700    INFO    [index-management]      idxmgmt/std.go:184      Set output.elasticsearch.index to 'winlogbeat-7.9.2' as ILM is enabled.
2020-09-11T18:52:42.317-0700    INFO    eslegclient/connection.go:99    elasticsearch url: https://10.0.0.1:9200
elasticsearch: https://10.0.0.1:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.12020-09-11T18:52:42.332-0700        INFO    [add_cloud_metadata]    add_cloud_metadata/add_cloud_metadata.go:89             add_cloud_metadata: hosting provider type not detected.
68.0.4
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
2020-09-11T18:52:42.450-0700    INFO    [esclientleg]   eslegclient/connection.go:314   Attempting to connect to Elasticsearch version 7.9.0
2020-09-11T18:52:42.489-0700    INFO    [license]       licenser/es_callback.go:51      Elasticsearch license: Basic
  talk to server... OK
  version: 7.9.2

Next up, we install the Winlogbeat service.

PS C:\winlogbeat-7.9.2> .\install-service-winlogbeat.ps1

Status   Name               DisplayName
------   ----               -----------
Stopped  winlogbeat         winlogbeat

Above we see the service was installed but its status says "Stopped". Let's start that service and verify it is running.

PS C:\winlogbeat-7.9.2> Start-Service winlogbeat
PS C:\winlogbeat-7.9.2> Get-Service winlogbeat

Status   Name               DisplayName
------   ----               -----------
Running  winlogbeat         winlogbeat

Looks good. Let's now switch back to the Kibana UI and verify that data is coming in.


Nice, we have data coming in.

Looking at the Security app.



All looks good.

Well that's it for this series. As a recap. In this eight part series, we installed, configured and provided basic security to Elasticsearch, Kibana, Metricbeat, Auditbeat, Filebeat, Packetbeat and Winlogbeat.


Posts in this series:

Security On The Cheap - Beginning Elastic Stack - Installing Elastic 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Installing Kibana 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Providing Basic Security to Elastic and Kibana 7.9 communication on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Auditbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04
Beginning Elastic - Installing, Configuring and Providing Basic Security to Packetbeat
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Winlogbeat

References:

https://www.elastic.co/downloads/beats/winlogbeat
https://www.howtogeek.com/670314/how-to-zip-and-unzip-files-using-powershell/

Beginning Elastic - Installing, Configuring and Providing Basic Security to Packetbeat - Elastic Stack 7.9 on Ubuntu 20.04

We have made significant progress so far, let's continue to building on it.

The first post, we installed Elasticsearch. In the second post we installed Kibana. This was followed by the third post where we provided basic security to Elastic and Kibana. In the fourth post, we installed, configured and secured Metricbeat and the fifth post, we installed, configured and secured Auditbeat. The sixth post, we installed, configured and provided basic security to Filebeat. 

Once again, keeping things simple and using the package manager.

root@securitynik-monitoring:~#  apt-get install packetbeat
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  packetbeat
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 26.2 MB of archives.
After this operation, 90.9 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/7.x/apt stable/main amd64 packetbeat amd64 7.9.2 [26.2 MB]
Fetched 26.2 MB in 1s (22.1 MB/s)
Selecting previously unselected package packetbeat.
(Reading database ... 176062 files and directories currently installed.)
Preparing to unpack .../packetbeat_7.9.2_amd64.deb ...
Unpacking packetbeat (7.9.2) ...
Setting up packetbeat (7.9.2) ...
Processing triggers for systemd (245.4-4ubuntu3.2) ...

Next up, we configure "packetbeat.yml". 

root@securitynik-monitoring:~# cd /etc/packetbeat/
root@securitynik-monitoring:/etc/packetbeat# cp packetbeat.yml packetbeat.yml.ORIGINAL

Here are the changes I made to the "packetbeat.yml" file.

root@securitynik-monitoring:~# cat /etc/packetbeat/packetbeat.yml | grep --perl-regexp "ssh|\[22\]|^\s+host|^\s+protocol|^\s+username|^\s+password"
- type: ssh
  ports: [22]
        host: "https://10.0.0.1:5601"
  hosts: ["10.0.0.1:9200"]
  protocol: "https"
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"

Like we did in the previous post, we take the last 8 lines from the "metricbeat.yml" and insert them into "packetbeat.yml"

root@securitynik-monitoring:~# tail --lines 8 /etc/metricbeat/metricbeat.yml >> /etc/packetbeat/packetbeat.yml
root@securitynik-monitoring:~# tail --lines 8 /etc/auditbeat/auditbeat.yml
# SSL Configuration enabled by Nik
ssl.enabled: true
output.elasticsearch.hosts: ["https://10.0.0.1:9200"]
output.elasticsearch.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]
setup.kibana.ssl.enabled: true
setup.kibana.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

Testing the Packetbeat configuration.

root@securitynik-monitoring:~# packetbeat test config

Config OK

Looking at the interfaces which I can capture on.

root@securitynik-monitoring:~#  packetbeat devices
0: enp0s25 (No description available) (10.0.0.1 fe80::224:e8ff:fef0:f679)
1: any (Pseudo-device that captures on all interfaces) (Not assigned ip address)
2: lo (No description available) (127.0.0.1 ::1)
3: nflog (Linux netfilter log (NFLOG) interface) (Not assigned ip address)
4: nfqueue (Linux netfilter queue (NFQUEUE) interface) (Not assigned ip address)

I also changed the interface for which Filebeat was listening on to represent what we are doing in this series.This more relates to the previous post on Filebeat.

root@securitynik-monitoring:~# cat /etc/filebeat/modules.d/netflow.yml
# Module: netflow
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.8/filebeat-module-netflow.html

- module: netflow
  log:
    enabled: true
    var:
      netflow_host: 10.0.0.1
      netflow_port: 2055

Load the Packetbeat dashboards in Kibana and the appropriate indexes.

root@securitynik-monitoring:~#  packetbeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

Enabling and starting the Packetbeat service.

root@securitynik-monitoring:~# systemctl enable --now packetbeat.service
Synchronizing state of packetbeat.service with SysV service script with /lib/systemd/systemd-sysv-install.

Executing: /lib/systemd/systemd-sysv-install enable packetbeat
Created symlink /etc/systemd/system/multi-user.target.wants/packetbeat.service → /lib/systemd/system/packetbeat.service.

root@securitynik-monitoring:~# systemctl status packetbeat.service
● packetbeat.service - Packetbeat analyzes network traffic and sends the data to Elasticsearch.
     Loaded: loaded (/lib/systemd/system/packetbeat.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2020-08-14 12:56:30 EDT; 9s ago
       Docs: https://www.elastic.co/products/beats/packetbeat
   Main PID: 34364 (packetbeat)
      Tasks: 10 (limit: 4563)
     Memory: 45.4M
     CGroup: /system.slice/packetbeat.service
             └─34364 /usr/share/packetbeat/bin/packetbeat -environment systemd -c /etc/packetbeat/packetbeat.yml -path.home /usr/share/packetbeat -path.config /etc/packet>

Looking at the Packetbeat  data.

Packetbeat
Looks like we have some data from Packetbeat. 


Well that is it for this post. See you in the next where we look at Winlogbeat, where we also wrap-up this series..

Posts in this series:

Security On The Cheap - Beginning Elastic Stack - Installing Elastic 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Installing Kibana 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Providing Basic Security to Elastic and Kibana 7.9 communication on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Auditbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04
Beginning Elastic - Installing, Configuring and Providing Basic Security to Packetbeat
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Winlogbeat

References:
https://www.elastic.co/beats/
https://www.elastic.co/downloads/beats/packetbeat

Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04

In this the sixth post in this series, we are looking at installing and securing Filebeat. The first post, we installed Elasticsearc. In the second post we installed Kibana while in the third post we provided basic security to Elastic and Kibana. In the fourth post, we installed, configured and secured Metricbeat. In the fifth post, we installed, configured and secured Auditbeat

In this post, we use the apt package manager to install Filebeat similarly to how we installed Auditbeat. Note you could have also download the .deb file similarly to what was done with Metricbeat install

root@securitynik-monitoring:~# apt-get install filebeat

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  filebeat
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 31.4 MB of archives.
After this operation, 112 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/7.x/apt stable/main amd64 filebeat amd64 7.9.2 [31.4 MB]
Fetched 31.4 MB in 2s (18.6 MB/s)
Selecting previously unselected package filebeat.
(Reading database ... 176095 files and directories currently installed.)
Preparing to unpack .../filebeat_7.9.2_amd64.deb ...
Unpacking filebeat (7.9.2) ...
Setting up filebeat (7.9.2) ...
Processing triggers for systemd (245.4-4ubuntu3.2) ...

With Filebeat installed, let's configure it. As always, I make a copy of the original file before editing it.

root@securitynik-monitoring:~# cd /etc/filebeat/
root@securitynik-monitoring:/etc/filebeat# cp filebeat.yml filebeat.yml.ORIGINAL

I then modified the entries to reflect below:

root@securitynik-monitoring:~# cat /etc/filebeat/filebeat.yml | grep --perl-regexp "^\s+host|^\s+username|^\s+password|^\s+protocol"
  host: "https://10.0.0.1:5601"
  hosts: ["10.0.0.1:9200"]
  protocol: "https"
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"

Finally, I copied the last eight lines from the Metricbeat configuration file into the file "filebeat.yml" file.

root@securitynik-monitoring:~# tail --lines 8 /etc/metricbeat/metricbeat.yml >> /etc/filebeat/filebeat.yml
root@securitynik-monitoring:~# tail --lines 8 /etc/filebeat/filebeat.yml
# SSL Configuration enabled by Nik
ssl.enabled: true
output.elasticsearch.hosts: ["https://10.0.0.1:9200"]
output.elasticsearch.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

setup.kibana.ssl.enabled: true
setup.kibana.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

Enable the "system" module

root@securitynik-monitoring:~# filebeat modules enable system
Enabled system

Load the Kibana dashboards

root@securitynik-monitoring:~# filebeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
Setting up ML using setup --machine-learning is going to be removed in 8.0.0. Please use the ML app instead.

See more: https://www.elastic.co/guide/en/elastic-stack-overview/current/xpack-ml.html
Loaded machine learning job configurations
Loaded Ingest pipelines

Cross our fingers and start Filebeat.

root@securitynik-monitoring:~# systemctl enable --now filebeat.service
Synchronizing state of filebeat.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable filebeat
Created symlink /etc/systemd/system/multi-user.target.wants/filebeat.service → /lib/systemd/system/filebeat.service.


root@securitynik-monitoring:~# systemctl status filebeat.service
● filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch.
     Loaded: loaded (/lib/systemd/system/filebeat.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2020-08-14 12:23:42 EDT; 23s ago
       Docs: https://www.elastic.co/products/beats/filebeat
   Main PID: 33756 (filebeat)
      Tasks: 14 (limit: 4563)
     Memory: 64.6M
     CGroup: /system.slice/filebeat.service
             └─33756 /usr/share/filebeat/bin/filebeat -environment systemd -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.d>

Looking to see if data is being received by Filebeat

With data now being received by Filebeat, let's now enable some modules we may need. Looking to see what modules exist.

root@securitynik-monitoring:~# filebeat modules list
Enabled:
system


Disabled:
activemq
apache
auditd
............

Enabling the ones I believe I will need later. Additionally, in this post, we installed Zeek on Ubuntu. By enabling Zeek below, we can now ingest these logs into Elastic via Filebeat.

root@securitynik-monitoring:~# filebeat modules enable  auditd elasticsearch iptables kibana netflow zeek
Enabled auditd
Enabled elasticsearch
Enabled iptables
Enabled kibana
Enabled netflow
Enabled zeek

From the blog on Zeek, the Zeek logs are stored in "/usr/local/zeek/etc/zeekctl.cfg".

root@securitynik-monitoring:~# cat /usr/local/zeek/etc/zeekctl.cfg  | grep LogDir
# Expiration interval for archived log files in LogDir.  Files older than this
LogDir = /usr/local/zeek/logs

With the above in mind, I then created a "bro" directory under "/var/logs/" and made a symbolic link to the currently configured Zeek log directory.

root@securitynik-monitoring:~# sudo mkdir /var/log/bro
root@securitynik-monitoring:~# sudo ln --symbolic /usr/local/zeek/logs/current /var/log/bro/

Next Zeek is configured to store its logs in JSON format

root@securitynik-monitoring:~# echo "@load policy/tuning/json-logs.zeek" >> /usr/local/zeek/share/zeek/site/local.zeek

Note at this point you may have to restart Zeek and or Filebeat. 

Below we see that Zeek's data is being received successfully.


Let's now look at other data received by Filebeat.


With that out of the way, let's move on to installing, configuring and securing Packetbeat.

Posts in this series:

Security On The Cheap - Beginning Elastic Stack - Installing Elastic 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Installing Kibana 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Providing Basic Security to Elastic and Kibana 7.9 communication on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Auditbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04
Beginning Elastic - Installing, Configuring and Providing Basic Security to Packetbeat
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Winlogbeat

References:

https://www.elastic.co/beats/filebeat
https://www.elastic.co/guide/en/beats/filebeat/7.8/setup-repositories.html#_apt
https://www.ericooi.com/zeekurity-zen-part-iii-how-to-send-zeek-logs-to-splunk/
https://www.elastic.co/blog/collecting-and-analyzing-zeek-data-with-elastic-security

Beginning Elastic - Installing and Providing Basic Security to Auditbeat - Elastic Stack 7.9 on Ubuntu 20.04

Continuing this journey where we install and configure the different components of Elastic Stack. In the first post, we installed Elasticsearc. The second post we installed Kibana. In the third post we provided basic security to Elastic and Kibana. In the fourth post, we installed, configured and secured Metricbeat. In this the fifth post, we are installing, configuring and securing Auditbeat.

Selecting Auditbeat from the list of logs sources and follow the directions.


Below I choose to Install Auditbeat from the repository.

root@securitynik-monitoring:~# apt-get install auditbeat
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  auditbeat
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 26.7 MB of archives.
After this operation, 92.4 MB of additional disk space will be used.
Get:1 https://artifacts.elastic.co/packages/7.x/apt stable/main amd64 auditbeat amd64 7.9.2 [26.7 MB]
Fetched 26.7 MB in 1s (19.8 MB/s)
Selecting previously unselected package auditbeat.
(Reading database ... 176029 files and directories currently installed.)
Preparing to unpack .../auditbeat_7.9.2_amd64.deb ...
Unpacking auditbeat (7.9.2) ...
Setting up auditbeat (7.9.2) ...
Processing triggers for systemd (245.4-4ubuntu3.2) ...

Make a copy of the original "auditbeat.yml" file.

root@securitynik-monitoring:~# cd /etc/auditbeat/
root@securitynik-monitoring:/etc/auditbeat# cp auditbeat.yml auditbeat.yml.ORIGINAL

Make the necessary configuration changes. 

root@securitynik-monitoring:~# cat /etc/auditbeat/auditbeat.yml | grep --perl-regexp "^\s+host|^\s+username|^\s+password|^\s+protocol"
  host: "https://10.0.0.1:5601"
  hosts: ["https://10.0.0.1:9200"]
  protocol: "https"
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"

Using a shortcut, let's ake the last 8 lines of the "metricbeat.yml" file and insert them into the "auditbeat.yml" file

root@securitynik-monitoring:~# tail --lines 8 /etc/metricbeat/metricbeat.yml >> /etc/auditbeat/auditbeat.yml

root@securitynik-monitoring:~#  tail --lines 8 /etc/auditbeat/auditbeat.yml
# SSL Configuration enabled by Nik
ssl.enabled: true
output.elasticsearch.hosts: ["https://10.0.0.1:9200"]
output.elasticsearch.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

setup.kibana.ssl.enabled: true
setup.kibana.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

Before loading up, we test that all is well with the configuration.

root@securitynik-monitoring:~# auditbeat test config
Config OK

Looking at "auditd-status

securitynik@securitynik-monitoring:~$ sudo auditbeat show auditd-status
[sudo] password for securitynik:

enabled 0
failure 1
pid 0
rate_limit 0
backlog_limit 64
lost 0
backlog 0
backlog_wait_time 15000
features 0x7f

Looking at "auditd-rules", we see no rules defined.

securitynik@securitynik-monitoring:~$ sudo auditbeat show auditd-rules
No rules

Guess we should ensure a few rules are defined, so that we have something to monitor. For simplicity sake, let's use the rules in the "auditbeat.yml" file. To do this, I uncommented the rules as they were previously commented. Below represents those changes.

securitynik@securitynik-monitoring:~$ sudo cat /etc/auditbeat/auditbeat.yml | grep "audit_rules:" --after-context=20
  audit_rules: |
    ## Define audit rules here.
    ## Create file watches (-w) or syscall audits (-a or -A). Uncomment these
    ## examples or add your own rules.

    ## If you are on a 64 bit platform, everything should be running
    ## in 64 bit mode. This rule will detect any use of the 32 bit syscalls
    ## because this might be a sign of someone exploiting a hole in the 32

    ## bit API.
    -a always,exit -F arch=b32 -S all -F key=32bit-abi

    ## Executions.
    -a always,exit -F arch=b64 -S execve,execveat -k exec

    ## External access (warning: these can be expensive to audit).
    -a always,exit -F arch=b64 -S accept,bind,connect -F key=external-access

    ## Identity changes.
    -w /etc/group -p wa -k identity
    -w /etc/passwd -p wa -k identity
    -w /etc/gshadow -p wa -k identity

With the above now configured, let's execute 

Verify we can connect to Elastic

root@securitynik-monitoring:~# auditbeat test output
elasticsearch: https://10.0.0.1:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 10.0.0.1
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.9.2

Next up, load the index template and appropriate dashboards, etc.

root@securitynik-monitoring:~# auditbeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

Enable and startup Auditbeat.

root@securitynik-monitoring:~# systemctl enable --now auditbeat.service
Synchronizing state of auditbeat.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable auditbeat
Created symlink /etc/systemd/system/multi-user.target.wants/auditbeat.service → /lib/systemd/system/auditbeat.service.

root@securitynik-monitoring:~# systemctl status auditbeat.service
● auditbeat.service - Audit the activities of users and processes on your system.
     Loaded: loaded (/lib/systemd/system/auditbeat.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2020-08-14 13:48:43 EDT; 2s ago
       Docs: https://www.elastic.co/products/beats/auditbeat
   Main PID: 35398 (auditbeat)
      Tasks: 11 (limit: 4563)
     Memory: 22.6M
     CGroup: /system.slice/auditbeat.service
             └─35398 /usr/share/auditbeat/bin/auditbeat -environment systemd -c /etc/auditbeat/auditbeat.yml -path.home /usr/share/auditbeat -path.config /etc/auditbeat ->

Verifying that data is coming into to Elastic.

   

We also verify now that we have some working rules.

root@securitynik-monitoring:~# vi /etc/metricbeat/metricbeat.yml^C
root@securitynik-monitoring:~# auditbeat show auditd-rules
-a never,exit -S all -F pid=707
-a always,exit -F arch=b32 -S all -F key=32bit-abi
-a always,exit -F arch=b64 -S execve,execveat -F key=exec
-a always,exit -F arch=b64 -S connect,accept,bind -F key=external-access
-w /etc/group -p wa -k identity
-w /etc/passwd -p wa -k identity
-w /etc/gshadow -p wa -k identity
-a always,exit -F arch=b64 -S open,truncate,ftruncate,creat,openat,open_by_handle_at -F exit=-EACCES -F key=access
-a always,exit -F arch=b64 -S open,truncate,ftruncate,creat,openat,open_by_handle_at -F exit=-EPERM -F key=access

Looking at some actual data now in Elastic


Ok. Let's move on to install, configure and secure Filebeat.

Posts in this series:

Security On The Cheap - Beginning Elastic Stack - Installing Elastic 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Installing Kibana 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Providing Basic Security to Elastic and Kibana 7.9 communication on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Auditbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04
Beginning Elastic - Installing, Configuring and Providing Basic Security to Packetbeat
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Winlogbeat

References:

https://www.elastic.co/downloads/beats/auditbeat
https://www.elastic.co/guide/en/beats/auditbeat/7.9/setup-repositories.html#_apt
https://www.elastic.co/guide/en/beats/auditbeat/current/auditbeat-overview.html
https://www.elastic.co/guide/en/beats/auditbeat/current/auditbeat-module-auditd.html
https://logz.io/blog/linux-auditbeat-elk/

Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04

Now that we have provided some basic security to Elasticsearch and Kibana communications, time to get some logs into the system.

Let's start this fourth post off with installing Metricbeat. Metricbeat can be downloaded directly from Elastic web site site. However, I prefer to use the instructions which can be found within my Kibana install.

Metricbeat

From the "Add Data" page, I then selected "Elasticsearch metrics". As we are running on Ubuntu, the instruction below is from the "Deb" tab. 

First download Metricbeat.

root@securitynik-monitoring:~$ cd /tmp/
root@securitynik-monitoring:~# curl -L -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-7.9.2-amd64.deb
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 37.2M  100 37.2M    0     0  23.4M      0  0:00:01  0:00:01 --:--:-- 23.4M

root@securitynik-monitoring:/tmp$ ls metricbeat-7.9.2-amd64.deb
metricbeat-7.9.2-amd64.deb

Next we install Metricbeat.

securitynik@securitynik-monitoring:/tmp$ sudo dpkg --install metricbeat-7.9.2-amd64.deb
Selecting previously unselected package metricbeat.
(Reading database ... 175727 files and directories currently installed.)
Preparing to unpack metricbeat-7.9.2-amd64.deb ...
Unpacking metricbeat (7.9.2) ...
Setting up metricbeat (7.9.2) ...
Processing triggers for systemd (245.4-4ubuntu3.2) ...

With Metricbeat installed, time to configure it.

securitynik@securitynik-monitoring:/tmp$ cd /etc/metricbeat/
securitynik@securitynik-monitoring:/etc/metricbeat$ sudo cp metricbeat.yml metricbeat.yml.ORIGINAL

Here are the changes I made

root@securitynik-monitoring:~# cat /etc/metricbeat/metricbeat.yml | grep --perl-regexp "^\s+host|^\s+protocol|^\s+username|^\s+password|^\s+protocol"
        host: "https://10.0.0.1:5601"
  hosts: ["https://10.0.0.1:9200"]
  protocol: "https"
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"

With that out of the way, we need to ensure Metricbeat is configured for SSL. As a result, the following lines were also added to the "metricbeat.yml" file.

root@securitynik-monitoring:~# tail --lines 8 /etc/metricbeat/metricbeat.yml
# SSL Configuration enabled by Nik
ssl.enabled: true
output.elasticsearch.hosts: ["https://10.0.0.1:9200"]
output.elasticsearch.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

setup.kibana.ssl.enabled: true
setup.kibana.ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

Once those were completed, I then enabled the Elasticsearch module

root@securitynik-monitoring:~# metricbeat modules enable elasticsearch
Enabled elasticsearch

Configuring the metricbeat Kibana module, I modified the "/etc/metricbeat/modules.d/elasticsearch.yml" to look as follow:


root@securitynik-monitoring:~# vi /etc/metricbeat/modules.d/elasticsearch.yml
root@securitynik-monitoring:~# cat /etc/metricbeat/modules.d/elasticsearch.yml
# Module: elasticsearch
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.9/metricbeat-module-elasticsearch.html

- module: elasticsearch
  #metricsets:
  #  - node
  #  - node_stats
  period: 10s
  hosts: ["https://10.0.0.1:9200"]
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"
  ssl.certificate_authorities: ["/etc/kibana/SecurityNik-CA.pem"]

  Then the Kibana configuration.

  root@securitynik-monitoring:~# cat /etc/metricbeat/modules.d/kibana.yml
# Module: kibana
# Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.8/metricbeat-module-kibana.html

- module: kibana
  #metricsets:
  #  - status
  period: 10s
  hosts: ["https://10.0.0.1:5601"]
  #basepath: ""
  username: "elastic"
  password: "WelcomeToSecurityNikElastic"

Run the Metricbeat setup command to load the Kibana dashboards.

root@securitynik-monitoring:/etc/metricbeat# metricbeat setup
Overwriting ILM policy is disabled. Set `setup.ilm.overwrite:true` for enabling.

Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

Now let's pray, cross our hands, legs, hair, etc. and start Metricbeat.

root@securitynik-monitoring:/etc/metricbeat# systemctl enable --now metricbeat.service
Synchronizing state of metricbeat.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable metricbeat
Created symlink /etc/systemd/system/multi-user.target.wants/metricbeat.service → /lib/systemd/system/metricbeat.service.

root@securitynik-monitoring:/etc/metricbeat# systemctl status metricbeat.service
● metricbeat.service - Metricbeat is a lightweight shipper for metrics.
     Loaded: loaded (/lib/systemd/system/metricbeat.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2020-08-14 11:22:33 EDT; 8s ago
       Docs: https://www.elastic.co/products/beats/metricbeat
   Main PID: 33050 (metricbeat)
      Tasks: 11 (limit: 4563)
     Memory: 19.9M

     CGroup: /system.slice/metricbeat.service
             └─33050 /usr/share/metricbeat/bin/metricbeat -environment systemd -c /etc/metricbeat/metricbeat.yml -path.home /usr/share/metricbeat -path.config /etc/metric>

Aug 14 11:22:38 securitynik-monitoring metricbeat[33050]: 2020-08-14T11:22:38.169-0400        INFO        [index-management.ilm]        ilm/std.go:139        do not generate ilm p>
Aug 14 11:22:38 securitynik-monitoring metricbeat[33050]: 2020-08-14T11:22:38.169-0400 INFO [index-management] idxmgmt/std.go:274 ILM policy success

Above suggests we are good to go. Let's confirm. 

Metricbeat receiving data

With Kibana letting us know that it is successfully retrieving data, let's enable some modules we know we will need.

To see a list of modules available, we execute.

root@securitynik-monitoring:/etc/metricbeat# metricbeat modules list

Enabled:
elasticsearch
system


Disabled:
activemq
aerospike
apache
....

We can see we have two modules enabled and a number of others disabled. If we go through the returned list, we may find one or more that we are interested in. As I am here, I will enable. 

root@securitynik-monitoring:/etc/metricbeat# metricbeat modules enable beat kibana linux 

Enabled beat
Enabled kibana
Enabled linux

At this point, you now have to configure the various modules as I did above for Kibana.

Next we setup up the Kibana dashboards.

root@securitynik-monitoring:/etc/metricbeat# metricbeat setup --dashboards
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards

With all of that out of the way, here is what the data looks like :

Metricbeat


Ok then. With this out of the way, now it is time to move on to Filebeat.

Posts in this series:

Security On The Cheap - Beginning Elastic Stack - Installing Elastic 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Installing Kibana 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Providing Basic Security to Elastic and Kibana 7.9 communication on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Auditbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04
Beginning Elastic - Installing, Configuring and Providing Basic Security to Packetbeat
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Winlogbeat

See you in the next post.

References:
https://www.elastic.co/beats/
https://www.elastic.co/pdf/introduction-to-logging-with-the-elk-stack.pdf
https://github.com/elastic/examples/tree/master/Reference/Beats
https://www.elastic.co/guide/en/beats/filebeat/current/configuration-ssl.html#key


Security On The Cheap - Beginning Elastic Stack - Providing Basic Security to Elastic and Kibana 7.9 communication on Ubuntu 20.04

In the first post, we installed Elasticsearch. In the second post, we installed Kibana. In this post, we now provide some basic security to the communication between the Elasticsearch and Kibana. Note, there is a lot more you can do to secure this environment, taking advantage of keystores, etc. 

In generating the Certification Authority (CA) certificate, I choose to use PKCS#12 format. In this format, this file contains both the CA certificate and its private key. This may be a cause for concern in some environments.

Additionally, I created one certificate file to be used by each component. This means the various components need to be able to read the file. This obviously makes the data in the file accessible by those users.  In this post, I am keeping it simple. If you are looking at thoroughly securing your environment, this post is something you can essentially build on if you wish.

If you are wondering why we need to secure the communication between Elastic and Kibana, here is a simple reason why.

root@securitynik-monitoring:~# tcpdump -nnti any host 10.0.0.1 and port 9200 -A
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
IP 10.0.0.1.60100 > 10.0.0.1.9200: Flags [P.], seq 3727928959:3727929473, ack 22660352, win 9203, options [nop,nop,TS val 4105791837 ecr 4105786838], length 514

E..6.G@.@.!"..........#..3...Y....#........
..i]..U.POST /.reporting-*/_search HTTP/1.1
content-type: application/json
Host: 10.0.0.1:9200
Content-Length: 374
Connection: keep-alive

{"seq_no_primary_term":true,"_source":{"excludes":["output.content"]},"query":{"bool":{"filter":{"bool":{"minimum_should_match":1,"should":[{"term":{"status":"pending"}},{"bool":{"must":[{"term":{"status":"processing"}},{"range":{"process_expiration":{"lte":"2020-08-14T01:02:35.764Z"}}}]}}]}}}},"sort":[{"priority":{"order":"asc"}},{"created_at":{"order":"asc"}}],"size":1}

IP 10.0.0.1.9200 > 10.0.0.1.60100: Flags [P.], seq 1:247, ack 514, win 512, options [nop,nop,TS val 4105791839 ecr 4105791837], length 246
E..*..@.@...........#....Y...3.......u.....
..i_..i]HTTP/1.1 200 OK
content-type: application/json; charset=UTF-8
content-length: 159
....

As shown above, using tcpdump, we were able to sniff the traffic on the wire, thus gaining visibility into actual communication occurring on the network. At this point, if we can see the data, anyone else can.

With that out of the way, let's get to providing some basic security to this environment via Transport Layer Security (TLS).

First up, we modify our "elasticsearch.yml" file to include "xpack.security.enabled: true". Here is what my configuration looks like.

root@securitynik-monitoring:~# cat /etc/elasticsearch/elasticsearch.yml | grep "xpack.security.enabled: true"
xpack.security.enabled: true

For TLS to work properly, we need a certificate. There are many ways to get certificates. However, for us, we will use the built in "elasticsearch-certutil" utility, to generate our own certification authority.

root@securitynik-monitoring:~# /usr/share/elasticsearch/bin/elasticsearch-certutil ca ca-dn securitynik.local
This tool assists you in the generation of X.509 certificates and certificate signing requests for use with SSL/TLS in the Elastic stack.

The 'ca' mode generates a new 'certificate authority'. This will create a new X.509 certificate and private key that can be used to sign certificate when running in 'cert' mode.


Use the 'ca-dn' option if you wish to configure the 'distinguished name' of the certificate authority

By default the 'ca' mode produces a single PKCS#12 output file which holds:

    * The CA certificate
    * The CA's private key


If you elect to generate PEM format certificates (the -pem option), then the output will be a zip file containing individual files for the CA certificate and private key

Please enter the desired output file [elastic-stack-ca.p12]: SecurityNik-CA.p12

Enter password for SecurityNik-CA.p12 :

Above, my CA cert is named "SecurityNik-CA.p12". Also you have the option to specify a password for the CA cert. I did not specify a password. In my example, the file was stored in:

root@securitynik-monitoring:~# ls /usr/share/elasticsearch/SecurityNik-CA.p12 -al
-rw------- 1 root root 2527 Aug 13 21:49 /usr/share/elasticsearch/SecurityNik-CA.p12

With the CA certificate generated, let's now generate a certificate for our node at "10.0.0.1".

root@securitynik-monitoring:~# /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca /usr/share/elasticsearch/SecurityNik-CA.p12  --days 1825 --dns monitoring,monitoring.securitynik.local --ip 10.0.0.1 --keysize 2048 --name 10.0.0.1 --out /usr/share/elasticsearch/10.0.0.1.p12 --pass "" --silent

Enter password for CA (/usr/share/elasticsearch/SecurityNik-CA.p12) :

Below we see the file which was created.

root@securitynik-monitoring:~# ls -al /usr/share/elasticsearch/10.0.0.1.p12 -al
-rw------- 1 root root 3529 Aug 13 22:23 /usr/share/elasticsearch/10.0.0.1.p12

Copy this certificate to the "/etc/elasticsearch/" folder.

root@securitynik-monitoring:~# cp /usr/share/elasticsearch/10.0.0.1.p12 /etc/elasticsearch/ -v 
'/usr/share/elasticsearch/10.0.0.1.p12' -> '/etc/elasticsearch/10.0.0.1.p12'

Next I changed the permission of the certificate so it is world readable. Not necessarily the best thing to do but this is for simplicity.

root@securitynik-monitoring:~# chmod 644 /etc/elasticsearch/10.0.0.1.p12
root@securitynik-monitoring:~# ls -al /etc/elasticsearch/10.0.0.1.p12
-rw-r--r-- 1 root elasticsearch 3529 Aug 13 22:32 /etc/elasticsearch/10.0.0.1.p12

As our certificate is in PKCS#12 format, we will add the following lines to our "elasticsearch.yml".

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: 10.0.0.1.p12
xpack.security.transport.ssl.truststore.path: 10.0.0.1.p12

Here is what my Elasticsearch configuration now looks like after those changes.

root@securitynik-monitoring:~# tail --lines 6 /etc/elasticsearch/elasticsearch.yml
# Configuration added by Nik for security
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: "10.0.0.1.p12"
xpack.security.transport.ssl.truststore.path: "10.0.0.1.p12"

Time to restart Elasticsearch and pray that everything works as expected.

root@securitynik-monitoring:~# systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2020-08-13 22:40:44 EDT; 3min 50s ago
       Docs: https://www.elastic.co
   Main PID: 24533 (java)
      Tasks: 61 (limit: 4563)
     Memory: 1.2G
     CGroup: /system.slice/elasticsearch.service
             ├─24533 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddres>
             └─24725 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Aug 13 22:40:17 securitynik-monitoring systemd[1]: Starting Elasticsearch...
Aug 13 22:40:44 securitynik-monitoring systemd[1]: Started Elasticsearch.

Above, all looks well. Looking at the network ports via "ss". Note you can also use "netstat". 

root@securitynik-monitoring:~# ss --numeric --listen --tcp | grep 9200
LISTEN  0       4096     [::ffff:10.0.0.1]:9200              *:*

Awesome! 

Next step, let's now generate a certificate for HTTP communication. For example, between Kibana and Elastic and ultimately between our browser and Kibana. Once again, we use "elasticsearch-certutil", this time with the "http" argument.

root@securitynik-monitoring:~# /usr/share/elasticsearch/bin/elasticsearch-certutil http --silent
## Elasticsearch HTTP Certificate Utility
## Do you wish to generate a Certificate Signing Request (CSR)?
Generate a CSR? [y/N]N
## Do you have an existing Certificate Authority (CA) key-pair that you wish to use to sign your certificate?
Use an existing CA? [y/N]Y
## What is the path to your CA?
CA Path: /usr/share/elasticsearch/SecurityNik-CA.p12
Password for SecurityNik-CA.p12:
## How long should your certificates be valid?
For how long should your certificate be valid? [5y]
## Do you wish to generate one certificate per node?
Generate a certificate per node? [y/N]N
## Which hostnames will be used to connect to your nodes?

securitynik-monitoring

securitynik-monitoring.securitynik.local

You entered the following hostnames.

 - securitynik-monitoring
 - securitynik-monitoring.securitynik.local

Is this correct [Y/n]y

## Which IP addresses will be used to connect to your nodes?
10.0.0.1

You entered the following IP addresses.

 - 10.0.0.1
Is this correct [Y/n]y
## Other certificate options
Key Name: securitynik-monitoring
Subject DN: CN=securitynik-monitoring

Key Size: 2048

Do you wish to change any of these options? [y/N]n
## What password do you want for your private key(s)?
Provide a password for the "http.p12" file:  [<ENTER> for none]
## Where should we save the generated files?

What filename should be used for the output zip file? [/usr/share/elasticsearch/elasticsearch-ssl-http.zip]

Confirming the file was successfully created.

root@securitynik-monitoring:~# ls /usr/share/elasticsearch/elasticsearch-ssl-http.zip -l
-rw------- 1 root root 7334 Aug 13 23:22 /usr/share/elasticsearch/elasticsearch-ssl-http.zip

First we install "unzip". With unzip installed, we are now able to look into the zip file

root@securitynik-monitoring:/usr/share/elasticsearch# apt install unzip
root@securitynik-monitoring:/usr/share/elasticsearch# unzip -l elasticsearch-ssl-http.zip
Archive:  elasticsearch-ssl-http.zip
  Length      Date    Time    Name
---------  ---------- -----   ----

        0  2020-08-13 23:22   elasticsearch/
     1091  2020-08-13 23:22   elasticsearch/README.txt
     3499  2020-08-13 23:22   elasticsearch/http.p12
      657  2020-08-13 23:22   elasticsearch/sample-elasticsearch.yml
        0  2020-08-13 23:22   kibana/
     1306  2020-08-13 23:22   kibana/README.txt
     1200  2020-08-13 23:22   kibana/elasticsearch-ca.pem
     1056  2020-08-13 23:22   kibana/sample-kibana.yml
---------                     -------
     8809                     8 files

Extracting the contents from the "elasticsearch-ssl-http.zip" into a folder named "certs" and verifying the extraction.

root@securitynik-monitoring:/usr/share/elasticsearch# unzip -d certs/ elasticsearch-ssl-http.zip
Archive:  elasticsearch-ssl-http.zip
   creating: certs/elasticsearch/
  inflating: certs/elasticsearch/README.txt
  inflating: certs/elasticsearch/http.p12
  inflating: certs/elasticsearch/sample-elasticsearch.yml
   creating: certs/kibana/
  inflating: certs/kibana/README.txt
  inflating: certs/kibana/elasticsearch-ca.pem
  inflating: certs/kibana/sample-kibana.yml
root@securitynik-monitoring:/usr/share/elasticsearch# ls certs/
elasticsearch  kibana

Time to copy "certs/elasticsearch/http.p12" file to "/etc/elasticsearch/" folder. 

root@securitynik-monitoring:~# cp /usr/share/elasticsearch/certs/elasticsearch/http.p12 /etc/elasticsearch/ -v 
'/usr/share/elasticsearch/certs/elasticsearch/http.p12' -> '/etc/elasticsearch/http.p12'

I then added the following lines to the bottom of my "elasticsearch.yml".

# This turns on SSL for the HTTP (Rest) interface
xpack.security.http.ssl.enabled: true
# This configures the keystore to use for SSL on HTTP
xpack.security.http.ssl.keystore.path: "http.p12"

For consistency I also rename the "elasticsearch-ca.pem" file to "SecurityNik-CA.pem"

root@securitynik-monitoring:/usr/share/elasticsearch/certs# cd /etc/kibana/
root@securitynik-monitoring:/etc/kibana# mv elasticsearch-ca.pem SecurityNik-CA.pem
root@securitynik-monitoring:/etc/kibana#

Next the file "kibana/SecurityNik-ca.pem" was copied to the "/etc/kibana" folder

root@securitynik-monitoring:/usr/share/elasticsearch/certs# cp /usr/share/elasticsearch/certs/kibana/SecurityNik-ca.pem /etc/kibana/ -v
'/usr/share/elasticsearch/certs/kibana/SecurityNik-ca.pem' -> '/etc/kibana/SecurityNik-ca.pem'

Next I modified my "kibana.yml" file changing "elasticsearch.hosts: ["http://10.0.0.1:9200"]" to "elasticsearch.hosts: ["https://10.0.0.1:9200"]". Note the https. I also added "elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/SecurityNik-CA.pem" ]"

With these configurations now out of the way, let's restart both Elasticsearch and Kibana.

root@securitynik-monitoring:/etc/kibana# systemctl stop elasticsearch.service
root@securitynik-monitoring:/etc/kibana# systemctl start elasticsearch.service
root@securitynik-monitoring:/etc/kibana# systemctl stop kibana.service
root@securitynik-monitoring:/etc/kibana# systemctl start kibana.service

Let's now configure Kibana so that we connect to it via HTTPS. 

First I copy the "10.0.0.1.p12" node certificate which was previously created to the Kibana folder. 

root@securitynik-monitoring:~# cp /etc/elasticsearch/10.0.0.1.p12 /etc/kibana/ -v 
'/etc/elasticsearch/10.0.0.1.p12' -> '/etc/kibana/10.0.0.1.p12'

Once again, modifying "kibana.yml", we add the following lines.

server.ssl.keystore.path: "/etc/kibana/10.0.0.1.p12"
server.ssl.enabled: true
server.ssl.keystore.password: ""

Once completed, restarted Kibana 

root@securitynik-monitoring:/etc/kibana# systemctl stop kibana.service
root@securitynik-monitoring:/etc/kibana# systemctl start kibana.service

Before we login, let's setup the default users using "elasticsearch-setup-passwords".

root@securitynik-monitoring:~# /usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive --silent

Please confirm that you would like to continue [y/N]y
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana_system]:
Reenter password for [kibana_system]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:

Now that the users are setup, the Kibana configuration file must now reflect the values for username and password for "kibana_system" account.

root@securitynik-monitoring:~# cat /etc/kibana/kibana.yml | grep "kibana_system" --after-context 1
elasticsearch.username: "kibana_system"
elasticsearch.password: "WelcomeToSecurityNikElastic"

Below represents some of those changes I've made to the "kibana.yml"

root@securitynik-monitoring:~# tail --lines 8 /etc/kibana/kibana.yml

# Below added by Nik
server.ssl.enabled: true
elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/SecurityNik-CA.pem"]
server.ssl.keystore.path: "/etc/kibana/10.0.0.1.p12"
server.ssl.keystore.password: ""
xpack.encryptedSavedObjects.encryptionKey: 'fhjskloppd678ehkdfdlliverpoolfcr'
xpack.security.session.idleTimeout: "30m"
xpack.security.session.lifespan: "8h"

With those changes in place, we should now be good to go.

Let's authenticate to Kibana using the username and password we setup for the "elastic" user. Once authenticated, you should create a few additional users based on their roles.

HTTPS Login Page


Once you authenticate successfully, you should now see the following.

Before we go, if you remember, when we started, we were able to see the clear text data crossing the wire. If we run tcpdump again, we don't see any data in the clear.

root@n3-monitoring:/etc/elasticsearch# tcpdump -nnti any host 10.0.0.1 and port 9200 -A -c 5
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
IP 10.0.0.1.36344 > 10.0.0.1.9200: Flags [P.], seq 2879060748:2879061003, ack 1452891267, win 512, options [nop,nop,TS val 2724200057 ecr 2724168569], length 255
E..3y.@.@.>...........#.....V.\......~.....
._.y._.y.........@.../....Ls}..E...
.l.;........x.^..u..3S...A.,.O...2e.h...../.).*@.a15ou.u.^D...~.3..Hy\W'.../....D.ahL..H...q;...j....$..L.0<%J..._..k]..TMQj.B
m....n.id.O....5.....S...=1.Iq.|Ox...}.mOP......z@..L.&...&X.>....R.u'I.-U*!|.Z..._.Z.[%7=...K.6m...
IP 10.0.0.1.9200 > 10.0.0.1.36344: Flags [P.], seq 1:238, ack 255, win 512, options [nop,nop,TS val 2724200061 ecr 2724200057], length 237
E..!..@.@...........#...V.\..........l.....
._.}._.y..............{.m6!....ZK^...
.j..$-.......4(..\Y6...y=j.....T0v..J.yG..8.I.....'.+%w.6...y..z...r...*K..L......t..cc.Pk.\..RZ.%.N.....l...Zq......qw.W..!05."...{...I..g..)\(.H..........Q...H..<X...1...X4.W..).
.eoB...w..i.....[......+.W
IP 10.0.0.1.36344 > 10.0.0.1.9200: Flags [.], ack 238, win 511, options [nop,nop,TS val 2724200061 ecr 2724200061], length 0
E..4y.@.@.?...........#.....V.]p...........
._.}._.}
IP 10.0.0.1.36294 > 10.0.0.1.9200: Flags [.], ack 2500636561, win 512, options [nop,nop,TS val 2724200281 ecr 2724199268], length 0
E..43.@.@.............#.3.1................
._.Y._.d
IP 10.0.0.1.9200 > 10.0.0.1.36294: Flags [.], ack 1, win 512, options [nop,nop,TS val 2724200281 ecr 2724199268], length 0
E..4..@.@...........#.......3.1............

At this point, it is time to move on, as we were able to provide some basic security to Kibana and Elastic.

Posts in this series:

Security On The Cheap - Beginning Elastic Stack - Installing Elastic 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Installing Kibana 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic Stack - Providing Basic Security to Elastic and Kibana 7.9 communication on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Metricbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Auditbeat - Elastic Stack 7.9 on Ubuntu 20.04
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Filebeat - Elastic Stack 7.9 on Ubuntu 20.04
Beginning Elastic - Installing, Configuring and Providing Basic Security to Packetbeat
Security On The Cheap - Beginning Elastic - Installing and Providing Basic Security to Winlogbeat


References:

https://www.elastic.co/blog/getting-started-with-elasticsearch-security
https://www.elastic.co/guide/en/elasticsearch/reference/7.8/configuring-tls.html#node-certificates
https://www.elastic.co/guide/en/elasticsearch/reference/current/get-started-built-in-users.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/get-started-kibana-user.html
https://www.youtube.com/watch?v=nMh1HWWe6B4&feature=youtu.be
https://www.elastic.co/guide/en/elasticsearch/reference/current/trb-security-sslhandshake.html

https://www.elastic.co/guide/en/kibana/7.9/using-kibana-with-security.html