Thursday, April 7, 2022

Installing & configuring Elasticsearch 8 and Kibana 8 on Ubuntu

In a previous post, we installed Elastic 7.1x. In this post, we are installing the new shiny toy from Elastic, Elastic 8.1

First up, install Elastic public signing key. 

securitynik@securitynik:~$ wget -qO - | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg

Install the apt-transport-https package

securitynik@securitynik:~$ sudo apt-get install apt-transport-https
Preparing to unpack .../apt-transport-https_2.0.6_all.deb ...
Unpacking apt-transport-https (2.0.6) ...
Setting up apt-transport-https (2.0.6) ..

Save the Elastic repo information

securitynik@securitynik:~$ echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] stable main

Install Elasticsearch 8.1

securitynik@securitynik:~$ sudo apt-get update && sudo apt-get install elasticsearch

The following NEW packages will be installed:
Preparing to unpack .../elasticsearch_8.1.1_amd64.deb ...
Creating elasticsearch group... OK
Creating elasticsearch user... OK
Unpacking elasticsearch (8.1.1) ...
Setting up elasticsearch (8.1.1) ...
--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : Laqr4gkhwa-Do=Ctia15

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

Make configuration change to customize this deployment for our environment. First make a backup copy of the configuration file.

securitynik@securitynik:~$ sudo cp /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.ORIGINAL

Here are the changes to my elasticsearch.yml. 

securitynik@securitynik:~$ sudo grep --invert-match "^#" /etc/elasticsearch/elasticsearch.yml n3-elastic securitynik.n3.local
node.attr.rack: ServerCloset /var/lib/elasticsearch
path.logs: /var/log/elasticsearch securitynik.local
http.port: 9200 true true
  enabled: true
  keystore.path: certs/http.p12
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: ["securitynik"] [_local_, _site_]

Adjusting the Java Virtual Machine (JVM) Heap Size by first creating a file.

securitynik@securitynik:~$ sudo cat /etc/elasticsearch/jvm.options.d/jvm.options

Add new host information to my host file, just in case DNS is not working.

securitynik@securitynik:~$ sudo bash -c "echo peeping-tom peeping-tom.n3.local >> /etc/hosts"
securitynik@securitynik:~$ grep peeping-tom /etc/hosts securitynik securitynik securitynik.local

Make a copy of the CA and HTTP certs to /etc/ssl/certs, so that it is in a location easily readable by the rest of the applications.

securitynik@securitynik:~$ sudo cp /etc/elasticsearch/certs/http_ca.crt /etc/ssl/certs -v

Reload systemd daemon, enable and verify Elasticsearch service

securitynik@securitynik:~$ sudo /bin/systemctl enable elasticsearch.service
Created symlink /etc/systemd/system/ → /lib/systemd/system/elasticsearch.service.

securitynik@securitynik:~$ sudo systemctl start elasticsearch.service
securitynik@securitynik:~$ systemctl status elasticsearch.service
● elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-03-30 19:53:03 EDT; 18s ago
   Main PID: 76746 (java)
      Tasks: 80 (limit: 38298)
     Memory: 16.9G
     CGroup: /system.slice/elasticsearch.service
             ├─76746 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cac>
             └─77051 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Mar 30 19:52:47 securitynik systemd[1]: Starting Elasticsearch...
Mar 30 19:53:03 securitynik systemd[1]: Started Elasticsearch.
lines 1-13/13 (END)

Confirming the Elasticsearch ports are listening for incoming communication.

securitynik@securitynik:~$ sudo ss --numeric --listening --tcp --processes | grep --perl-regexp "9300|9200"
LISTEN   0        4096       [::ffff:]:9200                 *:*       users:(("java",pid=76746,fd=386))

LISTEN   0        4096         [::ffff:]:9200                 *:*       users:(("java",pid=76746,fd=385))

LISTEN   0        4096                      [::1]:9200              [::]:*       users:(("java",pid=76746,fd=384))

LISTEN   0        4096       [::ffff:]:9300                 *:*       users:(("java",pid=76746,fd=382))

Connecting to the Elasticsearch service via https

securitynik@securitynik:~$ sudo curl --cacert /etc/elasticsearch/certs/http_ca.crt -u elastic
Enter host password for user 'elastic':
  "name" : "securitynik.n3.local",
  "cluster_name" : "n3-elastic",
  "cluster_uuid" : "KDh-JRfXQtuXjXo2hniQjg",
  "version" : {
    "number" : "8.1.1",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "d0925dd6f22e07b935750420a3155db6e5c58381",
    "build_date" : "2022-03-17T22:01:32.658689558Z",
    "build_snapshot" : false,
    "lucene_version" : "9.0.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  "tagline" : "You Know, for Search"

Good stuff! We have validated Elasticsearch is working as expected.

Installing Kibana on 

Considering all the heavy lifting was done above, time to install Kibana.

securitynik@securitynik:~$ sudo apt-get update && sudo apt-get install kibana
Preparing to unpack .../kibana_8.1.1_amd64.deb ...
Unpacking kibana (8.1.1) ...
Setting up kibana (8.1.1) ...
Creating kibana group... OK
Creating kibana user... OK
Created Kibana keystore in /etc/kibana/kibana.keystore

Make a copy of the Kibana configuration file.

securitynik@securitynik:~$ sudo cp /etc/kibana/kibana.yml /etc/kibana.yml.ORIGINAL

Generate a token

securitynik@securitynik:~$ sudo /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token --scope kibana

Generate encryption keys for SavedObjects, Reports, Dashboards, etc.

securitynik@securitynik:~$ sudo /usr/share/kibana/bin/kibana-encryption-keys generate
## Kibana Encryption Key Generation Utility

The 'generate' command guides you through the process of setting encryption keys for:

    Used to encrypt stored objects such as dashboards and visualizations

    Used to encrypt saved reports
    Used to encrypt session information

Already defined settings are ignored and can be regenerated using the --force flag.  Check the documentation links for instructions on how to rotate encryption keys.
Definitions should be set in the kibana.yml used configure Kibana.

xpack.encryptedSavedObjects.encryptionKey: f4667a5634faf22053dbd40d91afa8b5
xpack.reporting.encryptionKey: f03f17de223aced044cd3afb42de3137 f17be84bbaa17dc9cb8a06cb95e0d5be

Add the last 3 lines from above, to the kibana.yml file and start the Kibana service.

securitynik@securitynik:~$ sudo /bin/systemctl daemon-reload
securitynik@securitynik:~$ sudo /bin/systemctl enable kibana.service
securitynik@securitynik:~$ sudo systemctl start kibana.service

securitynik@securitynik:~$ sudo systemctl status kibana.service
● kibana.service - Kibana
     Loaded: loaded (/lib/systemd/system/kibana.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-03-30 22:56:14 EDT; 8s ago
   Main PID: 102001 (node)
      Tasks: 11 (limit: 38298)
     Memory: 231.7M
     CGroup: /system.slice/kibana.service
             └─102001 /usr/share/kibana/bin/../node/bin/node /usr/share/kibana/bin/../src/cli/dist

Mar 30 22:56:14 securitynik systemd[1]: Started Kibana.
Mar 30 22:56:21 securitynik kibana[102001]: [2022-03-30T22:56:21.275-04:00][INFO ][plugins-service] Plugin "metricsEntities" is disabled.
Mar 30 22:56:21 securitynik kibana[102001]: [2022-03-30T22:56:21.345-04:00][INFO ][http.server.Preboot] http server running at http://192>
Mar 30 22:56:21 securitynik kibana[102001]: [2022-03-30T22:56:21.372-04:00][INFO ][plugins-system.preboot] Setting up [1] plugins: [inter>
Mar 30 22:56:21 securitynik kibana[102001]: [2022-03-30T22:56:21.374-04:00][INFO ][preboot] "interactiveSetup" plugin is holding setup: V>
Mar 30 22:56:21 securitynik kibana[102001]: [2022-03-30T22:56:21.399-04:00][INFO ][root] Holding setup until preboot stage is completed.
Mar 30 22:56:21 securitynik kibana[102001]: i Kibana has not been configured.
Mar 30 22:56:21 securitynik kibana[102001]: Go to to get started.

Open the URL identified above in a browser and add the previously created token for Kibana.

Once the token is added, we should see below.

With the above completing successfully. Time to login to the UI, using the initially created user.

After all the changes, here is what my kibana.yml looks like

securitynik@securitynik:~$ sudo grep --perl-regexp --invert-match "^#" /etc/kibana/kibana.yml ""

server.publicBaseUrl: "" "kibana.n3.local"

      type: file
      fileName: /var/log/kibana/kibana.log
        type: json
      - default
      - file

pid.file: /run/kibana/

xpack.encryptedSavedObjects.encryptionKey: d2667a5634faf33053dbd40d91afa8c9
xpack.reporting.encryptionKey: f03f17de223aced044cd3afb42de4398 f17be84bbaa17dc9cb8a06cb95e0f437

elasticsearch.hosts: ['']
elasticsearch.serviceAccountToken: BBEAAWVsYXN0aWMva2liYW5hL3Vucm9sbC1wcm9jZXNzLM2va2VuLTE2NDg2OTU1ODcwMDg6OVlHSWhfaFlRQXVzMFhVcWZqSTdNZw
elasticsearch.ssl.certificateAuthorities: [/var/lib/kibana/ca_1648695587748.crt]
xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: [''], ca_trusted_fingerprint: e224a19239031533623921f032b6a06e070d673964a43ff9d99e9977fb8526bd}]

No comments:

Post a Comment