How to Install and Configure ELK Stack on Ubuntu 16.04

ELK stands for Elasticsearch, Logstash and Kibana and is a robust open source data analysis and visualization solution for search. Elasticsearch is distributed, RESTful search and analytics based on Lucene, Logstash is a data processing pipeline for event and log management, and Kibana is a web application for data visualization in Elasticsearch. This ELK Stack installation should work just as well on other Linux VPS systems, but has been tested and written for an Ubuntu 16.04 VPS.

Requirements

There are several requirements for this article:

  • Ubuntu 16.04 VPS
  • Sudo user

Updating the system and installing the required packages

sudo apt update && apt -y upgrade
sudo apt install apt-transport-https software-properties-common wget

Make sure you always update the software on your Linux VPS or set up automatic updates.

Installing Oracle Java JDK Using PPA

We will be using the PPA repository maintained by the Webupd8 Team. The installation script will ask you to accept the license agreement and it will download the Java archive file from the Oracle download page and configure everything for you.

To add the Webupd8 Team PPA repository, run the following commands on the server:

sudo add-apt-repository ppa:webupd8team/java
sudo apt update

Now you can install JDK8 using the following command:

sudo apt install oracle-java8-installer

To check that everything is correct, run the command:

java -version

and you should see something like the following:

java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Installing and configuring Elasticsearch

We will be installing Elasticsearch using a package manager from the Elastic repository.

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
sudo apt update
sudo apt install elasticsearch

After the installation is complete open the file elasticsearch.yml and restrict remote access to the Elasticsearch instance:

sudo nano /etc/elasticsearch/elasticsearch.yml
network.host: localhost

Start the Elasticsearch service and set it to start automatically at boot:

sudo systemctl restart elasticsearch
sudo systemctl enable elasticsearch

Installing and configuring Kibana

Also Elasticsearch we will install Kibana using the package manager from the Elastic repository.

sudo apt install kibana

After the installation is complete open the file kibana.yml and restrict remote access to the Kibana instance:

sudo nano /etc/kibana/kibana.yml
server.host: "localhost"
Start the Elasticsearch service and set it to start automatically on boot:
sudo systemctl restart kibana
sudo systemctl enable kibana

Kibana will now run on localhost on port 5601

Installing and Configuring Nginx as a Reverse Proxy Server

We will use Nginx as a reverse proxy to access Kibana from an IP address. To install Nginx, run the following command:

sudo apt-get install nginx

Creating a basic authentication file using the OpenSSL command:

echo "admin:$(openssl passwd -apr1 YourStrongPassword)" | sudo tee -a /etc/nginx/htpasswd.kibana

Note:

Note: Always use a strong password.

Generating Self-Signed SSL Certificates:

Remove the default Nginx virtual host:

sudo rm /etc/nginx/sites-enabled/default

and create a virtual host config file for our Kibana instance:

sudo nano /etc/nginx/sites-available/kibana
server {
    listen 80 default_server;
    server_name _;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 default_server ssl http2;
 
    server_name _;
 
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    ssl_session_cache shared:SSL:10m;
 
    auth_basic "Restricted Access";
    auth_basic_user_file /etc/nginx/htpasswd.kibana;
 
    location / {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}

Activate the server block by creating a symbolic link:

sudo ln -s /etc/nginx/sites-available/kibana /etc/nginx/sites-enabled/kibana

Checking Nginx configuration and restarting Nginx:

sudo nginx -t
sudo service nginx restart

Installing Logstash

The last step is to install Logstash using a package manager from the Elastic repository.

sudo apt install logstash

The Logstash configuration depends on your personal preference and the plugins you will be using. You can find more information on how to configure Logstash here: https://www.elastic.co/guide/en/logstash/current/configuration.html.

Sidebar