Graylog is a powerful open source log management platform. It collects and extracts important data from server logs that are often sent using the Syslog protocol. It also allows you to search and visualize logs in the web interface.
In this article, we will install and configure Graylog on Ubuntu 16.04, and create a simple login that will handle the system logs.
Before starting this tutorial, you need to:
- One Ubuntu 16.04 server, at least 2GB of RAM, private networks included, and no root user. This can be created using the article: Initial Ubuntu 16.04 Server Setup.
- Installed Oracle JDK 8.
- Elasticsearch 2.x. Certain versions of Graylog only work with certain versions of Elasticearch. For example, Graylog 2.x does not work with Elasticsearch 5.x. This tutorial uses Elasticsearch 2.4.4 and Graylog 2.2.
- Installed MongoDB.
Step 1 – Configuring Elasticsearch
We need to modify the Elasticsearch configuration file to match the cluster name with one set in the Graylog configuration file. For simplicity, we will set the Elasticsearch cluster name to the default Graylog name.
graylog… You can set whichever you want, but make sure you update the Graylog config file to reflect this change.
Open the Elasticsearch configuration file in an editor:
sudo nano /etc/elasticsearch/elasticsearch.yml
Find the following line:
cluster.name: <CURRENT CLUSTER NAME>
cluster.name на value
Save the file and exit the editor.
Since we changed the configuration file, we must restart the service for the changes to take effect.
sudo systemctl restart elasticsearch
Now that you’ve configured Elasticsearch, let’s move on to installing Graylog.
Step 2 – Installing Graylog
In this step, we will be installing the Graylog server.
First, download the package file containing the Graylog repository configuration. Visit Graylog download page and find the current version number. We will use the version
2.2 for this tutorial.
Then set the repository config from the package file
.deb, replace again
2.2 to the version you downloaded.
sudo dpkg -i graylog-2.2-repository_latest.deb
Now that the repository configuration has been updated, we need to extract the new package list. Run the following command:
sudo apt-get update
Then install the package
sudo apt-get install graylog-server
Finally, start the automatic Graylog download at system startup with the following command:
sudo systemctl enable graylog-server.service
Graylog is now successfully installed, but it’s still a start. We need to configure it before it works.
Step 3 – Configuring Graylog
Now that we have Elasticsearch configured and Graylog installed, we need to change a few settings in the default Graylog config file before we can use it. The Graylog configuration file is located at
First, we need to set the value
password_secret… Graylog uses this value to protect stored user passwords. We will use a randomly generated value of 128 characters.
We will use
pwgen to create a password, so set it if it isn’t already set:
sudo apt install pwgen
Generate password and place it in Graylog config file. We will use the program
sedto set the value
password_secret to the Graylog config file. This way we don’t have to copy and paste any values. Run this command to create a secret and save it to a file:
sudo -E sed -i -e "s/password_secret =.*/password_secret = $(pwgen -s 128 1)/" /etc/graylog/server/server.conf
Next, we need to set the value
root_password_sha2… it SHA-256 hash your desired password. Once again, we will use the command
sed to modify the Graylog config file so we don’t have to manually generate the SHA-256 hash using
shasum and paste it into the config file.
Run this command and replace
password below with the desired default admin password:
Note: There is a leading place in the command that prevents the password from being stored in plain text in the Bash history.
sudo sed -i -e "s/root_password_sha2 =.*/root_password_sha2 = $(echo -n 'password' | shasum -a 256 | cut -d' ' -f1)/" /etc/graylog/server/server.conf
Now we need to make a couple more changes to the config file. Open the Graylog configuration file with your editor:
sudo nano /etc/graylog/server/server.conf
Find and change the following lines, uncomment them and replace
graylog_public_ip to the public IP of your server. This can be an IP address or a fully qualified domain name.
... rest_listen_uri = http://your_server_ip_or_domain:9000/api/ ... web_listen_uri = http://your_server_ip_or_domain:9000/ ...
Save the file and exit the editor.
Since we have changed the configuration file, we must restart (or start) the service
graylog-server… The reboot command will start the server even if it is currently stopped.
sudo systemctl restart graylog-server
Then check the server status.
sudo systemctl status graylog-server
The output should look like this:
● graylog-server.service - Graylog server Loaded: loaded (/usr/lib/systemd/system/graylog-server.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2017-03-03 20:10:34 PST; 1 months 7 days ago Docs: http://docs.graylog.org/ Main PID: 1300 (graylog-server) Tasks: 191 (limit: 9830) Memory: 1.2G CPU: 14h 57min 21.475s CGroup: /system.slice/graylog-server.service ├─1300 /bin/sh /usr/share/graylog-server/bin/graylog-server └─1388 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSCon
You should see
active in status.
If the output says the system is not working, check
/var/log/syslog на presence of errors. Make sure you have Java installed when installing Elasticsearch and that you changed all values in step 3. Then restart the Graylog service again.
If you have configured your firewall with
ufw, add a firewall exception for TCP port,
9000 so that you can access the web interface:
sudo ufw allow 9000/tcp
Once Graylog is up and running, you should be able to access using your web browser. You may have to wait up to five minutes after rebooting before launching the web interface. Also, make sure MongoDB is running.
Now that Graylog is working as expected, we can move on to processing the logs.
Step 4 – Create Login
Let’s add a new entry to Graylog to receive logs. The inputs will tell Graylog which port to listen on and which protocol to use when fetching the logs. We will be adding a Syslog UDP input which normally uses the logging protocol.
When you visit your browser at http: // your_server_ip: 9000, you will see the login page. Use the username admin and the password entered in step 3 for your password.
After logging in, you will see a page titled “Getting Started” that looks like this:
To view the login page, click System in the dropdown menu in the navbar and select inputs…
After that, you will see a drop down box containing the text Select Input… Please select Syslog UDP from that dropdown list and then click on the button Launch new input…
A modal form should appear. Fill in the following details to create your input:
- For Node, select a server. It should be the only item on the list.
- For Title, enter a suitable name, for example
Linux Server Logs…
- For Bind address, use the private IP of your server. If you would like to be able to collect logs from external servers (not recommended since Syslog does not support authentication) you can set it to
- For Port, enter
8514… Please note that we are using the port
8514in this article because the ports
1024can only be used by the root user. You can use any port number above
1024 иeverything should work as long as it doesn’t conflict with other services.
Click the button Save… The local inbound list will update to show its new login as shown in the following image:
Now that the login has been created, we can send some of the Graylog.
Step 5 – Configuring Servers to Send Logs to Graylog
We have an input configured and listening on a port
8514but we’re not sending any input yet, so we won’t see any results.
rsyslog this is a utility used to forward logs and is pre-installed on Ubuntu, so we will configure sending logs to Graylog. In this tutorial we will configure the Ubuntu server to run Graylog so that we just created our logins to send our logins, but you can follow these steps on other servers.
If you want to send data to Graylog from other servers, you need to add a firewall exception for UDP port
sudo ufw allow 8514/udp
Create and open a new config file
rsyslog in the editor.
sudo nano /etc/rsyslog.d/60-graylog.conf
Add the following line to the file, replacing
your_server_private_ip to the private IP of your Graylog server.
Save and exit the editor.
Restart the service
rsyslogfor the changes to take effect.
sudo systemctl restart rsyslog
Repeat these steps for each server where you want to send logs.
You should now be able to view the logs in the web interface. Click the tab Sources in the navigation bar to view the source graph. It should look something like this:
You can also click the tab Search in the navigation pane to view an overview of the most recent logs.
You now have a working Graylog server with an input source that can collect logs from other servers.
Next, you can look at configuring dashboards, alerts and streams. Dashboards provide a quick overview of logs. Streams categorize messages that you can monitor with alerts. To learn more about configuring advanced Graylog features, you can find instructions at Graylog documentation…