You can download this article in PDF format via the link below to support us.Download the guide in PDF formatClose
Grafana Loki is a tool that provides you with a panel for indexing system logs and visualizing them on the dashboard. Grafana Loki does not index the content of the log, but only the tags of the log. This reduces the workload of processing and storing logs.
Like Prometheus, Promtail is Loki’s log collector, which sends log tags to Grafana Loki for indexing.
In this article, we will introduce the following:
- Install Grafana
- How to install Loki
- How to install Promtail
- How to configure Loki data source and browse
Let’s quickly start the installation steps:
Step 1-Install Grafana Monitoring Tool
In this section, we will introduce Grafana on Ubuntu . Use the link below, it will guide you how to install Grafana on different systems:
How to install Grafana 7 on CentOS 7
How to install Grafana on CentOS 8 / RHEL 8
Install Grafana on Kubernetes for cluster monitoring
Update your Ubuntu system and restart:
sudo apt-get update
sudo apt-get upgrade
sudo reboot
Add the Grafana GPG key, and then install the Grafana APT repository:
## Add gpg key
sudo apt-get install -y gnupg2 curl
curl https://packages.grafana.com/gpg.key | sudo apt-key add -
## Add Grafana APT repository
sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
Install Grafana
sudo apt-get update
sudo apt-get -y install grafana
Start Grafana service
sudo systemctl start grafana-server
sudo systemctl enable grafana-server
Grafana is now installed and can be accessed through the server’s IP and port 3000. (https://server_IP:3000)
You need to allow port 3000 to pass through the firewall.
For Ubuntu:
sudo ufw allow proto tcp from any to any port 3000
Then you can access the Grafana web dashboard,
Step 2-Install Grafana Loki Log Aggregation System
Now, we follow the steps below to install Loki:
- Go to Loki Publish page And choose the latest version of Loki
- navigation assets And download the Loki binary zip file to your server. During the publication of this article, v2.0.0 is the latest.
curl -s https://api.github.com/repos/grafana/loki/releases/latest | grep browser_download_url | cut -d '"' -f 4 | grep loki-linux-amd64.zip | wget -i -
Install and unzip
# Ubuntu /
Debian
$ sudo apt install unzip
# CentOS / Fedora / RHEL
$ sudo yum -y install unzip
3. Unzip the binary file to /usr/local/bin
unzip loki-linux-amd64.zip
sudo mv loki-linux-amd64 /usr/local/bin/loki
Confirm the installed version:
$ loki --version
loki, version 2.0.0 (branch: HEAD, revision: 6978ee5d)
build user: [email protected]
build date: 2020-10-26T15:54:56Z
go version: go1.14.2
platform: linux/amd64
4. Create a YAML file in Loki below /usr/local/bin
Create the required data directory:
sudo mkdir -p /data/loki
Create a new configuration file.
sudo vim /etc/loki-local-config.yaml
Add the following configuration to the file:
auth_enabled: false
server:
http_listen_port: 3100
ingester:
lifecycler:
address: 127.0.0.1
ring:
kvstore:
store: inmemory
replication_factor: 1
final_sleep: 0s
chunk_idle_period: 5m
chunk_retain_period: 30s
max_transfer_retries: 0
schema_config:
configs:
- from: 2018-04-15
store: boltdb
object_store: filesystem
schema: v11
index:
prefix: index_
period: 168h
storage_config:
boltdb:
directory: /data/loki/index
filesystem:
directory: /data/loki/chunks
limits_config:
enforce_metric_name: false
reject_old_samples: true
reject_old_samples_max_age: 168h
chunk_store_config:
max_look_back_period: 0s
table_manager:
retention_deletes_enabled: false
retention_period: 0s
5. Create Loki service:
Create the following files below /etc/systemd/system Guarding Loki services:
sudo tee /etc/systemd/system/loki.service<<EOF
[Unit]
Description=Loki service
After=network.target
[Service]
Type=simple
User=root
ExecStart=/usr/local/bin/loki -config.file /etc/loki-local-config.yaml
[Install]
WantedBy=multi-user.target
EOF
6. Reload the system daemon, and then start the Loki service:
sudo systemctl daemon-reload
sudo systemctl start loki.service
You can check whether the service has started successfully:
$ sudo systemctl status loki
● loki.service - Loki service
Loaded: loaded (/etc/systemd/system/loki.service; disabled; vendor preset: enabled)
Active: active (running) since Mon 2020-12-21 11:49:49 UTC; 2min 37s ago
Main PID: 15223 (loki)
Tasks: 7 (limit: 1137)
Memory: 13.6M
CGroup: /system.slice/loki.service
└─15223 /usr/local/bin/loki -config.file /etc/loki-local-config.yaml
Dec 21 11:49:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:49:49.330959628Z caller=table_manager.go:476 msg="creating table" table=index_2658
Dec 21 11:49:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:49:49.331092225Z caller=table_manager.go:476 msg="creating table" table=index_2549
Dec 21 11:49:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:49:49.331220486Z caller=table_manager.go:476 msg="creating table" table=index_2562
Dec 21 11:49:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:49:49.331347316Z caller=table_manager.go:476 msg="creating table" table=index_2615
Dec 21 11:49:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:49:49.331471475Z caller=table_manager.go:476 msg="creating table" table=index_2643
Dec 21 11:49:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:49:49.327278535Z caller=module_service.go:58 msg=initialising module=ring
Dec 21 11:49:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:49:49.331950866Z caller=module_service.go:58 msg=initialising module=distributor
Dec 21 11:49:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:49:49.332140208Z caller=module_service.go:58 msg=initialising module=ingester-querier
Dec 21 11:49:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:49:49.332342162Z caller=loki.go:227 msg="Loki started"
Dec 21 11:51:49 ubuntu loki[15223]: level=info ts=2020-12-21T11:51:49.311922692Z caller=table_manager.go:324 msg="synching tables" expected_tables=141
You can now access Loki indicators by https://server-IP:3100/metrics
Step 3-Install Promtail Agent
Promtail is an agent that sends local log content to a private Loki instance or Grafana Cloud.
- Download the Promtail binary zip from the following location Publish page
curl -s https://api.github.com/repos/grafana/loki/releases/latest | grep browser_download_url | cut -d '"' -f 4 | grep promtail-linux-amd64.zip | wget -i -
After downloading the file, unzip it to /usr/local/bin
unzip promtail-linux-amd64.zip
sudo mv promtail-linux-amd64 /usr/local/bin/promtail
Check version:
$ promtail --version
promtail, version 2.0.0 (branch: HEAD, revision: 6978ee5d)
build user: [email protected]
build date: 2020-10-26T15:54:56Z
go version: go1.14.2
platform: linux/amd64
2. Create a YAML configuration file for Promtail on the page. /usr/local/bin table of Contents:
sudo vim /etc/promtail-local-config.yaml
3. Add the following to the file:
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /data/loki/positions.yaml
clients:
- url: https://localhost:3100/loki/api/v1/push
scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/*log
4. Create Promtail service:
sudo tee /etc/systemd/system/promtail.service<<EOF
[Unit]
Description=Promtail service
After=network.target
[Service]
Type=simple
User=root
ExecStart=/usr/local/bin/promtail -config.file /etc/promtail-local-config.yaml
[Install]
WantedBy=multi-user.target
EOF
5. Reload and start the Promtail service
sudo systemctl daemon-reload
sudo systemctl start promtail.service
Confirm whether the service is running:
$ systemctl status promtail.service
● promtail.service - Promtail service
Loaded: loaded (/etc/systemd/system/promtail.service; disabled; vendor preset: enabled)
Active: active (running) since Mon 2020-12-21 11:57:41 UTC; 3s ago
Main PID: 15381 (promtail)
Tasks: 6 (limit: 1137)
Memory: 8.8M
CGroup: /system.slice/promtail.service
└─15381 /usr/local/bin/promtail -config.file /etc/promtail-local-config.yaml
Dec 21 11:57:41 ubuntu systemd[1]: Started Promtail service.
Dec 21 11:57:41 ubuntu promtail[15381]: level=info ts=2020-12-21T11:57:41.911186079Z caller=server.go:225 http=[::]:9080 grpc=[::]:35499 msg="server listening on>
Dec 21 11:57:41 ubuntu promtail[15381]: level=info ts=2020-12-21T11:57:41.911859429Z caller=main.go:108 msg="Starting Promtail" version="(version=2.0.0, branch=H>
So far, we have installed Grafana, Loki and Promtail.
The next step is to configure the Grafana dashboard and visualize the logs using Loki.
Step 4-Configure Loki data source
- Log in to the Grafana web interface and select ‘explore’ . You will be prompted to create a data source.
2. Click Add data source Then choose Loki from the available options:
4. Enter the following values for Loki:
Name: Loki
URL: https://127.0.0.1:3100
See the screenshot below.
5. Click Save and test . You should see a notification that the data source has been successfully added.
Step 5-Use Loki to visualize the logs on Grafana
Now, we can use Loki to visualize the log.
Click on explore Then choose Rocky In the data source
You can now select Log tag Options.
in conclusion
We have successfully installed Grafana Loki with Promtail and can visualize logs on the Grafana dashboard.
In addition to Promtail, Grafana Loki can use more log forwarding tools, such as FluentBit. You can also use Grafana Loki to monitor Kubernetes clusters. Please enjoy!
You can download this article in PDF format via the link below to support us.Download the guide in PDF formatClose
