Installing Prometheus on Centos 8, NGINX Basic Auth

Prometheus is a free software application used for event monitoring and alerting. It writes metrics in real time to a time series database built using an HTTP request model, with flexible requests and real time alerts.

Installing Prometheus

Add the system user prometheus

                      
                        $ sudo useradd -M -s /bin/false prometheus
                      
                    

Create the necessary directories for prometheus

                      
                        $ sudo mkdir /etc/prometheus /var/lib/prometheus
$ sudo chown prometheus /var/lib/prometheus/
                      
                    

Download the latest version of prometheus to the / tmp directory

                      
                        $ wget https://github.com/prometheus/prometheus/releases/download/v2.19.3/prometheus-2.19.3.linux-amd64.tar.gz -P /tmp
$ cd /tmp
                      
                    

Unpack

                      
                        $ sudo dnf install tar
$ tar xvzf prometheus-2.19.3.linux-amd64.tar.gz
                      
                    

Install prometheus

                      
                        $ cd prometheus-2.19.3.linux-amd64
$ sudo cp prometheus  /usr/local/bin
$ sudo cp promtool  /usr/local/bin
$ sudo cp prometheus.yml /etc/prometheus/
                      
                    

Editing the prometheus configuration file

                      
                        $ sudo nano /etc/prometheus/prometheus.yml
# my global config
global:
  scrape_interval:     15s
  evaluation_interval: 15s

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093 
# Load rules once and periodically evaluate them according to the global 'evalu$
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"
# В дальнейшем будем пользоваться схемой: новое правило - новый файл
# rule_files:
#   - /etc/prometheus/rules.d/*.rules.yml

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
    - targets: ['localhost:9090'] 

# В дальнейшем добавим авторизацию в node_exporter
  - job_name: 'node'
#    basic_auth:
#      username: prometheus
#      password: password
    static_configs:
    - targets: ['localhost:9100'] 
                      
                    

We open port 19090. We are using a non-standard Prometheus port, so that in the future we would add Basic auth via NGINX

                      
                        $ sudo firewall-cmd --add-port=19090/tcp --permanent
$ sudo firewall-cmd --reload
                      
                    

Create a System Unit

                      
                        $ sudo nano /etc/systemd/system/prometheus.service
[Unit]
Description=Prometheus Time Series Collection and Processing Server
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus 
    --config.file /etc/prometheus/prometheus.yml 
    --storage.tsdb.path /var/lib/prometheus/ 
    --web.console.templates=/etc/prometheus/consoles 
    --web.console.libraries=/etc/prometheus/console_libraries 
    --web.external-url https://localhost:19090 
    --web.route-prefix=/

[Install]
WantedBy=multi-user.target
                      
                    

Add the service to autoload and start it

                      
                        $ sudo systemctl daemon-reload
$ sudo systemctl enable --now prometheus
                      
                    

Installing NGINX and setting up Reverse proxy

Installing the dnf-utils utility

                      
                        $ sudo dnf -y install dnf-utils
                      
                    

Add NGINX repository

                      
                        $ sudo nano /etc/yum.repos.d/nginx.repo
[nginx-stable]
name=nginx stable repo
baseurl=https://nginx.org/packages/
                        
                          centos
                        
                        /$releasever/$basearch/
gpgcheck=1
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

[nginx-mainline]
name=nginx mainline repo
baseurl=https://nginx.org/packages/mainline/centos/$releasever/$basearch/
gpgcheck=1
enabled=0
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true
                      
                    

The stable version will be used by default. If you need a mainline version, switch

                      
                        $ sudo dnf config-manager --set-enabled nginx-mainline
                      
                    

Install NGINX

                      
                        $ sudo dnf -y install nginx
                      
                    

Disable the default config

                      
                        $ cd /etc/nginx/conf.d/
$ sudo mv default.conf default.conf.disable
                      
                    

Create config prometheus.conf

                      
                        server {
    listen       19090;
    listen       [::]:19090;
    server_name  _;
    
                        
                          location / {
                        
                        
                          proxy_set_header Accept-Encoding "";
                        
                        
                          proxy_pass https://localhost:9090/;
                        
                        
                          proxy_set_header Host $host;
                        
                        
                          proxy_set_header X-Real-IP $remote_addr;
                        
                        
                          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                        
                        
                          proxy_set_header X-Forwarded-Proto $scheme;
                        
                        
                          auth_basic "Prometheus";
                        
                        
                          auth_basic_user_file "/etc/nginx/prometheus.htpasswd";
                        
                        
                           }
                        
                        
}
                      
                    

Installing the httpd-tools utility

                      
                        $ sudo dnf -y install httpd-tools
                      
                    

Generating a password

                      
                        $ sudo htpasswd -c /etc/nginx/prometheus.htpasswd mypomethuser
    New password: password
    Re-type new password: password
    Adding password for user mypomethuser
                      
                    

SELinux configuration

Configuring SELinux

                      
                        $ cd /tmp
$ sudo grep nginx /var/log/audit/audit.log | grep denied | audit2allow -m nginxlocalconf > nginxlocalconf.te
$ sudo grep nginx /var/log/audit/audit.log | grep denied | audit2allow -M nginxlocalconf
   
                        
                          ******************** IMPORTANT ***********************
                        
                        
                             To make this policy package active, execute:
                        
                        
                             semodule -i nginxlocalconf.pp
                        
                        
$ sudo semodule -i nginxlocalconf.pp
                      
                    

Add the Nginx service to autoload, start it, see the status

                      
                        $ sudo systemctl enable --now nginx
$ sudo systemctl status nginx
                      
                    

Done, prometheus with authorization is available at http: //% your_ip%: 19090

Related Posts