To deny specific paths or URLs in Nginx to protect content, you can use fail2ban to deny 403 requests to Nginx. For example, if you renamed the login page using a plugin, then usually, hacker bots will receive a 403 forbidden error when trying to log in. Likewise, if you have to completely disable xmlrpc.php.

This guide will show you how to use fail2ban to automatically ban bots at the server level using IPTables. Be careful not to prevent Google or other search engines from indexing your site!

Configuring fail2ban to deny requests in Nginx 403 Forbidden

Installation overview

• Generating log data
• Setting up Fail2ban filter and jail

Generating log data for 403 errors

Go to the url that is denied by your nginx virtual host to generate log data

Show the last 50 lines of your Nginx log file

tail -n 50 /var/log/nginx/logfile

We found these entries to be used as the basis for the fail2ban filter to ban users who generate a 403 forbidden error

2016/09/15 17:01:18 [error] 3176#3176: *1 access forbidden by rule, client: 189.167.27.53, server: guides.andreyex.ru, request: "GET /wp-login.php HTTP/1.1", host: "guides.andreyex.ru", referrer: "android-app://com.Slack"
2016/09/15 17:01:54 [error] 3176#3176: *1 access forbidden by rule, client: 189.167.27.53, server: guides.andreyex.ru, request: "GET /wp-login.php HTTP/1.1", host: "guides.andreyex.ru", referrer: "android-app://com.Slack"

Now that we have the log data we can create a filter for fail2ban.

Configuring fail2ban for Nginx 403 Forbidden error

We need to create a Fail2ban filter that matches the errors from the log file. Then we create a jail to use this filter and deny users.

Creating a Fail2ban filter for forbidden requests in Nginx

Then create a filter for Nginx

sudo nano /etc/fail2ban/filter.d/nginx-forbidden.conf

Add this regex which will match 403 forbidden error in Nginx logs

[Definition]
failregex = ^ [error] d+#d+: .* forbidden .*, client: <HOST>, .*$

ignoreregex = 

Ctrl + X, Y + Enter to save changes and exit.

We can now test the Nginx HTTP authentication filter by scanning the error log listed in the Nginx virtual host.

fail2ban-regex /var/log/nginx/wpbullet.error.log /etc/fail2ban/filter.d/nginx-forbidden.conf

You will see this output which shows you found the failed login attempts that we generated earlier.

Running tests
=============

Use   failregex file : /etc/fail2ban/filter.d/nginx-forbidden.conf
Use         log file : log

Results
=======

Failregex: 2 total
|-  #) [# of hits] regular expression
|   1) [2] ^ [error] d+#d+: .* forbidden .*, client: <HOST>, .*$
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [2] Year/Month/Day Hour:Minute:Second
`-

Lines: 2 lines, 0 ignored, 2 matched, 0 missed

Making Jail in Fail2ban for Denied Requests in Nginx

Make sure you have a jail folder in Fail2ban

sudo mkdir -p /etc/fail2ban/jail.d

Create Nginx jail Fail2ban config file for HTTP authentication

sudo nano /etc/fail2ban/jail.d/nginx-forbidden.conf

Paste in this configuration, which uses the filter we created earlier that scans all Nginx log files and bans users for 6000 minutes who generated an error 3 times in a 60 second period.

[nginx-forbidden]
enabled = true
filter = nginx-forbidden
port = http,https
logpath = /var/log/nginx/*error*.log
findtime = 60
bantime = 6000
maxretry = 3

Now that we know we will jail, check the Fail2ban syntax to make sure it all works

sudo fail2ban-client -d

If you do not see any errors (OK warnings), then you can restart fail2ban

service fail2ban restart

Checking nginx status for forbidden in fail2ban

Fail2ban client can be used to show statistics of their places of detention

sudo fail2ban-client status nginx-forbidden

While testing on virtual machines, I managed to get the banned gateway.

Status for the jail: nginx-forbidden
|- filter
|  |- File list:        /var/log/nginx/andreyex.error.log /var/log/nginx/error.log
|  |- Currently failed: 0
|  `- Total failed:     3
`- action
   |- Currently banned: 1
   |  `- IP list:       192.168.60.1
   `- Total banned:     1

You can also list IPTables

sudo iptables -L -n

This shows the iptables chain for nginx banned in jail

Chain f2b-nginx-forbidden (2 references)
target     prot opt source               destination
REJECT     all  --  192.168.0.1          0.0.0.0/0            reject-with icmp-port-unreachable
RETURN     all  --  0.0.0.0/0            0.0.0.0/0
RETURN     all  --  0.0.0.0/0            0.0.0.0/0

Any bots that scan for and cause the 403 forbidden error in Nginx will now be automatically banned in Fail2ban.