10 wget command examples
We will show you 10 practical examples of the wget command. wget is a free utility that can be used to extract files using HTTP, HTTPS and FTP, which are considered the most widely used protocols on the Internet.
Its name comes from World Wide Web + get. wget has many features, making it a very easy task when it comes to getting large files, recursive downloads, multiple file downloads, or mirroring entire websites or FTP.
wget is not interactive, which gives a lot of flexibility when using it. It can be called from scripts, cron, terminals, etc. It can run in the background even if the user is not logged in. This allows you to start downloading the file and disconnect from the system, allowing wget to finish.
In this article, we will demonstrate the use of wget through a few practical examples that you can use to accomplish some of the most common tasks such as downloading files or even mirroring entire websites.
For this demonstration, we will install wget on Ubuntu 16.04 VPS.
Note that even though this has been tested on Ubuntu 16.04, the instructions can also be used on any other Linux distribution.
Login to server and install wget
The first step is to log into the server via SSH.
You can also ensure that your server is up to date with the following commands:
apt-get update apt-get upgrade
After the updates have been installed, you can install the wget software package using the following command:
apt-get install wget
After the installation is complete, you can start using the wget command on the server.
1. wget command to download one file
The most common and simple use of wget is to download a single file and save it in the current directory.
For example, to download the latest version of WordPress you can use the following command:
wget https://wordpress.org/latest.zip
This is the output you get while downloading the file:
--2017-10-24 16:46:24-- https://wordpress.org/latest.zip
Resolving wordpress.org (wordpress.org)... 66.155.40.249, 66.155.40.250
Connecting to wordpress.org (wordpress.org)|66.155.40.249|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8912693 (8.5M) [application/zip]
Saving to: 'latest.zip'
latest.zip 100%[=============================================================================>] 8.50M 3.83MB/s in 2.2s
2017-10-24 16:46:27 (3.83 MB/s) - 'latest.zip' saved [8912693/8912693]
[email protected]:~#As you can see, it also shows download progress, current download speed, size, date, time and filename.
In our case, this command will download the file and save it in the current directory under the name “latest.zip”.
2. wget command to download the file and save it under a different name
You can save the file under a different name. To do this, you can use the option -Oas in this example:
wget -O wordpress.zip https://wordpress.org/latest.zip
It will download and save the latest WordPress installation in the current directory called “wordpress.zip”.
3. wget command to download a file and save it to a specific directory
In order to download the file and save it in another directory, you can use the option -P, eg:
wget -P /opt/wordpress https://wordpress.org/latest.zip
It will download and save the file in the / opt / wordpress directory on the server.
4. wget command to set download speed
If you accidentally download a huge file that takes longer than it takes to fully download, you can limit the download speed to prevent wget from using its full possible bandwidth.
To limit your download speed, for example to 300k, you can use the following command:
wget --limit-rate=300k https://wordpress.org/latest.zip
5. wget command to continue interrupted download
Sometimes, when you download a very large file that will take longer to fully download, you may temporarily lose your Internet connection and the download will be interrupted.
In order to avoid restarting the entire download, you can continue where the download was interrupted from using the option -c:
wget -c https://wordpress.org/latest.zip
If the download is interrupted and you start downloading everything again without the -c option, wget will append “.1” to the end of the file, since the filename with the previous name already exists.
6. wget command to download in the background
For large files, you can also use the option -bto download the file in the background.
wget -b http://example.com/big-file.zip
The output will be written to the “wget-log” file in the same directory, and you can always check the download status with the following command:
tail -f wget-log
7. wget command, increase retries
If you are having problems connecting to the Internet and the download gets interrupted several times, you can increase the retry attempts to download the file from the option -tries:
wget -tries=100 https://example.com/file.zip
8. wget command to download multiple files
If you want to download multiple files at the same time, you can create a text file (for example download.txt) where you put all the URLs of the files you want to download. Follow these steps to create a text file:
touch download.txt
After that you can edit the file with nano and enter all urls of all files you want to download:
nano download.txt
http://example.com/file1.zip http://example.com/file2.zip http://example.com/file3.zip
After saving the file, you can use the option -ito load all the files saved in this text file:
wget -i download.txt
9. wget command for FTP upload
You can also use wget to upload a file directly via FTP using a set of username and password using the following command:
wget --ftp-user=username --ftp-password=password ftp://url-to-ftp-file
10. wget command to download the entire site
You can even use wget to download the entire website, which you can browse locally, offline, without the need for an internet connection. To do this, you can use the following command:
wget --mirror --convert-links --page-requisites ----no-parent -P /path/to/download https://example-domain.com
FROM –mirror, you enable all options required for mirroring.
FROM –Convert-links, you will convert all links so that they work offline.
FROM –Page-requisites, all the necessary files, such as CSS stylesheets and images, required to render the page properly offline, will be loaded.
FROM –No-parent, you can restrict downloads to only a specific part of the site.
Alternatively, you can set the path to where we want to upload the files with the -P command followed by the directory path.
We have covered some of the most common uses of the wget command. To find out more about wget, you can always check its page with the command man wget…