wget Command: 30 Practical Examples

Wget is a non-interactive, free utility designed for downloading files from the web. It supports HTTP, HTTPS, and FTP protocols, and can retrieve files through HTTP proxies.

The basic syntax of wget is as follows:

wget [option] [URL]

Within the above syntax:

• The command wget is the trigger for deploying the utility.
• [option] acts as a discretionary parameter enabling you to define various configurations. Wget comes with a vast array of options that allow users to modify its operation. From enabling background downloads to limiting download speeds, a wide assortment of settings can be fine-tuned to suit your needs.
• [URL] signifies the web address of the file you are aiming to download. This is where you input the specific location on the web from where the utility fetches the file.

Detailed Wget Command Examples

1. Downloading a Single File

The most straightforward use of wget is to download a single file from a specified URL.

wget https://example.com/file.zip

This command will download the file named file.zip from example.com and save it in the current working directory.

2. Downloading in the Background

For downloading large files, wget can be run in the background using the ‘-b’ option.

wget -b https://example.com/largefile.zip

This command initiates the download of largefile.zip in the background, allowing you to continue using the terminal for other tasks.

3. Limiting Download Speed

To prevent wget from consuming all available bandwidth, you can limit the download speed using the ‘–limit-rate’ option.

wget --limit-rate=200k https://example.com/file.zip

This command restricts the download speed to 200 KB/s, ensuring other online activities are not disrupted.

4. Downloading Multiple Files

Wget can download multiple files at once. This is achieved by creating a text file with a list of URLs, and using the ‘-i’ option.

wget -i urls.txt

In this example, urls.txt is a text file containing the URLs of all the files to be downloaded. The command downloads all the files listed in urls.txt.

5. Downloading an Entire Website

Wget can be used to download an entire website for offline viewing. This is done using the ‘–mirror’ or ‘-m’ option.

wget --mirror https://example.com

This command mirrors the entire example.com website, downloading all its pages and assets for offline use.

6. Downloading Files with Specific Extensions

To download files with specific extensions from a website, use the ‘-r’ option for recursive download, ‘-A’ to specify the file types.

wget -r -A.pdf https://example.com

This command downloads all PDF files from example.com and its subdirectories.

7. Downloading Files Under Different Names

Using the ‘- O’ option, you can save a file under a different name.

wget -O myFile.zip https://example.com/file.zip

This command downloads file.zip from example.com and saves it as myFile.zip in the current directory.

8. Saving Files in a Specified Directory

You can specify a directory to save the downloaded file using the ‘-P’ option.

wget -P /path/to/directory https://example.com/file.zip

This command downloads file.zip from example.com and saves it in the directory specified by /path/to/directory.

9. Setting Retry Attempts

You can set the number of retry attempts in case the download fails using the ‘–tries’ option.

wget --tries=3 https://example.com/file.zip

This command attempts to download file.zip from example.com up to 3 times before giving up.

10. Downloading via FTP

Wget can also download files via FTP. You need to specify the username and password.

wget --ftp-user=USERNAME --ftp-password=PASSWORD ftp://example.com/file.zip

This command downloads file.zip from the FTP server at example.com using the specified username and password.

11. Continuing Interrupted Downloads

If your download gets interrupted, you can continue it using the ‘-c’ option.

wget -c https://example.com/file.zip

This command resumes the download of file.zip from example.com from where it left off.

12. Retrieving Whole Websites

You can download the entire content of a website for offline viewing using a combination of options.

wget --mirror --convert-links --page-requisites --no-parent -P /path/to/directory https://example.com

This command downloads the entire example.com website and its assets, converts the links for offline use, and saves everything in the directory specified by /path/to/directory.

13. Locating Broken Links

Wget can be used to find all broken URLs that display a 404 error on a specific website.

wget -o wget-log -r -l 5 --spider http://example.com

This command checks all links on example.com up to 5 levels deep and logs the results in wget-log.

14. Downloading Numbered Files

If you have files or images numbered in a specific list, you can easily download all of them.

wget http://example.com/images/{1..50}.jpg

This command downloads the images 1.jpg to 50.jpg from example.com/images.

15. Downloading Files with a Timeout

With the ‘– timeout ‘ option, you can set a timeout for wget to stop trying to connect or read from a site.

wget --timeout=30 https://example.com/file.zip

This command attempts to download file.zip from example.com, but will stop trying if it cannot connect or read from the site within 30 seconds.

16. Ignoring Case When Matching Files/Directories

The ‘-nc’ or ‘–no-clobber’ option prevents wget from overwriting existing files or directories.

wget -nc https://example.com/file.zip

This command downloads file.zip from example.com only if a file with the same name does not already exist in the current directory.

17. Downloading Only Certain File Types

You can instruct wget to download only specific file types from a site using the ‘-r’ and ‘-A’ options.

wget -r -A jpeg,jpg,bmp,gif,png https://example.com

This command downloads all JPEG, JPG, BMP, GIF, and PNG files from example.com and its subdirectories.

18. Downloading All Files Except Certain File Types

You can instruct wget to download all files except certain file types from a site using the ‘-R’ or ‘–reject’ option.

wget -r -R pdf https://example.com

This command downloads all files from example.com and its subdirectories, except for PDF files.

19. Downloading Files from Websites That Use Cookies

Some websites require users to accept cookies before they can access files. You can use the ‘–load-cookies’ option in this case.

wget --load-cookies /path/to/cookies.txt https://example.com

This command downloads files from example.com using the cookies stored in cookies.txt.

20. Downloading Files from Password Protected Sites

If a site requires username and password authentication, you can pass them using the ‘–user’ and ‘–password’ options.

wget --user=USERNAME --password=PASSWORD https://example.com

This command downloads files from example.com using the specified username and password.

21. Downloading Files from a List of URLs

You can download files from a list of URLs using the ‘-i’ option.

wget -i list.txt

This command downloads files from the URLs listed in list.txt.

22. Downloading Files to a Specific Directory

Using the ‘- P ‘ option, you can specify the directory where the downloaded files will be saved.

wget -P /path/to/directory -i list.txt

This command downloads files from the URLs listed in list.txt and saves them in the directory specified by /path/to/directory.

23. Downloading Files with a Specific File Extension

You can download files with a specific file extension using the ‘-A’ option.

wget -r -A .pdf https://example.com

This command downloads all PDF files from example.com.

24. Downloading Files from a Specific Directory

You can download files from a specific directory on a website using wget.

wget -r -np -nH --cut-dirs=3 -R index.html https://example.com/dir/subdir/

This command downloads all files from the subdir directory on example.com.

25. Downloading Files with a Specific Pattern in Their URL

You can download files with a specific pattern in their URL using wget.

wget -r -np -nH --cut-dirs=3 -A '*.pdf' https://example.com/dir/subdir/

This command downloads all PDF files from the subdir directory on example.com.

26. Downloading Files from a Website Using FTP

You can download files from a website using FTP with wget.

wget --ftp-user=USERNAME --ftp-password=PASSWORD ftp://example.com/

This command downloads files from example.com using FTP with the specified username and password.

27. Downloading Files from a Website Using HTTPS

You can download files from a website using HTTPS with wget.

wget https://example.com/file.zip

This command downloads file.zip from example.com using HTTPS.

28. Downloading Files from a Website Using HTTP

You can download files from a website using HTTP with wget.

wget http://example.com/file.zip

This command downloads file.zip from example.com using HTTP.

29. Downloading Files from a Website Using a Proxy

You can download files from a website using a proxy with wget.

wget -e use_proxy=yes -e http_proxy=PROXY_URL:PORT https://example.com/file.zip

This command downloads file.zip from example.com using the specified proxy.

30. Downloading Files from a Website and Saving Them with a Different Name

You can download files from a website and save them with a different name using wget.

wget -O newname.zip https://example.com/file.zip

This command downloads file.zip from example.com and saves it as newname.zip