How To Use Wget Linux

This tutorial will show you how to use wget command in Linux. We also give examples of 12 useful wget commands. As a result, you will see all the uses of wget and be able to download files from the World Wide Web.

Read more:

• How To Check If A File Is A Directory
• How To Block Messages On Macbook

What is Wget Command?

Wget is a computer tool created from the GNU Project. You can use it to extract data and content from many different web servers. Its name is a combination of the World Wide Web and the word get. It supports downloading over FTP, SFTP, HTTP, and HTTPS.

Wget is made in portable C and can be used on any Unix system. It can also be deployed on Mac OS X, Microsoft Windows, AmigaOS, and other popular platforms.

How to install Wget?

For this wget command tutorial, we will be using Ubuntu 16.04. But the command structure can be used on other Linux distributions.

First, access the server via SSH. Execute the following command to install wget on your computer:

When the installation is finished, you can start using it. Also, the basics of SSH can be of great help.

Wget Command example

We give you 12 examples of wget command that you will probably use every day. Please note that you can also use this function from scripts and cron jobs!

Use Wget Command to download each file

One of the most basic wget commands is to download the file and save it in the current directory. For example, if you want to download the latest version of WordPress, use the following command:

wget https://wordpress.org/latest.zip

Result

--2018-02-23 12:53:10-- https://wordpress.org/latest.zip
Resolving wordpress.org (wordpress.org)... 198.143.164.252
Connecting to wordpress.org (wordpress.org)|198.143.164.252|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9332728 (8,9M) [application/zip]
Saving to: 'latest.zip'

latest.zip 100%[===================>] 8,90M 6,88MB/s in 1,3s

2018-02-23 12:53:14 (6,88 MB/s) - 'latest.zip' saved [9332728/9332728]

In this example, a file named latest.zip will be loaded into the directory you are using. You will see additional information such as, trim progress, speed, size, time and date.

Use Wget Command to download multiple files

We can use wget to download multiple files at once. To do this, you need to create a text file and put URLs to download the file into. In this example, we will download the latest versions of WordPress, Joomla, and Drupal. Enter the following command:

nano example.txt

This command will create a file named example.txt and open the text ditor. Paste the following links into:

https://wordpress.org/latest.zip

https://downloads.joomla.org/cms/joomla3/3-8-5/Joomla_3-8-5-Stable-Full_Package.zip

https://ftp.drupal.org/files/projects/drupal-8.4.5.zip

Then you can use -i to get all the files contained in the example file:

wget -i example.txt

Wait for a while the download will complete and you have the 3 most popular versions of Content Management.

Use Wget Command to download files under a different name
In this wget example we'll save the file with a different name with the -O option:

wget -O wordpress-install.zip https://wordpress.org/latest.zip

In this case, the download is saved as wordpress-install.zip instead of the original name.

Use Wget Command to save the file in a specified directory
You can use wget to put the file in another directory using the -P option:

wget -P documents/archives/ https://wordpress.org/latest.zip

The files you have downloaded will appear in the documents / archives / folder.

Use Wget Command to limit download speed

With wget, you can limit the download speed. This is useful when you are downloading a large file and avoid it using up your bandwidth. The example below is limited to 500k:

wget --limit-rate=500k https://wordpress.org/latest.zip

Use Get Command to set the number of reload attempts

Internet connection may cause simple paragraph errors. For this, we can increase the number of reload attempts using the -tries option:

wget -tries=100 https://wordpress.org/latest.zip

Use Wget Command to load files in the Background

For very large files, you can use function -b. It will run in the background

wget -b http://example.com/beefy-file.tar.gz

A wget-log will appear in the current directory, you can check the progress and status. This command will help you test it:

tail -f wget-log

Use Wget Command to download files via FTP

This command can be used with FTP. You just need to set the correct name and password as in the following case:

wget --ftp-user=YOUR_USERNAME --ftp-password=YOUR_PASSWORD ftp://example.com/something.tar

Use Wget Command to resume interrupted file downloads

Downloading can be interrupted if there's a problem with the connection. This usually happens when you are downloading a large file. Instead of reloading from scratch, you can continue with function -c:

wget -c https://example/very-big-file.zip

If you proceed without the -c function, the new file will have .1 added at the end as it already exists.

Use Wget Command to download the entire website

You can also use the wget command to download the entire site. It will show you on your computer without internet connection. Use the following command:

wget --mirror --convert-links --page-requisites --no-parent -P documents/websites/ https://some-website.com

Let's parse the fields in the wget command:

–mirror	It helps you to download in recursive format.
–convert-links	All links will be converted to offline links.
–page-requisites	This command will include all the necessary files such as CSS, JS, and images
–no-parent	It ensures that there are no directories above the downloaded website directory
-P documents/websites/	Make sure all content is saved to the specified folder.

When done, you can open the website on your computer and find all the necessary files in the directory documents/websites/

Use Wget Command to locate the error link

Use commands in a more advanced way. You can use the wget command to define broken URL that shows a 404 error on your website. By executing the following command:

wget -o wget-log -r -l 5 --spider http://example.com

-o	Group all outputs into one file for later use
-l	Determine the recursive level
-r	Make the download recursive
–spider	Put wget in spider mode

We will continue to investigate more wget-log file to determine the link error. Here's the command to do this:

grep -B 2 '404' wget-log | grep "http" | cut -d " " -f 4 | sort -u

Use Wget Command to download files by number

If you have images or files numbered according to a certain list, you can download them all using the following structure:

wget http://example.com/images/{1..50}.jpg

See more:

How To Undo Deleted Text In Gmail App
How To Burn Large Iso To DVD

Congratulations. By completing this tutorial you learned how to use wget professionally. You can utilize it to download one or more files. Furthermore, you can also use its advanced feature of downloading the entire website and identifying broken links. For more information, see also this official documentation section

Do you have any tips or tricks you want to share? Feel free to leave your comments below!

Source: https://blebees.com

source https://blebees.com/how-to-use-wget-linux/