Tips to download file From Linux [Practical Examples]

In Linux, each process has more than one alternative. In operating systems such as Windows and MAC, most file downloads are done with applications with a graphical interface, while in Linux, this ratio is mostly on the terminal. In this article, we will discuss “How to download file from Linux”.

Different tools available to download file from Linux

In Linux operating systems with a graphical interface(GUI), file downloads can be made via web browsers (Firefox, Chrome, Opera, etc.). In server systems that do not have any GUI, different applications are used for this process. The most frequently used ones are the following packages;

• Wget : The non-interactive network downloader.
• Curl: curl is  a  tool  to transfer data from or to a server
• Httpie: cURL-like tool for humans

The file to be downloaded must be on a network accessible from your system. For example, if you are going to pull data from a web page on the internet, the system must be able to access the internet.

Another issue is that you must have the url information of the file to be downloaded.

Pre-requisite: Download Packages

On a Debian based system (Ubuntu, Pardus, Kali etc.), you can download the package you want with the following steps;

For wget;

ubuntu@ubuntu:/tmp$ apt install wget -y

For curl;

ubuntu@ubuntu:/tmp$ apt install curl -y

For httpie;

ubuntu@ubuntu:/tmp$ apt install httpie -y

In Redhat based operating systems (Centos, AlmaLinux, Rocky Linux etc);

For wget;

foc@fedora:/tmp$ yum install wget

For curl;

foc@fedora:/tmp$ yum install curl

For httpie;

foc@fedora:/tmp$ yum install httpie

Method-1: Using Wget Tool

Download Single File

To download a single file with the wget package, run the following command in the terminal;

ubuntu@ubuntu:~$ sudo wget https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png 
--2022-07-07 20:51:35--  
https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png 
Resolving www.golinuxcloud.com (www.golinuxcloud.com)... 
104.19.155.92, 104.19.154.92, 2606:4700::6813:9b5c, ... 
Connecting to www.golinuxcloud.com (www.golinuxcloud.com)|104.19.155.92|:443... connected.
HTTP request sent, awaiting response... 200 OK Length: 104507 (102K) [image/png] 
Saving to: ‘disk-format-package-list-1.png’ disk-format-package-list-1.png                        
100%[======================================================================================================================>] 102.06K   
544KB/s    in 0.2s 2022-07-07 20:51:36 (544 KB/s) - ‘disk-format-package-list-1.png’ saved [104507/104507]

After this command, a single file is downloaded.

Download files with defined directory depth

Now you may not want to download all sub-directories from the URL. In such case you can define the depth of the directories you want to download from Linux using wget. The following command is written to download the desired directory and its subdirectories;

ubuntu@ubuntu:/tmp/test$ wget -r --no-parent https://github.com/farukomercakmak/ansible-example

The directory and sub-files are downloaded with a directory structure as follows;

ubuntu@ubuntu:/tmp/test$ tree github.com
github.com/
├── farukomercakmak
│   └── ansible-example
├── features
│   └── actions
├── fluidicon.png
├── index.html
├── manifest.json
├── mobile
├── opensearch.xml
├── robots.txt
└── signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F<user-name>%2F<repo-name>&source=header-repo

2 directories, 9 files

Download single directory (without sub-directories)

To download a directory (not subdirectories) using wget, the following command should be run;

sudo wget -r -l1 --no-parent [directory]

For example;

ubuntu@ubuntu:/tmp/test$ wget -r -l1 --no-parent https://github.com/farukomercakmak/ansible-example

If the --no-parent parameter is not given, it starts downloading starting from the first step of the url (For example github.com). Therefore, do not forget to give this parameter.

Resume download

You may encounter problems downloading the file. This is an undesirable problem, especially with large files. In order for the download to continue from where it left off, you need to use -c parameter along with wget in the command you use.

To continue the package download with wget;

ubuntu@ubuntu:~$ :~$ wget -c https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png

Method-2: Using Curl Tool

Download Single File

If you want to download a single file using curl, the following command should be run;

ubuntu@ubuntu:~$ curl -O https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png

Resume download

In the file download process using curl, the following command is written to continue the download.

ubuntu@ubuntu:~$ :~$ curl -C - -O https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png

Multiple file download

File download may not always be for a single file. Examples of usage of related packages for multi-file download are given below. For multi-file download with wget, the url of the file to be downloaded must be written to a file.

You can print different url information to the file with the example below;

echo "https://www.golinuxcloud.com/wp-content/uploads/container-2-1.jpg" >> urls.txt

You can print the url information you want to download between the “” signs. The final version of the urls file should be as follows;

ubuntu@ubuntu:/tmp# cat urls.txt
https://www.golinuxcloud.com/wp-content/uploads/container-2-1.jpg
https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png

There are 2 different uses in the curl package. First, to give two url information one after the other with the -O parameter;

ubuntu@ubuntu:/tmp# curl -O https://www.golinuxcloud.com/wp-content/uploads/container-2-1.jpg -O https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 41095  100 41095    0     0  87997      0 --:--:-- --:--:-- --:--:-- 87809
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  102k  100  102k    0     0   325k      0 --:--:-- --:--:-- --:--:--  325k

In the second use, it can be downloaded by reading the url information from a file, as in the wget package. The xargs command is also needed for this;

ubuntu@ubuntu:/tmp# xargs -n 1 curl -O < urls.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 41095  100 41095    0     0   142k      0 --:--:-- --:--:-- --:--:--  142k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  102k  100  102k    0     0   382k      0 --:--:-- --:--:-- --:--:--  382k

Download the file with a different name (rename)

We can use -o followed by the output path and the file name you want to use for the file which gets downloaded. For example I will rename the image to myimage.png while downloading the same using curl:

curl https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png -o /tmp/test.png

Method-3: Using Httpie Tool

Download Single File

For single file download using httpie;

ubuntu@ubuntu:~$ http --download  https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png

Resume download

For resume download using httpie;

ubuntu@ubuntu:/tmp# http --download  https://www.golinuxcloud.com/wp-content/uploads/disk-format-package-list-1.png --output disk-format-package-list.png --continue

http: error: --continue requires --output to be specified

References

Finally, each package has features other than the above. You can check the man pages in the terminal to find out;

man page for wget
man page for curl
man page for httpie