Downloading Files with Wget Command in Linux

Welcome to our comprehensive guide on downloading files using the Wget command in Linux! The wget command is a powerful, versatile tool that helps to automate file downloads from the internet. Whether you're new to Linux or looking to optimize your download processes, this guide will walk you through basic to advanced Wget usage.

Getting Started with Wget

Before we dive into examples, let's ensure you have Wget installed on your Linux system. Open your terminal and type:

wget --version

If Wget isn't installed, follow these steps based on your Linux distribution:

• Ubuntu/Debian:

  sudo apt-get update
  sudo apt-get install wget
  

• CentOS/RHEL:

  sudo yum install wget
  

• Fedora:

  sudo dnf install wget
  

Once installed, you're ready to start downloading files!

1. Basic File Download

Downloading a file with Wget is simple. Use the command:

wget <URL>

Example: To download a file from example.com, type:

wget http://example.com/file.zip

This command downloads file.zip into the current directory.

2. Downloading Multiple Files

Wget can download multiple files at once. Create a text file with each URL on a new line. Then use the -i option:

wget -i urls.txt

Example:

If urls.txt contains:

http://example.com/file1.zip
http://example.com/file2.zip

Running the above command will download both files listed in urls.txt.

3. Recursive Downloads

To download an entire website or directory, use the -r option for recursive downloads:

wget -r <URL>

Example:

wget -r http://example.com/dir/

This downloads all files in the directory and subdirectories at example.com.

4. Resuming Downloads

Have an interrupted download? No worries! Resume downloads using the -c option:

wget -c <URL>

Example:

wget -c http://example.com/bigfile.zip

This resumes downloading bigfile.zip from where it left off.

5. Download with Custom Filename

To save a file with a different name, use the -O (uppercase letter O) option:

wget -O newname.zip <URL>

Example:

wget -O myfile.zip http://example.com/file.zip

This downloads the file and saves it as myfile.zip.

6. Authentication and Cookies

For files that require login credentials, pass them using --user and --password:

wget --user=username --password=password <URL>

Example:

wget --user=jdoe --password=secret http://example.com/securefile.zip

Use cookies for websites requiring them:

wget --load-cookies cookies.txt <URL>

7. Limiting Download Speed

To control bandwidth usage, limit the download rate with --limit-rate:

wget --limit-rate=200k <URL>

Example:

wget --limit-rate=100k http://example.com/largefile.zip

This restricts the download speed to 100KB/s.

8. Storing Downloads in Specific Directory

Set a directory for saving files using -P:

wget -P /path/to/directory <URL>

Example:

wget -P /home/user/downloads http://example.com/file.zip

This saves the file directly into the specified downloads directory.

9. Using Wget in Scripts for Automation

The power of Wget shines in scripts, allowing you to automate the downloading process, which is extremely useful for repetitive tasks.

Example Script for Automated Downloads

Below is a simple bash script that demonstrates how to automate file downloads with Wget. Save this script as download_script.sh:

#!/bin/bash

# List of URLs to download
urls=(
    "http://example.com/file1.zip"
    "http://example.com/file2.zip"
    "http://example.com/file3.zip"
)

# Download directory
download_dir="/home/user/automated_downloads"

# Create the download directory if it doesn't exist
mkdir -p "$download_dir"

# Loop through URLs and download files
for url in "${urls[@]}"; do
    wget -P "$download_dir" "$url"
done

echo "All files have been downloaded to $download_dir"

How to Use the Script:

1. Save the script as download_script.sh.
2. Make it executable: chmod +x download_script.sh.
3. Run the script: ./download_script.sh.

This script downloads a series of files listed in urls into a specific directory.

Conclusion

The Wget command is a fantastic tool for downloading files in Linux, offering simplicity and versatility. From downloading single files to automating complex tasks, Wget covers it all. With these examples and tutorials, you're equipped to handle downloads more effectively. For more options, check out the Wget manual page by typing man wget in your terminal. Happy downloading!