Working with files remotely is a common task, especially for developers, system administrators, and even power users. While web browsers offer a user-friendly interface for downloading, the command line provides a powerful, flexible, and often faster alternative, particularly when dealing with automation, scripting, or working on headless servers. One of the most versatile tools for this purpose is curl.
What is curl?
curl is a command-line tool and library for transferring data with URLs. It supports a vast range of protocols, including HTTP, HTTPS, FTP, FTPS, SCP, SFTP, and many more. Its primary use for many is simply downloading files, but its capabilities extend far beyond that.
Why Use curl for Downloading?
Using curl offers several advantages:
- Automation: Easily integrate downloads into scripts (Bash, Python, etc.) for recurring tasks.
- Speed and Efficiency: Often faster for direct downloads without the overhead of a browser interface.
- Control: Fine-grained control over the download process using various options.
- Availability: Pre-installed on most Linux and macOS systems, and available for Windows.
- Server Environments: Essential for downloading files on servers where a GUI browser isn’t available.
Basic File Download with curl
The simplest way to download a file is to provide the URL:
curl [URL]
For example:
curl https://example.com/path/to/myfile.zip
By default, curl will output the content of the file directly to your terminal. This is useful for viewing text files or source code but not for saving binary files or large documents.
Saving the Downloaded File
To save the downloaded content to a file, you have a couple of primary options:
Save with a Specific Name: Use the
-o(or--output) option followed by the desired local filename.curl -o mylocalfile.zip https://example.com/path/to/myfile.zipThis downloads the file from the URL and saves it as
mylocalfile.zipin your current directory.Save with the Original Filename: Use the
-O(or--remote-name) option. This tellscurlto figure out the filename from the URL (or the server’s response headers) and save it using that name.curl -O https://example.com/path/to/myfile.zipThis will download the file and save it as
myfile.zipin your current directory. This is often the most convenient method.
Tracking Download Progress
When downloading large files, you’ll want to see the progress. curl provides options for this:
The default behavior often shows a basic progress meter for larger files.
Use the
-#or--progress-baroption for a more visual progress bar.curl -O -# https://example.com/largefile.tar.gz
Handling Redirects
Sometimes, the URL you provide might redirect to another location. By default, curl doesn’t follow redirects. To ensure you download the final content, use the -L (or --location) option.
curl -O -L https://shorturl.at/examplefile
Resuming Interrupted Downloads
A very useful feature is the ability to resume a download that was stopped (due to network issues, server problems, or canceling the command). Use the -C - (or --continue-at -) option. The hyphen (-) tells curl to automatically figure out where to resume based on the existing local file.
curl -O -C - https://example.com/verylargefile.iso
If verylargefile.iso already exists in the current directory, curl will attempt to continue downloading from the point where it left off.
Important Considerations
- Security: Always download files from trusted sources.
curlsupports HTTPS, providing encrypted transfers, but the integrity and safety of the file itself depend entirely on the source website. Be cautious when downloading executables or archives from unknown sites. - Permissions: Ensure you have write permissions in the directory where you are trying to save the file.
- Error Handling: In scripts, always check the exit status of
curl(using$?) to ensure the download was successful.
Mastering curl for downloading files is a valuable skill for anyone working with computer systems. Its flexibility and power make it an indispensable tool for countless tasks, from simple file retrieval to complex automation workflows. Experiment with the options discussed here to streamline your command-line file transfers.
Source: https://www.redswitches.com/blog/how-to-use-curl-to-download-files-from-the-command-line/
