Wget Command in Linux With Examples - Nixcraft
Wget Command in Linux With Examples - Nixcraft
This page explains how to use the wget command with valuable examples and
comprehensive explanations of the most common options to become a pro-CLI
user. Let us get started with wget.
ADVERTISEMENT
The syntax is as follows for wget command in Linux, macOS, and Unix system:
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 1/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
wget url
wget [options] url
The url where files are located is always the last parameter. For instance:
https://www.cyberciti.biz/files/somefile.tar.gz. Let us see some common Linux
wget command examples, syntax and usage.
SUSE/OpenSUSE Linux user should type the zypper command to install the wget
command:
Arch Linux users should type the pacman command when you wish to install the
GNU wget command:
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 2/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
Alpine Linux users try the apk command to install the GNU wget command
instead of using busybox wget:
FreeBSD users try the pkg command to install the GNU wget command:
macOS user first install Homebrew on Mac OS to use the brew package manager
and then type the following brew command for installing the GNU wget
command:
OpenBSD and NetBSD users try the following pkg_add command to install the
wget command:
Now that the GNU wget command is installed on your Linux or Unix-like system,
it is time to learn how to use the wget command to download stuff from the LAN,
WAN, or Internet. Let us gets some hands-on experience to increase our
productivity.
Type the following command to grab the latest Linux kernel and compile and
install the Linux Kernel from source code:
$ wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-
5.15.5.tar.xz
Click to enlarge
The above output clearly shows that the wget command connects to the
cdn.kernel.org server to download the file. It also displays the download
progress, file name, DL speed, and other information. By default, the file is
downloaded in the current working directory. However, you can save the
downloaded file under a different directory, including the file name.
$ wget http://www.cyberciti.biz/here/lsst.tar.gz
$ wget -q ftp://ftp.freebsd.org/pub/sys.tar.gz
$ wget -O output.file
http://nixcraft.com/some/path/file.name.tar.gz
$ wget --output-document=file=rhel9.iso
'https://access.cdn.redhat.com/content/origin/23e6decd3160473/
rhel-baseos-9.0-beta-0-x86_64-dvd.iso?_auth_=xyz'
$ wget
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 4/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
http://www.kernel.org/pub/linux/kernel/v2.6/testing/linux-
2.6.36-rc3.tar.bz2
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 5/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
In other words, download file and write it to given FILE named file.txz. To save
the downloaded file under a different name and /tmp/, pass the -O option as
follows:
$ wget -O /tmp/file.tzx \
'https://domain/foo-bar-xyz-long-file.tzx?key=xyz'
You can set directory prefix to prefix by passing the -P prefix option. The
directory prefix is the directory where all other files and subdirectories will be
saved to. In other words, the top of the retrieval tree. The default is . (the
current directory):
$ wget -o download.log \
-O rhel9.iso \
'https://access.cdn.redhat.com/content/origin/23e6decd3160473/
rhel-baseos-9.0-beta-0-x86_64-dvd.iso?_auth_=xyz'
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 6/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
$ cat download.log
more download.log
## or use the grep command/egrep command ##
$ grep 'error' download.log
$ egrep -iw 'warn|error' download.log
Use the following wget syntax when you wish to download stuff from multiple
URLs. For example:
$ wget http://www.cyberciti.biz/download/lsst.tar.gz \
ftp://ftp.freebsd.org/pub/sys.tar.gz \
ftp://ftp.redhat.com/pub/xyz-1rc-i386.rpm
You can create a shell variable that holds all urls and use the ‘BASH for loop‘ to
download all files using the wget:
URLS = "http://www.cyberciti.biz/download/lsst.tar.gz \
ftp://ftp.freebsd.org/pub/sys.tar.gz \
ftp://ftp.redhat.com/pub/xyz-1rc-i386.rpm \
http://xyz-url/abc.iso"
for u in $URLS
do
wget " $u "
done
How do I read URLs list from a text file and grab files using
wget?
You can put all urls in a text file and use the -i option to the wget to download
all files. First, create a text file as follows using a text editor such as nano or
vi/vim:
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 7/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
$ vi /tmp/download.txt
https://www.cyberciti.biz/download/lsst.tar.gz
ftp://ftp.freebsd.org/pub/sys.tar.gz
ftp://ftp.redhat.com/pub/xyz-1rc-i386.rpm
http://xyz-url/abc.iso
$ wget -i /tmp/download.txt
You can also force wget to get a partially-downloaded file i.e. resume
downloads. This is useful when you want to finish up a download started by a
previous instance of wget, or by another program. For instance:
$ wget -c http://www.cyberciti.biz/download/lsst.tar.gz
$ wget -c -i /tmp/download.txt
Please note that the -c option only works with FTP / HTTP / HTTPS servers
that support the “range” header.
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 8/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
OR
The nohup runs the given COMMAND (in this example wget) with hangup signals
ignored, so that the command can continue running in the background after you
log out.
You can limit the download speed to amount bytes per second when using the
wget command. Amount may be expressed in bytes, kilobytes with the k suffix,
or megabytes with the m suffix. For example, --limit-rate=100k will limit the
retrieval rate to 100KB/s for your wget download session. This is useful when,
for whatever reason, you don’t want the wget to consume the entire available
bandwidth. Hence, this is useful when you want to download a large file file, such
as an ISO image from mirrors:
Use the m suffix for megabytes ( --limit-rate=1m ). The above command will
limit the retrieval rate to 50KB/s. It is also possible to specify disk quota for
automatic retrievals to avoid disk DoS attack. The following command will be
aborted when the disk quota is (100MB+) exceeded.
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 9/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
Please note that Wget implements the limiting by sleeping the appropriate
amount of time after a network read that took less time than specified by the
rate. Eventually this strategy causes the TCP transfer to slow down to
approximately the specified rate. However, it may take some time for this
balance to be achieved, so don’t be surprised if limiting the rate doesn’t work
well with very small files.
Another way to specify username and password is in the URL itself. For example:
$ wget 'http://username:password@cyberciti.biz/file.tar.gz
$ ps aux
Sample outputs:
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 10/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
To prevent the passwords from being seen, store them in .wgetrc or .netrc, and
make sure to protect those files from other users with chmod command. If the
passwords are really important, do not leave them lying in those files either edit
the files and delete them after the wget has started the download. Under Linux
you can hide processes from other users and ps command. Same can be done
with FreeBSD to prevent users from seeing information about processes owned
by other users using the ps command or top command/htop command.
Generally you can use shell special character aka wildcards such as *, ?, [] to
specify selection criteria for files. Same can be use with FTP servers while
downloading files. For instance:
$ wget ftp://some-dot-com/url/pub/downloads/*.pdf
$ wget ftp://somedom-url/pub/downloads/*.pdf
OR
$ wget -g on ftp://somedom-com/pub/downloads/*.mp4
In this example, the wget command will grab the file to stand output device ( -O
- ) and pipe it to the shell command of your choice (such as the tar command:
Some websites/ftp servers may throttle requests or ban your IP address for
excessive requests. So use this option carefully and do not overload remote
servers.
$ wget -m https://url/
$ wget --mirror https://url/
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 12/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
$ wget --no-check-certificate \
https://www.cyberciti.biz/robots.txt
lftp fetches HTTP URLs in a manner similar to wget, but segments the retrieval
into multiple parts to increase download speed. It gets the specified file using
several connections. This can speed up transfer, but loads the net heavily
impacting other users. Use only if you really have to transfer the file ASAP.
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 13/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
Not a fan of wget CLI? Try the following GUI based tools:
Conclusion
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 14/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
always type the following man command or help command to read docs on your
Linux or Unix server prompt:
$ man wget
$ wget --help
# get a short help for wget options using the egrep command as
filter #
$ wget --help | grep -Ewi -- '-(i|w|e)'
Hi! 🤠
I'm Vivek Gite, and I write about Linux, macOS, Unix, IT, programming,
infosec, and open source. Subscribe to my RSS feed or email newsletter
for updates.
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 15/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
I remember there was a way to download all .txt file from a URL however I
haven’t been able to find it. You mention that wget http://../../*.txt will do it
however it generate an error saying:
HTTP request sent, awaiting response… 404 Not Found
↩ ∞
Try to use -i option as described above to fetch list of files from text file.
wget -i list.txt
↩ ∞
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 16/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
if some site not giving ( showing its full URL )and also talking http passwd and
user login than
how to get the data directory and how can i use these with wget command
plz mail me on my id i cant remember this site name .
↩ ∞
↩ ∞
↩ ∞
↩ ∞
Lol, these instructions are great for my dedicated server for spreading out my
RS.com downloads.
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 17/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
Cheers!
↩ ∞
Very interesting thanks for the information on Wget my favorite way of uploading
to my dedicated servers. Be it an over complicated and complex way but my
favorite none the less
↩ ∞
↩ ∞
↩ ∞
↩ ∞
↩ ∞
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 18/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
Nice hints!
What I am missing (and still searching for) is –> how to limit file size of a file to
download? One of my current bash scripts is ‘spidering’ certain sites for certain
archive files. IF encountered a 5GB *.zip file, It would happily download it, which I
don’t want. So: what would be a good practise to limit downloads to, say, 2 MB?
Cheers
↩ ∞
Hi, i need to retrieve a file from an http site, which asks for username and
password to access the site. How can i retrieve the file using wget? please guide
me.
↩ ∞
what i get is
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 19/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
Looks like you are pressing Control-C from your shell copy here
↩ ∞
↩ ∞
↩ ∞
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 20/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
now merge
dd if=image2.iso of=image.iso bs=1024 count=100000 skip=100000
dd if=image3.iso of=image.iso bs=1024 count=100000 skip=200000
…
^^
↩ ∞
$wget -cb -o
********** (10stars)
↩ ∞
How can I use wget to download Samba in Linux. I’ve tried several times but
have been unsuccessful?
↩ ∞
Try MultiGet, it also splits the file up into threads to download faster.
↩ ∞
I send the login details and store the cookie. I then rotate through n numbers to
copy the data into a new file. The saved file is always blank i.e 0kb file size.
My website stores data on individual pages e.g.: (i have changed my actual site
name to “mywebsite”)
‘http://admin.mywebsite.com/index.php/print_view/?html=true&order_id=50
I am trying to rotate through the numbers 50 to 1 and extract the data from each
page.
#!/usr/bin/perl
$x = 50;
while ($x <= 1) {
system ("wget –wait=400 –post-data 'html=true&order_id=50' –
referer=http://admin.mywebsite.com/ –cookies=on –load-cookies=cookie.txt –
keep-session-cookies –save-cookies=cookie.txt
http://admin.mywebsite.com/index.php/print_view/");
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 22/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
$x++;
}
How do I modify the code above so data is pulled correctly and the saved files
are not blank? Thank you
↩ ∞
Sir,
I have used the command given as example but error is comming as given
below.
wget url
–21:55:38– url
Resolving http://www.cyberciti.biz 75.126.153.206, 2607:f0d0:1002:51::4
Connecting to http://www.cyberciti.biz|75.126.153.206|:80… failed: Network is
unreachable.
Connecting to http://www.cyberciti.biz|2607:f0d0:1002:51::4|:80… failed:
Network is unreachable.
↩ ∞
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 23/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
variable
yyyy=year, mm=month, dd=day, hh=hour, nn=minute
I want to change the owner of the file with the wget command, what ever the
filles wiil be downloaded needed to change the owner.
↩ ∞
The easiest way to avoid changing owner/group of download files via wget is to
use sudo (run process as other user) or set the download path to a mount point
with specific uid/gid (e.g. uid=henry,gid=henry)
↩ ∞
Hello,
how can i use the wget command if the following situation arises
1) when connected to a particular server the wget command will download the
file. for this if we set a crontab then at the mentioned time the download will
happen. but at time if there is a problem with the server and is not getting
connected the wget command will overwrite the existing file with a dummy file
there by loosing the contents
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 24/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
Is there any way to prevent this?? i.e. when not connected to the server the wget
command should not create or overwrite.
↩ ∞
Thanks,
Bipin Bahuguna
↩ ∞
When I use wget to download 5 files from server using a script it sends 1 GET
request and waits for server to respond then sends the 2nd and so on. I want the
GET to be sent simultaneously irrespective of the response from the server at
the same moment. How to do this? Any insights? Thanks
↩ ∞
You can write a small shell script such that for 5 different files u write 5 wget
command inside the shell script and finally run the shell script.
↩ ∞
URLS=â€http://www.cyberciti.biz/download/lsst.tar.gz
ftp://ftp.freebsd.org/pub/sys.tar.gz
ftp://ftp.redhat.com/pub/xyz-1rc-i386.rpm
http://xyz.com/abc.iso”
for u in $URLS
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 25/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
do
wget $u
done
↩ ∞
how can extract file are in the format of 111.zip.bz2 and abc.zip.bz2 , so is this
possible to bzip2 and unzip a these files in at time command,
↩ ∞
↩ ∞
And what when it comes to get the list of links contained in a web page ?
$ lynx -dump -listonly file > html-list
^^
↩ ∞
hi
and how can i (most easyli) get files from sourceforge?
this “automatic closiest proxy” selection is awful!
↩ ∞
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 26/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
hi,
I have to download a .txt file on daily basis through our one of server, these all
exercise i have done manualy. and save the file in to unix system. i want to create
a automate script who can download and save file into unix directory.
Thanks in advance
↩ ∞
I’ve been with windows my whole life until I started pc science in college I
started playing around with linux its a whole new world!1
↩ ∞
↩ ∞
↩ ∞
I am using the wget command to pull the files from external website. It executes
successfully but, I see a file which has html code rather file that I am expecting.
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 27/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
Thank you
↩ ∞
Hi! May i ask you something? So, if i dont join patreon, is that mean this website
will tracking me?
↩ ∞
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 28/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
No. The Patreon content comes from a separate domain. This site is always
supported by ads.
HTH.
↩ ∞
Leave a Reply
Your email address will not be published. Required fields are marked *
Comment *
Name
Post Comment
Use HTML <pre>...</pre> for code samples. Your comment will appear only after approval by
Next post: Linux Iptables Block All Incoming Traffic But Allow SSH
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 29/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
🔎 SEARCH
🔎 To search, type & hit enter...
🔥 FEATURED ARTICLES
1 30 Cool Open Source Software I Discovered in 2013
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 30/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
👀 /etc
➔ Howtos & Tutorials
➔ RSS/Feed
➔ About nixCraft
➔ nixCraft Shop
➔ Mastodon
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 31/32
29/07/2024, 18:45 Wget Command in Linux with Examples - nixCraft
https://www.cyberciti.biz/tips/linux-wget-your-ultimate-command-line-downloader.html 32/32