How to use wget

Posted on March 13th, 2008 in Applications, Basics, Commandline Tools by admin

GNU Wget is a free software package for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols. It is a non-interactive commandline tool, so it may easily be called from scripts, cron jobs, terminals without X-Windows support, etc. GNU Wget has many features to make retrieving large files or mirroring entire web or FTP sites easy. If it’s not already installed, chances are it’s in your repositories. If not you can download it here:

To download a file just type (the file extension does not matter)

wget http://websitename/file.jpg

But you cannot resume broken downloads. So you would need to use the  -c option to start resumable downloads

wget -c http://websitename/file.jpg

You can also mask the program as web browser using -U. This helps when the sites doesn’t allow download managers.  In this case we mask it as a mozilla browser.

wget -c -U Mozilla http://websitename/file.jpg

You can download an entire website using -r option.

wget -r http://websitename.com

Be careful with this option. It downloads the entire website for you. Since this tool can put a large load on servers it obeys robot.txt

You can mirror a site on you local drive using -m option.

wget -m http://websitename.com

You can select the levels up to which you can dig into the site and downloads using -l option.

wget -r -l3 http://websitename.com

This will download only up to 3 levels. Suppose you want download only sub folders in a website url use –no-parent option. With this option wget downloads only the sub folders and ignores,the parent folders

wget -r –no-parent http://websitename/subfldr/subfolder

Post a comment