Perl Webget Project

Links and download

Note: that the most up to date version is in version control. See development page instructions how get latest source code.

Main project page (never changes; bookmark this one)
Development page (Source code, bug tracker etc.)
Downloads
Manual
ChangeLog Generated by editing the code in Emacs and pressing key binding (C-x 4 a).

Other pages

Old program binary from days of early development. Please do not use file from Perl CPAN archive http://cpan.perl.org/modules/by-authors/id/J/JA/JARIAALTO/

Description

The pwget is a tool that can help to keep track of programs and package releases around the Internet. The major differences to well known wget utility are as follows:

Configuration file(s). Program supports file downloads as with wget by specifying the URLs in the command line. But the advantage is to automate the periodic downloads from your favourite locations by adding entries to configuration file(s). You refer to named tag in order to download new files from developer's site.
The configuration file can use unlimited number of hierarchy levels: like a) "linux" b) sub-level "linux-gnome". If you request tag "linux", everything under that hierarchy level is downloaded.
Heuristics. As developers release new packages, the package name is subject to changes. The de facto packaging standard is package-version.suffix, like in package-12.1.tar.gz. You do not need to know the exact URL, because program is able to determine latest version. If you had instructed to download emacs-19.28.tar.gz and supplied option option --new, the newest kit would have been found. Record the initial URL to a configuration file, instruct to monitor newer versions, and you can start keeping your favourite programs up to date
Content check. If a page contains many files (like directory listing), it is possible to download only files whose content match a regular expression. For example, it is possible to request author's name in the file with option --Regexp-content='(?i)Author:\s*Mr.\s*Foo'
Extract. After download, program can unpack the downloaded package in a civilized manner. If archive includes a root directory, then everything is ok, if not, root directory is created upon unpack. This is desireable, because some archives may not include proper package-version/ root directory.

All in all, pwget is designed to be your companion to keep in touch with the changing package releases in the software release world. The aim was to develop an utility which can go to an URL, sniff around a bit to make the decision what to download. Debian has developed .deb install archives and Red Hat a .rpm format respectively, but there is a need to handle .tar.gz, .tgz, .tar.bz2 and other file downloads as well. pwget may help with tracking URLs easier.

Test run - creating Emacs site-lisp

There is an example configuration which contains URLs of Emacs Lisp packages. Emacs is a development environment that can bextended with additional packages (an example). The example configuration contains presets for /usr/share/*/site-lisp/net hierachy (the install location is configurable). It contains instructions how to download many custom Emacs packages. The results, what generated directory structure would be, can be seen at log file.

    perl pwget [--dry-run] --verbose --Tag elisp

Contact

There is no mailing list for the project. See Development page how to contact maintainer and submit feature requests and bug reports.

All files in this project are licensed under GNU GPL.

This project, as well as many other projects is hosted by Savannah.

W3C HTML valid

W3C CSS valid

W3 validated.