It will follow it's path, even if it is virtually invisible. Walking from leaf to leaf it will find all the servers out there.

get all the stuff your ever dreamed of...

News

18.03.2004: New release v0.1.1 is availiable now. It is a minor usability update: Progress indicators for fetching the grouplist and the article headers are added and the levels of verbosity are reorganized to be both, more informative and more readable. See ChangeLog for details.

13.03.2004: New release v0.1 is availiable now. It is a major improvement in usability and fixed many bugs from the pre-alpha version 0.0.1. Now it reached full alpha status: It is usable, but major changes in the commandline and the code are very likely.

Overview

nntpspider searches for public accessable NNTP servers. It reads a list of known servers. All these servers are checked for public access, queried for a list of supplied newsgroups and for information about additional nntp servers. These new servers are recursivly queried. The resulting list of currently availiable newsservers is saved to a file. For every public host the list of supplied newsgroups is saved to a file.

How does it work?

All servers with public or unknown status are queried for a list of supplied newsgroups. If access is granted, the Path: lines from the message headers of Usenet postings in a couple of groups are scanned for information about the nntp servers used posting these ones. All found servers are queried until no more unknown servers are found.

The search starts with at least one default newsserver you have access to, usually the one at your ISP. The list of known server is read from the given files or read from stdin.

Prerequisitories

  • Perl (tested with Perl 5.8.3 but should work with every version of Perl 5)
  • Net::NNTP module (avaliable at CPAN, included in most standard Perl distributions)
As being a Perl programm, it should be OS-independed. I'm developing on Linux only, but since no special OS-features are used it should run flawlessly with every system witch offers you a perl environment, including OS-X, some PDAs and even Windows. Please send me a short report about the environment you use NNTPspider - successful or not! (See below for my mail adress)

Bugs

Many. It is still an alpha version. See the Man-page and the TODO-file for details.

Please report all bugs to the bug tracker on the project web site.

Anonymous SVN (Subversion) Access

This project's BerliOS Developer SVN repository can be checked out through anonymous (svnserve) SVN. The following instruction will checkout all source files to a new directory nntpspider:

svn checkout svn://svn.berlios.de/nntpspider/trunc nntpspider

More information

can be found in the man page, the README and the TODO.

Roadmap

See the TODO-file for details on planned features and for which version they are sheduled.

Acknowledgments

Writing this tool was inspired by the windows based NewsWolf

Author

Michael Mayer: mmayer (at) users (dot) berlios (dot) de