perl-WWW-Robotrules http://search.cpan.org/~gaas/WWW-RobotRules-6.02/ Osman Erkan osman.erkan@pisilinux.org Artistic GPLv2 programming.language.perl app:console

database of robots.txt-derived permissions

This module parses /robots.txt files as specified in "A Standard for Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html> Webmasters can use the /robots.txt file to forbid conforming robots from accessing parts of their web site. http://search.cpan.org/CPAN/authors/id/G/GA/GAAS/WWW-RobotRules-6.02.tar.gz perl perl-URI perl-WWW-Robotrules perl perl-URI /usr/lib /usr/share/doc /usr/share/man /usr/share 2018-07-26 6.02 Rebuild for new toolchain Pisi Linux Community admin@pisilinux.org 2017-01-24 6.02 Rebuild for new toolchain Pisi Linux Community admin@pisilinux.org 2016-06-09 6.02 Release Bump Pisi Linux Community admin@pisilinux.org 2016-03-26 6.02 First release Alihan Öztürk alihan@pisilinux.org