Letter P

perl-WWW-RobotRules - Database of robots.txt-derived permissions

Website: http://search.cpan.org/dist/WWW-RobotRules/
License: GPL+ or Artistic
Vendor: Scientific Linux
Description:
This module parses /robots.txt files as specified in "A Standard for Robot
Exclusion", at <http://www.robotstxt.org/wc/norobots.html>. Webmasters can
use the /robots.txt file to forbid conforming robots from accessing parts
of their web site.

Packages

perl-WWW-RobotRules-6.02-5.el7.noarch [17 KiB] Changelog by Daniel Mach (2013-12-27):
- Mass rebuild 2013-12-27

Listing created by Repoview-0.6.6-1.el6

Rabisu Mirror Service We provide mirrors to support Open source communities. Our mirror server is located in Istanbul/Turkey region.

Please do not hesitate to contact mirror@rabisu.com for new open source mirror submissions.