October 2007
Robots.txt Generator - McAnerin International Inc.
by lukeslytalker & 3 othersGood Robot.txt generator
October 2005
Robots.txt Generator | mcanerin.com
by simon_bricolo & 2 othersCreate Robots.txt to allow/disallow search crawlers in certain directories on your site
1
(2 marks)