Subject: Re: Specific pages not being copied (the ones I want)
Author: Christopher
Date: 10/17/2011 12:40
I am including the robot.txt file here. This is what it has:
User-agent: *
Disallow: /cgi-bin/
Disallow: /stats/
Disallow: /webstats/
Disallow: /scripts/
