| Subject: Re: Too many robots.txt |
Author: gatekat |
Date: 07/23/2008 11:41 |
| | I'm having the same issue. It's trying to download the entire web. I have
'maximum external depth' set to 0, 'get non-HTML files related to a link'
unselected (even though I'd like it on), 'No external pages' selected, and
'Global travel mode' set to 'Stay on the same domain'. What do I need to
change to keep it on one domain? | |
|
|
|
|
1
Created with FORUM 2.0.11
