| Subject: Robot.txt what is it about |
Author: jacob |
Date: 04/08/2007 01:26 |
| | i have tried to download a website and whenever i try it it finishes way too
quickly, i am unable to download pdf files and zip files. if i set up the
software to ignore will it download them | |
|
|
|
|
3
Created with FORUM 2.0.11
