| > plz tell me how to down load .asp files .... and yes i
did
> add +'.asp'
Try:
+*.asp
> in the options and here is the site iwant to
> copy w3schools.com and what is this about the robot.txt
> rules something which occuring in my log
The site is then forbidding the capture of page(s) to
crawlers.
If you want to download it, ensure that you have
reasonnable settings (such as bandwidth limit, 2
simultaneous connections), and disable robots.txt rules
(Set options / Spider / Spider)
But, AGAIN: setup REASONNABLE bandwidth settings!
| |