| > Could it be robots.txt?This is what i found in my
> file log.
Ah yes, now that you've posted the log it's obvious (to me). The /media/
folder is being blocked and that contains a lot of the images and PDFs.
Turn off robots.txt with Options >> Spider >> Spider: "No robots.txt rules"
but please be kind to their server by setting limits (speed, connections) like
I mentioned previously.
| |