| Subject: Re: crawl single folder but save all external images |
Author: WHRoeder |
Date: 12/24/2012 03:16 |
| | > So if I remove all +filters (+*.png +*.gif +*.jpg
> +*.css +*.js -ad.doubleclick.net/*
> -mime:application/foobar) it should stop crawling
> other website unnecessarily? When I had looked at
You need to learn to read. I said they were unnecessary. I didn't say they
would change anything.
> the progress in the morning it was crawling a
> completely different domain. I'm going to try again
Based on what you've posted it shouldn't be.
| |
|
|
|
|
7
Created with FORUM 2.0.11