I have a dynamic CGI driven site I want to create a static HTML copy of. The
bulk of the data is images. I can easily copy the image directories to the
static copy myself. However when I run the crawl even if I exclude the image
directories such as -*/images/* it still downloads all the images. The only
way around it is to rename the image directories so all the images are 404 not
found but then that means the site is broken while I do the crawl and it's a
big site.
|