| Subject: Duplicate Images |
Author: Mike Schinkel |
Date: 07/12/2007 16:39 |
| | Hi:
I'm getting tens of thousands of duplicate images because the website has
chosen for some reason (log analysis maybe?) to include a query string on
their images, i.e.:
example.com/foo.gif?x=000000
example.com/foo.gif?x=000001
example.com/foo.gif?x=000002
,,,
example.com/foo.gif?x=999999
Is it possible to generically say that if an image file has a parameter that
you only download one? It seems like this would need to be a heuristic
optimization that would need to be built into HTTrack... | |
|
|
|
|
6
Created with FORUM 2.0.11