| | It makes two HEAD calls and then a GET. This only happens
when it's a redirect from www.tv2.dk and -u2 is turned on.
Does checking the file type cause another download? That
sounds strange...
-Lars
[pc770 HTTrackCrawls]$ cat hts-ioinfo.txt
[0] request for www.tv2.dk/:
<<< HEAD / HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: www.tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9
[0] response for www.tv2.dk/:
code=301
>>> HTTP/1.0 301 Moved Permanently
>>> Date: Fri, 06 Feb 2004 09:09:48 GMT
>>> Server: Apache/1.3.26 (Unix) (Red-Hat/Linux) PHP/4.3.1
>>> Location: <http://tv2.dk/>
>>> Content-Type: text/html; charset=iso-8859-1
>>> Age: 20
>>> X-Cache: HIT from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive
[1] request for tv2.dk/:
<<< HEAD / HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9
[1] response for tv2.dk/:
code=200
>>> HTTP/1.0 200 OK
>>> Date: Fri, 06 Feb 2004 09:09:56 GMT
>>> Server: Apache/1.3.26 (Unix) (Red-Hat/Linux) PHP/4.3.1
>>> X-Powered-By: PHP/4.3.1
>>> Cache-Control: public, max-age=60, s-maxage=60
>>> Content-Length: 45482
>>> Content-Type: text/html
>>> Age: 12
>>> X-Cache: HIT from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive
[2] request for tv2.dk/:
<<< HEAD / HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9
[2] response for tv2.dk/:
code=200
>>> HTTP/1.0 200 OK
>>> Date: Fri, 06 Feb 2004 09:09:56 GMT
>>> Server: Apache/1.3.26 (Unix) (Red-Hat/Linux) PHP/4.3.1
>>> X-Powered-By: PHP/4.3.1
>>> Cache-Control: public, max-age=60, s-maxage=60
>>> Content-Length: 45482
>>> Content-Type: text/html
>>> Age: 12
>>> X-Cache: HIT from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive
[3] request for tv2.dk/robots.txt:
<<< GET /robots.txt HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9
[4] request for tv2.dk/:
<<< GET / HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9
[3] response for tv2.dk/robots.txt:
code=404
>>> HTTP/1.0 404 Not Found
>>> Date: Fri, 06 Feb 2004 09:10:08 GMT
>>> Server: Apache/1.3.26 (Unix) (Red-Hat/Linux) PHP/4.3.1
>>> X-Powered-By: PHP/4.3.1
>>> Pragma: no-cache
>>> Cache-Control: no-cache
>>> Last-Modified: Sun, 06 Nov 1994 08:49:37 GMT
>>> Content-Length: 2265
>>> Content-Type: text/html
>>> X-Cache: MISS from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive
[4] response for tv2.dk/:
code=200
>>> HTTP/1.0 200 OK
>>> Date: Fri, 06 Feb 2004 09:09:56 GMT
>>> Server: Apache/1.3.26 (Unix) (Red-Hat/Linux) PHP/4.3.1
>>> X-Powered-By: PHP/4.3.1
>>> Cache-Control: public, max-age=60, s-maxage=60
>>> Content-Length: 45482
>>> Content-Type: text/html
>>> Age: 12
>>> X-Cache: HIT from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive
| |
