HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: receive-header called multiple times
Author: Lars Clausen
Date: 02/06/2004 10:16
 
It makes two HEAD calls and then a GET.  This only happens
when it's a redirect from www.tv2.dk and -u2 is turned on. 
Does checking the file type cause another download?  That
sounds strange... 

-Lars

[pc770 HTTrackCrawls]$ cat hts-ioinfo.txt 
[0] request for www.tv2.dk/:
<<< HEAD / HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: www.tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9


[0] response for www.tv2.dk/:
code=301
>>> HTTP/1.0 301 Moved Permanently
>>> Date: Fri, 06 Feb 2004 09:09:48 GMT
>>> Server: Apache/1.3.26 (Unix)  (Red-Hat/Linux) PHP/4.3.1
>>> Location: <http://tv2.dk/>
>>> Content-Type: text/html; charset=iso-8859-1
>>> Age: 20
>>> X-Cache: HIT from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive


[1] request for tv2.dk/:
<<< HEAD / HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9


[1] response for tv2.dk/:
code=200
>>> HTTP/1.0 200 OK
>>> Date: Fri, 06 Feb 2004 09:09:56 GMT
>>> Server: Apache/1.3.26 (Unix)  (Red-Hat/Linux) PHP/4.3.1
>>> X-Powered-By: PHP/4.3.1
>>> Cache-Control: public, max-age=60, s-maxage=60
>>> Content-Length: 45482
>>> Content-Type: text/html
>>> Age: 12
>>> X-Cache: HIT from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive


[2] request for tv2.dk/:
<<< HEAD / HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9


[2] response for tv2.dk/:
code=200
>>> HTTP/1.0 200 OK
>>> Date: Fri, 06 Feb 2004 09:09:56 GMT
>>> Server: Apache/1.3.26 (Unix)  (Red-Hat/Linux) PHP/4.3.1
>>> X-Powered-By: PHP/4.3.1
>>> Cache-Control: public, max-age=60, s-maxage=60
>>> Content-Length: 45482
>>> Content-Type: text/html
>>> Age: 12
>>> X-Cache: HIT from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive


[3] request for tv2.dk/robots.txt:
<<< GET /robots.txt HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9


[4] request for tv2.dk/:
<<< GET / HTTP/1.1
<<< Connection: Keep-Alive
<<< Host: tv2.dk
<<< User-Agent: HTTrack 3.30.92 (non-archiving test version,
see www.netarkivet.dk/website/info.html)
<<< Accept: image/png, image/jpeg, image/pjpeg,
image/x-xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9,
utf-8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9


[3] response for tv2.dk/robots.txt:
code=404
>>> HTTP/1.0 404 Not Found
>>> Date: Fri, 06 Feb 2004 09:10:08 GMT
>>> Server: Apache/1.3.26 (Unix)  (Red-Hat/Linux) PHP/4.3.1
>>> X-Powered-By: PHP/4.3.1
>>> Pragma: no-cache
>>> Cache-Control: no-cache
>>> Last-Modified: Sun, 06 Nov 1994 08:49:37 GMT
>>> Content-Length: 2265
>>> Content-Type: text/html
>>> X-Cache: MISS from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive


[4] response for tv2.dk/:
code=200
>>> HTTP/1.0 200 OK
>>> Date: Fri, 06 Feb 2004 09:09:56 GMT
>>> Server: Apache/1.3.26 (Unix)  (Red-Hat/Linux) PHP/4.3.1
>>> X-Powered-By: PHP/4.3.1
>>> Cache-Control: public, max-age=60, s-maxage=60
>>> Content-Length: 45482
>>> Content-Type: text/html
>>> Age: 12
>>> X-Cache: HIT from balnode1.webdmz.tv2.dk
>>> Connection: keep-alive
 
Reply Create subthread


All articles

Subject Author Date
receive-header called multiple times

02/05/2004 15:29
Re: receive-header called multiple times

02/05/2004 23:32
Re: receive-header called multiple times

02/06/2004 10:16




2

Created with FORUM 2.0.11