| | It looks like "Cache-control" is set to "private" in the
responses from my website. Is this being interpreted by
HTTrack as a non-cachable piece of content and therefore
it doesn't issue the If-modified-since header?
Below is a Log extract where:
* localhost:99 is my proxy
* this is the second run with "-C2" Httrack setting
* It looks like there are no If-modified-since headers
issued
Thanks for your help with this,
Mark
====Log starts====
[0] request for localhost:99/robots.txt:
<<< GET /robots.txt HTTP/1.1
<<< Cookie: $Version=1;
ASPSESSIONIDCCDTRBSA=DPMMPEMACKIBMKPMNHNCMKJC; $Path=/
<<< Connection: Keep-Alive
<<< Host: localhost:99
<<< User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows
NT 5.1; .NET CLR 1.1.4322)
<<< Accept: image/png, image/jpeg, image/pjpeg, image/x-
xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9, utf-
8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9
[1] request for localhost:99/:
<<< GET / HTTP/1.0
<<< Cookie: $Version=1;
ASPSESSIONIDCCDTRBSA=DPMMPEMACKIBMKPMNHNCMKJC; $Path=/
<<< Connection: close
<<< Host: localhost:99
<<< User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows
NT 5.1; .NET CLR 1.1.4322)
<<< Accept: image/png, image/jpeg, image/pjpeg, image/x-
xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9, utf-
8;q=0.66, *;q=0.33
[0] response for localhost:99/robots.txt:
code=302
>>> HTTP/1.1 302 Object moved
>>> Server: Microsoft-IIS/5.0
>>> Date: Mon, 11 Oct 2004 08:49:46 GMT
>>> Location: ../../index.asp
>>> Content-Length: 121
>>> Content-Type: text/html
>>> Cache-control: private
[1] response for localhost:99/:
code=200
>>> HTTP/1.1 200 OK
>>> Server: Microsoft-IIS/5.0
>>> Date: Mon, 11 Oct 2004 08:49:47 GMT
>>> Connection: Keep-Alive
>>> Content-Length: 12480
>>> Content-Type: text/html
>>> Cache-control: private
[1] request for localhost:99/index.asp?pageid=2:
<<< HEAD /index.asp?pageid=2 HTTP/1.1
<<< Referer: <http://localhost:99/>
<<< Cookie: $Version=1;
ASPSESSIONIDCCDTRBSA=DPMMPEMACKIBMKPMNHNCMKJC; $Path=/
<<< Connection: Keep-Alive
<<< Host: localhost:99
<<< User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows
NT 5.1; .NET CLR 1.1.4322)
<<< Accept: image/png, image/jpeg, image/pjpeg, image/x-
xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9, utf-
8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9
[2] request for localhost:99/index.asp?pageid=3:
<<< HEAD /index.asp?pageid=3 HTTP/1.1
<<< Referer: <http://localhost:99/>
<<< Cookie: $Version=1;
ASPSESSIONIDCCDTRBSA=DPMMPEMACKIBMKPMNHNCMKJC; $Path=/
<<< Connection: Keep-Alive
<<< Host: localhost:99
<<< User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows
NT 5.1; .NET CLR 1.1.4322)
<<< Accept: image/png, image/jpeg, image/pjpeg, image/x-
xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9, utf-
8;q=0.66, *;q=0.33
<<< Accept-Encoding: gzip, identity;q=0.9
[3] request for localhost:99/index.asp?pageid=2:
<<< GET /index.asp?pageid=2 HTTP/1.0
<<< Referer: <http://localhost:99/>
<<< Cookie: $Version=1;
ASPSESSIONIDCCDTRBSA=DPMMPEMACKIBMKPMNHNCMKJC; $Path=/
<<< Connection: close
<<< Host: localhost:99
<<< User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows
NT 5.1; .NET CLR 1.1.4322)
<<< Accept: image/png, image/jpeg, image/pjpeg, image/x-
xbitmap, image/svg+xml, image/gif;q=0.9, */*;q=0.1
<<< Accept-Language: en, *
<<< Accept-Charset: iso-8859-1, iso-8859-*;q=0.9, utf-
8;q=0.66, *;q=0.33
[2] response for localhost:99/index.asp?pageid=3:
code=200
>>> HTTP/1.1 200 OK
>>> Server: Microsoft-IIS/5.0
>>> Date: Mon, 11 Oct 2004 08:49:49 GMT
>>> Content-Length: 16771
>>> Content-Type: text/html
>>> Cache-control: private
| |
