HTTrack Website Copier
Free software offline browser - FORUM
Subject: Suggestion - httrack dynamic url capture
Author: Rasman
Date: 01/12/2006 06:52
 
I have a problem common to a number of people using sites which require
authentication.

After using the url capture feature (which is a fantastic feature) Changing
cookie values stuff further spidering and subsequent pages are redirected to
the homepage or an error page.

First some questions:
What stops httrack getting the changing cookie values? is it just not coded to
do this?Is there any reason why httrack couldn't update the cookie values?How
does a browser deal with the cookie changes?
now some ideas:

The URL capture facility involves configuring the browser to use httrack as a
proxy. 
httrack doesn't actualy act like a proxy but instead just captures the
request. 

What if httrack forwarded the request and handled any responses? 

I dont have a deep enough understanding of http, httrack or cookies, however
it makes sense that httrack could then examine all data coming in and going
out, and just as with the "url capture" feature, could update the cookie and
continue to the next link.

Maybe httrack could then only do one link at a time, but thats better than
nothing. Its also possible the httrack spidering algorythem might need to be
re-coded to operate in this manner - but a simple progress-regress algorythem
(which I suspect httrack might already use) would do the trick.

so to conclude - I think this would be a valuable feature which shouldn't be
too difficult to implement. I would be happy to discuss algorythem and coding
changes to the specific modules if this would help the development process.

So far a fantastic tool - I hope I can help you improve it.
 
Reply


All articles

Subject Author Date
Suggestion - httrack dynamic url capture

01/12/2006 06:52
Re: Suggestion - httrack dynamic url capture

01/12/2006 06:56




8

Created with FORUM 2.0.11