HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Need helping copying www.howstuffworks.com
Author: Bjarne Andersen
Date: 04/27/2004 19:40
 
It looks like that websites uses a lot of javascript.
No webcrawler like that ! - if you look at the source code 
for the frontpage it includes javascript from 2 external 
files: spider.php and main.js - my guess is that the first 
checks wheter you look like a spider - the second does a 
lot of javascripting.
The first step for you is to ensure that you do not look 
like a webcrawler - so set useragent to be some kind of 
mozilla.
If the javascript in main.js is important for navigation 
on the page and if HtTrack cannot parse it for links 
(which it possibly cannot) this website cannot be copied.
 
Reply Create subthread


All articles

Subject Author Date
Need helping copying www.howstuffworks.com

04/27/2004 11:41
Re: Need helping copying www.howstuffworks.com

04/27/2004 19:40
Re: Need helping copying www.howstuffworks.com

05/04/2004 07:15




3

Created with FORUM 2.0.11