HTTrack Website Copier
Free software offline browser - FORUM
Subject: a local copy of robots.txt is not saved/kept
Author: chris
Date: 04/11/2008 06:41
 
How do I get HTTrack to save the copy of the robots.txt file it downloads.  If
I'm mirroring a site, I want the public files including the robots.txt.  Is
there an option I can use to change the this behavior so the robots.txt will
be saved locally?
Thanks!
 
Reply


All articles

Subject Author Date
a local copy of robots.txt is not saved/kept

04/11/2008 06:41
Re: a local copy of robots.txt is not saved/kept

04/11/2008 06:49




2

Created with FORUM 2.0.11