How do I get HTTrack to save the copy of the robots.txt file it downloads. If
I'm mirroring a site, I want the public files including the robots.txt. Is
there an option I can use to change the this behavior so the robots.txt will
be saved locally?
Thanks! |