HTTrack Website Copier
Free software offline browser - FORUM
Subject: Robots.txt
Author: Janus
Date: 12/30/2005 10:40
I try to mirror a site (and yes, I have permission of the owner!)

In the past there were no problems. Now WinHTTrack stops after a 4 files and
nothing is saved??
I think it has something to do with "robots.txt"

How can I mirror this site ??
Thank you very much for your help!

All articles

Subject Author Date

12/30/2005 10:40
Re: Robots.txt

12/30/2005 10:46
Re: Robots.txt

12/30/2005 15:18
Re: Robots.txt

12/30/2005 15:21
Re: Robots.txt

01/21/2014 21:10


Created with FORUM 2.0.11