HTTrack Website Copier
Free software offline browser - FORUM
Subject: robots.txt
Author: twj
Date: 10/05/2015 18:06
 
I have successfully used this product for many, many year until today.

I have never seen this before:

2:04:01 Warning:  Note: due to pcglobal.ca remote robots.txt rules, links
beginning with these path will be forbidden: / (see in the options to disable
this)
12:04:01 Warning:  Link pcglobal.ca/ not scanned (follow robots meta tag)

Any ideas would be greatly appreciated.

Thanks

Twj
 
Reply


All articles

Subject Author Date
robots.txt

10/05/2015 18:06
Re: robots.txt

10/08/2015 00:22
Re: robots.txt

11/13/2015 19:11
Re: robots.txt

11/13/2015 19:16




a

Created with FORUM 2.0.11