| Hi Guys,
I noticed HTTrack obeys robots.txt but I have a doubt.
I need HTTrack doens't obey this file to mirror a folder included in
robots.txt and I tried the option "include links" in Options, Scan Rules
without success.
It's possible configure HTTtrack to skip robots.txt?
Thanks in advance
Mike | |