| Hello,
I am trying to rip some mobile advertisement landing pages. These guys are
sneaky, if they detect httrack, they will not show you the landing page
anymore. I am getting an error that is mentioning something about robots.txt.
How can I prevent them from seeing me as Httrack. I am already using a user
agent of a mobile phone. | |