HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: How to completely ignore 'robots.txt'?
Author: marty
Date: 02/19/2013 19:52
 
please share with me the solutions to this question by email.
i need to find some old info archived on the net for real and important
research. these days the government and global corporations are blocking
everything such as news articles and websites that was archived so people
cannot find info to defend themselves or prove the government or corporations
wrong. robots.txt needs to be bypassed for this obvious logical reason. please
send your replies how to bypass robots.txt by sending replies to
martdot7@aol.com  thank you for your kind assistance.
 
Reply Create subthread


All articles

Subject Author Date
How to completely ignore 'robots.txt'?

11/30/2001 11:41
Re: How to completely ignore 'robots.txt'?

11/30/2001 17:43
Re: How to completely ignore 'robots.txt'?

10/03/2010 02:01
Re: How to completely ignore 'robots.txt'?

02/19/2013 19:52
Re: How to completely ignore 'robots.txt'?

01/11/2015 22:18
Re: How to completely ignore 'robots.txt'?

03/18/2015 15:10
Re: How to completely ignore 'robots.txt'?

05/23/2015 22:13
Re: How to completely ignore 'robots.txt'?

05/25/2017 03:56
Re: How to completely ignore 'robots.txt'?

01/16/2020 19:36
Re: How to completely ignore 'robots.txt'?

04/21/2023 23:47




5

Created with FORUM 2.0.11