HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Banning robots
Author: Xavier Roche
Date: 10/30/2003 11:47
> What a pity !
> now it seems to be providers too
> (instead of offering a faster or better service which would
> stop a lot of mirrors by people who hate slow connections).

And it is also bad for webmasters: many og them just can not
backup their contents anymore in case of problems (backup
crash, for example)
The feature to change the Browser ID is then useful, even if
the inital goal was to capture browser-dependant sites (such
 as IE-DHTML or Netscape-DHTML depending on the User-agent
string sent by the remote client)

> What will be the next step to reduce bandwidth usage ?
Hum, a full-javascript site, working with only the latest
Internet Explorer version? I already saw few of them ..

Reply Create subthread

All articles

Subject Author Date
Error: "Bad Request" (400) at link

10/29/2003 08:55
Re: Error: "Bad Request" (400) at link

10/29/2003 19:36
Banning robots

10/30/2003 10:31
Re: Banning robots

10/30/2003 11:47
Re: Error: "Bad Request" (400) at link

11/02/2003 22:49


Created with FORUM 2.0.11