HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Can no longer download (some) websites.
Author: danday3953
Date: 12/02/2008 11:00
Hi, I think you helped me as well with your question.  Perhaps you should view
the warning and error messages.  I would think this is similar to what I've
been studying.  Your post may even be a bit abnormal considering I obtained
similar results and posted here only a few minutes before... weird!

Although I know very little about IT specifics, I think I'm getting the
basics+.  Upon my completion, or should I say failure of a scan, I checked
tosee if I was BL'd at any of the well-known sites.  Nope! This could infact
take a few hours for me to query the NS for a poor rating,
but I think it's more likely that I've attempted to spoof my user-agent.

This spoof may trigger a deny to occur if the UA was BL'd!  What if we tried
to decrease the threads and increase the TTL/WAIT time, while using a browser

It may also help increase our chances of success if we can find a proxy that
will work for our attempts.  But can you proxy this crawler even if you could
mask your IP address via proxy?
This is a good RE: to get us both started I think.

P.S. It's also funny that I was crawling a media site as well!  Perhaps
coincidence... But I'll find the cash_icon.jpg or 138117.jpg first! Muhahahaha

Reply Create subthread

All articles

Subject Author Date
Can no longer download (some) websites.

12/02/2008 09:56
Re: Can no longer download (some) websites.

12/02/2008 11:00
Re: Can no longer download (some) websites.

12/03/2008 10:27


Created with FORUM 2.0.11