| Hi,
Thank you for your answer.
I'm sorry to answer so late but apparently
I had lost the thread :) !
>> I'd guess proxy and httpS doesn't mix.
It looks like you are right:
- I can capture a website which URL has "HTTP"
for protocol,
- I can't capture a site which URL has "HTTPS"
for protocol.
>> Also the site has robots.txt (deny all)
>> you'll have to override that also.
As I can capture a website which URL has "HTTP"
for protocol, I think there is no problem with
the website's "robots.txt". Am I wrong?
Thank you for your help.
Sincerely,
--
Lmhelp | |