> Trying to download the site www.erowid.org
> Here are my Scan Rules -* -*www.erowid.org_/*
> +*www.erowid.org/*
Don't tell us what you think you did, post the actual command line used (log
file line 2 to doit.log) as you did in the subsequent post.

The default is to download from the starting site only so your -* +*site/*
does nothing
the -*site_/* also does nothing as there is no such site

> After some time, there are Warning, and Error in
> hts-log.txt
> Warning: file not stored in cache due to bogus state
> (incomplete type):
> Error: "Unknown (not HTTP/xx) response structure"
> (-1)
Sounds like you crashed the server - reduce the connections/sec to one.

> Warning: 	file not stored in cache due to bogus
> state (incomplete type):
> www.erowid.org/cgi-bin/messages/message_view_record.
> php?message_id=117&page_url=/plants/coffee/coffee.sh
> tml
Especially since you overrode robots.txt which specifically excluded cgi-bin.