| Bill,
Would you be so kind as to lend your expert eye to the latest iteration of our
htrackrc file?
These are the "base" settings for the 1,400 sites we mirror. We add
site-specific settings to the command line.
Anything here catch your eye as odd or useless?
Thanks!
-- Cliff
set footer "<-- This file mirrored for eGranary Digital Library [from host %s
[file %s [at %s]]] -->"
set user-agent "Mozilla [en] eGranary Digital Library"
set do-not-log
set cache=2
set test
set extended-parsing
set near on
set robots=0
set check-type=1
set store-all-in-cache
set updatehack
set urlhack on
set structure=0
set search-index=0
set priority=3
set can-go-down
set stay-on-same-address
set keep-links=0
set host-control=0
set index=0
set keep-alive
set single-log
set max-rate=25000
set assume=cgi,php,php3,asp=text/html
set httpproxy-ftp=0
set advanced-flushlogs
set quiet
set purge-old
set advanced-maxlinks=5000000 | |