| Is there some way to have a max depth that is less strict than mirror depth but
more strict than external depth?
I have multiple pages, like webpage.com/favorites/(page#), that has lists of
links I would like to download. Conveniently, each of these links follows a
very similar format: webpage.com/content/*, but often these pages will link to
other pages of the same format, which in turn link to more
webpage.com/content/* links. The two possible approaches I can think of are
disallowing webpage.com/content/*, but allowing an external depth of 2 or so,
or allowing webpage.com/content*, but only allowing a mirror depth slightly
greater than the number of pages in favorites. However, both of these results
in more scanning than I would like. Is there any way to narrow down the number
of pages scanned in my situation?
Basically I would like to download everything of the format *content* that
*favorites* links to, but nothing of the format *content* that any *content*
links to.
Thanks in advance for the help. | |