> However, I still get only the site's home page and
> the single PDF offered on it, with no other
> content.

If you do not download html files where links are referenced, you won't catch
anything ; ie. you should also allow html pages where the files are found.