| I am using this program to download short stories off of a website. it has
different categories and the links are numbered 1-40 in the link structure. so
it's like "stories/stories_by_category.php?category=5"
but the stories all come up under "stories/showstory.php?id=12345"
while scanning the story pages there are links to similar stories, but i don't
want to go to those pages. i placed filters to block the categories i don't
want, but it still follows the similar stories links. a lot of the stories are
several pages long, and i want the whole thing to read, but it crawls all the
other stories because they all have the same "showstory.php?id=12345" but no
category filter of any sort. the pages with multi page stories have links to
other pages for the story like "/stories/showstory.php?id=12345&page=2"
so how do i block the other pages on the story page except for the links with
the &page=2 in the link, while getting all the stories in the category i
choose? | |