> Creating files like this would be very easy in a unix
> shell with some shell programming/related tools
> knowledge, how to do it in Windows, no idea. Basic links
> could even be created in DOS with some simple .BATting,
> or even using BASIC :-)

On Windows, just install cygwin tools (bash, grep..)

But the main problem with an exhaustive link scan is that 
it tends to HAMMER the website (and website logs)