With
  <http://127.0.0.1/site/index.php>
  DocumentRoot "d:/site"
  "d:/site/robots.txt", 
I could mirror the local site,

but there was still a path problem as the URL should have
been <http://127.0.0.1/index.php>

I tested with this URL and the file robots.txt was correctly
read and followed

I still don't understand why I can enter
<http://127.0.0.1/site/index.php> or
<http://127.0.0.1/index.php> to mirror the site

I will have to modify the DocumentRoot when mirroring
another local site

Thanks for your help

Dan