Oops.
Nevermind. I found a bug in the sites PHP code, that had no effect on online
browsing, but affeced the spider...