Oops. Nevermind. I found a bug in the sites PHP code, that had no effect on online browsing, but affeced the spider...
Created with FORUM 2.0.11