If the articles are in the /assetRoot/ directory on this site, here is the
reason:  (www.direct.gov.uk robots.txt file)

User-agent: UKOnline
Disallow:

User-agent: *
Disallow: /assetRoot/

You need to Mirror, Modify Options and click the Spider tab.  in the
robots.txt combo box, select "no robots.txt rules"