> Okay it seems, you are not actually looking deeply
> into the SET OPTION after entering the URL in the
> Box.
> 
> Click on set option -> Go to Spider ->Do Not Follow
> Robots.txt 
> 
> Done.
> 
> Hope it should be easy and helpful.

Unable to find 'set option'