Subject: Re: Using Options to disable remote robots.txt ruls
Author: SV
Date: 11/26/2017 14:03
Okay it seems, you are not actually looking deeply into the SET OPTION after
entering the URL in the Box.
Click on set option -> Go to Spider ->Do Not Follow Robots.txt
Done.
Hope it should be easy and helpful.
