I believe that the website is using the robot.txt file to block agents from
scraping, I am facing the same issue here but I am not sure that it is the
reason.