Search engines like google and yahoo will likely not know how to proceed While using the URL . It truly is unclear to them whether they're allowed to obtain. When directives usually are not crystal clear to Google, they're going to go Using the minimum restrictive directive, which in this https://rankingseotoolz.com/robots-txt-generator