Learn Something About Robots.txt In An Easy Way - Testing and Verification | Part 2/2
SEO Episode 30:
I explained in the first part in the last seo episode a lot about robots.txt by showing examples like Nike.com that has a logo at the end because # means nothing will be counted after #. I also explained almost all the key elements Allow: Disallow: User-agent etc as well as the disadvantage being not used robots.txt for the secret information on your site - the example of NTS site, students data leaked.
How to Check A Specific File is Being Blocked By Robots.txt Or Not?
Matlab Agr Robots.txt Block Kar Rha Hai to Google Engine Crawl Nhi Kary Ga Aur Phr Search Result Main Show Bhi Nhi Hoga.
First of all, go to Webmaster Tool for Robot.txt Testing