Learn Something About Robots.txt In An Easy Way - Testing and Verification | Part 2/2
SEO Episode 30:
I explained in the first part in the last seo episode a lot about robots.txt by showing examples like Nike.com that has a logo at the end because # means nothing will be counted after #. I also explained almost all the key elements Allow: Disallow: User-agent etc as well as the disadvantage being not used robots.txt for the secret information on your site - the example of NTS site, students data leaked.
How to Check A Specific File is Being Blocked By Robots.txt Or Not?
Matlab Agr Robots.txt Block Kar Rha Hai to Google Engine Crawl Nhi Kary Ga Aur Phr Search Result Main Show Bhi Nhi Hoga.
Ab Suppose Apki Site Par Ek JPG Hai (Ya PDF) Jo aap Chahty Han Block Ho Jye Jo Google Search Main Show Na Ho. Is Ka Direct Link ye Hoga
To Aap Is Ko esy Likhy Gy Robots.txt Main
Disallow: /wp-content/uploads/2016/12/Seo-Episodes.jpg
Jis Ka Look Esy Hoga Search Console main, See Screenshot Here.
Ab Check Karny Ky Liye Yahan Paste Karain Link, Then Aap Dekhain Gy Aapko Show Hoga "Blocked" Matlab Google Is File Ko Crawl Nhi Kary Ga Aur Na Hi Search Result Main Show Kary Ga. See here.
Agr Allow A Jye to Matlab Aap Ky Code Main Kye Mistake Hye Hai Jesy Mery Sy ye Hye Thi Ky Main yahan \\ Lga Diya Tha To Is Ki waja Sy Block Nhi Howa Aur Allow Ho Gya. See here.
Isi Tarha Aap Ek Directory Ko Block Kary Then Check Kary Asal Main Wo File Ya Directory Block Hye Bhi Ya Nhi Ye Mainy Wesy Hi Likh Di. Ye Ek Tester Hai Jo Bta Dy Ga. Phr Aap Apni Site Ky Robots.txt Editor Main Ja Kr Changes Kar Lain Ya Direct Hosting Ki File Manager Main Change Kar Lain Then Yahan A Kar Submit Bhi Kar Dain Ta Ky Google Ko Pta Chal Jye Aap Ny Changes Ki Hai. See here
If you haven't read its first part, then you will not get anything. Read that for the best understanding.
Note:
Sometimes Google shows result even you include the file in Robots.txt; the reason is that when a website gives a link (backlink) to the blocked file, Google crawls that and index. So, don't worry, if it's getting indexed. You can make use of Google Webmaster Tools in order to remove the result from the Google Search.
Sometimes Google shows result even you include the file in Robots.txt; the reason is that when a website gives a link (backlink) to the blocked file, Google crawls that and index. So, don't worry, if it's getting indexed. You can make use of Google Webmaster Tools in order to remove the result from the Google Search.
---
Have You Missed the Previous 30 SEO Episodes and Videos?
--
Then Go To Learn Something About Robots.txt In An Easy Way | Part 1/2
Have You Missed the Previous 30 SEO Episodes and Videos?
--
Then Go To Learn Something About Robots.txt In An Easy Way | Part 1/2
#SEOEPISODE30
Written By: Muhammad Aamir Mursleen