A Blog where you can get Informative Guides Based on WordPress, Blogger, SEO, and Google Adsense tips & tricks.
Get a high ranking on Google with your Blogspot blog. How? Just Configure the Robots.txt, Meta Tags, Custom redirect, and Custom Robot header tag. It’s easy and interesting to understand!
These configurations of Custom Robot header tags, Robots.txt, Custom redirects & description meta tag of your blog in the right way can help your blog to increase your search engine visibility and also, can lead to de-indexing your blog from Search Engine if you misconfigured it. So, I will recommend you to please be serious and more careful while reading this Guide. I am not scaring you but warning you that what the wrong configuration of Meta tags & Robots.txt can lead.
For Setting up Crawlers and Indexing in Blogger | Robots.txt, Meta tag & More follow these steps:
*Login to your Blogspot dashboard
*Go to Setting
*Now, go to search preferences section
Table of Contents
This is where you can write an informative short description about your blog in 150 words. Writing better and enough Good description can lead you better search engine ranking and clicks. But don’t forget to add your main keywords. For example Wikiijunction – A Blog where you can get Informative Guides Based on WordPress, Blogger, SEO, and Google Adsense tips & tricks.
In my Meta description, I have told about what my site is about and included my main keywords on which I want to rank. Don’t exceed your meta description more than 150 words as the crawler bots will ignore characters after 150 words.
After adding meta description the crawlers will crawl your blog and shows up the description on the search engine result page.
Robots.txt is a file which contains some lines of code which give proper instruction to the crawler to how to index, which part of your blog not to index or block crawler bots from some of the pages of crawl your blog. For example, I do not want to crawl some of the pages or posts such as archive, tags, etc so I will add
disallow: /URL in the robots.txt file.
Don’t add or edit the robotx.txt file if you don’t know. Otherwise, it can lead to de-indexing of your blog. Crawlers read the robots.txt file before crawling any blog or website.
This video made by Ex-Google Engineer, Matt Cutts will give you enough basic information about Blogger robots.txt file:
User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search Allow: / Sitemap: http://example.blogspot.com/sitemap.xml
Don’t know what is sitemap and how to create it? Read this post “What is SEO and How it works? | Google SEO Guide For Beginners” to create your blog sitemap.
*[User-agent: *]: –This code is for all bots as it is marked with (*).
*[User-agent: Mediapartners-Google]: –This code is for the Google partners such as Google Adsense. If you don’t have the Google Adsense on your blog then leave it.
*[Disallow: /search]: – This code means not to index the links which contain “/search” term. For example https://www.wikiijunction.com/search/label/wordpress. If we remove this the crawlers will index all the links.
*[Allow: / ]: – This code is for allowing all bots to crawler the homepage of your site.
*[sitemap: ]: – This code is for referring bots the sitemaps of your Blog. Sitemaps are where your all posts, pages, categories, & Images are listed in the form of XML language.
How to make your sitemap of your blog? Go to XML-Sitemap generator and download the sitemap, it will in the form of XML. Add the sitemap on your site and provide the link in the robots.txt file.
Now, your robots.txt is ready! Now, its time to add in the blogger.
In this, I am gonna tell you how to add the custom Robots.txt in Blogger. So, follow the below steps to add the robots.txt:
Check your robots.txt file by putting robots.txt after your blog URL.
Custom Redirects gives you options for the redirection of your broken links to the right URL. Optimizing your broken links will you give a boost to your blog traffic and helps your readers to come to the right location. You can make any redirection as temporary (302) or permanent (301) by just tick the permanent checkbox or left it if you want to make it 302 temporary redirections.
These flags are used to set the robots header tags served to search engines. These tags are used to control an individual page indexing for better serving to the users in SERP. These robots meta tags and description tags in the <head> section of your blog posts make it search engine friendly.
Let me explain all the directives and these are not case sensitive.
Custom Robots Headers tags, Meta Tags, and Robots.txt gives the powerful boost in SEO to Blogspot blog. Blogger makes it easy to configure the Custom Robots headers tags and give the proper way to tell the Search Engine bots to What to index or what they have to follow or not.
These Setting gives a huge increase in your traffic and also increase your Search engine ranking. Without the knowledge of coding the, anyone can tell the spiders that they do or do not want to index or not to index the pages, posts, and categories.
Google also gives the proper information about all these tags and how to utilize them for your blog to make it search engine friendly. You can find the link below the read the full guide give my the Google. It will help you a lot to customize your Meta tags and custom robots header tags.
For any query, you can post the comment below. If you like the post then feel free to share!
Blogging is my hobby. I am sure that my Blog will provide value to the readers. Wiki Junction is only made for the people who like to read Informative Articles, Unknown Facts, Blogging Tips and Much More.
I am Harsh – Author of Wiki Junction – A Place where you can get Informative Contents Based on WordPress, Blogger, SEO, and Google Adsense tips and tricks.