Blogger

How to Add Meta Tags, Robot txt & Custom robots header tags in Blogger?

Setting up Crawlers and Indexing in Blogger | Robot.txt, Meta tag & More

Before you submit your blog to Google Search Console you need to do some changes and configurations in blogger’s dashboard so that your blog will be get indexed by Google and start ranking in SERP. These settings and configurations are necessary for a blog to perform well in the Google Search Result Page. It includes Meta tags, Robot txt and Custom robots header tags that are available readymade to configure in blogger.

It’s simple and easy to understand!

This Guide is dedicated to newbie bloggers who just started their blogs on blogger and want to understand & configure the Indexing of their Blog.

These configurations can increase your blog’s visibility in SERP and misconfigurations can lead your blog deindexed by Google. So, be careful while implementing these configurations.

Let’s get started!

How to add Meta Tags in Blogger?

Meta Tags are the short summary of what your blog is all about. So, you need write a short detailed description of your blog so that whenever your blog appears in Google result page then, the visitor can know what is your blog is all about and what service you are providing.

To do that login to your Dashboard>>Setting>>Search Preferences>>Meta Tags. Now, Click on it and confirm that you are going to add this.

Now, write a short detailed description of your blog and include your main keywords for SEO perspective. It must be within 150 characters otherwise Google bot will not read the extended words and it is bad for SEO.

For an Example:

add meta tags description in blogger

Now, Save the Changes you have made. After adding meta Tag the crawlers will crawl your blog and shows up the description on the search engine result page.

How to create Robots.txt file in Blogger?

Robots.txt is a file which contains some lines of code which give proper instruction to the crawler to how to index, which part of your blog not to index or block crawler bots from some of the pages of crawl your blog. For example, I do not want to crawl some of the pages or posts such as archive, tags, etc so I will add disallow: /URL  in the robots.txt file.

Don’t add or edit the robotx.txt file if you don’t know. Otherwise, it can lead to de-indexing of your blog. Crawlers read the robots.txt file before crawling any blog or website.

This video made by Ex-Google Engineer, Matt Cutts will give you enough basic information about Blogger robots.txt file:

Default robots.txt file

User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: /search

Allow: /

Sitemap: http://example.blogspot.com/sitemap.xml

Don’t know what is sitemap and how to create it? Read this post “What is SEO and How it works? | Google SEO Guide For Beginners” to create your blog sitemap.

Some Useful Robots.txt Rules for Create your own Robots.txt file

custom header tags

 *[User-agent: *]: –This code is for all bots as it is marked with (*).

*[User-agent: Mediapartners-Google]: This code is for Google partners such as Google Adsense. If you don’t have the Google Adsense on your blog then leave it.

 *[Disallow: /search]: –  This code means not to index the links which contain “/search” term.

For example, https://www.wikiijunction.com/search/label/wordpress. The Crawler will not index this link as it contains “/search” term.

*[Allow: / ]: – This code is for allowing all bots to crawler the homepage of your site.

*[sitemap: ]: – This code is for referring bots the sitemaps of your Blog. Sitemaps are where your all posts, pages, categories, & Images are listed in the form of XML language.

Don’t have any sitemap of your blog? Then, create one. Follow the link to create a sitemap for your blog.

For creating an advance robots.txt file for your blog, you need to follow to this link.

Now, your robots.txt file is ready! Now, its time to add in the blogger.

How to Add the Custom Robots.txt in Blogger?

In this, I am gonna tell you how to add the custom Robots.txt in Blogger. So, follow the below steps to add the robots.txt.

Login to your Dashboard>>Setting>>Search Prefences>>Custom Robots txt>>Save it.

That’s it!

Add Robots txt in blogger

Check your robots.txt file by putting robots.txt after your blog URL.

https://www.yourdomain.blogspot.com/robots.txt

How to Set up Custom Robots Header Tags in Blogger?

Custom Robots header tags are used to set the robots header tags served to search engines. These tags are responsible for controlling an individual page indexing for better serving to the users in SERP. These robots meta tags and description tags in the <head> section of your blog posts make it search engine friendly.

Let me explain all the directives and these are not case sensitive.

  • All:- It will allow the bots to crawl everything and index.
  • noindex:- It will tell the bots to crawl but not to index it.
  • nofollow:- It is utilized for telling the bots not follow the links in posts or pages.
  • none:- It is used for providing the information to the bots that nofollow as well as noindex it.
  • noarchive:- This is used for telling the bots not to show cache links in the search results.
  • nosnippet:- This is used for telling the bots that not to show snippet in the search results.
  • notranslate:- This is used for telling the bots not to translate the text into other languages.
  • noodp:- this option can not allow search engines to use metadata from the DMOZ directory.
  • noimageindex:- This is used for telling the spiders not to index the images.
  • unavailable_after: This is very useful as it can be used for deindexing the zombie pages that are too old by just providing the date/month/year after the term.

How to Custom Robots Header Tags in Blogger?

  • Login to your blogger dashboard.

add meta tags, custom robot header tags in blogger

  • Setting >> Search Preferences.
  • Now Navigate Custom Robots header tags and edit it.

How to add custom robot header tags in blogger

  • Make the changes and save it.

add custom robot header tags in blogger

  • That’s it!

Bottom Line

Custom Robots Headers tags, Meta Tags, and Robots.txt gives the powerful boost in SEO to Blogspot blog. Blogspot makes it easy to configure the Custom Robots headers tags and give the proper way to tell the Search Engine bots to What to index or what they have to follow or not.

These Setting gives a huge increase in your traffic and also increase your Search engine ranking. Without the knowledge of coding, anyone can tell the spiders that they do or do not want to index or not to index the pages, posts, and categories.

Google also gives the proper information about all these tags and how to utilize them for your blog to make it search engine friendly. You can find the link below the read the full guide give my the Google. It will help you a lot to customize your Meta tags and custom robots header tags.

link: https://developers.google.com/search/reference/robots_meta_tag?hl=en

For any query, you can post the comment below. If you like the post then feel free to share!

5 comments

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.