​Robots.txt is a text file which contains few lines of simple code. It is saved on the website or blog’s server which instruct the web crawlers on how to index and crawl your blog in the search results. That means you can restrict any web page …