Robots.txt file is very important for Search Engine. If this file is not witten in proper way then search engine will crawl your site and your website never come in search engine result so you need to be very care ful about this file.
how to check robots.txt file in website
For Checking robots.txt file you should go for following website I really like this website.
http://tool.motoricerca.info/robots-checker.phtml
You should write your robots.txt file following way.
User-agent: * Disallow: Sitemap: https://purabtech.in/sitemap.xml.gz # WordPressAPI.com 18 April 2010
