A robots file is a file on a website in the root directory which determines which search engines have access to which pages within a website. Search engine genie understands the need for robots file because search engines can’t crawl your sites without robots file. Hence we pay a lot of importance to the proper creation of robots file.