Your robots.txt tells search engines, such as Google, which pages they should and should not search on your website. Although you rarely have to do anything in a robots.txt, there may be situations where you do not want to have a part of your website visited by search engines or other bots. You can indicate this in your robots.txt.

Since the last update you can easily customize your robtots.txt yourself via the front-end of Webanizr. You can read how to do this in the manual about placing your robots.txt.