Log in
Log in with USB
Inloggen met vingerafdruk wordt binnenkort mogelijk.
This manual points out how to make changes to the robots.txt file of your website.
First log in to the front-end editor in edit mode. The button for the settings tab is then visible in the upper blue bar. It is the button with the sliders.
Click within the settings tab again within on the settings. In the field under "Extra Robots.txt regels" put your prefered robots.txt content. Then click on Save.
The use of robots.txt comes in handy when you want to control how bots will act on your website. See the example on the right, which shows how you can prevent e.g. the Google AdsBot from indexing any content.
If you want to verify or validate your robots.txt, a convenient way is to check with the robots.txt tester from Google. It allows you to simulate different bots and shows which part of the robots.txt file matches.