Problems finding and installing robots.txt

[mjenich]

I’m not a computer guy…I purchased the latest update (backlight-5-Installer-119) and extracted the files. I am looking for the robot.txt file. I went to backlight/modules and found the folder empty-no reference to Custom-Sources.

[volvoxturbo]

Hi,
after installing backlight on your webspace you will find the robots.txt here: backlight>modules>custom-resources>robots.txt

Best regards,
Oliver

[Matthew]

I’ve also posted the contents of the file here in the forum:

Blocking AI Bots in Backlight

Our next version of Backlight (v5.4.1) will include updates to our distributed robots.txt file to block AI web crawlers from indexing your site, and I’ve already updated our robots.txt documentation to include new information about it, and links to external resources. But as it’s a manual task to implement the robots.txt on your site, there’s no need to wait for the update. You can implement the new rules immediately by adding all of this to a robots.txt file at the root of your site. User-age…