Setting a noindex flag in Backlight 6

[Matthew]

Backlight ships with a prepared robots.txt file, and I recommend using that as your baseline.

We also have documented robots.txt pretty thoroughly:

robots.txt | Backlight

And I have a thread here on the forum where I’ve been updating our default robots.txt file in real time:

Blocking AI Data Scrapers in Backlight

Our next version of Backlight (v5.4.1) will include updates to our distributed robots.txt file to block AI web crawlers from indexing your site, and I’ve already updated our robots.txt documentation to include new information about it, and links to external resources. But as it’s a manual task to implement the robots.txt on your site, there’s no need to wait for the update. You can implement the new rules immediately by adding all of this to a robots.txt file at the root of your site. This lis…

If you want to block everything, then I think this:

User-agent: *
Disallow: /