Salastil/nitter
1
0
Fork 0
mirror of https://github.com/zedeus/nitter.git synced 2025-12-06 03:55:36 -05:00
Code Activity

Block search engines via robots.txt (#631)

Browse Source 
Prevents instances from being rate limited due to being senselessly
crawled by search engines. Since there is no reason to index Nitter
instances, simply block all robots. Notably, this does *not* affect link
previews (e.g. in various chat software).
This commit is contained in:
minus
2022-06-04 15:48:25 +00:00
committed by GitHub
parent 778c6c64cb
commit c543a1df8c
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
public/robots.txt Normal file
View File

@@ -0,0 +1,2 @@
User-agent: *
Disallow: /