From a3a2fa7173833b6c5182b19ce9b8094be751d3eb Mon Sep 17 00:00:00 2001 From: Royiex Date: Sun, 17 Aug 2025 02:19:24 +0200 Subject: [PATCH 1/2] update robots.txt (#3940) disallow crawlers from /assets/ they treat it as junk/bad content --- docs/public/robots.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/public/robots.txt b/docs/public/robots.txt index c2a49f4fb..5f51faa5f 100644 --- a/docs/public/robots.txt +++ b/docs/public/robots.txt @@ -1,2 +1,3 @@ User-agent: * +Disallow: /assets/ Allow: / From 49f816991d5a49cbba091ae4e5599fe9d218539c Mon Sep 17 00:00:00 2001 From: Royiex Date: Sun, 17 Aug 2025 02:52:54 +0200 Subject: [PATCH 2/2] Update robots.txt (#3941) * update robots.txt disallow crawlers from /assets/ they treat it as junk/bad content * Update robots.txt make it so image files in the base directory don't get indexed --- docs/public/robots.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/public/robots.txt b/docs/public/robots.txt index 5f51faa5f..3bfd2659b 100644 --- a/docs/public/robots.txt +++ b/docs/public/robots.txt @@ -1,3 +1,6 @@ User-agent: * Disallow: /assets/ +Disallow: /*.png$ +Disallow: /*.svg$ +Disallow: /*.ico$ Allow: /