Update robots.txt (#3941 )

* update robots.txt disallow crawlers from /assets/ they treat it as junk/bad content * Update robots.txt make it so image files in the base directory don't get indexed
update robots.txt (#3940 )
2025-11-28 00:41:20 +11:00 · 2025-08-16 17:52:54 -07:00 · 2025-08-16 17:19:24 -07:00
1 changed files with 4 additions and 0 deletions
--- a/docs/public/robots.txt
+++ b/docs/public/robots.txt
@ -1,2 +1,6 @@
 User-agent: *
+Disallow: /assets/
+Disallow: /*.png$
+Disallow: /*.svg$
+Disallow: /*.ico$
 Allow: /