From 9ea54de470563e14434d7108497776aaa55e64c3 Mon Sep 17 00:00:00 2001
From: Leo Balter <301201+leobalter@users.noreply.github.com>
Date: Tue, 30 Apr 2024 18:19:58 -0700
Subject: [PATCH] Add no-index policy
---
content/policies/crawlers.mdx | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/content/policies/crawlers.mdx b/content/policies/crawlers.mdx
index ea48de1f245..ea20ae8b183 100644
--- a/content/policies/crawlers.mdx
+++ b/content/policies/crawlers.mdx
@@ -10,3 +10,9 @@ npm's [website](https://www.npmjs.com) also has package metadata available. We a
If you do not wish to install CouchDB to manage replication, we provide [open source software](https://github.com/npm/concurrent-couch-follower) that makes it easy to sync to the registry's public feed.
If you attempt to access package metadata by high-velocity crawling of the npm website, we reserve the right to rate-limit or ban your IP, user-agent or both.
+
+## Website no-indexing rules for new packages
+
+The npm website uses the `X-Robots-Tag: noindex` tag in HTTP response header for new packages to avoid spam. The no-index header eventually goes away when the packages get a certain level of adoption.
+
+We understand this might not be desirable for newly published and well intended packages, but this is a measure we've had to take due to active abuse campaigns against the registry.
pFad - Phonifier reborn
Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.
Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.
Alternative Proxies:
Alternative Proxy
pFad Proxy
pFad v3 Proxy
pFad v4 Proxy