Our scanner, and how to stop it

What it does

• It reads. It never writes. Every request is a plain GET of a public page. We never submit a form, never add anything to a cart, never create an account and never sign in to anything.
• It obeys your robots.txt, matched against the exact URL. Not the URL family — if your file disallows /cart/ with a trailing slash we treat /cart as allowed, and vice versa, because those are different rules. If we cannot read your robots.txt we do not follow links on your site at all.
• It never touches checkout. Not because robots.txt says so, though it usually does — because a read-only scan has no business there.
• It loads each page once and resizes the browser window to measure at three widths, rather than fetching the page once per width.
• It backs off harder than you ask it to. If you return a 429 or a 403 we stop, honour your Retry-After as a minimum rather than a target, and extend it further on each refusal.
• It limits itself per site, regardless of how many people ask. A repeat request inside the window returns the stored report rather than crawling you again.
• It does not try to look like something else. No fingerprint spoofing, no stealth plugins, no residential proxies, no CAPTCHA solving. If you block it, it stays blocked.

How to stop it

Any of these works, and the first two need nothing from us:

1. Your robots.txt. We read it before every crawl and match it per URL. Disallowing the paths you care about is the fastest route and it takes effect on our next request.
2. Block us at your edge. A 403 is respected permanently for the run and triggers the backoff described above. We will not route around it.
3. Email us and we will add your domain to a denylist. bot@gradientt.space — send the domain, nothing else needed. We do not ask you to prove ownership for a request to be left alone.

One thing we are honest about

Our crawler currently sends an ordinary desktop Chrome user agent rather than a user agent naming itself Gradient. That is a tension we have not resolved rather than one we are hiding: a self-identifying agent is the honest thing to send, and it is also very likely to be blocked by rules that have nothing to do with our behaviour. We would rather say that here than quietly claim a transparency we have not implemented. The behaviour described above is what the code does either way, and you can verify all of it from your own logs.