Skip to content
Can I personalize?

Our scanner

What our scanner fetches, and how to stop it.

Someone typed your domain into Can I personalize? to see who your site sells to. Here's exactly what we read when they do.

How it identifies itself

Every request carries this user agent:

RightMessageScanner/1.0 (+https://canipersonalize.com/bot)

What we fetch

  • Your robots.txt, first. If it disallows us, we stop.
  • Your sitemaps, including sitemap indexes. We read up to 5,000 URLs and sort them by what the address says, without fetching those pages.
  • Your homepage, your pricing page, and up to 4 pages written for a specific audience. That's 8 pages at most per scan.

We only fetch pages on the domain that was entered and its www version. We use plain GET requests with no cookies, never fill in forms and never log in.

Limits

  • 10 seconds and 2 MB per page.
  • About 45 seconds for the whole scan.
  • A scanned domain is cached for 7 days, so repeat visits don't fetch your site again.
  • Each domain gets at most one fresh scan a day.

When a page needs JavaScript

If a page's HTML has almost no readable text, we load that one page in a headless browser (Cloudflare Browser Rendering) with images, fonts and media blocked. If that also fails, or we hit a bot check, we stop and tell the person your site blocked us. We don't try to get past challenges.

Robots.txt rules we follow

We obey Disallow rules written for RightMessageScanner and for *. To block us everywhere:

User-agent: RightMessageScanner
Disallow: /

To let us in when a broader rule blocks bots:

User-agent: RightMessageScanner
Allow: /

Want a report taken down?

Reports are shared by link and hidden from search engines unless the site owner opts in. If you'd rather your site's report didn't exist at all, request removal.