ConformataResearchBot

If you found ConformataResearchBot/1.0 in your server logs, this page explains it. It is a small, slow research crawler run occasionally by Conformata to understand how public checklist libraries are organized.

What it collects

• Page structure only: page titles and headings, category paths, how many sections and items a checklist has, which standards are named, and which page features exist (for example a print button or a PDF download).
• It does not keep page text. To make sure our own checklists are not similar to anyone else's, it computes short one-way fingerprints of checklist item text in memory and stores only those fingerprints; the text is discarded.
• It never signs in, submits forms, clicks download or “run” buttons, calls APIs, or visits paid areas.

How it behaves

• It identifies itself honestly in its User-Agent and never pretends to be a browser.
• It reads your terms of use and does not crawl a site whose terms prohibit automated access.
• It obeys robots.txt, including Disallow, Allow and Crawl-delay, for the group that matches its name or otherwise for all agents.
• It uses one connection per site and waits at least 10 seconds between requests, longer if your Crawl-delay asks for it, with a fixed limit of requests per site per run.
• It stops crawling a site at the first 403 response or after three consecutive errors, and honours Retry-After.
• It never uses proxies, rotating addresses, CAPTCHA solving or other ways around a block.

How to opt out

Add this to your robots.txt; the crawler checks it at the start of every run:

User-agent: ConformataResearchBot
Disallow: /

Last updated: September 25, 2026.