What can an AI crawler actually read on your site?

Before any assistant can name you, it has to be able to read you. We fetch three things — your home page, /llms.txt and /robots.txt — and tell you what we find.

Every finding shows the file it came from, so you can check it yourself instead of taking our word for it. Nothing here asks an engine anything, so nothing here is a guess about what an assistant would say — that is the paid report, and it is a different question.

Three requests to your site. No account, and no engine calls — so nothing here is a guess about what an assistant would say.

Why these three files

They are the only things a crawler can read before it decides whether to read anything else. A crawler that is refused in robots.txt never sees the rest of the site, and a page whose text only arrives after JavaScript has nothing to quote however good the writing is.

We check as 16 named agents, one at a time, using the same parser the paid product uses. The list is in our methodology.