A page is indexable when it returns 200, is not blocked by robots.txt, has no noindex directive and is its own canonical.
A page can be crawlable but not indexable, for example when it carries noindex or canonicalises to another URL.
Pages that are not indexable cannot rank. Accidental non-indexability is one of the most expensive technical SEO mistakes.
VisibilityKit Crawler checks status, robots rules, meta robots and canonicals for every URL, so you can see which pages are indexable and why the others are not.
Crawlable means a bot can fetch the page. Indexable means it is also allowed to be stored and shown in results.
Check for noindex, a canonical to another URL, thin content or duplicate content.
A directive telling search engines not to include a page in their index.
A text file at the root of a site that tells crawlers which paths they may and may not fetch.
An HTML link element that tells search engines which URL is the preferred version of a page.
Three-digit codes a server returns with every response to say whether the request succeeded, redirected or failed.
The process by which AI platforms catalog and store web content for retrieval during response generation.
A desktop SEO crawler for macOS, Windows and Linux. Find what is broken, assign the work, then verify every fix. Crawl data stays on your machine.
Get Crawler free