Auditing a small site
Input: example.com
Output: 184 indexable URLs · homepage indexable: Yes · 2 sitemaps found
A tidy footprint; cross-check against Search Console for the real indexed total.
Domains Tools
Indexability Checker
Estimate how many pages of a site are publicly indexable by discovering its robots.txt and XML sitemaps, counting the URLs they list, and checking the homepage's index status.
Step by step
How to use the Indexability Checker
- Enter the domain (e.g. example.com) to inspect
- Click "Estimate indexed pages" to discover robots.txt and sitemaps
- Read the estimated indexable URL count and the sitemap sources found
- Check sample URLs and the homepage indexable flag
Pro tip: This counts sitemap-listed URLs, not Google's real index; a missing sitemap.xml drags the estimate to zero.
How the estimate is built
Enter a domain and the tool fetches the homepage, then looks for the site's robots.txt to find any Sitemap: lines it declares. If robots.txt lists sitemaps, those are used; if it does not, the tool falls back to the conventional /sitemap.xml location. It then walks the sitemaps it finds — following a sitemap index to its child files where one exists — and counts the unique URLs listed inside the <urlset> documents. The headline number is that count of indexable URLs.
Alongside the count, the tool checks whether the homepage itself is indexable by reading the robots meta tag in the page head and flagging a noindex directive if present. It reports how many sitemap files it discovered, the robots.txt status, and a sample of the first URLs it found so you can eyeball what is actually listed. This is deliberately different from typing site: into Google. It measures what the site itself declares as indexable, which you control, rather than what a search engine currently has stored.
What the results show
- Indexable URLs (sitemap) — the count of unique addresses listed across the sitemaps that were read.
- Homepage indexable — Yes or No, based on whether the homepage carries a noindex robots meta tag.
- Sitemaps found — how many sitemap files were successfully fetched and parsed.
- A discovery table with the resolved homepage, the robots.txt status, and the sitemap file URLs.
- A sample of the first URLs (up to 50) so you can confirm the right pages are listed.
Why measure the declared footprint
- Spot index bloat — far more URLs than you expected often signals thin, duplicate, or auto-generated pages.
- Confirm a new or updated sitemap is actually being served and is reachable from robots.txt.
- Check that important sections appear in the sitemap and are not accidentally left out.
- Catch a homepage that has been set to noindex by mistake after a launch or migration.
Tips for getting an accurate read
Run the check on the canonical version of your domain (for example the https, non-www form you actually serve) so the homepage and robots.txt resolve cleanly. If the count looks far lower than your real page total, open the discovery table and confirm the sitemap was found and parsed — a single broken or gzipped sitemap can leave many pages uncounted. Use the sample URL list as a quick quality gate: if you see staging paths, tag archives, or parameter URLs you did not expect, that is a cue to tidy your sitemap before search engines crawl it.
Common mistakes
The most common misunderstanding is reading this number as "pages Google has indexed". It is not — it is a count of what the site advertises as indexable through its own sitemaps. A second pitfall is assuming a low number means a problem with the tool when the real cause is a missing, misnamed, or unreferenced sitemap. Finally, remember that listing a URL in a sitemap is a suggestion to crawlers, not a guarantee of indexing; a page can be in the sitemap yet excluded by Google for quality, duplication, or canonical reasons.
Honest limitations
It cannot report how many pages Google or Bing have truly indexed — only the search engine knows that, and the official place to see it is Search Console's Indexing report (or Bing Webmaster Tools). Sites with no sitemap, or with sitemaps that omit pages, will produce a low or incomplete estimate. To keep each check fast the tool reads a limited number of sitemap files rather than crawling unbounded sitemap sets, so very large sites may be partially sampled, and the sample URL list is capped. Compressed (.gz) sitemaps or unusual XML formats can also reduce the count. The fetches run on our edge against the public site, so no login-only or private pages are seen.
Worked examples
Examples straight from the tool
Catching a noindex homepage
Input: relaunched-site.com
Output: 210 indexable URLs · homepage indexable: No · 1 sitemap found
The sitemap looks healthy, but a leftover noindex on the homepage needs fixing fast.
A site with no discoverable sitemap
Input: tiny-brochure-site.com
Output: 0 indexable URLs · homepage indexable: Yes · 0 sitemaps found
The pages may exist, but without a sitemap there is nothing to count — add one.
FAQ
Questions we get about Indexability Checker
Is this the number of pages Google has indexed?
No. It estimates indexable URLs from the site's own sitemaps and robots.txt. Only Google Search Console (or Bing Webmaster Tools) reports the true indexed count.
Where does it find the sitemaps?
It reads Sitemap: lines from robots.txt first, and if none are listed it falls back to the standard /sitemap.xml location, following a sitemap index to its child files.
What does "homepage indexable" mean?
It reports whether the homepage carries a noindex robots meta tag. "No" means a noindex directive was found, which would keep the homepage out of search results.
What if the site has no sitemap?
Without a discoverable sitemap there is little to count, so the estimate will be low or zero even if the site has many pages.
Why is the count lower than my real page total?
The tool reads a limited number of sitemap files for speed, and broken, gzipped, or unreferenced sitemaps are not counted, so large or misconfigured sites can be undercounted.
Does a URL in the sitemap mean it will be indexed?
No. A sitemap entry is a suggestion to crawlers. Search engines may still skip a page for quality, duplication, or canonical reasons.
Why does it differ from a site: search?
A site: query reflects a search engine's current index; this tool reflects what the site declares as indexable. The two measure different things.
Site standards
How HighSEOTools handles data and methodology
Our editorial and data-source notes explain how each check works and where estimates come from.