Website Tracking Tools

Spider Simulator

See the text-oriented view a basic search-engine crawler extracts from a page's returned HTML — its meta tags, heading outline, internal and external link counts, and visible text, with no JavaScript executed.

Website Tracking ToolsServer-backed/spider-simulator

Shows what a search engine spider reads: meta, headings, links, and visible text.

Step by step

How to use the Spider Simulator

  1. Enter the page URL you want to inspect into the Page URL field
  2. Click Simulate crawler to fetch the live HTML server-side
  3. Review the crawler-visible title, meta, headings, links, and extracted body text
  4. Check the indexable status and word count to confirm crawlers see real content
How to use the Spider Simulator — tool screenshot
The Spider Simulator on HighSEOTools

Pro tip: If visible text is thin (under 150 words) or noindex appears, crawlers may skip the page despite a rich-looking browser view.

Seeing the page like a crawler

Visitors see your page after the browser has applied styles, images, and scripts. A simple crawler sees something far plainer: the raw HTML stripped down to text and structure. This simulator reproduces that view by fetching the page from our Cloudflare edge and extracting the meta tags, the heading outline, the links, and the visible text from the markup that was returned.

Laying the page out this way exposes a gap many audits miss: content that looks rich to a human but is sparse or disordered once the styling is gone.

What the extraction returns

  • Crawl signals — the title, meta description, canonical, robots directive, and the HTML lang attribute a crawler reads first, with indexability derived from the robots tag.
  • Word count — how many words of visible body text were found in the HTML, flagged when it falls below roughly 150 words.
  • Heading outline — the heading sequence with its levels, up to the first fifteen, which reveals whether the page has a logical, scannable structure.
  • Link counts — how many links resolve to the same host (internal) versus another host (external), summarised as a single in / out figure.
  • Visible text preview — the readable body content extracted from the HTML with tags stripped, shown up to roughly the first 900 characters.

The JavaScript blind spot

This is the most important thing to understand about the tool. It does not run JavaScript. If your headings, copy, or links are injected by a framework after the page loads in a browser, they exist only in the rendered DOM, not in the HTML our edge receives, so they will not appear in the extracted view, and the word count will look thin even on a page that feels full.

That is precisely the value of the simulation: when expected content is missing here, it warns you that a basic crawler relying on the raw HTML may miss it too, and that you may be depending on rendering that not every crawler performs.

Reading the results honestly

A thin word count and an empty heading outline on a visually full page is a signal to check whether your content is server-rendered. A healthy extraction, with a sensible heading order and a meaningful word count, means the essentials are present in the HTML itself. Watch the robots line too: if it contains noindex, the page is asking to be kept out of search regardless of how good the content is. Bear in mind this models a basic, text-only crawler; the most capable search engines do render JavaScript, so a gap here is a caution flag rather than proof a page is invisible everywhere. As with the other tools, the fetch uses an identified tool user agent and only public URLs are reachable, so pages that block automated requests may return little to analyse.

Worked examples

Real inputs and their results

A server-rendered article

Input: https://example.com/guide

Output: Indexable: Yes · Words seen: 1,180 · Headings: 6 · Links: 14 in / 4 out • Title and description found, HTML lang: en • Outline: H1 "The Complete Guide" then five H2s • Visible text preview populated.

The content lives in the HTML, so even a basic crawler reads it fully.

A JavaScript-rendered app shell

Input: https://example-app.com/dashboard

Output: Indexable: Yes · Words seen: 12 · Headings: 0 · Links: 1 in / 1 out • Title found, no description • Outline: empty • Visible text preview: (no readable body text found in the HTML response).

The near-empty extraction warns that simple crawlers see an empty shell.

FAQ

Spider Simulator: common questions

Why does the page look empty here but full in my browser?

Your browser ran the JavaScript that builds the page; this tool does not. A near-zero word count and empty outline mean the content is not in the raw HTML our edge received.

Does Google see exactly what this shows?

Not exactly. This models a basic text-only crawler. Google can render JavaScript, so a gap here is a warning to investigate rather than proof the content is hidden from every engine.

How does it count internal versus external links?

It resolves each link in the HTML and compares its host to the page's host: same host counts as internal, a different host counts as external, shown as a single in / out figure.

Why is the word count flagged as thin?

The tool flags visible body text under roughly 150 words because that is often a sign the real content is loaded by JavaScript or the page is genuinely sparse for a crawler to read.

Is the full visible text shown?

No. The preview is capped at roughly the first 900 characters with tags stripped, which is enough to confirm whether real body text exists in the HTML without dumping the whole page.

How is this different from the Website Checker?

The Website Checker summarises core health signals like status and tag lengths; this tool reconstructs the fuller text-and-structure view a crawler extracts, including the heading outline, link split, and a body-text preview.

Site standards

How HighSEOTools handles data and methodology

Our editorial and data-source notes explain how each check works and where estimates come from.