Indexing: Why Crawled Doesn’t Mean Included

This is the gate most beginners don’t know exists. Google can crawl a page perfectly and still choose not to index it.

Indexing is Google’s decision about whether a page provides enough unique, useful value to justify storing it in the searchable database. Thin content, duplicate content, and low-quality signals all trigger exclusion here.

Evidence points toward thin or duplicate content being the leading cause of “crawled — currently not indexed” status in Search Console — though the exact threshold for “thin enough to exclude” isn’t publicly documented by Google.

In our experience the failure usually happens earlier than people assume: pages get written, published, and crawled successfully, then quietly excluded weeks later once Google’s quality systems fully evaluate them against similar existing content.

The fix isn’t technical — it’s editorial. Add genuine depth, differentiate from near-duplicate pages on your own site, or merge thin pages together rather than leaving them as separate weak signals.

There’s a second, less obvious driver behind exclusion: canonicalization signals. If a page’s canonical tag points to a different URL — even accidentally, through a templating error — Google may index the canonical target instead of the page you intended, leaving your actual page invisible despite being technically crawlable.

This happens more often on large sites than people expect, particularly after a CMS migration or theme change that alters how canonical tags get generated. A single misconfigured template can silently de-index thousands of pages at once.

Duplicate content doesn’t have to mean copy-pasted text from elsewhere. Internal duplication — multiple pages on your own site covering near-identical ground — splits relevance signals and forces Google to choose which version, if any, deserves the index slot.

The practical sequencing matters here too. Indexing checks happen continuously, not just at first crawl — a page indexed today can be re-evaluated and dropped later if surrounding content quality declines, links pointing to it disappear, or duplicate pages elsewhere on the site dilute its uniqueness.

Pro Tip: In Google Search Console, go to Pages → “Why pages aren’t indexed” → filter “Crawled — currently not indexed.” If this count exceeds 10% of your total page count, prioritise content depth fixes before publishing new pages.

Ranking: How Google Orders What’s Already Indexed

Once a page clears crawling and indexing, ranking determines its position for a specific query — and this is where hundreds of weighted signals come into play.

Not every signal matters equally for every query. A recipe search weighs different factors than a “best accounting software” search — Google’s ranking systems adapt signal weighting to query type and detected intent.

Core ranking factors include relevance to query intent, content quality and depth, page experience (Core Web Vitals), mobile-first usability, and authority signals like backlinks and E-E-A-T. The full mechanics of E-E-A-T and content trust are covered in our dedicated Google’s EEAT guidelines piece.

Mobile-first indexing means Google primarily evaluates your mobile site version — not desktop — for both indexing decisions and ranking signals, a shift fully rolled out since 2019.

A small UK accountancy firm’s site ranked well on desktop testing tools but had a broken mobile navigation menu. Fixing the mobile menu alone moved their primary keyword from position 14 to position 6 within 5 weeks (GSC, Q4 2025).

What separates ranking from the two earlier gates is that it’s relative, not absolute. A page can improve in every measurable way and still drop in position — simply because a competitor improved faster, or a new page entered the same competitive set.

This relativity is why ranking volatility frustrates site owners more than crawling or indexing problems. Crawling and indexing are largely within your direct control; ranking depends partly on what everyone else publishing similar content does too.

Google also runs continuous core algorithm updates that re-weight these signals over time, sometimes significantly. A page that ranked well under one weighting scheme can lose position under a refreshed one — not because the page changed, but because the evaluation criteria did.

This is why monitoring ranking changes against Google’s documented update timeline matters more than reacting to a single day’s fluctuation. Short-term position swings are common and often self-correct; sustained drops aligned with a known algorithm update deserve real investigation.

Long-Tail Search: How Specific Queries Get Matched

Specific, longer queries follow the same three-gate pipeline — but ranking competition differs enormously based on query specificity.

“SEO” competes against millions of indexed pages with broad relevance. How search engines rank long-tail queries for new sites” competes against a tiny fraction of that, because fewer pages cover that exact intent precisely.

That’s the practical reason long-tail content ranks faster for new or smaller sites — not because Google treats long-tail queries differently in its pipeline, but because the competitive pool indexed for that specific intent is smaller.

The technical pipeline is identical: crawl, index, rank. The difference is the size and quality of the competing page set Google is choosing between at the ranking stage.

This has a direct strategic implication for new sites specifically. A brand-new domain attempting to rank for broad, high-competition terms is fighting established pages with years of accumulated authority signals at the ranking gate — a battle the technical pipeline alone can’t win.

Targeting long-tail variations first lets a new site clear all three gates against a much smaller, less entrenched competitive set, building the indexed history and authority signals needed before attempting broader terms later.

Verifying Your Pages Pass Every Gate

Most site owners check rankings and stop there. Checking all three gates separately catches problems rankings alone won’t reveal.

Start with crawling: Google Search Console → Settings → Crawl Stats. Confirm Googlebot is actively crawling your site and response times stay under 1,000ms.

Then check indexing: Search Console → Pages report. If “Indexed” sits below 90% of submitted pages, diagnose the “why pages aren’t indexed” breakdown before assuming a ranking problem.

Finally check ranking: Search Console → Performance report → filter by query. If a page is indexed but shows zero impressions for its target keyword, the issue sits in relevance or competition — not the technical pipeline.

If indexed percentage is below 90% and crawled pages exceed indexed pages by a wide margin — pause new content production and fix the content depth or duplication issue driving exclusion first.

A useful recurring habit: run this three-gate check monthly rather than only when traffic drops. Catching an indexing decline early, before it compounds across dozens of pages, is far cheaper to fix than recovering after months of silent exclusion