What Google Has to Do Before It Can Rank You

The five steps

Worth understanding because the remedies differ entirely depending on which one failed.

Discovery. Finding out the page exists, usually through a link or a sitemap.

Crawling. Fetching it, which requires the server to respond and the rules to permit it.

Rendering. Running any scripts to see what the page actually shows.

Indexing. Deciding the page is worth storing, and storing what it says.

Ranking. Choosing where it appears for a given search.

Almost all advice is about the fifth. Most small site problems are in the first four.

Discovery

A page nobody links to and no sitemap mentions may never be found at all.

The common cases: a page added but never linked from the navigation, a new site with no inbound links anywhere, and a page reachable only through a form or a search box.

The remedies are ordinary: link to it from somewhere on your own site, include it in a sitemap, and submit the sitemap in Search Console.

For a brand new site with no links from anywhere, discovery can take days to weeks, and there is nothing wrong that time will not fix.

Crawling

Where a surprising number of small sites block themselves.

A robots file left over from a staging site, disallowing everything, is the classic. It is a two-line file and it makes an entire site invisible.

Password protection or an under-construction setting that was never removed.

A server returning errors intermittently, or so slowly that crawling is reduced.

And a hosting setup that blocks automated requests, which occasionally catches search engines along with unwanted traffic.

All of those are checkable in minutes and all of them produce total invisibility, which people usually attribute to competition.

Rendering

The step most people do not know exists, and the one that catches modern sites.

If a page's content is assembled by scripts after loading, a search engine has to run those scripts to see anything.

That happens and it is slower and less reliable than reading plain content, and it occasionally fails.

The result is a page that looks full to a visitor and close to empty to a search engine.

Which is why content that matters should be in the page rather than assembled afterwards, and why a site built entirely as an application has a disadvantage a plain one does not.

Indexing

Being crawled does not mean being kept.

A page can be fetched and then not indexed, and Search Console will say so in exactly those terms.

Common reasons: the page duplicates another, it is very thin, it is marked with an instruction not to index, or it points elsewhere with a canonical tag.

The instruction-not-to-index case is worth checking first, because it is frequently left over from development and it is invisible in a browser.

A page carrying it will never rank for anything, no matter how good it is, and no amount of optimisation will change that.

A worked example

A business whose new site had been live for five months and appeared for nothing at all.

Two rounds of paid search work had produced no change, which is unsurprising in hindsight.

The robots file, copied from the development server, disallowed everything.

Removing two lines and requesting indexing produced results within about a fortnight.

Nobody had checked, because everybody had assumed the problem was ranking, and ranking work does not reveal a crawling problem.

The whole diagnosis took under ten minutes once somebody looked at the right thing.

Checking which step failed

1. Search for your exact address with the site prefix to see what is indexed.
2. Open Search Console and look at the coverage report.
3. Use the URL inspection tool on a page that should rank.
4. Read what it says about crawling, indexing and any blocks.
5. Check the robots file by visiting it directly.
6. View the rendered page in the inspection tool to see what was seen.

That sequence identifies the failing step in most cases, and it requires no expertise beyond reading what the tool reports.

The counter-case

Where the problem genuinely is ranking.

A page indexed, appearing for its own title, and simply not competitive for the searches you want. That is a content and authority question rather than a technical one.

A competitive term where established sites hold the positions and the realistic target is a narrower search.

And a new site, correctly indexed, which needs time and links rather than fixes.

Establishing that you are in this situation rather than one of the earlier ones is the point of the checks above, and it prevents spending money on the wrong problem.

Sitemaps and what they do

Worth being clear, because they are frequently over-credited.

A sitemap helps discovery. It lists your pages and tells a search engine they exist.

It does not cause indexing, does not improve ranking, and does not override an instruction not to index.

Which means submitting one is worth doing, takes minutes, and will not fix a site that is blocked or thin.

It is most valuable for large sites, new sites, and pages that are poorly linked internally, and close to irrelevant for a well-linked five-page site.

How long each step takes

Useful for judging whether something is broken or simply not finished.

Discovery of a new page on an established site is usually days. On a brand new site with no links, it can be weeks.

Crawling after discovery is typically quick, and requesting it in Search Console shortens it further.

Indexing can follow within days or take considerably longer, and a page may sit crawled-but-not-indexed for weeks before a decision settles.

Ranking movement after a change is the slowest of all, typically weeks to months.

Which means patience is genuinely part of the diagnosis. A page missing after three days is normal; the same page missing after two months with no explanation in Search Console is a problem worth investigating.

What to take from this

When a site is invisible, establish which step failed before doing anything else.

The remedies for discovery, crawling, rendering, indexing and ranking share almost nothing, and applying the wrong one produces months of effort and no change.

Ten minutes in Search Console answers the question, and it is free.

The content side of the same problem is covered in a local site that ranks for nothing.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.