URL Inspection in Search Console

What arrived yesterday

Google began rolling out a new Search Console tool yesterday, called URL Inspection. It is part of the new Search Console, still in beta, and should reach every account over the coming weeks.

The idea is simple. You paste in the address of one page on a site you have verified, and Search Console reports what Google has on file for it.

For a small business the useful moments are specific: a new page that never appears, an old title that will not go away, a page that was fixed and still shows the problem.

What it tells you

• Whether the page is on Google, stated plainly at the top.
• When Google last crawled it, and whether that crawl worked.
• Why it was left out, such as an instruction not to index it.
• Which address Google treats as the main version of the page.
• Enhancements it found, such as a linked AMP page or recipe markup.

The fourth surprises people most. When several addresses show the same content, Google picks one as the main version, called the canonical, and it is not always the one you intended.

The secure and plain versions of an address, or two near-identical service pages, can end with Google choosing differently. The tool shows your choice and Google's.

Google's copy, not your page

The tool reports what Google recorded on its last visit, not what is on your server right now.

If you fixed a page this morning and Google last crawled it three weeks ago, the tool still describes the old version. That is not an error, and not a sign the fix failed.

Which makes the last crawl date the first thing to read.

When to use it

A new page that has not appeared in search after a few weeks is the obvious case. The tool says whether Google knows about it, and if so, why it left it out.

A page showing an old title in search results is another. If the last crawl predates your change, Google has not seen it yet.

After a move to HTTPS or a redesign that changed addresses, inspect the pages that matter most. And when the coverage report lists a group of pages with an error, inspecting one usually explains the rest.

Reading the usual answers

Most results fall into a few patterns, each with an ordinary cause.

A noindex instruction usually means a setting left on from when the site was being built, or a box ticked in a plugin by mistake.

A page blocked by robots.txt is being kept out by a file at the root of the site, often written for a staging copy and carried across at launch.

A page crawled but not indexed has been judged not worth including for now, which often points to thin or duplicated content.

And where Google chose a different canonical, the page is being treated as a copy of another one.

A worked example

A two-person chimney sweep business added a page in the spring about wood stove inspections, aimed at the people who book in late summer.

By June it had never appeared in search, even for its own exact title, while the rest of the site ranked as it always had.

Inspecting the address showed the page was known to Google and excluded by a noindex instruction. It had been saved as a draft with its search setting switched off, and the setting stayed off at publication.

Whoever maintains the site switched it back on, checked the other pages added that year, and asked Google to crawl the page again.

A week later the last crawl date had moved and the page was on Google.

Asking Google to look again

For a single page, the established route is Fetch as Google in the old Search Console, followed by its request indexing button. Requests are limited, so save them for pages that matter.

For many pages with the same problem, the coverage report in the new Search Console has a button to validate a fix, which asks Google to recheck the affected pages.

Either way it is a request, not an instruction. Google decides when to crawl and whether to index, and an inspection a week later is how you find out.

How the page looks to the phone crawler

For how a page looks to Google on a phone, the separate Mobile-Friendly Test is the tool. It loads the page as the smartphone crawler would, shows a screenshot, and lists anything it could not load.

That matters more now that Google has begun moving sites to mobile-first indexing, where the phone version is the one it reads. A page that looks broken in that screenshot is the page to fix first.

What it will not tell you

It does not report clicks or positions; the performance report in the same Search Console does. Nor will it explain why an indexed page ranks below a competitor.

And it only works for sites you have verified, which is one more reason to make sure your business, and not only a former supplier, has access to its own Search Console.

The counter-case

On a small site where everything is indexed and nothing changes, there may be little reason to open it at all.

Searching for an exact phrase from a page still answers whether it is there, and the coverage report gives the overview for the whole site.

The tool earns its place when something is wrong or new: a missing page, a stale title, a fix that does not seem to have taken.

Inspect the page before guessing, and read the crawl date before the verdict.

Next time a page goes missing

1. Paste its address into URL Inspection.
2. Read the last crawl date first.
3. Note the reason it was left out.
4. Fix the cause, not just that page.
5. Request a fresh crawl for the page.
6. Check the phone version in the Mobile-Friendly Test.
7. Inspect again a week later.

Step four is the one that prevents a repeat. A setting left on in one page is usually left on in others, and a robots.txt rule rarely blocks only one address.

Helping Google read the business details on those pages is covered in structured data for a local business.

Frequently asked questions

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.