Googlebot now renders pages with an up-to-date Chrome, so modern JavaScript no longer confuses it. It still does not click, scroll or wait forever, so put important text and links in the page itself.

What Google announced

At its developer conference yesterday, Google said that Googlebot now renders pages using Chromium 74, the same engine as the current version of Chrome.

It also said the crawler will be kept up to date from now on, following new releases of Chrome rather than staying fixed on one version.

Until now, Googlebot rendered pages with Chrome 41, a version from 2015. Features that current browsers take for granted were missing from it.

Developers worked around that by rewriting modern JavaScript into older forms and adding polyfills, small scripts that fill in missing features, sometimes for Googlebot's sake alone.

That gap is now closed. A page that works in a current browser should work for Googlebot too.

Crawling, then rendering

Googlebot reads a page in two stages. First it fetches the HTML, the text and code your server sends. Then, when it has capacity, it runs the page's scripts to see what a browser would show.

That second step is rendering, and it is the part that has changed.

What has not changed is the order. Google describes rendering as a separate step that can come some time after the first fetch, so what is in the HTML is seen first.

For a page that changes rarely, later is fine. For a new page, or prices that change weekly, it is a reason to keep the important parts in the HTML.

What still stops Googlebot

A current browser engine solves the old problem of unsupported code. It does not make Googlebot behave like a person.

The first is the easiest to build by accident. Googlebot loads the page as it arrives; it does not open tabs or press buttons.

Hidden is not the same as missing

One distinction decides whether a tab is a problem at all.

Text that is already in the page but folded away, in a tab or an accordion that opens without fetching anything, is part of the HTML. Google reads it like any other text.

Text that is fetched from somewhere else only when the tab is opened is not in the page until somebody clicks. Googlebot never clicks, so for search it does not exist.

The two look identical to a visitor. Only the page's code tells them apart.

Links Googlebot can follow

Googlebot finds pages by following links, and it follows only ordinary ones: an address in a standard link that a browser could open on its own.

Menus and buttons that change the page through a script, without a real address behind them, may work perfectly for visitors and lead Googlebot nowhere.

Pages reachable only that way can go undiscovered. A sitemap helps Google learn they exist, but it is no substitute for real links between pages.

A worked example

A tile and flooring showroom in an industrial park had its product range on a single page, with a tab for each category: porcelain, natural stone, vinyl plank and hardwood.

Only the first tab's products were in the page when it loaded. The others were fetched from the supplier's catalogue when somebody clicked, which kept the page quick to open.

Customers who searched for one of the vinyl plank ranges by name found the supplier and two competitors. The showroom never appeared, though it stocked the range and had it on the floor.

The rebuild gave each category its own page, with its own address, its products written into the HTML, and ordinary links between them in place of the tabs.

Yesterday's announcement would not have helped. Googlebot now understands the code that fetched the products; it still does not click the tab that asks for them.

Other visitors that read only the HTML

Googlebot is not the only thing that reads your pages.

Other search engines have not said they render pages the way Googlebot now does. The services that build link previews for Facebook, LinkedIn and messaging apps read the HTML and stop there.

Some people still visit on Internet Explorer 11 from older office machines, where modern scripts can fail outright. Code written for older browsers may still be needed for them, even though Googlebot no longer needs it.

And on a weak phone connection, scripts are the part of a page most likely to arrive late or not at all.

Checking your own pages

The quickest test needs nothing but a browser on a computer.

Open a page, then choose to view its source, which shows the HTML as the server sent it. Search that for a sentence you can see on the page. If it is there, every crawler can read it.

If it is visible on the page but missing from the source, a script added it, and only crawlers that render will see it.

Then search Google for a distinctive sentence from the page in quotation marks. If the page does not come up, Google has not indexed that text, whatever the reason.

The counter-case

Some sites are applications, and should be.

A customer portal, a booking tool behind a login, or a quote calculator does its work in scripts, and most of what it shows was never meant to be found in search.

A site built properly with a modern framework, sending finished HTML from the server and adding scripts on top, is already in good shape, and this week's news simply removes old workarounds.

And nobody should rebuild a working site because of this announcement. The point is narrower: keep the words and links you want found in the page itself.

The check

  1. View the source of your key pages.
  2. Search it for a sentence you can see.
  3. Check every tab's text appears in the source.
  4. Hover over menu links to see real addresses.
  5. Search Google for a distinctive sentence in quotes.
  6. Ask about robots.txt and blocked scripts.
  7. Move key content into the HTML.

Step three is the one most likely to turn something up. Tabs, accordions and load-more buttons are where content quietly leaves the HTML, and nobody notices because the page looks complete to everyone who clicks.

Checking whether a page is in Google's index at all is covered in when pages drop out of Google for a week.

Frequently asked questions

What did Google announce about Googlebot?

That Googlebot now renders pages with Chromium 74, the engine of the current Chrome, and will keep updating it rather than staying on the 2015 version it used until now.

Does Google now see everything on my page?

Not necessarily. It understands modern code, but it still does not click tabs, press buttons, scroll for more or accept prompts, so content that needs any of those can be missed.

Is text inside tabs a problem?

Only if the tab fetches its text when opened. Text already in the page but folded away is read normally; text fetched on a click is never seen by Googlebot.

Why keep content in the HTML?

Rendering can happen some time after the first fetch, and other search engines and link preview services read only the HTML. Text in the page itself reaches all of them.

How can I check my own pages?

View the page source and search it for a sentence you can see on screen. Then search Google for a distinctive sentence in quotation marks to see whether it is indexed.

Should I rebuild my site because of this?

No. A working site needs no rebuild. The change removes old workarounds for developers; the practical rule for owners is to keep the words and links you want found in the page.

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.

Content that only appears after a click?

We will check which of your pages Google can read as text, and move what matters into the HTML.

Start a Conversation