A login page protects nothing if the files sit at public addresses. Test by opening a file's address in a private window, then move files behind a login the server checks before sending anything.

A price list in the wrong hands

A manufacturer's sales manager searches for the company's name, and the dealer price list comes up in Google's results, a PDF anyone can open with one click.

The dealer area on the website asks for a password. But a dealer forwarded the PDF's address to a colleague at another store, it opened without one, and at some point it reached a public page.

Now retail customers quote dealer prices back to the dealers, and a competitor can read the margins line by line.

The dealer area did what it was built to do. It just was not built to protect the files.

How older dealer areas were built

Many members areas built some years ago protect the door rather than the room. The login page asks for a password, and everything behind it sits in an ordinary folder on the server.

A web server sends any file it holds to anyone who asks for its address. Unless the server itself checks a login first, a PDF in the dealer folder is as public as the home page.

Some older areas check the password in the browser, with a script in the page itself. Anyone who views the page's source can read the password, or skip to the address it leads to.

Weak spots to look for

Any one of these is enough to expose the files, and older dealer areas often have several at once.

How Google found it

Google does not guess passwords. It follows links, and a link to the PDF only had to appear once on a page it could read: a dealer's own site, a forum post, a shared document.

Once Google has the address, it fetches the file like any other page. The PDF's text becomes searchable, and the file turns up in searches for the product names it contains.

Testing your own protected area

Log in, open the dealer area and copy the address of a file. Then open a private browsing window, where you are not logged in, and paste it. If the file opens, it is public.

View the source of the login page and look for the password, or for the address it sends you to. Then open the folder holding the files, without a file name, and see whether a list appears.

Last, search Google for your domain's PDF files with site: and filetype:pdf together, and read what comes back. Anything from the dealer area in that list is already out.

What a proper login involves

A proper login is checked on the server, before anything is sent. The server asks for a name and password, checks them against its own records, and remembers the session for that visitor.

Files sit outside the public web folder, or in one the server refuses to serve directly. A short script sends each file only after confirming the session belongs to a logged-in dealer.

Passwords are stored scrambled with a one-way hash, never as readable text. The login page and the dealer area run over HTTPS, so the password is not sent in the clear.

A worked example

A popsicle maker on the Sunshine Coast makes fruit and cream popsicles in a small production kitchen and sells them wholesale to coffee shops, grocers and caterers across the region.

Its website had a stockists page with a password, leading to a wholesale price list and an order form, both PDFs. A grocer mentioned finding the price list in Google while looking up the company.

The password was checked by a script in the page, and the PDFs sat in a public folder. The same password had gone to every stockist since the page was built.

Each stockist now has its own login, checked on the server. The PDFs moved outside the public folder and are sent only after login, and the old addresses now return a not-found page.

The maker asked Google to remove the old address and issued a new, dated price list, since copies already downloaded could not be called back.

Getting the file out of Google

Once the old address no longer serves the file, Google drops it the next time it checks. The Remove URLs tool in Webmaster Tools asks Google to take it out of results sooner.

Google may also hold a cached copy of the file, and the removal request clears that as well.

Removal from Google does not recall copies people saved. If the prices matter, issue a new list with a date on it, and tell dealers which one is current.

Who holds a login

Give each dealer an account of its own rather than one shared password. When a dealer stops carrying your products, close that account and nobody else is affected.

Review the accounts when a dealer's staff change, too. A salesperson who has moved to a competing store should not still hold a working login.

Individual accounts also show who downloaded what and when, which helps answer the question of how a file got out.

The counter-case

Not every document needs protecting. A retail price list, a catalogue or a product sheet meant for the public should simply be public, where search engines can find it.

If a document would do no harm in a competitor's hands, publishing it openly is better than protecting it badly.

Dealer pricing, terms and anything marked confidential are different. For those, the check has to happen on the server, every time a file is requested.

Locking the files, not just the page

  1. Open a private browser window and paste a file's address.
  2. View the login page's source for a password.
  3. Open the files folder without a file name.
  4. Search Google for your domain's PDF files.
  5. Move the files behind a login checked on the server.
  6. Give each dealer an account of its own.
  7. Ask Google to remove anything already indexed.

Step one takes a minute and answers the main question. If a file opens in a window where nobody is logged in, everything else on this list needs doing.

Google reached the price list the way it reaches any page, by following a link, which inventory hidden behind the site search form explains from the other side.

Frequently asked questions

Does a password on the dealer page protect the files?

Not by itself. If the files sit in an ordinary folder, the server sends them to anyone who has the address, whether they logged in or not.

How can we test whether our files are exposed?

Copy a file's address from the dealer area, open a private browsing window where you are not logged in, and paste it. If the file opens, it is public.

Why is a password checked in the browser unsafe?

Because the script that checks it is part of the page. Anyone who views the page's source can read the password or the address it leads to.

Does robots.txt keep files private?

No. It only asks search engines not to crawl a folder, and anyone can read it. It points curious visitors straight at the folder you meant to hide.

How do we get the price list out of Google?

Protect the file so it no longer opens without a login, then use the Remove URLs tool in Webmaster Tools. Copies already downloaded cannot be recalled.

Should every dealer share one password?

No. Give each dealer its own account. When a relationship ends, close that account, and you can see who downloaded what if a file escapes again.

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.

Dealer files behind a password?

Give us the address of your dealer area, and we will check whether its files open for someone who is not logged in.

Start a Conversation