Credit: Shutterstock via Computerworld.com
When Penguin, an update of Google's search algorithm, swam online five years ago, it just about sank StairSupplies.
"In the course of a week we went from 1,000-1,200 organic clicks a day [from Google search results] to 100-200," says Caleb Morris, marketing manager at the Goshen, Ind.-based vendor of stairway components for house remodelers. "The only traffic we had left was from paid ads."
StairSupplies was a victim of an ongoing, behind-the-scenes battle between search engines and the experts who try to decipher their algorithms - or game them, depending on whom you ask or the color of the expert's hat. Yes, hat, as in black hat versus white hat. Search engines to some extent operate in the Wild West.
Google, like other search engine proprietors, is constantly shifting, enhancing and tweaking its search algorithm, which determines who is listed in what order on a search results page, and some of those recalibrations are noteworthy enough to earn names, as in the case of Penguin (see the "Evolution of the Google search algorithm" at the end of this article). Those changes often try to nullify deceptive tactics that websites use to improve their Google rankings -- and can leave companies such as StairSupplies scrambling to regain lost ranking stature. Trying to manipulate search algorithms -- and coping with the tide of changes -- has been the goal of experts in search engine optimisation (SEO) for almost as long as we've had search engines.
The stakes are high. "Number one on the search results list will get double the traffic of number two, and number two gets triple the traffic of number five, so SEO really matters," says Bruce Clay, who heads up digital marketing optimisation firm Bruce Clay Inc.
Except that no one can actually define search engine algorithms. Google, for example, employs a kaleidoscope of hundreds of constantly shifting "signals" to rank pages. (Google declined to comment on its search algorithm for this story, which is in keeping with a thickening veil of secrecy that shrouds the process. The company stopped updating the public version of its database of page rankings in 2013 and cut off all access to it last year.)
"There are those who claim to be able to reverse engineer Google's search algorithm," says former Google search engineer Vanessa Fox, now head of Keylime Toolbox, a search analytics software firm. "But even Google doesn't know what it is at a given moment - algorithms are being used to change other algorithms. Teams work on different areas of it. There is no way to see it all at once. And it will change every few hours."
Sign up for Computerworld eNewsletters.