Subscribe / Unsubscribe Enewsletters | Login | Register

Pencil Banner

Your strategy for dealing with web bots has to take into accout business context

Renny Shen, senior product marketing manager, Akamai’s Web Experience & Security Division | Sept. 5, 2016
Simply blocking all bot visits can end up hurting your business

Second, distinguishing between good and bad bots can be extremely difficult, if not impossible. While you can always whitelist bots from a small number of authorized partners, the travel industry has many smaller search engines and travel sites. These smaller sites fall into a grey area of unauthorized partners that, while they may not have a formal relationship with airlines, still serve as a potential channel for selling to customers. This means that blocking unknown bots will likely catch these unauthorized partners and have an unexpected impact on sales.

So how do you solve the business problem in a meaningful way while remaining cognizant of the fact that many unknown bots may contribute to sales? Bot management needs to move beyond blocking and enable organizations to apply a wider range of management actions or bot responses.

For example, in the airline industry, the bot traffic itself is not the primary problem. The primary problem is the high cost of GDS lookups made by bots visiting the bookings site. You can alleviate this problem without trying to block bots en masse by using data stored in cache to respond to bot requests for flight schedules and prices. By serving bots with cached data we can dramatically reduce the number of calls being made to the GDS without alerting the bot operator that we’ve detected its bot. This helps minimize the high business costs associated with bots, while slowing down the rate of bot evolution and without shutting the door on incremental customers and sales revenue through unauthorized (and unknown) partners.

The situation above is by no means limited to the airline industry. The line between good and bad bots are increasingly blurred in most industries, making it difficult for any organization to employ a black-and-white approach like blocking.

Online fashion retail provides another fascinating example. Most fashion retailers today sell directly to consumers through their e-commerce sites, in addition to brick-and-mortar stores. With a B2C model, it is always tempting to think you could just block non-human visitors to your e-commerce site, under the assumption they don’t contribute to sales.

What this assumption misses is that consumer behavior is constantly changing, and how those consumers interact with your online brand and website is constantly changing as well. Consumers today no longer buy the majority of their clothing from a single brand, but instead cross-shop multiple brands. In addition, many are looking for an introduction to new and different brands. This means consumers are actively going to third parties for new ideas and recommendations.

Consider Fashiola. Fashiola is one of a new breed of specialized search engines focusing on a niche of the search market – in this case, online fashion retailers. While likely not yet on the radar of major brands – especially outside of Europe – Fashiola helps connect millions of users with over 1,800 different brands, representing a potential channel to reach consumers. If you were in the market for a pair of blue suede shoes, for example, you could just go to Fashiola, type in “blue suede shoes” and view the relevant offerings from over 1,800 brands.


Previous Page  1  2  3  Next Page 

Sign up for Computerworld eNewsletters.