Another company had their own system to determine which customers were sent certain offers. Yandex would use the recommendations of a statistical model produced by a machine learning algorithm to determine how a random slice of the customer base was contacted. The rest of the customers were contacted according to the previous system, and the company then compared the conversion rates of offers into sales.
The only problem was that the offers were sent to the control group on Friday and to the experimental group on the weekend. The diverse patterns of behaviour at the different times of contact made any comparison meaningless.
Business managers often ask Yandex whether they should take courses in machine learning or data science to understand how the technology could benefit their organisations.
"What we usually answer is actually no, it doesn't make any sense," says Zavalishina. "It won't make you data scientists, so it won't really help you. If you want to be able to apply the technology in your work, you are much better off learning the scientific method and measuring and experimentation. Basically, we need a more scientific approach in the business if you want this technology to bring results."
Businesses need to embrace the scientific culture. Negative results don't mean the work has failed, they only prove that the optimisation didn't work.
The responsibility within the corporate structure is another challenge. Yandex was once approached by a client hoping to optimise its advertising spending. The algorithm developed promised the same level of response while saving 20 percent in costs.
Implementing the results proved more challenging than attaining them. The staff responsible for this project were paid bonuses based on their plans and decisions behind what they should buy to achieve optimal results.
"So now they have this model, which provides them with recommendations, and mathematically it is proven that the recommendations were better, but the problem is it's their responsibility," explains Zavalishina.
Data science projects acknowledge for the different responsibilities and priorities that can exist in the same business. This team was expected to implement a model that could result in cuts to their bonuses.
"When it comes to a scientific approach it's much more rational, much more measurable, and this can be quite a conflicted situation," adds Khaytin.
"The usual decision-making purpose is going to be at least disrupted. For example, an expert can tell you 'I have an intuition, I have an idea, it's going to be that way'. In our hands you have some data science tool, some data science project and it's totally different, there is no intuition, there is no place for it."
Integrating business and scientific approaches is a complicated process that requires patience and understanding. Yandex also worked with a steel manufacturer on optimising the balance behind the mixture of materials used in the production process. The quality was improved by increasing the quantity of a certain substance, but the more of this substance that was added the more expensive the production.
Sign up for Computerworld eNewsletters.