- \n
- 80 %<\/strong> – approximately this amount of a data scientist\u2019s time is spent preparing data for advanced analysis and machine learning, according to recent articles published by The New York Times<\/a>, Harvard Business Review<\/a> or Forbes<\/a><\/li>\n<\/ul>\n\n\n\n
\nAI is not just one particular computer program or algorithm. Instead,\nit\u2019s a system of several mathematical models. The ability of this\nsystem to emulate the human decision-making process relies on the\nright input information. Companies often suppose that more data leads\nto a better model. Unfortunately, it is not that easy.<\/p>\n\n\n\n
Big Data does not necessarily mean you will have a good AI model<\/strong><\/p><\/blockquote><\/figure>\n\n\n\n
The result depends on how well the data covers key drivers of the problem we are trying to solve. If successful, AI can save tens of percent. McKinsey Global Institute published the results<\/a> of its research which attempts to simulate the impact of AI on the world economy:<\/p>\n\n\n\n
- \n
- $13 trillion <\/strong>– that\u2019s the amount of additional global economic activity which could be delivered by 2030<\/li>\n\n\n\n
- 20 % <\/strong>– that\u2019s the amount of decline in cash flow from today\u2019s levels nonadopters might experience by 2030, assuming the same cost and revenue model as today<\/li>\n<\/ul>\n\n\n\n
\n\n\n\n\n
Are you new to AI and Machine Learning? If so, we recommend the first article<\/a> in our “Unlocking AI potential in e-commerce and online business” dealing with basic concepts.<\/p>\n<\/blockquote>\n\n\n
\n![\"Garbage](\"https:\/\/www.pbi.ai\/blog\/wp-content\/uploads\/2019\/08\/garbage_in-garbage_out_vert-311x1024.png\")
<\/figure><\/div>\n\n\n
\n\n\n\n\nA very common beginner\u2019s mistake is spending most of the time on\ndeveloping complex algorithms instead of brainstorming about the\nright data.<\/p>\n\n\n\n
- \n
- A simple model based on good features always beats a complex model based on poor features<\/strong><\/li>\n<\/ul>\n\n\n\n
Collecting data at random with no idea of how exactly to utilize it often leads to data inconsistency and bad data structure. Features extracted from data collected in such conditions are almost unusable for any kind of analysis or machine learning. Having some data is better than having no data, but, without proper business case, mathematical modeling is like looking for a needle in a haystack.<\/p>\n\n\n\n
Data worth storing<\/h2>\n\n\n\n
\nBy building AI for e-commerce, we often automate tasks and procedures\nwhich are being done manually by domain experts. A common way to\nutilize data is a data analysis done by hand followed by some action\nin response to the new insights. Thanks to machine learning, we can\nreplace this process by a mathematical model which performs an action\ndirectly based on real-time and historical data.<\/p>\n\n\n\n
- \n
- The right data for machine learning is often that which yields similar information to what a human expert would use to solve the problem<\/strong><\/li>\n<\/ul>\n\n\n\n
Data and ontology<\/h3>\n\n\n\n
Let\u2019s think about how this works in a real case. In our last article<\/a> we introduced a fictional company called Fictional Online Fashion Store which would like to personalize its website content in real time and offer customers relevant products. We already illustrated how to approach the project from the management point of view. How to handle the data selection process? How would a human expert proceed in a regular brick and mortar fashion store? When a new customer comes in, a salesperson performs a quick brainstorming:<\/p>\n\n\n\n
- \n
- Is it a new or an existing customer? <\/li>\n\n\n\n
- Is it a man or a woman? <\/li>\n\n\n\n
- How old is he\/she? <\/li>\n\n\n\n
- What is his\/her fashion style? <\/li>\n\n\n\n
- How wealthy is he\/she? <\/li>\n\n\n\n
- Does he\/she view goods slowly or quickly? <\/li>\n\n\n\n
- Does he\/she view goods which share a particular feature? <\/li>\n\n\n\n
- Does the current day, time or season matter? Is late evening shopping typical for some particular customer segment? <\/li>\n\n\n\n
- What does our competition offer in this area? <\/li>\n\n\n\n
- What is popular among celebrities and influencers? <\/li>\n<\/ol>\n\n\n\n
These brainstorming questions form the basis for an ontology <\/strong>– a definition of entities, attributes and their relations in the given context.<\/p>\n\n\n\n
In our case we have at least two entities – customers and products. Each customer has such attributes as shopping history, gender, age, fashion style, income and shopping behavior (questions 1-7). Similarly, each product in our fictional fashion store has such attributes as fashion style or target customer segment.<\/p>\n\n\n\n
The values of these attributes are used by the human expert (salesperson) to recommend relevant products in the given context (questions 8 – 10). How do we represent such information using available data sources?<\/p>\n\n\n
\n![\"Ontology](\"https:\/\/www.pbi.ai\/blog\/wp-content\/uploads\/2019\/08\/ontology_crop.png\")
<\/figure><\/div>\n\n\nKnowledge representation<\/h3>\n\n\n\n
\nLet\u2019s split the information mentioned earlier into several\nsegments:<\/p>\n\n\n\n
- \n
- Demographics\n\t<\/strong>– customer gender, age and income\n\t<\/li>\n\n\n\n
- Shopping\n\tbehavior history<\/strong> – customer fashion style based on his or her\n\tclothes and accessories, history of purchases in our Fictional\n\tOnline Fashion Store and history of interaction with newsletters and\n\tmarketing campaigns\n\t<\/li>\n\n\n\n
- Current\n\tshopping behavior<\/strong> – real-time interaction with the Fictional\n\tOnline Fashion Store website (viewing products, banner clicks)\n\t<\/li>\n\n\n\n
- External\n\tcontext <\/strong>– shopping and fashion trends for the current season or\n\ttime, what the competition offers, trends among celebrities and\n\tinfluencers\n<\/li>\n<\/ul>\n\n\n\n
\nWhen building an AI model, we need to think about the significance of\neach information segment and how well we can cover it using available\ndata sources.<\/p>\n\n\n\n
Demographics<\/h4>\n\n\n\n
Demographics such as age or income could be tricky to obtain due to the consumer privacy and data protection regulations. We can use a registration form but we never know whether the customer is telling us the truth. Luckily, we can estimate this information using correlated data.<\/p>\n\n\n\n
We often know the\nlocation, mobile device type and operating system which reveals the\npossible income category. An iPhone owner from a big city probably\nspends more on fashion compared to a cheap phone owner from a rural\narea. In the case of an existing customer we can combine this\ninformation with historical purchases to get a more precise estimate.<\/p>\n\n\n\n
Based on the visit\ntime and location we can guess the age group. The daily schedule of\nan economically active population differs from the schedule typical\nfor kids or seniors.<\/p>\n\n\n\n
\n\n\n\n\n
- Shopping\n\tbehavior history<\/strong> – customer fashion style based on his or her\n\tclothes and accessories, history of purchases in our Fictional\n\tOnline Fashion Store and history of interaction with newsletters and\n\tmarketing campaigns\n\t<\/li>\n\n\n\n
- Demographics\n\t<\/strong>– customer gender, age and income\n\t<\/li>\n\n\n\n
- The right data for machine learning is often that which yields similar information to what a human expert would use to solve the problem<\/strong><\/li>\n<\/ul>\n\n\n\n
- 20 % <\/strong>– that\u2019s the amount of decline in cash flow from today\u2019s levels nonadopters might experience by 2030, assuming the same cost and revenue model as today<\/li>\n<\/ul>\n\n\n\n
- $13 trillion <\/strong>– that\u2019s the amount of additional global economic activity which could be delivered by 2030<\/li>\n\n\n\n
![\"Ondrej](\"https:\/\/www.pbi.ai\/blog\/wp-content\/uploads\/2020\/04\/B2B6716-1_crop1-948x1024.jpg\")
<\/figure>\n","protected":false},"excerpt":{"rendered":"