Alexey Grigorev, DataTalks.Club
As product managers, we always want the best experience for users. So one of the issues to tackle is displaying only the listings of available products. Though it absolutely meets common sense, it’s easier said than done. And in this situation, ML models come in handy.
Being one of the quickest-developing trading networks and uniting 20+ brands, OLX Group every day faces the necessity to perfect the system. In this case study, with the help of the principal Data Scientist from OLX Group Alexey Grigorev, we discover how to develop a model for detecting sold items and how to know if it really works.
Don't disappoint users with unavailable goods
When users enter a marketplace and find the fitting listing — they get their hopes up. Normally they would message the owner of the item to get in touch. But what if the item has already been sold? It’s good if the seller immediately answers or deactivates the listing themselves. In this situation, disappointment is moderate. But another case scenario is that the seller is just silent for hours or maybe even days. And therefore, the experience of the buyer with your platform is deteriorating.
If you ask yourself, how many listings on our platform are actually active, would you be able to give the correct answer?
How to limit unavailable offers
As the solution, OLX Group developed a model for predicting how fast an item may be sold. To do this they used all available data:
- Category: there are categories that sell fast, and there are those that sell slower.
- Location: in bigger cities there are higher chances to sell something.
- Keywords in the title: items that are sold cheaper or in a good condition may get sold faster.
- Price: the lower the price — the faster it’s sold.
- Publishing date.
- Delivery: it increases the chances to be sold, especially for the offers from small towns.
- Seller’s history: some sellers are more efficient than others.
Potentially, it would be helpful to use the data from personal chats, but it is hard to legally organize it.
So all these are sort of feature values that we have. They change very infrequently. But there are things that change frequently, such as the number of clicks. When somebody clicks on the thumbnail, we can track it. Popular items receive more clicks. And if they receive more clicks, then there is a probability of being sold faster.
Once you've identified the items that are believed to be sold, here's what you can do.
Remind seller to deactivate the item
Right after the item was potentially sold, the seller would get a push notification saying: “Looks like you sold the item. Do you want to deactivate it?” So maybe the seller doesn't realize that when an item is gone, they should go and actually remove it from the platform, so it will be a small motivation.
Lower the position of the listing
If we knew that some items are not available anymore, we could somehow give them a lower position in the search results. So we make sure that the items that appear at the beginning are actually active, and those closer to the bottom maybe not anymore. So in this case, the sellers who are looking for items, will go through this list and would first contact the promising sellers.
How to measure the impact
Developing such a system may be tricky for an online marketplace, as the total number of listings on the platform is one of the core KPIs. For example, if we detect 10% to be inactive, then the total number of listings will also drop by 10%.
Another process we need to think about is the number of conversations people have on the platform, as they lead to a successful deal. This number will also go down, if people are able to find what they're looking for faster, it means they have to contact fewer people.
So even though all these are negative data for the platform, they signify the efficiency of the marketplace.
The KPI’s to track to see if the algorithm works:
- Number of listings goes down;
- Number of conversations goes down;
- Time before closing the deal gets shorter;
- Accuracy of listing predictions gets more precise.
Of course, we also want to see how this system affects the user. Therefore, we can look at the number of sellers who deactivated their listing after receiving a push notification and run a sort of online accuracy test. If the user deactivated the listings after the push, it means that the model was accurate, if not — the prediction might be wrong.
To learn in detail how OLX Group uses ML for deal estimation and how they build their processes, please watch the full video. You can access it for FREE with our 7-days trial.
Get product growth insights straight to your inbox: 💌 No spam! Just one newsletter a week ⏳Only takes 5 minutes to read (and become smarter) 😄 100% FREE (and if you don't like it, unsub in 2 clicks)