Sadly once you like one song that’s been on the radio once, it starts spiralling into other songs (often good even) you know from the radio and 0 other songs. With things that kind of come in sets (like “songs that played often on X channel in the 90s”) it becomes quickly a game of complete the set rather than discovering new music you’d also like.
There collaborative filtering algorithms do tend to have a popularity bias. The other downside is that these algorithms also don’t help new music and musicians get found.
I’ve worked in recommender systems for news specifically myself for a couple years so if anyone has some questions that aren’t too identifiable, AMA I guess
We had a system that combined your reading history (from a tracking pipeline that already existed similar to google analytics) (though it could have just as easily been sent from the front-end with the request for recommendations as we didn’t precompute anything) with several scoring systems (from simple things like popularity score per article which ignores your history, to multiple complex pretrained models that use your history to calculate a score optimising for some variable), all of which can be weighed and then the scores are added/multiplied and sorted and bam, out rolls your personal list of recommendations.
We could easily tone down (even turn negative) the populatity bias, but it turned out that it was just a strong predictor for what people wanted to read (measured in both click through rate and dwell time on the clicked page), so we’re not entirely sure whether news is just different (if you spent a couple of minutes per day scrolling past headlines you’ll have seen everything from today, and clicked what you cared about, and left again) or we weren’t really catering to the crowd that would be helped by getting non-popular recommendations, because they’re drowned out by the crowd that’s just looking for whatever’s popular.
Sadly once you like one song that’s been on the radio once, it starts spiralling into other songs (often good even) you know from the radio and 0 other songs. With things that kind of come in sets (like “songs that played often on X channel in the 90s”) it becomes quickly a game of complete the set rather than discovering new music you’d also like.
There collaborative filtering algorithms do tend to have a popularity bias. The other downside is that these algorithms also don’t help new music and musicians get found.
Yes, though I’d argue that’s the same downside :D
I’ve worked in recommender systems for news specifically myself for a couple years so if anyone has some questions that aren’t too identifiable, AMA I guess
We had a system that combined your reading history (from a tracking pipeline that already existed similar to google analytics) (though it could have just as easily been sent from the front-end with the request for recommendations as we didn’t precompute anything) with several scoring systems (from simple things like popularity score per article which ignores your history, to multiple complex pretrained models that use your history to calculate a score optimising for some variable), all of which can be weighed and then the scores are added/multiplied and sorted and bam, out rolls your personal list of recommendations.
We could easily tone down (even turn negative) the populatity bias, but it turned out that it was just a strong predictor for what people wanted to read (measured in both click through rate and dwell time on the clicked page), so we’re not entirely sure whether news is just different (if you spent a couple of minutes per day scrolling past headlines you’ll have seen everything from today, and clicked what you cared about, and left again) or we weren’t really catering to the crowd that would be helped by getting non-popular recommendations, because they’re drowned out by the crowd that’s just looking for whatever’s popular.
Anyway, AMA
this is my annoyance too