Robot Rock: How deep learning is central to Spotify’s strategy
Credit: Engadget
Spotify’s Discover Weekly is extremely popular with users for finding new music. Many however don’t realise that deep learning is at the core of the feature.
Many feared that the internet would kill albums. Cherry-picking highlights from an LP is now common and AI is central to the next chapter in music’s evolution.
Nearly 60 years after the Beatles revolutionised the way that we think about pop music, we’ve arrived at the playlist. The playlist is now more dominant than the album: perhaps a natural progression from the days of recording the songs you heard on the radio onto a cassette tape.
Playlists offer customisation. They can allow listeners to pick their favourite songs from their favourite artists and place them into a potentially endless stream, based on mood, genre or even an event. Playlists are more than just compilation devices, though. For many, they have replaced radio DJs.
Music curation – like genre, styles and trends themselves – has been through its fair share of change. We began with individuals who compiled playlists that they thought others would like before they were replaced by Pandora’s tagging system. Tagging enabled users to search for a label they liked and gave them recommendations. However, tagging is time-consuming. Music is subjective. When it comes to something as personal and divisive as music, there is always going to be difference.
Spotify’s Discover playlists are some of the most accurate ways to get pinpoint music recommendations. So when you’re looking for new artists, how does the Swedish service manage to so accurately hit the nail on the head, without tagging and without DJs?
How does Spotify use deep learning?
Spotify has been working with machine learning for over a decade. What started out as one man working on AI in the company as a kind of side-project has blossomed into a much bigger aspect of the company. Rather like how machine learning itself is typically comprised of multiple algorithms challenging each other, Spotify has a number of ways that they generate music recommendations.
One of the primary ways that they do this is through Natural Language Processing (NLP). NLP analyses huge swathes of natural language data and this can include speech recognition. In the case of music recommendations, Spotify crawls the internet for written text about music to find out what words are used when talking about particular songs, finding out which other songs are mentioned in relation to them. So if, for example, there is a lot written on the internet about Beyoncé in relation to Rihanna, the NLP algorithms will take this into account and recommend Rihanna songs to Beyoncé fans.
Deep learning keeps pushing itself to learn more. It’s perfect for a streaming service that wants us to stay plugged in and listening.
Spotify also employs Convolutional Neural Networks (CNN) to assess the waveform data of a song. This is a form of deep learning that looks at a minimum of three layers of nodes: an input layer, a hidden layer and an output layer. CNN is most often used in image recognition, often for faces.
Spotify uses CNNs slightly differently though, as they apply them to sound rather than vision. Audio frames are analysed for attributes such as tempo, loudness and key so that Spotify can group together songs with similar characteristics.
Spotify also uses a Python library to run a matrix formula. It compares each user for the songs they listen to against other users and the songs that they listen to, so from here, you can compare a song’s data with other songs and find out which are similar.
Why is deep learning so important to entertainment?
Much to our parents’ disgust, music is ever-changing. The tastes that we had when we were 20 change by the time we’re 30. The songs that we can’t get enough of grow old very quickly and that’s a challenge for Spotify.
Deep learning keeps pushing itself to learn more, assess more data and come out with better results. It’s perfect for a streaming service that wants us to stay plugged in and listening. Deep learning across the entertainment industry, however, is offering deeper personalisation than marketing has ever managed before.
Deep learning identifies patterns in how people consume content. It assesses what you may call “micro-preferences”, from when a consumer is interested in a product to how they may react next. A feedback loop is a crucial part of AI learning, as the machine determines whether the decisions that it made were the right ones or not. Ultimately, the more data that a company has on you, the better a deep learning algorithm can “get to know you” and offer recommendations.
According to Netflix, 75% of what consumers watch on their service comes from their recommendation algorithms. Consider too that Netflix is well aboard the hyper-personalisation train. The service will generate a random thumbnail for a TV show based on the content that you watch. So if you’re particularly into horror movies, expect to see a darker movie poster than someone who is into romantic comedies: they might see a leading couple as their thumbnail, regardless of whether the series focuses around that couple.
We’re not that far away from movie trailers being cut and spliced based on what you’re more likely to respond to. If YouTube, for example, could harness the power of deep learning to learn what movies you enjoy most, it could easily randomly generate a trailer for a new movie based on the characteristics you’re likely to respond to.
Or, perhaps even generate a movie based on what you’d like.
Deep learning is integral to the future of music
Incredibly, Spotify CEO Daniel Ek didn’t originally like Discover Weekly.
“I would have killed that if it was just me, 100%,” said Ek of the feature. “I thought, ‘Oh, this is going to be a disaster.’ And then obviously it turned out to be something really successful. It’s one of the most loved product features that we have.”
It’s perhaps poetic that Ek didn’t rate deep learning, in the same way that many critics disliked the Beatles in the 1960s. Both AI and the Fab Four are revolutionary to popular culture: not just music.
Music is ever-changing but the wheel revolves. Trends spike into fashion before fading from view. The way that we consume music will continue to develop but with deep learning integrated into the world’s biggest music consumption platforms, recommendations will no doubt develop with the sounds we listen to.
