Big Data Spain

17th ~ 18th NOV 2016 MADRID, SPAIN #BDS16

Shortening the Feedback Loop: How Spotify’s Big Data Ecosystem has evolved to produce real-time insights

Thursday 17th

from 17:15 to 17:55

Theatre 25



Spotify launched in 2008, making an enormous catalog of music accessible to users in many countries all over the world. Users were amazed at the instantaneous nature of playback-- gone were the days of waiting for music to buffer before listening. Behind the scenes, deriving insights for events-- from clicks to searches, skips to favorites/saves-- was a much slower process, often taking multiple days.

In this talk, Josh will present an overview of the big data processing ecosystem at Spotify. He will then discuss how this infrastructure has evolved over the last few years to shorten the feedback loop from days to seconds, particularly how it has been able to leverage tools like Pub/Sub, Dataflow and BigQuery from Google’s Cloud in this effort.

Josh Baer foto

Josh Baer

SpotifyTechnical Product Owner - Real Time Data