Skip to content

How a Spotify-like music service might evolve

A catalogue of tens of millions of tracks, playback that starts instantly, and recommendations that feel personal. How catalogue management, audio delivery, ML pipelines and autonomous teams shape each other.

The user problem and the hard parts

A listener wants any song, instantly, on any device, and something great to listen to next without searching. Artists and labels want accurate royalties. The company wants hundreds of teams shipping independently.

The hard engineering problems:

  • Catalogue: tens of millions of tracks with metadata from labels and distributors in many formats, rights that differ by country and change over time, and duplicates of the same recording.
  • Streaming: audio files are small compared to video, but start latency is everything. Pressing play should feel instant, including skipping tracks quickly.
  • Recommendations: personalised playlists and home screens built from billions of listening events. This is a data and ML (Machine Learning) system as much as a product feature.
  • Royalties: every play must be counted correctly per track, country and rights holder. Listening events are financial data.
  • Team autonomy: hundreds of small teams must change the product daily without a central bottleneck.

How the architecture might evolve

A Spotify-like music service

A plausible architecture for a music streaming service at each scale. Spotify has publicly described its squads-and-tribes organisation, its Backstage developer portal and its move to Google Cloud; other details are illustrative.

  1. 1
    Stage 1 · Launch
    1M users · 5M tracks
    Desktop + mobile clients with local cacheBackend services for auth, catalogue, playbackRelational database for users and playlistsObject storage for encoded audioCDN for audio delivery

    Audio is cached aggressively on the client and served from a . The first track starts fast because clients prefetch the beginning of likely next tracks.

  2. 2
    Stage 2 · Growth
    30M users
    Catalogue ingestion pipeline (label feeds → normalise → rights per market)Search service over catalogueEvent logging of every play to a central pipelineBatch ML jobs for weekly personalised playlistsWide-column store for playlists and user data

    Listening events become the core data asset: they drive royalties, charts and recommendations. Batch pipelines are enough for weekly playlists.

  3. 3
    Stage 3 · Autonomous teams
    100M+ users · hundreds of squads
    Hundreds of microservices owned by squadsDeveloper portal and service catalogue (Spotify open-sourced Backstage)Event streaming backbone with long retentionFeature store + real-time recommendation signalsExperimentation platform

    Each squad owns services end to end. A service catalogue answers "who owns this, how do I call it, is it healthy?" Long event retention lets new teams replay history to build new features.

  4. 4
    Stage 4 · Audio platform
    Global · music, podcasts, audiobooks
    Managed cloud data and compute (Spotify has publicly described moving to Google Cloud)Unified content model for different audio typesML platform for training and serving many modelsGolden paths: templates, CI/CD and observability by default

    New content types reuse the same playback, catalogue and recommendation platforms. Platform teams make the right way the easy way, which keeps hundreds of autonomous services consistent.

Play, log, recommend

A plausible architecture for Stage 3 might look like this. One button press touches three loops: delivery, event logging and the recommendation feedback loop.

Click a component

Click any component to see what it does.

Engineering trade-offs

Full squad autonomy over tech choices

  • Teams move fast and own outcomes
  • Local experimentation
  • Fragmented stacks, duplicated effort
  • Hard for engineers to move between teams
Choose when: Early in a growth phase, with strong ownership culture.

Autonomy on a paved road

  • Shared templates, CI/CD, observability and service catalogue
  • Squads still own their services
  • Platform teams to fund
  • Some teams feel constrained
Choose when: Hundreds of services; consistency becomes a productivity multiplier.

Batch recommendations

  • Simple, cheap, easy to evaluate offline
  • Doesn't react to what the listener is doing right now
Choose when: Weekly discovery playlists, catalogue-wide similarity.

Real-time signals + batch candidates

  • Responds to the current session
  • Reuses expensive batch work
  • Streaming feature pipelines to operate
  • Harder to debug
Choose when: Home screens and radio-style queues where context matters.

Practical exercise

Counting plays for royalties

Royalties are paid per play counted after 30 seconds of listening. Mobile apps log plays while offline and upload them later; network retries sometimes upload the same batch twice; some plays are fraudulent bot loops. Design the pipeline from app to monthly royalty report.