Netflix has two sets of Apache Kafka cluster: Fronting Kafka and Consumer Kafka.

Fronting Kafka clusters are in charge of obtaining messages from producers, which essentially every Netflix application instance is. They serve as data collectors and buffers for systems farther down the line. Consumer Kafka clusters contain a subset of topics routed by Samza for real-time consumers.

In 2016, Netflix already operated 36 Kafka clusters whcih consist more than 4,000 broker instaces for both Fronting Kafka and Consumer Kafka. More than 700 billion messages are ingested with in a day.

Spotify is a digital music, podcast, and video service that gives you access to millions of songs and other content from creators all over the world. It has 82 Million songs on its platform while 422 million active Spotify users as of the first quarter of 2022.

Even though Spotify had decided to migrated from Apache Kafka to Google Cloud Pub/Sub in 2016, the way that Spotify uses Apache Kafka on their platform is still intriguing to observe: Whenever a user listens to a specific song or podcast, Spotify logs the experience as an event and utilizes it as data to learn more about the user's preferences. Additionally, Spotify records infrastructure-level events, such as when a logging server runs out of disk space. Events of more than 300 different types are gathered from Spotify users.