Apache Kafka

Apache Kafka is a distributed streaming platform that enables applications to publish, subscribe to, store, and process streams of records in real-time. It is designed for high-throughput, low-latency, and fault-tolerant handling of data feeds, commonly serving as a central nervous system for…

Key facts

First appeared
2011
Category
technology
Problem solved
Apache Kafka was created at LinkedIn to address the limitations of existing messaging systems in handling the company's massive scale of real-time activity data streams. Traditional message queues (e.g., ActiveMQ, RabbitMQ) struggled with the throughput, low-latency requirements, durable storage, and replayability needed for operational metrics, activity tracking, and data synchronization across distributed systems.
Platforms
Windows, Linux, macOS

Related technologies

Notable users

  • Uber
  • Netflix
  • New York Times
  • Pinterest
  • LinkedIn
  • Goldman Sachs
  • Walmart
  • Cisco
  • Spotify