Apache Spark Streaming
Apache Spark Streaming is an extension of the core Apache Spark API that enables scalable, high-throughput, fault-tolerant processing of live data streams. It ingests data in mini-batches, processes them using Spark's batch processing engine, and then outputs the results, providing a 'stream of…
Key facts
- First appeared
- 2013
- Category
- technology
- Problem solved
- Apache Spark Streaming was created to bridge the gap between batch and real-time data processing, allowing organizations to process continuous streams of data with the same high-level APIs and fault-tolerance guarantees as traditional batch jobs. It addressed the complexity and lack of fault tolerance inherent in earlier stream processing systems, enabling more reliable and scalable real-time analytics and transformations.
- Platforms
- Kubernetes, Apache Hadoop YARN, Python, Apache Mesos, R, Standalone Cluster, JVM (Java, Scala)
Related technologies
Notable users
- Uber
- Netflix
- Alibaba
- Databricks
- Various companies leveraging Spark for Big Data analytics