Kinesis Client Library (KCL)
The Kinesis Client Library (KCL) is a client-side library developed by Amazon Web Services (AWS) that simplifies the process of consuming and processing data from Amazon Kinesis Data Streams. It abstracts away complex tasks such as connecting to the stream, load-balancing processing across…
Key facts
- First appeared
- 2013
- Category
- technology
- Problem solved
- KCL was created to simplify the development of distributed stream processing applications. It addresses the significant challenges developers faced in managing shard discovery, distributing work across multiple consumer instances, checkpointing processed data to ensure fault tolerance and 'at-least-once' processing semantics, and handling dynamic stream changes like resharding, which were previously arduous to implement manually using the raw Kinesis Data Streams API.
- Platforms
- macOS, Windows, JVM (Java, Scala, etc.), Linux
Related technologies
Notable users
- Netflix
- Expedia
- Many enterprises using AWS for real-time data processing