Inception of Flume and Millwheel (Internal Google)
Google develops internal streaming systems like Flume and Millwheel to handle its massive real-time data processing needs, paving the way for the unified Dataflow model.
Significance
These internal systems demonstrated the power of stream processing at scale and highlighted the complexities of managing disparate batch and stream pipelines, directly inspiring the unified Dataflow approach.
Key facts
- Year
- 2010
- Type
- invention
- Location
- Google, Mountain View, CA