Apache ORC
Apache ORC (Optimized Row Columnar) is a self-describing, columnar file format optimized for big data workloads in the Hadoop ecosystem. It provides a highly efficient way to store data, offering significant improvements in compression and query performance compared to traditional row-oriented…
Key facts
- First appeared
- 2013
- Category
- technology
- Problem solved
- Apache ORC was created to address the inefficient storage and slow query performance of big data in Hadoop, particularly for analytical workloads processed by Apache Hive. Traditional row-oriented formats (like TextFile or SequenceFile) required reading entire rows, even when only a few columns were needed, leading to excessive I/O. Its direct predecessor, RCFile, offered columnar storage but had limitations in compression, complex type handling, and overall query optimization. ORC aimed to significantly improve upon these, providing better compression ratios, faster query execution through predicate pushdown, and more robust support for diverse data types.
- Platforms
- Hadoop Ecosystem, Windows, Linux, macOS
Related technologies
Notable users
- Netflix
- Google (internally uses similar columnar principles)
- Uber
- Any enterprise leveraging Apache Hive, Spark, or Presto for large-scale data analytics