Apache Hadoop HDFS

Apache Hadoop HDFS (Hadoop Distributed File System) is a highly scalable, fault-tolerant distributed file system designed to run on commodity hardware. It provides high-throughput access to application data and is a core component of the Apache Hadoop framework, ideal for storing massive…

Key facts

First appeared: 2005
Category: technology
Problem solved: Apache Hadoop HDFS was created to address the growing challenge of storing and processing extremely large datasets (terabytes to petabytes) generated by internet-scale applications, such as web search engines, where traditional centralized file systems and relational databases were prohibitively expensive, not scalable, or lacked fault tolerance for commodity hardware failures.
Platforms: Unix-like operating systems, Linux

Related technologies

Apache Hive
Apache HBase
Apache Hadoop MapReduce
Apache Flink
Apache YARN
Apache Pig
Apache Spark
Apache ZooKeeper
Apache Kafka

Notable users

Hulu
Twitter
eBay
Various financial institutions and government agencies
LinkedIn
Facebook
Netflix
Yahoo!