File Systems

HDFS (Hadoop Distributed File System) is a highly fault-tolerant, scalable distributed file system designed to run on commodity hardware. It provides high-throughput access to application data and is suitable for applications that deal with large datasets, leveraging a write-once, read-many…

Key facts

First appeared
2006
Category
technology
Problem solved
HDFS was created to solve the challenge of reliably storing and processing petabytes of data across large clusters of inexpensive, commodity hardware. Traditional file systems and relational databases struggled with the scale, fault tolerance, and throughput required for web-scale data processing, especially for batch analytics.
Platforms
Windows (primarily for development/testing, less common for production), Unix-like operating systems, Linux

Related technologies

Notable users

  • Twitter
  • Facebook
  • LinkedIn
  • Yahoo!
  • Netflix
  • Many large financial institutions
  • E-commerce companies