About HDFS:
HDFS is a Java-based file system that provides scalable and reliable data storage, and it was designed to span large clusters of commodity servers , in other words it is a distributed Java-based file system for storing large volumes of data.HDFS forms the management layer along with the YARN.HDFS has demonstrated production scalability of up to 200 PB of storage and a single cluster of 4500 servers, supporting close to a billion files and blocks.By distributing storage and computation across many servers, the combined storage resource can grow linearly with demand while remaining economical at every amount of storage.HDFS entails following features which ensures high availability , fault tolerance , scalability and efficient storage of data.
Rack awareness : Takes node's physical allocation for scheduling tasks.
Standby NameNode: Main component for providing redundancy and high availability
Less Data Movement: Processing of tasks takes place in the physical node where the data resides , hence data movement is reduced ,increasing high aggregate bandwidth
HDFS is a Java-based file system that provides scalable and reliable data storage, and it was designed to span large clusters of commodity servers , in other words it is a distributed Java-based file system for storing large volumes of data.HDFS forms the management layer along with the YARN.HDFS has demonstrated production scalability of up to 200 PB of storage and a single cluster of 4500 servers, supporting close to a billion files and blocks.By distributing storage and computation across many servers, the combined storage resource can grow linearly with demand while remaining economical at every amount of storage.HDFS entails following features which ensures high availability , fault tolerance , scalability and efficient storage of data.
Rack awareness : Takes node's physical allocation for scheduling tasks.
Standby NameNode: Main component for providing redundancy and high availability
Less Data Movement: Processing of tasks takes place in the physical node where the data resides , hence data movement is reduced ,increasing high aggregate bandwidth
 |
| Features of HDFS |
PROGRESS IN HDFS:
- HDFS version 2.3.0 provides centralized cache management heterogeneous storage, implementation of openstack swift and HTTP supports
- Version 2.4.0 provides metadata compatibility, rolling upgrades (allows upgrading individual HDFS daemons).
- Version 2.5.0 provides incremental data copy , extended attributes for accessing metadata.
No comments:
Post a Comment