Hadoop Cluster – The Anatomy of Hadoop Pipeline Write

Now we know about Big Data, HDFS, Map Reduce and different types of Hadoop Nodes. In this post, I will be touching the how the client writes the files with in the Hadoop Cluster with different options. A client is having a file 1 which is split into three blocks named A, B and C as depicted in the below figure.

Writing a block on the different data nodes with in same rack space:-
Step 1:- In the client will send its request to “name node” by saying I have three blocks of file 1. Please let me know how could I write these blocks to which nodes.
Step 2:- Name Node will reply by returning the names of data node 1 is used for Block A, data node 2 is for block B and data node 3 is for block C.
Step 3:- Client will write block A on data node 1. Block B writing will not start till the client will get the acknowledgement of block A. Thereafter it will write C.


Share on facebook
Share on twitter
Share on linkedin

Leave a Reply

Your email address will not be published. Required fields are marked *

Become a member

Full Access to 739 Lessons. New Lessons Added Every Week!

Awesome Deal! Get 2 Months for FREE!

No Obligations. Cancel At Any Time!