A Blockchain Based Distributed File System for the Integration in MPI

Distributed computing is an inevitable part of data processing as huge volume of data being produced from different sources like social sites, IoT, mobile data, sensor data, black box data and so on in an exponentially fast manner. Distributed File System defines how to organize and distribute files among different cluster computing nodes. One of the key points to be addressed was how to manage the metadata for the effective retrieval of the data distributed among different nodes, like Master File Table in NTFS. The popular distributed file systems such as Google File System and Hadoop Distributed File System stores metadata centrally. Hence the Single Point of Failure issue arise the need of backup and alternative solutions to recover the metadata on failure of metadata server. Name node server is built using an expensive and reliable hardware. For small sized clusters, maintaining an expensive hardware for the implementation of name node server remains difficult. The cheap commodity hardware substitutes the name node functionality, but they are prone to hardware failure. This thesisincorporates a novel DFS to distribute files over cluster of machines connected in a P2P network.