FPGA-enhanced consistency for distributed data storage in the big-data era

This master project will investigate novel use of programmable and re-configurable network adaptors for efficiently enabling data consistency which is vital for the performance of modern distributed data storage.
Master

Big data requires efficient data storage solutions. Commercial companies and non-profit organizations alike have become aware of the critical role of data storage growing at an accelerating pace, as well as the importance of fast and scalable data access for gaining a competitive edge in the market or providing high-quality service to the society. To achieve both scalability and capacity, distributed data storage solutions have been increasingly adopted. A main challenge is to accelerate access to massively distributed data, particularly when dealing with online services such as Facebook, Amazon, Microsoft, Google, etc. In connection with distributing the data over the memories of many machines in a data center, one of the latest strategies is to access the data through the RDMA technology. RDMA enables access to the memory of machines over the network, bypassing the operating system and CPU, thereby significantly reducing the latency of the process. However, such bypasses bring challenges to guaranteeing the correctness of data since different accesses are allowed from different machines over RDMA on the same memory.

Goal

To guarantee the correctness of data, also known as data consistency, RDMA supports atomic operations that each can modify a memory address while guaranteeing that other operations on this address do not occur between the modifications. However, from a performance perspective, atomic operations have some limitations. In this thesis, we are going to use a special type of network adaptor which is equipped with a Field Programmable Gate Array (FPGA) to enhance the performance of the RDMA atomic operations. The result of this master project will have the potential of improving all the systems that require correctness through atomic operations.

Learning outcome

In this thesis, you will learn state-of-the-art technologies that are heavily exploited in a large-scale data center. Specifically, you will learn RDMA fundamentals and programming an FPGA. Moreover, you will learn state-of-the-art techniques in distributing data over shared memory and using atomic operations.

Qualifications

It is beneficial, but not mandatory, if you have some knowledge on networking and operating systems as well as FPGA programming. It is however vital that you are interested in learning both the fundamental knowledge (such as RDMA fundamentals & atomic operations) and the latest technological solutions. Moreover, you will get support to organize the work and design new systems through FPGA programming.

Supervisors

  • Ernst Gunnar Gran
  • Xing Cai
  • Masoud Hemmatpour (main supervisor, Simula, mashemat@simula.no)

Contact person