High-Performance Data Storage for Large Solar Telescope Using Distributed File System

ABSTRACT

New Vacuum Solar Telescope (NVST), which is an altazimuth mounting telescope, is generating data more than 2 TB per day on fine observing conditions. These data impose a great pressure on the performance of real-time data transferring and scalability of the centralized data storage. Our investigation reveals that the combination of commodity servers and the distributed file system is a cost-effective alternative solution to these issues. In this paper, We study the distributed file system to handle the massive data generated by multi-channel data acquisition system under the solar observational environment. An aggregated FITS file based on the feature of the FITS format is employed to further improve the performance of distributed data storage. Experiments show that one single channel can provide the highest writing speed at 420 MB/s under 1 Gb network by using bonding technology. The real-time data transferring rate can supply data capture more than five times than the current in NVST. The works of this paper also provide a reference to highperformance data storage of other large telescopes.

Keywords: