Home My Page Projects FastBit indexing software
Summary Activity Lists Docs News SCM Files

Forum: NVST DATA ARCHIVING SYSTEM BASED ON FASTBIT NOSQL DATABASE

Posted by: John Kesheng Wu
Date: 2014-07-15 22:56
Summary: NVST DATA ARCHIVING SYSTEM BASED ON FASTBIT NOSQL DATABASE
Project: FastBit indexing software

Content:

Abstract: The New Vacuum Solar Telescope (NVST) is a 1-meter vacuum solar telescope that aims to
observe the fine structures of active regions on the Sun. The main tasks of the NVST are high resolution
imaging and spectral observations, including the measurements of the solar magnetic field. The NVST has
been collecting more than 20 million FITS files since it began routine observations in 2012 and produces a
maximum observational records of 120 thousand files in a day. Given the large amount of files, the effective
archiving and retrieval of files becomes a critical and urgent problem. In this study, we implement a new
data archiving system for the NVST based on the Fastbit Not Only Structured Query Language (NoSQL)
database. Comparing to the relational database (i.e., MySQL; My Structured Query Language), the
Fastbit database manifests distinctive advantages on indexing and querying performance. In a large scale
database of 40 million records, the multi-field combined query response time of Fastbit database is about
15 times faster and fully meets the requirements of the NVST. Our study brings a new idea for massive
astronomical data archiving and would contribute to the design of data management systems for other
astronomical telescopes.

http://jkas.kas.org/journals/2014v47n3/v47n3p115_wang.pdf
Latest News

IMG uses FastBit to accelerate Gene Context Analysis

John Kesheng Wu - 2013-10-15 17:15 -

FastBit used by Cold Spring Harbor Laboratory

John Kesheng Wu - 2013-10-15 17:07 -

Sifting Through a Trillion Electrons

John Kesheng Wu - 2012-10-19 05:47 -

Jochen Schlosser completed PhD with highest honor

John Kesheng Wu - 2011-03-30 15:20 -
Monitor Forum | Start New Thread Start New Thread