June 24, 2016

Data processing

Gaia data processing

The ground processing of Gaia data, which is to be carried out by the European scientific community, represents a very large-scale scientific and technical enterprise.

ESAC at Villafranca is in charge of the initial processing, the University of Cambridge, of the photometric processing, and the Geneva Observatory, of processing variable objects.

CNES planned and set a computer science architecture, hardware and software, to operate the spectroscopic, astrophysics, and classification processing software programmes for Gaia data during the entire mission, from 2014 to 2019. For every cycle (from five months to one year), this super-system will process all datasets of the Gaia preliminary catalogue. At the end of the mission, it will conduct the final processing that will exploit the data collected during the five-year mission, to create the final Gaia catalogue.

Process a billion objects in six months implies being able to process one object in 1/60 second! The enormous computation power needed to take up this challenge is evaluated at 6000 GFlops/sec (6000 billion operations per second). The cluster required to perform these calculus will reach 6000 processor cores at the end of the mission.

The CNES Gaia team is constituted of about 10 engineers, supported by the technical and industrial assistance.