Storage for scientific computing

The SCC operates storage systems for different purposes:

Storage systems at the clusters

So-called parallel file systems are directly connected to the clusters. These are characterized by a very high throughput performance and very good scalability. Since the beginning of 2005, Lustre has been used as the data system for the clusters at the SCC, which is currently used by the research high performance computers (ForHLR I and ForHLR II) and the state computer bwUniCluster. At present there are a total of 10 Lustre file systems with a storage capacity of 9573 TiB, 61 servers and 2600 clients on these HPC systems.

Further information:

  • User Guides of the various HPC systems.
  • Slides and presentations on productive Lustre installations on the page Lustre.

On-demand file systems
Future applications will place higher demands on the storage subsystem of HPC systems. To meet these demands, the SCC provides the functionality of an on-demand file system on current and future HPC systems, which will be created exclusively for an HPC job and will only be available on the allocated compute nodes during simulation. More information about this topic can be found here and on the corresponding documentation of the HPC systems.

Mass storage for scientific data (LSDF Online Storage)
The service "LSDF Online Storage" provides users of KIT with access to a data storage, which is especially designed for the storage of scientific measurement data and simulation results of data intensive scientific disciplines. The LSDF Online Storage is operated by the Steinbuch Centre for Computing. Access is guaranteed via standard protocols.

The backup and protection of the data is carried out according to the current state of the art. The service is not suitable for storing personal data.
More information: Mass storage for scientific data (LSDF Online Storage).

Archive storage (bwDataArchive)
The state service bwDataArchive offers a technical infrastructure for long-term archiving of scientific data.

bwDataArchive is available especially for members of universities and public research institutions in Baden-Württemberg. Data archiving is carried out at KIT and includes reliable storage of even large data sets for a period of ten years or more. The service enables a qualified implementation of the recommendations of the German Research Foundation (DFG) on good scientific practice (recommendation 7 on the securing and storage of research data).

Online storage for sharing data (bwSync&Share)
The state service bwSync&Share is an online storage service for employees and students of universities and colleges in Baden-Württemberg. It has been operated at KIT since January 1, 2014 and enables users to synchronize or exchange their data between different computers, mobile devices, and users. Access to the bwSync&Share portal via bwsyncandshare.kit.edu.