Large-Scale Data Management & Analysis
The amount of data collected by science is rising fast and in many sciences the increasing amount of is are reaching the limit of established data handling and processing. Data is the basis of modern research and the key to new knowledge and competitive development lies in efficient and effective data management and subsequent analysis. The projects LSDF and LSDMA address the requirements of modern science for special purpose facilities, infrastructures and support to handle the data life cycle for large amounts of data. LSDMA and LSDF are very close. LSDMA builds on services and infrastructure the LSDF provides.
With funding of the state of Baden-Württemberg the project bwLSDF evaluates technologies to allow research institutes and Universities state wide flexible access to central and federated storage and archives. At completion, bwLSDF will provide modern secure shared storage services accessible from pda, desktop or HPC clusters for the scientific community in Baden-Württemberg.
Big Data Spin Off
The startup company da-cons excels in applied analysis, visualisation and archival of big image data sets, primarily in the area of biology and medicine. Their novel software helps scientists to get information from images and is based on developments and expertise from various departments of KIT. Together with SCC researchers da-cons brings science to public enterprise. da-cons received funding from BMBF through the EXIST program.
Modern science and scientific computing is about data. In the process from collecting data till publication, the data has been moved, aggregated, selected, visualized and analyzed. Regarding the steadily increasing amounts of data this process must be organized and structured. Data management is the organization and structuration of the data life cycle which will allow faster results and dependable long term references.
ReaReaching the fundamental goal of sustainably improving data analysis chains and data life cycles also depends on availability of data management components and their development. Standardized and generic tools have to be provided and have to be promptly researched, developed and established in a joint R&D program, run by data specialists and driven by user communities.
These two activities are reflected by the LSDMA project structure: several Data Life Cycle Labs are closely connected to five of the six Helmholtz Association research fields enhanced with a Data Services Integration Team. Research focuses on:
- Data-Intensive Computing and Application
- Migration, Preservation und Curation
- Universal Data Access
- Storage System Design
The project LSDMA started on January 1st, 2012. Its initial phase ends on December 31st, 2016, but the project is projected to be integrated into the sustainable Program oriented funding framework of the Helmholtz Association. The project partners for the initial phase are four Helmholtz Association research centers, namely DESY, FZJ, GSI and KIT, as well as six universities, namely HTW Berlin, Technical University of Dresden, University of Frankfurt, University of Hamburg, University of Heidelberg, University of Ulm, and the German Climate Research Center DKRZ.
In September 2012 KIT hosted the first International LSDMA Symposium: The Challenge of Big Data in Science . In March 2013 the LSDMA Community Forum took place at DESY.
Contact: Christopher Jung, email: christopher jung ∂kit edu