We continue to review all events currently planned for the next sixty days and organizers will be notified if their event must be canceled, postponed, or held remotely. Please, check back on Indico during this time for updates regarding your meeting specifics.
As DOE O 142.3A, Unclassified Foreign Visits and Assignments Program (FVA) applies not only to physical access to DOE sites, technologies, and equipment, but also information, all remote events hosted by Fermilab must comply with FVA requirements. This includes participant registration and agenda review. Please contact Melissa Ormond, FVA Manager, with any questions.

Indico search will be reestablished in the next version upgrade of the software: https://getindico.io/roadmap/


   

This search is only for public events. Restricted events are not available.

 

18-22 May 2015
1200 Molecular Plant Science Building, Michigan State University
EST5EDT timezone
Registration and Abstract Submission Now Open!

NSLS-II Data Management

May 20, 2015, 4:45 PM
15m
1200 Molecular Plant Science Building, Michigan State University

1200 Molecular Plant Science Building, Michigan State University

15 minute contribution Services

Speaker

Arman Arkilic (Brookhaven National Lab)

Description

Each NSLS-II beamline can generate 72,000 data sets per day, over 2 M data sets in one year. The large amount of data files generated by our beamlines poses a massive file management challenge. In response to this challenge, we have developed filestore, as means to provide users with an interface to stored data. By leveraging features of Python and MongoDB, filestore can store information regarding the location of a file, access and open the file, retrieve a given piece of data in that file, and provide users with a token, a unique identifier allowing them to retrieve each piece of data. Filestore does not interfere with the file source or the storage method and supports any file format, making data within files available for NSLS-II data analysis environment.

Primary author

Arman Arkilic (Brookhaven National Lab)

Presentation Materials