TODAY Sep. 8 DUNE Data Management meets jointly with the Far Detector DAQ Performance Taskforce to discuss the contents of the metadata. This is the weekly DUNE Data Management Meeting. We discuss the Data Management Project for the upcoming DUNE experiment, and also data management operations issues. Attendees are people who are actually working on data management plus higher DUNE computing management as required.
Data management meeting notes 9/15/2020
Discussion with Edinburgh people
Understanding where the data is, and how to access the closest data in the job
Could play into the “Data Dispatcher”
Wenlong—should focus on the Rucio in the future
Who needs to be involved from US side, Steve, Ken, Robert.
Steve will send a doodle poll
—————
Metadata and trigger records across multiple files.
Challenge, how do we store thousands of integer numbers associated.
List of integers is allowed but may not scale well for syncing.
List are not scary anymore.
Indices will be comparable in size to the table itself
But not bigger than the data.
Look at the metadata schema, Igor shares
—\
Collaboration meeting agenda
Intro
Rucio Ingest
MetaCat
Data Dispatcher
Early plans for data location in analysis
Futures/ AOB