This is the weekly DUNE Data Management Meeting. We discuss the Data Management Project for the upcoming DUNE experiment, and also data management operations issues. Attendees are people who are actually working on data management plus higher DUNE computing management as required.
Data management meeting 7/9/24
5 TB/hour
Steps to rucio activities
(1) Have to set DAQ activity in Rucio
(2) have to set DAQ activity in FTS3. Both have to match
Suggested “DAQ” activity
Suggested “Neardet” activity
Suggested “Data Challenge” activity
User subscription stay
Suggested “Dataset Transfer” activity
Suggested “Data Processing” activity
Suggested “MC processing” activity
(3) need declaD to know about these activities
Steve to open Fermi service now ticket to Brandon
And CERN service now ticket open
—————
2x2 update
Minerva data doesn’t have a right time stamp
Can’t be processed by declaD
Something is an string that should be an int, Doug is investigating
2x2-light and 2x2-charge data has been declared
Should move some light data into the production area as soon as possible
Matt: directory for charge and light
/data/DAQ/rawdata
Have pulled from acd-daq01, acd-daq02, acd-daq04
Alex has been working on mainline ND production
ND-reco-sim meeting 9 AM Wed.
——
Discussion of moving histograms to Fermilab
(Heidi’s ticket)
It is taking more than 5 minutes per histogram file to copy them into a worker node live
Via rucio, will go faster if all are at FNAL. Will work on getting files to Fermilab in her absence.
End goal should be to bring the things into a production workflow.
Keep up processing—Jake will be talking to DRA conveners for final sign off on data processing
No issues with huge run time
————
Rucio clients—Steve gives Dennis Lee the go-ahead to deploy the rucio add-user script as is
Dennis needs a vm to upload his python_gfal2_python tarball to, this will be fermicloud744
Some unexpected code issues on Doug’s declad decladgpvm01 somehow the patches Steve made in his PR are not there and we don’t understand why. (They are in trunk when looking at GitHub).