DUNE Data Management Meeting

Tuesday 1 Sep 2020, 09:00 → 10:00 US/Central

203 (Feynman Computing Center)

This is the weekly DUNE Data Management Meeting. We discuss the Data Management Project for the upcoming DUNE experiment, and also data management operations issues. Attendees are people who are actually working on data management plus higher DUNE computing management as required.

Data management 2020 Sep 1

 

Discussion:

 

Plans for the collaboration meeting

 

Data mgmt

Network

Database

Framework

Tutorial

 

Next week metadata Taskforce

And every other week for about 4 meetings.

Plug in Igor

Plug in ND stuff

 

 

 

Comments on Peter’s question

 

Discussion—Igor — can local worker nodes have disk space

Robert/ don’t want to =support other weird SE’s

Steve—local bandwidth to disk is now a big issue, you are IOPS bandwidth-limited

Code tarballs, constants, etc.

 

Steve will get numbers for disk usage to Peter later today.

 

Kirby’s numbers off by a factor from what Heidi has in docdb.

 

—————————

 

 

Metadata server

 

Data Dispatchers — Igor and Steve and Robert and Brandon will meet tomorrow

 

Runs DB planning

 

Rucio deployment / ingest

Not all data yet in Rucio—need to get a plan to get it there.

Steve will try to flesh out the development plan.

 

Most important Rucio feature the non-deterministic path for archival storage.

 

Try to get Nicole’s stuff into Rucio this week.

 

——

Have to get IN2P3 stuff to the new xrootd-based SE

New SE may be different storage than the one currently in Rucio, Wenlong will check.

 

——

Have to reconcile SAM script with the other stuff.