DUNE Workflow/load Development 9 January 2023 --------------------------------------------- https://indico.fnal.gov/event/57664/ Present: Andrew McNab (AM) (chair/notes) Steve Timm (ST) Doug Benjamin (DB) Chris Brew (CB) Heidi Schellman (HS) Jake Calcutt (JC) Mike Kirby (MK) Intro+news ---------- - 2022 Data Challenge completed, for DD and WFS approaches Need to write this up for CHEP paper, LBNC (late March) etc - Rucio upgrade this week - HTCondor has released HTCondor 10, so support for 9 goes away on May 11th, which withdraws GSI proxy authentication. They can still be passed in jobs though. Need to think about non-OSG sites that cannot accept tokens for job submission. Need to make sure sites have DUNE tokens enabled. CERN factory uses CI Logon tokens. DUNE Global Pool ---------------- - New jobsub lite server will be brought up for global pool. -- Might already work with the new jobsub lite clients on dunegpvm etc - Meeting before Christmas about DUNE Global Pool -- Appeared to allow direct submission from RAL cloud for WFS -- Will allow offsite schedds for larger sites / regions User tools ---------- - HS: Existing documentation on getting token in jobsub lite is unhelpful - JC: Jobsub lite tutorials do have recordings available. - HS: Will try to write down what users need to do - ST: Will be in cvmfs as well as RPMs (1st Feb) - Still need clarify what can be accessed offsite - AM: Metacat client vs server synchronisation needs to be better - AM: wfs-cvmfs-upload. Need to think about user implications. - DB: What about dropboxes vs writing to Rucio managed storage? -- Discussion about different scenarios. We need to decide what to expose to users and how to achieve that. Use collaboration meeting for that. New Workflow System -------------------- - AM: Charts added to Workflow Dashboard. Currently just for files' progress for each stage of a request. Other topics ------------ AOB --- Meetings will continue in this slot, 11am CT Mondays