DUNE Workflow/load Development 28 Oct 2021
------------------------------------------

https://indico.fnal.gov/event/51574/

Present: Andrew McNab (AM) (chair/notes)
         Steve Timm (ST)        
         Ken Herner (KH)
         Raja Nandakumar (RN)
         Doug Benjamin (DB)

Intro+news
----------
- 

GlideInWMS/HTCondor
-------------------
- ST: Hope to do some testing before next week with global pool
      Have a working version of jobsub_submit in place now for this
      Need to test how it works when used at larger scales
- KH: Will ask prod team people to test it

User tools
----------
- ST: Above is jobsub lite beta
- AM: Thanks for ST for adding me to the RUCIO/firewall ticket
      Next step is to use RUCIO client from jobs

Workflow/production
-------------------
- Discussion about placing output files of jobs
  Worries about users uploading to managed storage and making a mess
  Worries about users not-uploading to managed storage and making a mess
  Suggestion that generic job should do the RUCIO upload, potentially using
  its higher-privileged credential
  Eventually generic jobs and bootstrap scripts they execute can have different
  tokens (bootstrap script inside a container to enforce this?)
- AM: will create an updated generic job script which discovers which output 
  files to upload using a series of filename patterns in the stage definition,
  and then (says it would) upload them itself. Metadata can be identified and 
  inserted into MetaCat before/during these uploads in future.
- RN: Have been making flow diagrams of the workflow system
 
HEPCloud
--------
- ST: KH has been preparing a run on NERSC using HEPCloud
      Oak Ridge also making capacity available

Pilot Factories
---------------
- 

Other topics
------------
- DB still talking to BNL about CRIC

AOB
---

Meetings continue in this slot, 8am CT Thursdays.