DUNE Workflow/load Development 29th July 2021
---------------------------------------------

https://indico.fnal.gov/event/

Present: Andrew McNab (AM) (chair/notes),
         Doug Benjamin (DB)
         Mike Kirby (MK)
         Raja Nandakumar (RN)
         Chris Brew (CB)
         Ken Herner (KH)
         Steve Timm (ST)

Intro+news
----------
- Joint meeting with Data Management

GlideInWMS/HTCondor
-------------------
  ST: Security issues with HTCondor, fix release yesterday
      - One affects all versions for HTCondor 
  MK: DUNE global pool. Maria says some large submissions done
      targetting various remote. Also jobs submitted from BNL.
      Submission nodes at FNAL and BNL. Two groups defined in 
      pool for now, with priorities. Some work done to run at
      NERSC through DUNE global pool. 
  ST: Main question to be decided is how users submit to it, 
      authentication. HEPCloud uses shared jobsub pool and to
      use NERSC HEPCloud would need to inspect DUNE global pool.
      At NERSC, there is a joint account, with a large fraction
      for DUNE. 
  AM: Can we do some of the production with the DUNE global pool
      now?
  KH: It can be tried now. Maybe with FD data.
      The pool should accept jobsub if all users are expected to
      use it in the future.
      A temporary dedicated jobsub server wouldn't be a long term
      committment since jobsub servers are going away with jobsub
      lite.

User tools
----------

MK: Need to make a push on jobsub lite. Would like it to be there. 
AM: Go straight to jobsub lite for users with DUNE global pool?

Workflow/production
-------------------

AM: We could make a Workflow Dispatcher design to allow the generic jobs
    we've been talking about. We need to have some design to put into the
    CDR very soon.
    
HEPCloud
--------

AM: How does HEPCloud work out whether to create VMs or pilots? Just
    job pressure?
ST: It uses job pressure plus rules about costs of different resources,
    rules about who can run on cloud. 
AM: So as long as the workflow system is submitting jobs, generic or
    otherwise, that are identifiably for NERSC etc, then HEPCloud can
    see them and provision resources.
ST: HEPCloud used to require that jobs specified their network etc 
    requirements. CMS didn't want that.
AM: If we have a workflow dipatcher, the client could say where it is
    and that it has limited network and ask for an appropriate job?

Pilot Factories
---------------


Other topics
------------



AOB
---

Meetings continue in this slot, 8am CT Thursdays, from later in August.