DUNE Global Computing Operations

US/Central
https://fnal.zoom.us/j/636941598

https://fnal.zoom.us/j/636941598

Andrew Mcnab, Heidi Schellman (Oregon state), Kenneth Herner (Fermilab), Michael Kirby (FNAL), Steven Timm (Fermilab), Stuart Fuess (Fermilab), peter clarke (University Edinburgh)
Description

Weekly operations meeting for sites doing DUNE computing

DUNE comp ops 9/27/2021

 

1 new ticket

Accounting at Bern

 

———

 

NP02—

re. Production

In release v9_30_00                 All the code for VD simulation and reconstruction is in

But some work to interface with pandora.

But memory foot print is quite high for wire cell, 5GB or even more

Need to work on this before heading to production.

Steve and Elisabetta to talk about structure for cold box data.

Also need to finalize metadata .

(Add Giovanna and Alessandro or make a 2nd meeting)

(Heidi Kirby, Ken also welcome)

 

NP04 — they are doing cleanup of the data servers in prep for taking cold box data

 

 

—————————

 

Production items

 

Working with inference as a server group

Tejin Cai @ york

Lots of failure last night

 

ND samples running now

 

Campaign launched without notification, re-education in progress

 

Last bit of pdspprod4a

 

Trying to move the near detector stuff for SAND to run at NERSC, only runs @ 2GB

Initial interactive tests ok

 

Some delay on FD sim/reco —still trying to get memory usage down, wire cell.

 

————————————

 

Data management ops

No movement on CERN on EOS fail

 

RAL and CTA?  Expect mid-october ready

Network routes could be ready as early as tomorrow

 

Testing under way trying to use FTS3 with tokens from FNAL->BNL

 

Discussion —does the non-deterministic path feature get us into trouble on tape sites?

Steve—at RAL we are ok—need non-dev path

—————————

 

Site roundtable

 

Gonzalo—PIC

CPU equipment has arrive and online

500 cores dedicated for DUNE @ PIC

Disk equipment has been purchased but HW is not yet here

 

Helo da motta — Brazil

Nothing to report @ CPBF

 

RAL—Raja , nothing special

 

GridPP—nothing

 

BNL—nothing other than what is mentioned above

 

————————

 

 

ETF—have been using preprod—should we go to prod—I think we should.

Rucio-lite, no recent update from James Perry

 

 

Steve

There are minutes attached to this event. Show them.
    • 08:30 08:40
      Open Tickets. GGUS/SNOW/Github 10m

      Github task tracker

      https://github.com/DUNE/dist-comp/labels/Operations

      Fermi and CERN ServiceNow

      DUNE GGUS tickets: http://cern.ch/go/p6s8

      Speakers: Dr Andrew McNab (University of Manchester), Dr Michael Kirby (FNAL), Dr Steven Timm (Fermilab)
    • 08:40 08:55
      Reserved for big presentation 15m
      Speaker: Dr Stuart Fuess (Fermilab)
    • 08:55 09:00
      Updates from running np04/np02 5m
    • 09:00 09:05
      Production Operation Items 5m
      Speaker: Dr Kenneth Herner (Fermilab)
    • 09:05 09:10
      Data Management Operation Items 5m
      Speakers: Dr Steven Timm (Fermilab), Wenlong Yuan
    • 09:10 09:20
      Computing Sites Roundtable 10m

      Any sites having issues to raise with DUNE operations can do it at this time.

    • 09:20 09:25
      Information Systems Operations (ETF, CRIC) 5m
      Speakers: Andrew Mcnab, Raja Nandakumar
    • 09:25 09:30
      Any Other Business / Announcements 5m