Weekly operations meeting for sites doing DUNE computing
https://indico.fnal.gov/event/47580/
GGUS tickets
BERN—enable DUNE to run
Manchester—permissions on the storage element
Sheffield—believed to be an auth problem
149395—BNL ticket job submission
149771 Liverpool ETF failure
OSG tickets—
NIKHEF—works but second CE isn’t working
Fermilab tickets—
One user close to ban hammer 10K jobs failed three times
Jasingh
CERN 3rd party copies issue is fixed
——
NP02 update—cooling will be back in mid-Feb
Wednesday will restart switching on storage machines
Will be doing upgrades on storage service
At end they will finish before test with cold box
Processing—Planning to do some reprocessing of NP02 data
Taking into account updates of using calorimetric information between views
On time scale of 2-3 weeks
\\
Move to reprocessing of all np02 data
Some simulation needs to be done for near vertical drift
Can use CCIN2P3 space to store reco data of NP02
And the results of virtual drift simulation
500TB available on DUNE_FR_CCIN2P3_XROOTD
Contact Denis for setup of NP02/EHN1 EOS
2 2021 testing activities cold box in summer
In fall voltage testing in current cryostat
—————
Np04
——
Production update
ic.ac.uk worker nodes—glidein went crazy yesterday
Major power outage @ imperial
Tata institute working now
A few data transfers from Fermilab->CBPF very slow or got stalled for hours
Things ok now.. Helio saw no messages over the weekend.
No update re. Fabio and campinas
Almost all 6GeV MC submitted, running
Almost all NERSC files done for the moment
Next need to run the merging of the anatuples
Some problems—need to get SFA policy in for michelnemoving files
Have already merged the files into reasonably sized chunks
Where are the big merged files—
Right now in the same directory since tagged by run number
Very end of phase 2.
———
Data management operations items
Cutting over all operations to the OKD-based rucio server
No off-site.
Will test once this is done by wiping everything for BNL and sending it again.
Robert—does FTS3 server have to have off-site exemption
Eventually yes
Rucio test account and scope getting enabled everywhere
——
Site roundtable
QMUL network outage, otherwise nothing to report
Brazil—nothing from Brazil sites
BNL—gave ticket to Paul
Jonathan—who to talk to re. Getting near detector test beam to main computing center
Start with Kirby—numbers should feed into Fermi Compute Resources Steering Group meeting which may be as soon as the end of this month.