attendance: Amir, Bernd, Maxim, John, Dario, Tom, Stu, Steve, others that I could not identify (please send me email for corrections)
-
next meeting: April 27th, 8 AM CDT
-
update from discussion with Andre, Mark, Tom
-
ProtoDUNEs are asking for 100M particles to get enough physics triggers: SP needs 5M, SP ?
-
SP: 4PB to 20PB ➜ factor 20
-
clarified, this is to do all physics and would be nice to have, this is not the minimum
-
maybe ProtoDune will have another run in 2021
-
Question: Do protoDUNE's computing requirements fit in reasonable resource envelopes at CERN and Fermilab?
-
CERN
-
Bernd can attach money tags to all resource types (cpu, disk, tape, services), using moderate scaling (Moore’s law is not valid anymore)
-
CERN request time schedule
-
2 deliveries scheduled for CERN per year
-
size of current protoDUNE request can be handled from normal purchases
-
1-3 PB EOS space: any time
-
timescale for larger purchases: 1 year lead time
-
10-20 PB EOS space: 1 year lead time
-
for having everything in place beginning of 2018, requests have to be in spring 2017
-
this is for CPU and DISK, tape is more flexible
-
Fermilab
-
The expectation was that there are infinite resources at Fermilab. Fermilab will do its share to make this project a success, but with 20 PB dat volumes this is going to be difficult within the intensity frontier computing envelope.
-
Clarified what happened at SC PMT: all experiments asked in sum for more resources than available, older experiments got told that their request cannot be fulfilled completely and that they have to look for opportunistic resources (for example on the OSG)
-
Stu will make SC PMT analysis public, show requests of all experiments
-
DUNE’s request for FY18:
-
CPU request, less than 20% of total FNAL intensity frontier request
-
Tape request: 10 PB of storage ➜ 25% of total FNAL intensity frontier request
-
discussion about funding: if independent funding is available, this would be handled like CMS (CMS is funded exclusively independently), a mode for protoDUNE would have to be found in this circumstance
-
Amir would like to have dollar values for Fermilab resources
-
General discussion in the light of significantly higher data volumes
-
asking for two tape copies (one Fermilab, one Castor) might be too ambitious
-
Cosmic data could be stored with less redundanc
-
could also remove zero suppression only on a part of the data
-
maybe rethink the trigger strategy?
-
could skim before processing
-
Technical questions:
-
Disk estimate for CERN: 2.5 PB is consequence of how many servers you need to sustain throughput
-
Bernd is more conservative, does not use 10 Gbps connection of the disk boxes
-
estimate 300 MB/s (1/3 of capacity) ➜ 20 servers, each server ~140 TB
-
Is there a document that one can refer to? ➜ this is more a case by case study
-
Want to compare to other experiments, Bernd will ask Castor and Eos teams
-
disk buffers need to be large enough to cover outage friday evening to Monday morning
-
transfers out, etc ➜ support on the weekend is best effort
-
in the last 10 years, CERN didn’t have a major issues where all buffers filled up
-
on the other hand, it is a limited time of data taking for test beam
-
model is SLA based
-
conclusion: this is consideration between CERN and DUNE
-
CVMFS master ➜ Fermilab has one, just needs to enable the osg repository on CERN machines
-
SL6 and SL7 issue
-
all CERN resources are SL6
-
migration plan to SL7 ➜ not clear, heavily depends on LHC experiments
-
neutrino platform has few hundred core cluster running SL7 ➜ no worth concentrating effort on enabling the software to run on SL7 for this
-
job submission and monitoring system at CERN
-
there are enough tools for lxbatch (LSF or condor). If you are on own VMs, you are on your own
-
need to put something on top of it to orchestrate production
-
lxbatch is grid accessible
-
test system
-
Stu and Maxim are handling test system between BNL and Fermilab as starting point
-
~ 6 month: have test system at CERN
-
CERN to Fermilab:
-
network paths are in place (for CMS)
-
just need a machine at CERN to orchestrate
-
Administrative questions:
-
Status of accounts at CERN
-
Tom got instructions per mail how to register for neutrino platform accounts
-
group volumes at CERN
-
AFS or EOS, majority is EOS
-
discussion about FUSE mounts for EOS
-
no interface people from DUNE at CERN yet
-
by the end of the week, will go out to the collaboration
-
nomination process, etc.
-
April 29th talk before collaboration
-
ECU will take vote
-
people will not be in place until May
There are minutes attached to this event.
Show them.