Minutes for LArSoft coordination meeting on September 24, 2019

Present | Vito Lynn Garren, Saba Sehrish, Erica Snider, Kyle Knoepfel, Herbert Greenlee, Robert Hatcher|

Remote | Katherine Lato, Heidi Schellman, Rob Kutschke, Patrick Gartung, Bruce Baller, Ben Morgan, Tracy Usher, Giuseppe Cerati, Paul Russo|

Release and Project status report [Erica Snider]

• Last week release was 08_31_00, this is the first release with root 6_18_04
  • Requires using native Cocoa graphics on Mac OS
• There are breaking changes
  • Lynn will update the releases page for breaking changes information.
• There are no requests so far this week.
• Python 3
  • FIFE intends to move to Python 3 early next year, jobsub script will be using older python
  • LArSoft already has py3 qualifier to select Python 3, was hoping to get it done sooner.
  • Experiments should use larsoft with py3 qualifier and report any issues to FIFE
• There was discussion about Mac OS support.
  • Scisoft plans to drop support for Mac OS due to low demoand, high effort required, loss of available effort
  • no time line was announced for the change, and is expected to occure by the end of this year
  • level of support will downgrade to known to work
  • Scisoft team will focus effort to modernizing our code, HPC etc
  • committed to ensuring that art/larsoft runs on any/all platforms needed by the experiments
  • Welcome contributions that would provide full, official support for any other OS
  • Will document contanierized solutions
  • Rob K mentioned that mu2e ran software tutorial, and there were 40 people who were able to use the docker container without a problem
  • Bruce Baller mentioned about the Xcode support, profiling and debugging tools that are convenient and better
  • There was discussion with Heidi Schellman about how to get information to end user e.g. about debugging tools that is made available by the Scisoft team
  • [TODO] We will send out the information about profiling, debugging tools.

DUNE data model and implications for LArSoft [Heidi Schellman (Oregon state)]

• DUNE is big
• APA, 2560 channels => 30 MB/readout/APA uncompressed, 12 MB compressed
• protoDUNE raw data size - about 75 MB
• Challenge: far detector is 25 times protoDUNE
  • 1-6 GB per 10kt module
  • 5000 per day
  • 3-10 ms readouts
  • Adds up to 1.6 GB/sec
  • 12 PB/year/module uncompressed x 4 modules
  • Expect one supernovae trigger per month, 100 s readout
  • 4 modules=460TB/event, takes 10hrs to read at 100 GB/s
  • Calibrations will be similar in size and scope
• Main problem is how to deal with supernovae events
  • Can break into APA chunks to better understand an interaction, but then need to reassemble them to interpret events
• Part size
  • Noise filtering requires > 100 samples for Fourier analysis
  • 2D deconvolution requires many adjacent wires
  • Signal processing is per APA
• Reco workflow
• Issues
  • what is an event?
  • For ND, need to stitch 2-3 detectors together
  • each APA take 30s, can do parallel, or sequential
• ICARUS is in the same boat, 55K channels.
  • current plan is to read out each TPC separately, changed simulation already
  • planning parallel processing of TPCs for some steps
• memory issue: SNB: Need to have 2 time slices in mem together to do transform w/o edge effects
• art vs LArSoft question
  • [Question] at what point are we hitting the framework limitations?
  • [Answer] Parallel processing not a problem. The what is an event question is the hard one. What is the best way of presenting a usable processing unit to the user for doing physics? .
• ON ICARUS side, deconvolution is on TPC basis, track reco is on cryostat basis, but analyzers want a unified view.