Topic: HEP-CCE2/SOP video conference
Join Zoom Meeting
https://cern.zoom.us/j/63214151964?pwd=OXNtZWJuZFBvT3FWN1RhWG9EMzE1QT09
Meeting ID: 632 1415 1964
Passcode: 528737
# Attendees
- Peter, Saba, Serhan, Amit, Chris, Dan, Jean Luca, Marcin, Matti, Rui, RobL, Shane, Paolo
# Introduction/Management News
Next week: RNTuple review continues
# Applying Lessons Learned to HEP Experiments, Mimicking Framework, Darshan and HDF5
Mimicking Framework, Darshan and HDF5
Amit Bashyal, Christopher Jones, Kenneth Herner, Patrick Gartung , Peter van Gemmeren, Rui Wang, Saba Sehrish, Shane Snyder
Shane:
a bunch of us hung out with ROOT team at Argonne;
ROOT mini-benchmarks / reporting
Next step: validate root reporting with darshan stats
Peter:
Compare performance between RNTuple and TTRee with ATHENA workflow using the Darshan Log.
Rui is working on this and hopefully some examples in the next meeting.
# Tracking and aiding the evolution of ROOT I/O, in particular RNTuple
Alaettin Serhan Mete, Amit Bashyal, Barnali Chowdhury, Christopher Jones, Daniel Riley, Kyle Knoepfel, Marcin Nowak, Matti Kortelainen, Peter van Gemmeren, Philippe Canal
Darshan work belongs in “applying lessons” more than here in “evolution of ROOT i/o”
Serhan enabled re-enabled multi threaded buffer writing using RNTuple in ATLAS
Peter: Following up with Chris’s last presentation, ATLAS did not see huge memory consumption like CMS sees.
Chris: CMS might have a lot more subfields.
Matti: What memory number do you (ATLAS check)?
Serhan: Checked Virtual memory size and RSS.
# Reduced Precision and Intelligent Domain-specific Compression Algorithms
Amit Bashyal, Meghna Bhattacharya, Peter van Gemmeren
Amit, uploaded slides.
LDRD proposal: use intelligent lossy compression against DUNE raw data. Approved for 3 months – supporting a student
# Object Stores and Strategies for Data Placement and Replication
Bo Jayatilaka, Doug Benjamin, Nicholas Smith, Rob Latham, Rob Ross, Saba Sehrish, Shane Snyder
Saba meeting with Bo (CMS storage – experiments with Ceph) next week
CMS and ATLAS have been doing work on this front. Nick presented last week and ATLAS has been doing object stores for their data.
RNTuple stores data in DAOS and the study of RNTuple support of DAOS falls under object store.
Rob L: Nothing to add for now.
# Optimized Data Delivery to HPC systems
Marco Mambelli, Saba Sehrish
Saba reached out to Marco but he was not able to reach out today. Marco sent an email and later Saba adds in note:
Marco’s update: Installed a Globus Connect Server on if-globusdtn-01.fnal.gov. The plan is to use this to process ANL data from the Fermilab experiment (transferred there from the DAQ host). Thomas Wester has been doing some transfer tests obtaining close to 700MBpb to/from ANL which is sufficient for their online rate. Transfer rate to/from this server and the DAQ server are currently the bottleneck. Thomas is using rsync, the plan is to test xroot.
We’d like suggestions for more rigorous transfer tests/testing tools
Thank you
|
Here the server details: |
# Any Other Business