Jun 16 – 22, 2022
US/Central timezone

Evaluating a novel, HEP distributed data service for NOvA neutrino candidate selection

Jun 21, 2022, 10:00 AM
One West (Fermilab)

One West



Dr Sajid Ali Syed (SCD)


In this work we evaluate the performance of the High-Energy Physics's new Object Store (hereafter referred to as HEPnOS) based on the mochi microservices architecture, that was designed specifically for HEP experiments and workflows. The use case we employ for the performance study is the task of NOvA neutrino candidate selection. This experimental setup consists of a HEPnOS server that holds the experimental data in an in-memory database and a set of client nodes that run the analysis by fetching the data from the server. While traditional analysis maps CPU cores to files (i.e. each core handles all events/slices within the file), the use of HEPnOS allows us to harness finer grained parallelism at the event level rather than at the file level. We show that this allows us improve strong scaling for this task, thereby allowing us to effectively harness available computational resources. Moreover, once the data is loaded into the server, the analysis can be run iteratively which can lead to speedups in higher level analysis routines like parameter fits.

Primary author


Andrew Norman (Fermilab) Shane Snyder (Argonne National Laboratory) Saba Sehrish (Fermilab) Rob Ross (ANL) Marc Paterno (Fermilab) Jim Kowalkowski (Fermilab) Steven Calvez Pengfei Ding (Fermilab) Rob Latham (Argonne National Laboratory) Dr Philip Carns (Argonne National Laboratory) Dr Jerome Soumagne (HDF Group) Dr Sajid Ali Syed (SCD)

Presentation materials