DAQ Coordination Meeting

Monday 15 Nov 2021, 09:00 → 10:45 America/Chicago

 https://fnal.zoom.us/j/369714575

09:00 → 09:10 News

Speakers: Alessandro Thea (STFC - RAL), Asher Kaboth (Royal Holloway University of London/Rutherford Appleton Laboratory), Giovanna Lehmann (CERN)

09:10 → 09:20 Specifications document status

Speaker: Giovanna Lehmann (CERN)

09:20 → 10:25 Round Table

09:20 FD Upstream DAQ

Speakers: Jim Brooke (University of Bristol), Mr Roland Sipos (CERN)

Past 2 weeks

• Firmware
  • FELIX Phase 2 FW build with TPG (2 links only) looks OK in very preliminary tests.  Lock up problem understood, and workaround applied.  Long term solution needed, will discuss with FELIX FW team.  Preparation of a FW candidate for integration tests underway.
  • 4.8Gbs links merged to FELIX base repo,.  We will rebase before Christmas.
• Software
  • ​​​​​​​all planned features (zero-copy SNB writing, TPG support, repackaging, debug/calib domain) now on one branch and undergoing tests.
• Timing
  • ​​​​​​​main firmware developer off work for 6 weeks following bike accident.  Commissioning uTCA crate at CERN will be delayed; will understand by how much.
• Ethernet readout
  • Xilinx QDMA core FW design built & basic tests OK.  Now working towards 1) throughput tests with QDMA+dpdk, 2) 100G+QDMA FW.

 

Plans

• Integration, integration, integration
• Presentation/discussion of Timing uTCA commissioning plans next Tuesday

 

Cross-WG issues

• What are VD PDR documentation requirements & timeline ?
• How/when will WIB2 interface be finalised ? (Comment in last UD meeting that packet definition is not yet final).

 

 

 

09:25 ND Upstream DAQ

Speakers: Patrick Dunne (Imperial College London), Patrick Dunne

09:30 FD Dataselection/Physics Performance

Speakers: Joshua Klein, Joshua Klein (University of Pennsylvania), Philip Rodrigues (University of Oxford)

• Working on implementing horizontal muon trigger for use in VD cold box
  • need to sort out channel map issues---implement within TA algorithm, rather than from TPs
  • need online generated TPs rather than (old) file output

09:35 ND Data Selection/Physics Performance

Speaker: Dr Jonathan Hays (Queen Mary University of London)

Recent work:

 - nothing to report this week - Alex is on vacation

Upcoming work:

 - starting to look at simulation for use with the ND-DAQ - we've got hold of an example ND-LAr simulated event data file to get started and managed to read it in and stream it to a ZMQ socket.

09:40 Dataflow

Speaker: Dr Kurt Biery (Fermilab)

Recent work:

• Minor changes in DataWriting as part of 2.8.2 (e.g additional file Attributes)
• Merges into NetworkManager branches
• Start of discussion of DFO for 2.10
• Integration test additions / improvements

Upcoming work:

• Update NetworkManager branches to include the changes in v2.8.2, including the separate DQM process; get these changes reviewed and merged to develop
• Work out the details of what we want for the DFO in 2.10
• Further HDF5Reader discussion
• Contribute to the testing/approval of Phil's config-gen tools

Cross-WG topics:

• Discussion with FarDet Dataflow about DFO ideas?
• Coordination of Readout package reorganization (hopefully already discussed).
• Bug with multiple split-off DQM processes in v2.8.2.
• np02_bde_coldbox and np04_coldbox raw data files have "JSON metadata" files being created (hence FTS-Light transfers).  If we can confirm the offline "run_type" fields for np02_arapucas and np04_vst, I can enable those.
  • np02_pds prefix with protodune-vd-pds?
• What do we expect to be needed at ICEBERG?
• CVMFS issues for UDAQ(Timing) folks?
• np04-srv-023 different behavior for different users (e.g. with regard to web proxy)
• is it OK to run nanorc in developer testing on the NP04 DAQ cluster?

Reminders of what I recall about the 2.10 release:

• NetworkManager and related changes, including possible DFO
• HDF5Reader
• Phil's configuration-generation tools
• Refactoring of the Readout package
• Non-empty TriggerPrimitive (and TriggerCandidate?) contents in data fragments from Readout and Trigger, and in the stream written out by Dataflow
• Firmware TP Generation?

09:45 CCM

Speakers: Alexander Tapper, Alexander Tapper (Imperial College London)

 Past work

• Further work on Run Control: Feedback  led to adding material to the run registry and interface with log book. Almost complete. Prototype FSM commands added for investigation of functionality etc.
• Further work on monitoring dashboards: Various technical improvements and consolidation of existing dashboards. 

Next

• Run control GUI to allow more  robust shifter use.
• Further consolidation of dashboards e.g. timing
• Follow up configuration work started by Phil.

Cross WG

• Kubernetes ideas being followed up slowly.
• In touch with ATLAS and CMS to exchange k8s experience in Dec.

09:50 Data Quality Monitoring

Speaker: Pip Hamilton (Imperial College)

Progress since last meeting:

• Stability improvements.
• Summed FFT (testing as we speak).

Next steps:

• HDF5 reading
• SSP data.
• Yann Donon departing.

 

09:55 Facility

Speakers: Prof. Alec Habig (Univ. of Minnesota Duluth), Bonnie King (FNAL)

6988_Det_and_Fac_Interfaces_DAQ.docx

ICD_JT_DAQ_Installation_R2_v5.docx

Working to get a versioned DAQ (2.8.x) onto Iceberg and Penn, BNL test stands.  Currently they're running an ad-hoc collection of tools.

Getting help from Pat Riehecky (Scientific Linux expert) on the CS8 datadisk machine missing drivers workaround for upgrading, and the best way to run the mainline kernel needed for the FELIX machines.

Reviewed specifications documents: see materials for commented up word docs for further discussion.

10:00 SW/FW coordination

Speaker: Alessandro Thea (STFC Rutherford Appleton Laboratory)

Past 2 weeks

• Release of dunedaq-v2.8.2
  • First patch-release for running;
  • A few iterations were required;
  • Eventually completed mostly smoothly, now deployed in data taking;
  • Exercised alternate CI to support the patch-release
• Spack integration continues in the background

Upcoming Work

• Provide C8S and docker images for v2.8.2
• Formalise patch-release workflow
  • and tools required during patch-releases preparation;
• Absorb the break-up of the readout package in develop;
• Make progress on the spack deployment model;
• Introduce a DUNEDAQ_RELEASE "variable" and its use in each release/work-area.

Cross-WG Issues

• Discussion of an update release workflow - planning to call for a software coordination meeting to discuss a proposal.

10:25 → 10:35 Topics for next general meeting