Please read these instructions before posting any event on Fermilab Indico

Indico will be down for maintenance on Thursday, May 16th from 6:00PM - 8:00PM CST.

EHN1 Operations and fddaq-v4.3.0 Release Preparation

US/Central
Kurt Biery (Fermilab), Wesley Ketchum (Fermi National Accelerator Laboratory)
Description
    • 07:30 07:50
      Operations 20m
      Speaker: Wesley Ketchum (Fermi National Accelerator Laboratory)

      DAQ Hardware Activities:

      • Moving np04-crt-001 from EHN1 barracks back to CRT rack: 6 Feb (Tu) afternoon, Wes
        • Complete
      • Moving VD coldbox setup back to Faraday cage: 7 Feb (W) morning, Matt, Will, Tiago, Wes
      • Installation of NP02 readout switch, and recabling of NP04 readout: 7 Feb (W) morning, Matt, Will, Tiago, Wes

      Standard DAQ monitoring runs:

      • Begin basic NP04 runs with all WIBs and available DAPHNE after recabling tomorrow
      • Plan to create new dev area based off of most recent nightly.

      Ongoing / to-be-done DAQ testing involving running with hardware ~this week:
      These all inform a baseline configuration for readout of NP04.

      • K8s-based testing: Tiago, Juraj, Pierre
        • Problem in the run control that came up while testing K8s configurations. Expect that RC being too strict on the application being alive and goes to error too quickly / doesn't retry. Tiago working on a fix to understand better the cause.
        • In theory, no need to work with any hardware, but the problem came up only when we started to test more with the WIBs, so may somehow be tied to the wibmod code, or ...
      • Dataflow with new np04-srv-005 server: Wes, (Kurt + Steve Timm to check metadata and transfer?)
        • ongoing
        • Kurt will look into the metadata
      • NP04 readout servers with multi-NIC running, use of new readout server (np04-srv-031): ???
        • Needs preliminary step for updating configuration with the pcie device ID
        • Wes will followup with Roland and Alessandro on updating the DRO maps
        • Should consider then moving a NIC to np04-srv-031 so it has two NICs? Or reserve it for testing
        • Wes to followup on what servers to use
      • CTB testing: Jon, Stoyan, Wes, + ??? (when?)

      Other testing for ~this week:
      Goal to have preliminary implementations ready for this week/early next week.

      • Automatic elog posting: Pierre, Roland, Enrico. <solution in sight, but need to do a fair bit for finalization>
        • Will plan to merge the elisa PR that's there, but then look into a solution utilizing submodules
        • What has worked is in a test instance, and will need to deploy this fully for the existing elogs (Enrico working to demonstrate having the PDSP elog in a dedicated project)
        • continue discussion offline
      • DQM justintime updates and deployment: Vitaliy, Wes
      • Readout servers hugepages/NIC configuration after reboot: Pat, Roland, Alessandro, ... this week, or put on pause?
        • Still missing a reasonable model on how to setup these machines from boot
        • Roland and Alessandro will try to converge quickly on how to bring things up in a known state, and then propagate the information/recipe to Pat and team

       

       

    • 07:50 08:10
      Release Preparation 20m

      First question: what do we want to include in fddaq-v4.3.0-rc1?

      • everything that is already on develop branches plus pending changes...
      • reference link: Miscellaneous Deliverables
      • known pending changes:
        • Watching for tardy TPs in tp_datalinkhandlers
          • waiting on Kurt code changes
        • Support for prescaling in TimingTriggerCandidateMaker
          • waiting on testing at EHN1?
        • Update(s) to build/release tools to handle new, suggested nightly names
        • Fix for TimeSlice numbers greater than 2^32
          • waiting for PR review?
      • Issues/PRs that have shown up on radar in the last ~24 hours
        • Adam & Ivana - fdreadoutlibs - AVX peak calc. in AbsRS, etc
        • Tiago - nanorc - WIB k8s pod fixes
        • Wes & Kurt - hdf5libs - annoying warning message, too many TR number digits
        • Eric - iomanager - bug fix (check pointer before dereference)
        • Pierre - elisa_client_api - update to new version, including a new external?
      • what else?

      If time permits, we can also start talking about starting to tag packages...

      • Tag Collector
      • Developers mentioned in the Tag Collector - are these reasonable?
        • MaR, ELF, AT, WRK, KAB, RS, AAA, ST, DA, AO, GC, JCF, GLM, PL
      • Question: are there new packages in addition to dqmtools, tpgtools, trgtools?
      • Reminder/question about bumping minor version number in package tags
      Speaker: Kurt Biery (Fermilab)