CCM Meeting

Wednesday 29 May 2024, 08:00 → 09:00 US/Central

Zoom: https://cern.zoom.us/j/66569029743 (usual password)

08:00 → 08:05 News

Speakers: Alexander Tapper (Imperial College London), Pierre Lasorak (Imperial College London)

Planning workshop next week at CERN

• v5.1.0 pencilled for 12th July
  • Targetting usability of OKS and Drunc at NP02
  • Tentative schedule in https://indico.fnal.gov/event/64576/#5-proposal-for-initial-v51-fea
• CCM server PRR work started (due end of this year)
  • Rough estimate of processes handled by the CCM
  • Expect more updates in meeting as we continue to make progress to understand the resources needed for the CCM to run!

08:05 → 08:10 Round table: control

Speakers: Pawel Plesniak (Imperial College London), Pawel Plesniak, Pierre Lasorak (Imperial College London), Tiago Alves (Imperial College London), Zbynek Kral (CTU Prague)

20240529_drunc_Lasorak.pdf

Pierre:

• drunc gantt chart https://docs.google.com/spreadsheets/d/1cdFtOPz-8KkDiNY7ZYbTIzWP3Z9NQdi3bNietXOjFCo/edit#gid=1115838130
• With Pawel:
  • renamed the FSMInterface to FSMHook
  • Rationalised the return error for children

Tiago:

• Fixed restart on the SSH process manager

08:10 → 08:15 Round table: config

Speakers: Alan Watson, Giles Barr (Oxford University), Giovanna Lehmann (CERN), Gordon Crone (University College London), Joeal Subash, Juraj Bracinik (University of Birmingham)

John

• Renaming OKS repos
  • genconfig, oksconfig, oksdbinterfaces
•  

Alessandro

• Readout schema brainstorming with Giovanna, will take a bit more time
• Schema editor improvements

08:15 → 08:20 Round table: k8s etc.

Speakers: Alessandro Thea (STFC Rutherford Appleton Laboratory), Bonnie King (FNAL), Patrick Riehecky (Fermilab), Pierre Lasorak (Imperial College London), Tiago Alves (Imperial College London), Tiago Alves (Imperial College London)

Tiago

• k8s at NP04:
  • Problem reported last week isn't really reproducible, so hard to debug
• Microservices
  • Large configuration used in TPG commissioning don't make it on the run registry, seems to be due to a timeout
  • Automated action in GH for microservice image creation.
    • Tiago has a tentative implementation of that but doesn't know how to test it
    • Looking at it with Alessandro
  • Pierre and Tiago to close outdated GH issues

Pat

• Nuke daqconfig, and postgres run service at NP04
• read only k8s dashboard (newest version). not deploying that

08:20 → 08:25 Round table: monitoring

Speaker: Marco Roda (University of Liverpool)

Marco

• DFO changes for trigger type monitoring done and tested. But the PR is not approved yet because a usage of a mutex raised concern. Eric probably is the right person to approve. He was tagged but he hasn't reviewed. 
  • There are no PRs yet on DataWriters and MLT.
• Grafana news.
  • Grafana instance had some issue this week while developing. We (Pat and I) are certain that's because the instance reached the limit of CPU consumption allowed by the pod configuration. We increased the limit. We also increased the limit of Memory usage, but that limit was not reached. This might take a few iteration to find the right value. As per the reason we reached the limit, it's probably because a lot of people while testing were looking at many open dashboards simultaneously.
  • The v4.4.1 grafana instance is now the new default as per Alessandro request. Although this is actually the development version for v4.4.2. Once the release is out, the names will be updated accordingly. 
  • In NP04, the static grafana instance (the one on port 31001) will not contain the DUNE DAQ dashboard anymore. 
• On v5.* matters.
  • Stoyan convinced me to support nested schema. So in the new system, a schema can contain another schema. They will be associated to the same OpMonId. The gain of this is that same block of can be used in different schemas. It's not a major change, I should be able to start working on that this week. 
  • I still don't have answers to the questions raised at the planning meeting:
    • Who is going to review the code for the monitoring API.
    • Which version am I expected to use for the changes in the API between Monitoring and appfwk.
• No news from Jake on the WIB monitoring. I hope I don't have to chase him.

08:25 → 08:30 Round table: others

No AOB