CCM Meeting

Wednesday 1 May 2024, 08:00 → 09:00 US/Central

Zoom: https://cern.zoom.us/j/66569029743 (usual password)

1 News

Speakers: Alexander Tapper (Imperial College London), Pierre Lasorak (Imperial College London)

Release and OKS

Planning workshop

• dates? 

2 Round table: control

Speakers: Pierre Lasorak (Imperial College London), Zbynek Kral (CTU Prague)

Pierre:

• Unified shell merging with Process Manager
•  
• JSON configuration not working anymore after OKS

Tiago:

• K8s PM: boot, restart, kill, ps all working. PR is here, the review is ongoing.
• UUIDs: same as what was discussed last week
• Discussion on restarting the pods

3 Round table: config

Speakers: Alan Watson, Giles Barr (Oxford University), Giovanna Lehmann (CERN), Gordon Crone (University College London), Joeal Subash, Juraj Bracinik (University of Birmingham)

Gordon:

• New release candidate for v5.0.0, one package was missed so retagged. Soon to be rebuilt (today)

Kurt:

• Older version of the connectivity server at NP04?
• Gordon tagged a new version
• Instructions for deploying the new service?
  • Cannot be done during a run?
  • Gordon says in principle it could be done during the run. But lets do it when quiet.
  • Need to make a new image (connectivityserver/deploy/Dockerfile)

4 Round table: k8s etc.

Speakers: Alessandro Thea (STFC Rutherford Appleton Laboratory), Bonnie King (FNAL), Patrick Riehecky (Fermilab), Pierre Lasorak (Imperial College London), Tiago Alves (Imperial College London), Tiago Alves (Imperial College London)

Pat:

• InfluxDB v2 in parallel with v1 to test both.
  • Marco: need a second instance of Telegraph
    • Pat says no need to have 2 Telegraph but 2 output sources
  • Hoping that InfluxDB v3 comes out in the mean time, so that we don't have to transition twice
  • Alex: what specific new feature do we want to have from v2?
    • Marco: None

Tiago

• Microservice deployed and running smoothly
  • Timezone fixed and everything is UTC
  • Pat: do we still want to have the 2 run services separate?
    • Need to discuss with Alessandro
  • Pat: Timezones should always be UTC in DBs
• K8s testing: a couple more tests, no problem
  • Wes suggested that we use K8s for monitoring runs.

5 Round table: monitoring

Speaker: Marco Roda (University of Liverpool)

- Grafana dashboards have been tagged for fddaq v4.4.0. The dashboards on grafana have been frozen, new dashboard with provisional tag v4.4.1 were created for development purposes. 

-  transition to new influx table in preparation for ProtoDUNE 2 run done

- After discussing with Alessandro, we started a google document where we can collect the lessons learned on monitoring in ProtoDUNE. This should help define the requirements for the DUNE, including servers specifications. The link to the document is here.

- Again, after discussion with Alessandro, we identified a new requirement for the monitoring update (v5). Our current tags are entirely software dependent, e.g. np04hd.runp04srv029eth0.datahandler_238 is reflecting the configuration, the machines where the software is running, etc, but not the hardware. We would like to add hardware information in the tags as well, e.g. channel. The proposed solution is to add a way for user to specify arbitrary tags when publishing opmon data. 

- Last friday in the F & I meeting we discussed a plan to study the feasibility of migrating from influxv1 to influxv2. Main reason for the transition being that influxv1 is currently unsupported. Pat and I are working on it, although not with super high priority given that the current system works. We are investigating the following
 - what changes needs to be done on the software side to use influx v2 
 - Can we import data from v1 into v2 so we don't loose data?
We believe all is doable and without too much effort, but the tests are going to confirm that. After that we can decide if we want to move. 

6 Round table: others