Please read these instructions before posting any event on Fermilab Indico

Indico will be unavailable on Wednesday, Nov 20th from 7-7:30 CST due to server maintenance.

CCM Meeting

US/Central
Description

Zoom: https://cern.zoom.us/j/66569029743 (usual password)

    • 1
      News
      Speakers: Alexander Tapper (Imperial College London), Pierre Lasorak (Imperial College London)

      Releases in progress etc.

    • 2
      Round table: control
      Speakers: Pierre Lasorak (Imperial College London), Zbynek Kral (CTU Prague)

      Pierre:

      • Nanorc work
        • Changes to logbook messages
        • Make sure default run type is production -- change of CLI for nanorc04
        • Merge timeout PR for k8s.
    • 3
      Round table: config
      Speakers: Alan Watson, Giles Barr (Oxford University), Giovanna Lehmann (CERN), Gordon Crone (University College London), Joeal Subash, Juraj Bracinik (University of Birmingham)

      Eric:

      • Presentation about engineering release in DAQ meeting
      • Work on timing and introduce OKS to more modules.
      • Found some things to improve in HSI libs etc.
      • Is latest tag ready for release? Release candidate. TBD.

       

       

       

       

    • 4
      Round table: k8s etc.
      Speakers: Alessandro Thea (STFC Rutherford Appleton Laboratory), Bonnie King (FNAL), Patrick Riehecky (Fermilab), Pierre Lasorak (Imperial College London), Tiago Alves (Imperial College London), Tiago Alves (Imperial College London)

      Tiago:

        K8s testing last week
        • Network issues last week
        • Changed proxy to microsocks -- most stable
        • Still see some failure in pings
        • Mitigated by setting longer timeout (10s)
        • Do we separate data and control networks? no (not in ssh either)
        • Run ping from inside k8s cluster? Tested. Seems fewer errors.
        • Upgrade k8s etc. versions? Rebuild cluster(s)? Yes, should take one hour. Request time from Wes
      • Microservices
        • Restart to pick up new image -- quick fixes in PR -- Wes tested and saw Oracle error -- others to test then merge and deploy.
    • 5
      Round table: monitoring
      Speaker: Marco Roda (University of Liverpool)

      Marco:

      • InfluxDB issues:
        • 100K maximum number of tags -- based on potential performance issues -- can be removed
        • Workaround for PD2 with new table
        • Upgrade to influxDB v2 (or v3 unknown when released) a possibility
        • Separate session and app from element for v5.1?
        • Conclusion: look into upgrade of v2 -- how much work? backward compatible?
        • Also, use as a opportunity to define procedures for database migrations?
      • General
        • CTB monitoring PR merged (added to trigger and HSI dashboard)
        • Redundant/obsolete plots -- contact HSI -- Stoyan, Diana et al.
    • 6
      Round table: others

      Meeting next week? Tentatively cancel next week unless Wes yells.