XCache DevOps Meeting Dec 06, 2018

Brian Bockelman, Derek Weitzel (University of Nebraska - Lincoln), Marian Zvada (US CMS)

Time: 11AM PT/1PM CT/2PM ET
Where: ZOOM.US

Join from PC, Mac, Linux, iOS or Android: https://unl.zoom.us/j/651969661

Or iPhone one-tap :
    US: +16699006833,,651969661#  or +14086380968,,651969661#
Or Telephone:
    Dial(for higher quality, dial a number based on your current location):
        US: +1 669 900 6833  or +1 408 638 0968  or +1 646 876 9923
    Meeting ID: 651 969 661
    International numbers available: https://unl.zoom.us/zoomconference?m=wxCNSgMZiA-cVKSNowGYlQ

    • News
      Conveners: Brian Bockelman, Mr Marian Zvada (US CMS)

      BrianB summarized:

      Based on some discussions at the USATLAS Facilities meeting this week, we'd like to propose some changes to the StashCache meeting in order to make it more inclusive.

      First, we are renaming it from "StashCache / XCache" to simply "XCache DevOps Meeting".  The focus of the meeting will be both development topics pertaining to XCache (Xrootd development, XCache plugin development, testing & release schedules, OSG packaging of XCache, OSG documentation of XCache) as well as operational concerns for the three infrastructures that use XCache (CMS XCache, ATLAS XCache, and StashCache).

      Marian will continue to run the meeting; he has put together a draft agenda here:


      With Indico, we'll have a place for everyone to post files and slides as necessary (and, importantly, meeting minutes), but don't take it as a requirement to produce slides for each week.  We'll create a new list for this meeting to reflect the broadened scope as this one is literally called "stashcache", but that likely won't happen until next week.

      There were two other goals proposed at the workshop:
      1.  By end-January, have OSG publish a XCache docker image in DockerHub that SLATE uses as the base for their XCache Helm chart.
      2.  By mid-2019, have OSG publish a XCache application (we left it ambiguous whether it's Docker, k8s, or Helm) that can participate in the ATLAS XCache, CMS XCache, and StashCache infrastructure.

    • XCache development

      General status of development, XRootD releases with new cache features, bug fixes, etc...

      Conveners: Mr Andrew Hanushevsky (SLAC National Accelerator Laboratory), Matevz Tadel (UCSD), Wei Yang (SLAC)

      Matevz: things TODO for XRootD File Cache Statistics exchanged over the email and another input documented here: https://docs.google.com/document/d/19v-t8CZ9QZzEJJW587Q2FzGgLeqy1cbhAfCr0gjDRtI/edit#heading=h.mp8y7vs3c8xr

      Andy: xrootd v4.9.0.rc2 coming out tomorrow or early next week. Depending on test, official release cut before Xmas or early New Year.

      Wei: RUCIO plugin ready if if OSG interested it can be moved on into their package management. On another note, there are several memory leak concerns that Wei would like to have addressed in the coming RC2. Andy says it should be the case.

      BrianL: Documentation how to install xrootd in OSG land with all authenticated portions is relatively done, couple of pull requests waiting for review. Continuting progress during the "Thursday afternoon OSG doc session".

    • ATLAS XCache deployment
      Conveners: Dr Ilija Vukotic (LAL), Wei Yang (SLAC)

      Wei: He doesn't know much of final deployment plans of ATLAS XCaches at the moment. A question has been raised: "What will be prefered way to deploy XCache/StashCache? Is it Kubernetes". Definitelly that is direction to go, there will be still packages version but deployment should be in flavor of Kubernetes, still evaluating Helm/Slate approach and that should be in hands of OSG Software team.

    • CMS XCache deployment
      Conveners: Diego Ciangottini (Universita e INFN, Perugia (IT)), Prof. Frank Wuerthwein (UCSD)

      Marian: Welcome to Diego, Diego is joining the group as well as mailing list, he has been implementing XCache for CMS as part of the Italy Stash Federation. As he is based at CERN it might be late for him but he tries to be available as time permits. Recent presentation on his CMS XCache contribution: https://indico.cern.ch/event/764787/

      Brian: What's status of CMS xrootd monitoring? Marian: We've implemented shovel to transport xrootd data streams from RMQ topic from GRACC to AMQ test topic at CERN. This part is validated and they receive the data, however, it requires change in the header of .json file we produce. Will work on this in a next days.

    • OSG StashCache deployment

      Status of OSG deployment, origin/caches operations, stashcp, usage, etc...

      Conveners: Brian Bockelman, Brian Lin (University of Wisconsin-Madison), Dr Derek Weitzel (University of Nebraska - Lincoln), Mr Edgar Fajardo Hernandez (UCSD), John Hicks (Internet2), Mr Marian Zvada (US CMS), Mr Matyas Selmeci (University of Wisconsin-Madison)

      Marian: New GATech cache registered.

      Edgar and Marian: Switched UNL cache talk to Kansas City redirector and UChicago cache talk to Chicago I2's redirector. Removed from the CVMFS_EXTERNAL url from OASIS.

      Derek: See the slides attached for the usage of StashCache.

    • AOB

      Marian: Frank offered discussion about XCache project evolution together with Internet2 in the next 2 years. We will primarily have his talk plus I2 people connected on the next week meeting. Looking forward to it!