Timing Week Ahead
Zoom: https://cern.zoom.us/j/65611907988?pwd=ak9jK04xdFY0OThzN3RRbEdVZkhNZz09
David Cussans is inviting you to a scheduled Zoom meeting.
Topic: DUNE Timing System Week Ahead
Time: Jan 17, 2022 11:00 AM London
Every week on Mon, 20 occurrence(s)
Jan 17, 2022 11:00 AM
Jan 24, 2022 11:00 AM
Jan 31, 2022 11:00 AM
Feb 7, 2022 11:00 AM
Feb 14, 2022 11:00 AM
Feb 21, 2022 11:00 AM
Feb 28, 2022 11:00 AM
Mar 7, 2022 11:00 AM
Mar 14, 2022 11:00 AM
Mar 21, 2022 11:00 AM
Mar 28, 2022 11:00 AM
Apr 4, 2022 11:00 AM
Apr 11, 2022 11:00 AM
Apr 18, 2022 11:00 AM
Apr 25, 2022 11:00 AM
May 2, 2022 11:00 AM
May 9, 2022 11:00 AM
May 16, 2022 11:00 AM
May 23, 2022 11:00 AM
May 30, 2022 11:00 AM
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://cern.zoom.us/meeting/u5ErduihrjwjEty6rVlnv8tn7IyhqaavFeey/ics?icsToken=98tyKu-tqDorE9SSuRGPR_MMBIj4d_PziGZEj7dEyg3TDy5FOhf4YctbOqN-Ec72
Join Zoom Meeting
https://cern.zoom.us/j/65611907988?pwd=ak9jK04xdFY0OThzN3RRbEdVZkhNZz09
Meeting ID: 656 1190 7988
Passcode: 959532
One tap mobile
+33170372246,,65611907988# France
+33170379729,,65611907988# France
Dial by your location
+33 1 7037 2246 France
+33 1 7037 9729 France
+33 1 8699 5831 France
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+41 43 210 71 08 Switzerland
+44 131 460 1196 United Kingdom
+44 203 481 5237 United Kingdom
+44 203 481 5240 United Kingdom
Meeting ID: 656 1190 7988
Find your local number: https://cern.zoom.us/u/cUwyxIgTf
Join by SIP
[email protected]
[email protected]
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 656 1190 7988
Passcode: 959532
NP04 - Aaron will (hopefully) meet up with Wes Tomorrow and check fibres etc. on NP04
ST has put 1ms setting on TS into 4.x ( production branch ). Will need testing at CERN.
DGC has received requests for help from DAPHNE firmware. Needs to respond to.
Need to estimate travel to CERN for NP04 , NP02. Estimate one 1-week trip per month. (Not always the same person). DGC will create calendar for availability and circulate ( as suggested by DA ).
----------
Prep for iPRR
DGC still iterating on safety case. DGC plans to respond by this time next week.
DGC has response from Roland S. w.rt. QA/QC. Some comments but minor
SP - writing up results into a document. Has added jitter measurement. SP will get technical details from DGC regarding jitter measurements (next week).
SP points out that we haven't discussed procurement Doc. Suggests meeting next week ( SP, ST, DC). DC reminded - only for AFC.
DGC reminded - we have decided to have Docs ready end March.
---------
Round Table
ST - Looked at MMC. Not getting 3V3MP. Comparted schematic with (working) AFC circuitry. Investigations still ongoing. ( DGC and ST baffled at moment, but with some avenues to pursue ).
DL and ST ran tests at CERN. Measuring timing delays. Did this by testing endpoint monitoring service at CERN. ST modified to add info print out.. Ran on 5 x DAPHNE , 4 APAs of WIBs ( 8 working ... ). Some of DAPHNE were in state-6 (not 8). Measured RTT for devices that were in state-8 using endpoint monitoring service. On 2 APAs none of the WIBs gave RTT. On the APA that were working, both had one WIB that didn't work. Roger (from CE) pointed out that some PTC versions have a mistake in Tx/Rx and were patched with solder/wires.
Secondary issue - couldn't read endpoint state via our timing fibre from master.
Action from run meeting - organize debug session(s) with WIB expert(s).
5 DAPHNEs were able to be moved to state-8 (by EP monitoring services)
3 DAPHNEs with endpoint addresses, but DL hasn't been able to physically locate them.
1 WIB on NP02 has issues
Some of the DAPHNE on NP04 addresses clash with VD BDE WIBs addresses ( 0x21 , 0x22 , 0x23 ).
Good news - endpoint monitoring is working. Ready to be run continuously on NP04. Do we add this to Graphana dashboard? ( Will require several days of effort into legacy CCM. ). DL says we need to decide how we are going to store list of EP addresses, list of RTT, etc. ST suggests coming up with a way of running reliably at NP04 and then revisiting. ST suggests using log messages before now, but waiting until new CCM API stabilizes before integrating with dashboard.
If EP doesn't respond EP mon will put error in dashboard. ( DL asked have we tested this if EP goes to state 0xE ?). If EP is in state 6,7,8 won't put message in log. Get log message for first RTT. Get log if RTT changes. DL suggests flagging error if RTT changes.
DL and ST already have ability to do this. DGC suggested that he and DA are briefed ( to maximize bus number/ shifters ).
ST has verified that we can run Alma 9 images on kipper ( with local directory access and access to hardware ). At the moment need sudo to run docker. ST suggests investigating non-sudo access. DGC will ask Luke K / Rhys M for magic to access docker without sudo, if it isn't easy then just use sudo. DGC's understanding ( from Luke K) is that it will be a while before we have a UoB Alma 9 build.
sudo docker run -v ${host_path}:/${docker_path} -it ghcr.io/dune-daq/frozen-release-alma9:fddaq-v4.3.0-a9
source cvmfs/dunedaq.opensciencegrid.org/setup_dunedaq.sh
setup_dbt latest
#dbt-create fddaq-v4.3.0-a9 work/fdaq_v430
dbt-setup-release fddaq-v4.3.0-a9
dtsbutler -h
Need more testing before annuncing that timing system doesn't need Alma 9. Aim to have tested in time to drop CentOS8 for next 4.x release.
DA - (really) close to finishing off initial version of timing config in OKS. Will merge in . Has been trying to run nanorc / nanotimingrc and having issues (
==> Error: Config file is not readable: /cvmfs/dunedaq-development.opensciencegrid.org/nightly/NFD_DEV_240321_C8/spack-0.20.0/etc/spack/defaults/linux/compilers.yaml
ERROR: [03/21/24 11:47:55] [dbt-setup-tools.sh:263]: There was a problem calling spack load systems@NB_DEV_240321_C8
ERROR: [Thu Mar 21 11:47:55 GMT 2024] [/cvmfs/dunedaq.opensciencegrid.org/tools/dbt/v7.7.0/scripts/dbt_setup_tools.py:71]: There was a problem running "/cvmfs/dunedaq.opensciencegrid.org/tools/dbt/v7.7.0/scripts/dbt-clone-pyvenv.sh /cvmfs/dunedaq-development.opensciencegrid.org/nightly/NFD_DEV_240321_C8/.venv 2>&1" (return value 100); exiting...
)
Aiming to merge for release candidate tomorrow.
ST points out that this is an "engineering release" (won't be fully functional until other packages also ported).
ST asks that DA includes the updated UML into the pull request. DA said OK.
DGC asks if DA could assemble two Enclustra+FMC
DGC - Working on Safety , QA/QC , FIB design.
Jon Sensig - he and James Shen have been busy with CTB support, collaboration meeting, etc. Aims to have version ready for testing in Bristol by end of next week .
Includes routing of 1PPS to hardware debug pin.
Includes registers to load TS from IPBus or from IRIG.
Include ability to select between starting counter from either IRIG end of frame strobe or IPBus.
Include ability to read out current TS value (atomically)
Nuno Barros - has received hardware shipped from Bristol . Doesn't look broken. Expects to be asking for assistance. Working on including endpoint code from 7.2.0 into CIB . Getting problems building on Zynq US+ . Feedback multiplier had to be adjusted ( freq was out of range ). ( Had other problem: Trenz routed a clock pin to High-Density not High-Performance pin , which causes criticial warnings from Vivado ). Has assembled two version of the firmware. Would like instructions on how to run dtsbutler. DGC asked if NB could post to #timing-integration.
JPM - Has made more measurements around phase of 0, 180 degrees errors round about phase difference 400-800ps. Deviation from linear -40ps - 200ps in this range. Using 32-tap delay line. Nominal value 72ps/tap. JPM sees deviation.
AoB - Dennis away for next 2 weeks, then IoP ( so away for next 3 weeks)
DA will be leaving from Malta on 28th for 2 weeks.
TLU was in Faraday cage. Back at cold box tomorrow.
Present: David Cussans(DGC), Diana Antic (DA), Dennis Lindebaum(DL),Jon Sensenig(JS), Sudan P, Jim Mateyack (JM), Stoyan T, Aaron B., Jean-Pierre Martin (JPM)
Update on plans for NP04 - DGC showed some slides with his assumptions about ramp-up to NP04 (see attached slides). Corrections from DL.
Update on iPRR - DGC showed the "charge letter" and invited comments. DGC will check with Alessandro Thea the state of recommendations from timing system FDR.
Technical Coordination meeting - We are asked for a short presentation on preparation for iPRR. DGC will prepare and circulate tomorrow.
Round Table.
ST - Started debugging MIB MMC in uTCA crate. No communication with crate with either main MMC code or the image downloaded code. With the MIB plugged into the crate, the AVR programmer didn’t see the 3.3V. Possibly the uC is not getting power.
SP - No time for DUNE work in last week. Will have a couple of hours tomorrow to work on iPRR documentation.
DL - Documenting what timing hardware is where. No new failures in soak test ( except for one endpoint lost lock. Happened at the same time that DA and DGC were moving equipment about ). GIB - MIB RTT measurement still unreliable.
JS - No time to work on GIB firmware in last week. Hopefully this week.
AB - Investigating routing of timing fibres etc. ( DL pointed out a spreadsheet that exists showing mapping)
JM - He and Kevin Fahey have "signed off" on QA/QC plan. DGC expressed thanks and said that it would be passed onto new technical lead when one is appointed.
DGC - Has been working through detailed comments from Terri Shaw and Linda Bagby about electrical safety of AFC. ( e.g. power trace/via current handling. Calculating power pin de-rating from Samtec datasheets and FNAL guidance ).
JPM- Reported on progress with precision phase measurement. Replicated the phase measurement circuit with 90-degree clock to be able to work around 0 and 180 degree non-linearity. Intermittent problems with I2C ACK failures. ( DGC urged JPM to post details to #timing-developers the next time faults occur )
DA - Chasing customs clearance of units shipped to CERN and LIP. Documenting what has been shipped where. Working on OKS. Still struggling to get a clean build. Asked Gordon Crone for some "pair programming" time.