Topic: Timing System Meeting
Time: Sep 26, 2024 03:00 PM London
Join Zoom Meeting
https://cern.zoom.us/j/65611907988?pwd=ak9jK04xdFY0OThzN3RRbEdVZkhNZz09
Meeting ID: 656 1190 7988
Passcode: 959532
One tap mobile
+33170372246,,65611907988# France
+33170379729,,65611907988# France
Dial by your location
+33 1 7037 2246 France
+33 1 7037 9729 France
+33 1 8699 5831 France
+41 31 528 09 88 Switzerland
+41 43 210 70 42 Switzerland
+41 43 210 71 08 Switzerland
+44 131 460 1196 United Kingdom
+44 203 481 5237 United Kingdom
+44 203 481 5240 United Kingdom
Meeting ID: 656 1190 7988
Find your local number: https://cern.zoom.us/u/cUwyxIgTf
Join by SIP
65611907988@188.184.89.188
65611907988@188.185.118.153
Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 656 1190 7988
Passcode: 959532
DUNE AFC v4 status
MIB v3 status
MIB tongue v2 status
FIB v2 status
GIB v2 status
Schedule status
Documentation status
28/Nov
AFC
Close action to review AFC v4 - too late.
No update on Stage2 funding
ACTION: DGC to ask Magnus L about progress on modifying AFC v4 to our spec.
MIB
Still chasing order for pre-production boards. ( Hi5 need to supply financial information to SBS)
ACTION: If no progress by Tuesday DGC will investigate cancelling order with Hi5 and order with Newbury. ( DGC will try phoning ).
GIB
Waiting for requote - needed to substitute package of TTL-> LVPECL converters on BNC inputs
Safety Documentation: ACTION: DGC to start writing
GIB in Bristol showing hardware faults - 3.3V power sometimes fails and needs power cycling. PLL and SFPs not responding to I2C
ACTION: ST will contact Wes and ask him to connect up ( or check connections ) GIB to a fanout.
ACTION: DGC - will order a couple more Si5395 - although will be difficult to rework.
PRR Schedule
Sandeep has sent us an updated schedule. Need to check his working. ACTION: ST , SP , DGC to review updated schedule.
----------------------------
AFC status:
Pre-production: Order received. Delivery by mid-Feb '25
( --> Production delivery mid-August + time taken to verify --> Say, mid-October )
Reminder: Happy to use Bristol modified AFC v4 boards for PRR
Magnus can continue doing modifications to AFC v4 - if not done by mid of next week, Action: DC to chase
Action: arrange internal review of DUNE mods to AFC v4
MIB v3 status:
Requisition raised. Delay due to chosen supplier (Hi5) not being in purchasing database.
Action: DC to discuss design changes to MIB tongue-2 with PH.
Fib v2 status:
On bench. Initial testing done.
Full being done by ST. Two out of 8 so far....
Need to add LED status lines to firmware ( but LEDs go out when AFC programmed so probably OK)
Gib v2 status:
Went for quote 15/Nov/24
Documentation:
Action: Electrical Safety documentation - draft by early December
Addendum: Discussion of review actions.
Probably have further work to do after iPRR.
i.e. Need to perform some system tests at ProtoDUNE . Job - which features??
Job - what impact on PRR date?
Action : ST to organize an internal meeting to discuss
Action : ST to draft words w.r.t. redundancy.
Action: DGC draft words w.r.t. partitioning. Calibration campaign - do in lab first. Difficult at PD2 . Don't have . Discrepancy between two GPS system. Meta-stability - explain we shouldn't have any . Not doing hot-swap any more. ( Need to align pleisisynchronous clocks though )/
November 24th
ST gone as far as he can ( given GIB h/ware failure). ACTION: ST to merge s/ware changes for GIB to develop and include in patch release 5.2.1
-------------------------------
ST testing TS transfer in MIB - works ( can configure GIB-->MIB endpoint . receives TS. TS from GIB used to initialize master in MIB and then TS from MIB master tx to AFC/FIB then
Then HSI connected to FIB compare with HSI connected to GIB. Looking at TS captured by pulses from Pgen. Difference between the two stayed the same between system resets. (Hurrah!). Difference is about 60 ticks.
About 100 ticks between MIB and endpoint.
Found issue with reset logic in GIB and MIB ( if call io reset the master doesn't get reset correctly. Doesn't go back to waiting for TS initialization. Also doesn't reset counters for outgoing messages correctly.
ST has written software to support MIB swapping between upstream sources.
------------------------
Notes from 14/Nov/24
ST: after release,
1. should get back to this after fib v2
2. then timestamp full change
3. then timestamp with TAI
Update: current gps oscillator just died after 3 yrs, DC contacted another manufacturer and we are getting new one on load with 5 yr guarantee
24/Nov
Waiting for
1. functional GIB
2. Control of fine phase blocks in AFC/FIB
----------------------------
5.x : No progress on OKS.
DA will work on OpMon when she returns.
ST has tested GIB and MIB reset with 5.2.0. So, consider regression testing complete. ( Release out in any case)
Patch release in progress - date unclear.
ST has got EP mon service working with fanout firmware on fanout hardware. ( Has tested with MIB --> AFC/FIB. Not yet tested with fanout on pc059 hardware. ) Action: ST to test on fanout hardware.
Timing FSM mods - Hopefully will be done by team working on overall FSM - no action for "timing team" (other than testing)
Gaia firmware/sware : Sitting behind fixing newly identified rst bug and also fanout on fanout firmware.
-------------------------------------------------------------------------------------------------
Notes from 14/Nov
DUNEDAQ - Finish regression testing of 5.2.0 ( Action: ST to finish regression testing at same time as developing/testing GIB/MIB timestamp transfer)
OpMon for HSI - DA will do this when she gets back ( and/or ask Marco R to help )
Job - get EP mon services working with fanout firmware working on fanout hardware ( currently running on master firmare on fanout unit)
Job - have to update timing FSM to keep in step with overall FSM
Gaia firmware/sware on GIB: ST may implement epoch switching as part of 5.2.x patch release.
-----------------------
HSI:
Diana: Working on testing HSIReadout using drunc on hardware (Boreas on fmc)
Action: ST + DA to work together on getting this working
Update: ST and DA worked on getting drunc running with hsi data file, some raw data events are not being sent, DA moving files and updating data files, ST to fix raw data issues
Opmon:
Action: After OKS stuff working, ST, DA and Marco Roda to meet to talk about getting Opmon back
Timing-drunc integration:
FSM is now configurable st we can request FSM for ourselves and we won't need to use enable triggers and other commands that we don't need
Custom commands which take args; ST tested a couple weeks ago and works - implemented cmd to send timing msg
Action: to add rest of commands after this release
Gaia high level support:
ST: there's a param to decide which timestamp source you're using (implemented) and the other is to decide which IRIG source to config IRIG epoch you want (to be done for next release)
ST: OKS config branches merged (woop); OPMon work will be deferred due to serious issues in latency buffer code; release has been delayed by a week
Action: testing drunc integration (next week, ST to do that)
Action for (delayed) dunedaq 5.2: testing for timing config + usual regression testing
24/Nov
No update
(ST aiming to have requirements document out for comment by end of year)
----------------------------
ST: May have to de-scope design of timing as service
-------------------------
Notes from 14/Nov
ST: Pierre and ST discussed this
ST presented outcome of discussion at CCM meeting. Not clear if audience understood issues involved.
Action: ST and PL to update spec in light of feedback from presentation.
For PRR should have design for this (UML diagram)
28/Nov
JP - Can only build *.bit with Vivado 2018.3 , but ipbb won't build with Vivado that old.
DGC - sees two options either build at Montreal and test in Bristol , or find another ZCU102 ( e.g. one at CERN )
ST points out that a different carrier should have a different carrier ID. DGC claims that all the IPBus registers will be the same with the same address map. ( Since targeting pc069 )
Action: JP to build *.bit file with 2022.2 and we will test in Bristol.
JP has double checked - TCA9800 will allow level shifting from 1.8V to 1.8V
Action (not high priority) DGC will modify pc069a design and pass modified design to ND ( SAND ) team to manufacture.
ST - Tests with pc069 show that Tx disable line doesn't work. Need to check before we manufacture more. ( DL thinks that we have seen this working. ACTION: DL will double check ELOG and post to Slack )
-----------------------------------------------------------------
21/Nov
J.P - ipbb doesn't build *.bit file with 2018.3
DGC thinks this is due to IP *.xco file format change (c.f. Penn issues )
Action: J.P. will build a simple *.bit file with 2018.3 and attempt to load to Oxford ES ZCU102
----------------------------------------------------------------------
ZCU102 : Oxford ES version arrived in Montreal.
Needs Vivado 2018.3 to program FPGA. Compatible with EP firmware??
Action: DA to check if SFP interface on ZCU in Bristol works
JPM: to use his branch to test this ^ - v simple EPT with 4 resistors
Action: JP to check if TCA9800 is a suitable level translator. Are there any other chips need changing to cope with 1.8V VCCIO ? DGC thinks "yes" , "no".
NP02
FMC needs
24/Nov
Action: DGC to ask when we can recover NP04 timing hardweare
Half of NP02 CRT , new, Jose Soto, we will supply one ( or hopefully two timing endpoint)
Other half Grenoble , Jean-Sebastian. Will use IRIG-B and 10MHz from nearest GPS-DO
Action: ST, DC to review list of FMCs we need for production testing.
Action: ST,SP,DC to discuss if we port to different FPGA carrier now. ( Or buy full stock of Enclustras now)
--------------------------------
Recover endpoint used for NP04 CRT and give to NP02 CRT
Action: DGC Ask #daq-ct : how long NP04 will be in operation - need to recover timing hardware. Ask when we can swap fibres to other fanout ( the one with new fans ). Want to free up a fanout for GIB --> Fanout --> endpoint tests.
Action: DGC to ask Cheng-Ju for Enclustras - since we can use with the FMCs we have now.
DGC recommends replacing fans in pc059 unit serving NP04 in January
ST - once firmware ready commission GIB + pc059 at CERN. Connect to test endpoints ( e.g. Charon ) see if the GIB at CERN shows same failure mode as one in Bristol ( 3.3V rail sags to 2.7V and SFP stops transmitting. Brought back by power cycling ).
Action: ST , DC to put together a change request for manufacture of more FMCs ( as part of extra requirements for project hardware not costed so far.) Action: ST to schedule a meeting.
------------------------------------------------------------------
Notes from 14/Nov
NP02: Contact from N02 CRT - will try just 1PPS to interface. Need endpoint hardware. Gave them our spare.
CRP factories: Don't need further units until towards the end of 2025.
Need to decide what to do about Enclustra order.
-----------------------
NP04:
Done.
DL: to make list of what went well/not while writing up thesis- 2 weeks to get draft done
Action: all of us to have a think and catch up in 2 weeks on this ^
NP02:
Action: DC to figure out if support we currently have (Aaron, Wes) will be enough/around for NP02 run
No beam until March...
Dario pushed back WR integration to end of year...
ICEBERG:
No news (good news?) - DC to check if fans are working
Everyone at fermi had to take August off...
Shipping Timing Unit to CRP factories:
Action: DA to do after HSI work, to fix cables and lid etc...
DC at CERN: replaced 2 fans, only old fanout has old fan
DA: to try config fanout PLL