Please read these instructions before posting any event on Fermilab Indico

Europe/London English (United Kingdom) Login

Timing System Meeting

→ Europe/London
Description

Timing Week Ahead

Zoom: https://cern.zoom.us/j/65611907988?pwd=ak9jK04xdFY0OThzN3RRbEdVZkhNZz09

 

David Cussans is inviting you to a scheduled Zoom meeting.

Topic: DUNE Timing System Week Ahead
Time: Jan 17, 2022 11:00 AM London
        Every week on Mon, 20 occurrence(s)
        Jan 17, 2022 11:00 AM
        Jan 24, 2022 11:00 AM
        Jan 31, 2022 11:00 AM
        Feb 7, 2022 11:00 AM
        Feb 14, 2022 11:00 AM
        Feb 21, 2022 11:00 AM
        Feb 28, 2022 11:00 AM
        Mar 7, 2022 11:00 AM
        Mar 14, 2022 11:00 AM
        Mar 21, 2022 11:00 AM
        Mar 28, 2022 11:00 AM
        Apr 4, 2022 11:00 AM
        Apr 11, 2022 11:00 AM
        Apr 18, 2022 11:00 AM
        Apr 25, 2022 11:00 AM
        May 2, 2022 11:00 AM
        May 9, 2022 11:00 AM
        May 16, 2022 11:00 AM
        May 23, 2022 11:00 AM
        May 30, 2022 11:00 AM
Please download and import the following iCalendar (.ics) files to your calendar system.
Weekly: https://cern.zoom.us/meeting/u5ErduihrjwjEty6rVlnv8tn7IyhqaavFeey/ics?icsToken=98tyKu-tqDorE9SSuRGPR_MMBIj4d_PziGZEj7dEyg3TDy5FOhf4YctbOqN-Ec72

Join Zoom Meeting
https://cern.zoom.us/j/65611907988?pwd=ak9jK04xdFY0OThzN3RRbEdVZkhNZz09

Meeting ID: 656 1190 7988
Passcode: 959532
One tap mobile
+33170372246,,65611907988# France
+33170379729,,65611907988# France

Dial by your location
        +33 1 7037 2246 France
        +33 1 7037 9729 France
        +33 1 8699 5831 France
        +41 31 528 09 88 Switzerland
        +41 43 210 70 42 Switzerland
        +41 43 210 71 08 Switzerland
        +44 131 460 1196 United Kingdom
        +44 203 481 5237 United Kingdom
        +44 203 481 5240 United Kingdom
Meeting ID: 656 1190 7988
Find your local number: https://cern.zoom.us/u/cUwyxIgTf

Join by SIP
65611907988@188.184.89.188
65611907988@188.185.118.153

Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 656 1190 7988
Passcode: 959532

Hide

Present: David Cussans(DGC),  Diana Antic, Dennis Lindebaum(DL),Jon Sensenig(JS),  Sudan P, Jim M, Stoyan T, Aaron B.

Round Table.

ST - Work started on design of DTS as service. ST will write a spec for internal discussion in timing group then wider DAQ group. Already in discussion with Pierre L.

DA - Working on OKS. Submitted PRR. Gordon Crone reviewed it. Aiming to finish by Thursday 29th. DA thinks she is probably responsible for porting HSILibs to OKS. ST points out that there are dependencies on other packages. DGC requested help in lab when OKS PR for timinglibs is out of the door and before starting on HSI

DL - Sorted out script to monitor iPRR test system. ( 7 endpoints , 2 FIB/AFC , 1MIB , 1GIB , WR-LEN).

1. Scripts sets up all devices using dtsbuter

2. Script to read out all endpoint status' (including MIB, AFC/FIBs)

3. Script measure RTT from masters to all endpoints. ( Currently MIB --> endpoints)

4. Script to periodically (every 5 mins ) execute scripts 2,3.

No errors measuring RTT so far ( since Tuesday ). EP status all OK between Friday-Monday. Will convert results to lower limit on MTBF (if we see no errors.... )

AB - All quiet at the moment. ( Now an official ProtoDUNE guide. Ask him for Merch ).

SP - Will start including results in documentation template.

JS - Progress on GIB: JS can load timestamp via s/ware. NTP/IRIG converter working. Tested GIB IRIG decoder. Can decode correct time and load into master. Can read out timestamp from endpoint. Not entirely confident about IRIG-B decoding firmware, since IRIG source only puts out IRIG data stream, not 10MHz. (In DUNE/ProtoDUNE we will have 10MHz and IRIG locked together. The 62.5MHz in the GIB is derived from the 10MHz input. Will test in Bristol towards the end of next week. JS requests a review of code before we merge to develop. General consent that this is a good idea.

JM - Will pass on comments from him and Kevin F. w.r.t. draft AFC/FIB QA/QC plan.

Report from James Shen: An update for the GIB: Jon and I was able to test the gib with our new NTP to irig converter, and was able to successfully pass the timestamp from the irig source to the enclustra endpoint. Two slight issues with our setup:

  1. we are seeing that the irig block sometimes translate the wrong time code. We think this is due to the fact that our irig converter does not have a 10Mhz clock built in, and we are supplying an out-of-sync 10Mhz clock from a function generator. We are fairly certain it works as expected with a proper gps setup since Jon tested it with a proper whiterabbit irig source two years ago.
  2. Jon's irig block calculates the timestamp using the TAI epoch, which is about a decade off from the unix epoch. However, on the software side the timestamps are translated to human readable times assuming the unix epoch. So while the output timestamp from the endpoint is the expected one, the software timestamp is off by about a decade, giving us a time around when dune fd will probably start collecting data:)
next steps currently is to add the proper registers in the ts_source block to report the time stamp back to software. I will hopefully have some time to work on it tomorrow or next week. 

 

--------------------------

Old minutes:

Present: David Cussans(DGC),  Diana Antic, Dennis Lindebaum(DL),Jon Sensenig(JS), ,Jean-Pierre Martin(JPM)

Work on interim PRR

  • Date for interim-PRR set for 19th April, 15:00 - 17:00 CET.
  • Stoyan Trilov (ST) has made a tag of firmware and software for full chain tests.
    • test/full-chain/b1  for timing  (software)
    • test/full-chain/b1  for firmware
  • DGC has loaded tagged firmware onto GIB, MIB and two AFC/FIB (reproducing ST,SP setup). See https://webapps-pp.bris.ac.uk/elog/DUNE/110
    • Looked at clocks on two endpoints
      • One endpoint on AFC/FIB in slot 4 , the other endpoint attached to AFC/FIB in slot 8
      • Seeing ~ 5ps (sigma) jitter between the two endpoints(250MHz LVDS)
      • Seeing ~ 8ps (sigma) jitter between WR-LEN(10MHz) and an endpoint (250MHz LVDS)
  • Want to reproduce:
    • Test two Endpoints on same fibre (from FIB). Measure relative clock jitter
    • Test two endpoints on same FIB in separate SFPs. Measure relative clock jitter.
    • Test two endpoints on separate FIBs. Measure relative clock jitter. (DONE)
    • NB. Add optical splitters.

  • Functional tests. Measure:

    • Measure RTT

    • Apply delay and see 1PPS move w.r.t. each other.

    • Look at stability of WR-LEN 1pps cf. Endpoint Charon design.

    • (Measure how much jitter from each stage is introducing. WR-LEN - GIB, GIB - MIB , MIB - AFC , AFC/FIB - Endpoint)

    • Moving clock with fine phase.

    • Moving AFC in crate and looking for change in delay

    • Soak test - assemble as many endpoints as possible and leave running.

    • Do we want to test maximum attenuation / minimum optical power before system fails?

    • Do we want to measure optical "eye diagram"? Perhaps with 30GHz sampling scope just to show off.

  • Dennis working on making his testing scripts usable with full chain

  • DGC working on safety review (need to chase Terri).

  • DGC will prepare a draft of QA/QC documentation for AFC v4. (From template from Jim M. )

Other work:

  •  Diana to finish testing the three Enclustra/pc069a combinations packages in 1U cases. Use for full-chain tests then ship to CERN.

 

Round Table

  • JS - Got NTP/IRIG converter working. Next step - plug into GIB and check f/ware still able decode IRIG. DGC suggested connecting end of frame (second) strobe from IRIG decoder in GIB to GIB GPIO. JS this may already be done. Then able to initialize ts_source (and hence master) from IRIG
  • JPM - Has added IPBus readout to fine-phase measurement and 2ns phase measurement. Has produced an example Python script to read out registers.
    • JPM has prepared a short report.
  • DL - All quiet on the PD-2 front.
    • (Will want to taper out and concentrate 100% on thesis ~ October )
  • DA - has been working on OKS. Implementing plugins. Next steps defined.
  • DGC - Placing order for NAT NATIVE-C8 crate ( same model at TDE )

AoB

  • Integration with CRT coming up at some point
    • Will use 62.5MHz and a 1PPS strobe. (Probably.... )
  • Need to revisit integration with TDE before NP02 tests.
  • When do we set up GIB/MIB/AFC+FIB system at CERN? In time for NP02 tests.

 

 

 

 

 

 

--------------------------------------------------------------------------

Present: David Cussans(DGC), Jon Sensenig(JS), Sudan Paramevaran(SP),Jean-Pierre Martin(JPM), Diana Antic, Terri Shaw(TS), Jim Mateyak(JM), James Shen(JaSh), Aaron Bundock(AB).

Interim-PRR ( will cover AFC v4 only)

Documentation template - Sudan

ST has produced a template in Overleaf ( https://www.overleaf.com/read/rdqkpqwnwhts#a368f6 )

What items to test?

* SP has produced skeleton for test results.

* Test two Endpoints on same fibre ?

* Test two endpoints on same FIB

* Test two endpoints on separate FIBs

Functional tests. Measure:-

* Clock jitter

* Measure RTT

* Adjust delay and see 1PPS move.

* Measure how much jitter from each stage is introducing

* Moving clock with fine phase.

* All done - except -

- re-measure clock jitter with LVDS (not TTL)

- jitter on two AFC/FIB boards.

- Jitter on individual clock ( i.e. TIE with software CDR)

- Soak test for a week (does clock move. Do we see bit errors ? )

*** ST will make tags for f/ware and s/ware to give consistent versions for testing.

(Was hoping to merge back to develop before tagging, but unlikely for s/ware. F/ware already merged)

*** Continue to add "cut and paste" instructions for each test in elog.phy.bris.ac.uk/elog

 

Electrical safety review.

Documents now split into 5

1. Overview

2. uTCA (joint with Dario, except for MCH - we only need 1GBit/s and JSM - which we use and TDE don't)

3. AFC/FIB

4.GIB

5.MIB

AHJ ( Authority Having Juristiction)

QA/QC documentation.

- Information from Creotech (can add to QA/QC docs "as is")

- Need to document how we will test AFC/FIB combination.

( e.g.

* Test voltages on FIB

* Test SFP I2C for all slots

* Test LOS,LOL lines from SFPs

* Check UID reading from FIB

* Check optical signal from SFPs ( eye diagram)

* Functional test of endpoint (Dennis' scripts)

... but written in DUNE QA/QC template

)

 

Round table:

DL Operations . Daphne issues. (turned out to be firmware ). TLU moved back up to Faraday cage. Probably possible to use one of "new" units in Faraday case permanently.

DA Tested one of the three new 1U units. FOr OKS, implemented into plugins. Next step testing and (in parallel) seeking review by Gordon C.

ST Carrying on with support for fanout design in full chain test (for high-level s/ware). Will have one config file and send one command which will result in all MIB/AFC/FIB being configured. Consolidation work for full chain test.

SP Documentation for testing/results.

DC Review documentation.

AB - no tasks

JaSh - Work on GIB: Completed tests 2 weeks ago can communicate with endpoint. Just purchased NTP--IRIG h/ware (VCL-3045). Requested changes to timing source (since this will propagate everywhere). ST points out that there are sever branches. JaSh confimed that gib-rearrange is the most recent. JaSh asks what registers do we need. JaSh points out clock domain crossing from 10MHz -- 62.5MHz is still an issue. Error could be up to 8 clock cycles of 62.5MHz. Is this an issue? ST points out that spec for FD is 1us. So, ignore for now.

JS - Providing consultancy to JaSh.

JPM - Now able to read IPBus registers. (Took a little while) Has to be able to read out edge information to be able to resolve phase ambiguity. DGC asks if JPM can tidy this up and write some example Python scripts on using phase measurement firmware block.

JM - points out justification of tests also needed (rather than just a list of them).

AOB

ST points out that DUNEDAQ release fddaq-4.3.0. Will need to run regression tests. DL asks which firmware shall we use? He suggests turning relval/7.2.1 (which fixes endpoint addressing bug) into a release. ST will make 7.2.1 before he makes a relval/7.2.2 from develop straight afterwards.

---------------------- OLD

 

Present: David Cussans(DGC), Stoyan Trilov(ST), Dennis Lidebaum(DL),  Jon Sensenig(JS), James Shen(JaSh) Sudan Paramevaran(SP), Aaron Bundock(AB), Jean-Pierre Martin(JPM), Diana Antic, Kevin Fahey.

Full -chain test

ST - fixed some technical issues. Changed architecture of fan-out on AFC/FIB. Feeds data from back-plane into PLL, rather than  (to avoid clock/data going out of phase).

Messages passed from MIB --> AFC/FIB --> Endpoint and back.

Clock jitter between two endpoints measured - 50ps.

Can adjust FPGA

Can't use address pins on back-plane to set IP address. DGC to modify soft-core to read IP address from PROM. DGC to move soft core code C for AFC from Enclustra repo. before we make tags.

Issues with locking upstream CDR in MIB to returning data from endpoint.  DL suggests using "switch-and-lock" command. ST can't use this directly, since no longer using the physical Mux on board. ST will investigate using

ST will do more work before merging software back to develop. Will test with Nano timing RC.

SP - continuing to write template for tests. Needs another week.

-----------------------

Supporting PD-2

 

DL and DC talked to:

Nuno - decided not to pass timing messages to laser calibration system. ( Use CCM for veto message)

Zelimir - about using timing system messages to trigger SSP. Want a system in ~ 2-3 weeks that records when timing system emits command.

ST asks will we need to support new hardware (full timing chain).

Old PTCs - tx path from WIBs not properly connected. Problem identified.

ST - need to have a look at Grafana dashboards. Timescale by 2-3 weeks. Action - DL will do the smallest job that will provide useful data.

 

Kevin Fahey

Present: David Cussans(DGC), Stoyan Trilov(ST), Dennis Lidebaum(DL),  Jon Sensenig(JS), Sudan Paramevaran(SP), Aaron Bundock(AB), Jean-Pierre Martin(JPM)

Current Actions:

* @J.P. Martin - Use DTS test system in Montreal to integrate fine phase measurement with pdts_upstream_cdr. STATUS: In progress. Next step - connect registers to IPbus , as well as ILA.

* J.P. Martin - Test 2ns phase measurement (initial tests unsuccessful in Bristol and no time to debug.) STATUS: works at SCTR (checking with ILA) . Next step - check this works at IPBus level.


* @dgc - Get back to @Ioannis Xiotidis about issues with ND-Gar endpoint. STATUS: Not done


* @Stoyan Trilov - debug MMC. STATUS - Discovered issue with reset. Paused until after full chain test.


* @dgc - Investigate why hardware input into pc069 not working. STATUS - Fixed. (bugs in I/O block and constraints file. Need complete MR 100. Waiting until consolidation for full chain finished.


* @Diana Antic - Liaise with @kurt.biery about getting the "nanotimingrc in parallel with nanorc" modifications back into dune-daq test framework. STATUS -


* @Diana Antic - Investigate putting all outstanding actions for timing system (hardware/firmware/software/system) into issue tracking system. STATUS -


* @sudan - develop template for recording test results, full chain test (in preparation for interim-PRR ). STATUS - SP has initial template in Overleaf. Will start to fill out in coming weeks.


* @sudan - to link from existing pages to new "Page-1". STATUS - Done.


* @Diana Antic - Work on porting timing system configuration schema to OKS. STATUS - Temporarily on pause. Will go into fddaq 5.0.


* @James Shen - Refactor IRIG decoder. STATUS - Firmware written. Pushed to branch. Needs testing in hardware. : UPDATE: James back at Penn. Done preliminary tests. Plan for more extensive tests in coming week. Jon S. expects IRIG decoder to work, since it worked before refactor and current code passes CI.

 

Round Table:

* DGC - Finished first draft of documentation for "Safety Engineering Design Review" and sent for comments. Figured out how (technically) to transfer FIB/MIB/GIB designs to Oxford. Submitted documentation requesting permission to share something produced with Europractice licenced Cadence s/ware. Working with Keith Clarke (Bristol) to package three Enclustra/pc069a FMC combinations in 1U crates to enhance hardware available at CERN (pc069a has more I/O which may make it easier to integrate with CRT). At collaboration meeting hope to talk to Terri Shaw (safety approval) , Jack Fowler (GPS antenna location site evaluation at SURF) and Asher / ND team (ND timing system).

* ST - Refactored firmware fanout design ( on AFC/FIB ) in process of testing. Also working on software

* DL - All quiet on the NP front (after burst of activity setting up new scripts for DAPHNE . Old scripts needed modifications since they used fanout, rather than TLU which is connected to DAPHNE). Action: Send message to DAPHNE team to point out can use endpoint monitoring service ( rather than using dtsbutler scripts).

* JS - Reminded us that CTB will be used for data selection triggering. No timing signal at moment. DL said he or DGC could check this when they were at the collaboration meeting next week.

* SP -

* AB -

* JPM

 

AOB - None.

 

---------------------------------------------------------------------------------------------------------------------------------------

Old:

 

Present: Jim Mateyack (JM) , Jon Sensenig (JS), Stoyan Trilov(ST) , Diana Antic(DA), David Cussans(DC) , Sudan Paramevaran(SP),Dennis Linebaum(DL),Jean-Pierre Martin (JPM)

DC presented some slides on the goals and key dates for 2024 ( see attached PPT). Discussion resulted in:

  1. Basic code tidy by end of month to allow for full-chain test of GIB --> MIB  --> AFC/FIB --> Endpoint still looks achievable (ST).
  2. Full chain test for interim PRR should concentrate on hardware/firmware features needed to provide confidence about AFC v4. I.e.
    1. full chain clock test ( GIB --> Endpoint )
    2. timing data from MIB --> Endpoint
    3. timing data from Endpoint --> MIB with ability to switch between endpoints (only SFPs with direct connection from SFP to AFC FPGA)
  3. If the ability to initialize the DTS time-stamp from GPS is available we will include testing this as a "stretch goal" when choosing tests for interim PRR
    1. James Shen reports (via Slack) that the IRIG decoding firmware and timestamp counter firmware have been re-factored and the code synthesizes. Will be tested after James returns to Penn next week.
  4. We can support Proto-DUNE2 with fanout hardware. The fanout can accept and external clock (not tested, but a simple circuit). Will need to convert from LVTTL --> LVDS, but this is relatively straight forwards. Won't need a TLU, since the CTB has been reconfigured to time-stamp the accelerator signals
  5. JM ran through a list of documentation we will need for i-PRR. This includes
    1. procurement plan
    2. list of tests that will be performed by the supplier of the AFCs
    3. list of acceptance tests that will be performed by us
    4. design documents ( schematic, BoM, etc.)

Round Table

  • DL has re-started timing sessions ( nanotimingrc sessions had been shut down over Christmas to avoid risk of error message flooding )
  • DA prepared presentation for DUNE-UK meeting. Now restarting OKS work. Will post UML diagram in the next few days
  • DC has been working on Safety Engineering Design Review documentation. Plans to post draft to Slack early next week. ( Has also been working on getting Oxford working on GIB/MIB/FIB respin but forgot to say in the meeting)
  • JPM reported on work measuring phase of returning data from endpoint w.r.t. outgoing data. (See attached PPT)
    • There are ambiguities in phase. DGC asked if these could be resolved using the 2ns bin phase measurement already present in firmware (but not working).
    • If the clock used to sample the outgoing/incoming data is close to a simple multiple of 62.5MHz then the beat-period will be very long and there will be problems measuring phase.
      • This has been avoided on the MIB by deliberately having a reference oscillator that doesn't have a simple ratio to 62.5MHz. However, the main issue is on the AFC. Need to investigate
  • ST was at DUNE-UK but (re)starting code tidy work for basic full chain tests.
  • SP has redirected pointers to FMC documentation to the new page(s).

 

There are minutes attached to this event. Show them.
The agenda of this meeting is empty
Powered by Indico v3.1