Frequently Asked Questions
about the White Rabbit PTP Core
-
Frequently Asked Questions
-
WR PTP Core general questions
- Q: Loading the FPGA bitstream seems to work, but executing spec-cl or spec-vuart results in a segmentation fault. What am I doing wrong?
- Q: How can I feed WR PTP Core running on SPEC board with 1-PPS and 10MHz signals?
- Q: Is it possible to generate a 10MHz reference from a SPEC FMC-DIO card synchronized to a white rabbit switch?
- Q: How can I synthesize WRPC gateware to run on SPEC in standalone mode?
- Q: Is alpha parameter a fixed constant in the WRPC-code somewhere or is it somehow estimated from measurement?
- Q: Synthesis without wrc.ram takes less time, how can I embed WR PTP Core software binary into the synthesized bitstream ?
- Q: How can I make the LM32 cross compiler work under Scientific Linux 7?
- Q: Why I have problems with synthesizing the WRPC in Vivado?
- Q: After updating the WRPC version to v3.0 I get errors when trying to read/write init script/sfp database/MAC address.
-
MAC addresses
- Q: I'm trying to run WR PTP Core on my personal FPGA boards and the MAC address for each of them is the same.
- Q: How is the MAC address set?
- Q: I would like to check if the communication with 1-wire serial number chip is working properly. It is used for MAC address generation, does it mean that it works well if the link works (it sends frames with source MAC address) or could it just take a default value?
- Inter-operability
-
WR PTP core internals
- Q: Why is the PLL locking the local clock to the physical link clock implemented with the help of an LM32 processor and is it not just implemented in hardware?
- Q: What about all those oscillators?
- Q: Could I use less external oscillators?
- Q: I'm trying to figure out how to add timestamps to samples from an ADC (25Ms/s) on a standalone Specv4 board.
- Q: How can I phase align a non-125 MHz clock locked to WR 125 MHz reference?
- Q: Which clocks I need to generate for WRPC on a Kintex7?
- Q: What is the purpose of the different clocks of the platform wrapper in WR PTP Core reference designs?
- Q: Any other helpful documentation on how to implement the WRPC?
- Files
-
WR PTP Core general questions
WR PTP Core general questions
Q: Loading the FPGA bitstream seems to work, but executing spec-cl or spec-vuart results in a segmentation fault. What am I doing wrong?
A: First of all make sure you run the spec-cl or spec-vuart as root (e.g. using sudo). If you have more than one SPEC board in your computer please check the spec-sw documentation for the parameters specifying which board spec-cl and spec-vuart should use.
Q: How can I feed WR PTP Core running on SPEC board with 1-PPS and 10MHz signals?
A: First of all you need to have your SPEC carrying a Digital IO FMC board (DIO). The DIO has several LEMO connectors you will be using. You just need to plug your 10MHz signal to LEMO connector no.5 and 1-PPS signal to LEMO connector no.4. After setting the WR PTP Core mode to GrandMaster (please check WRPC building manual for instructions on how to do this) it will synchronize its internal clock to your reference and will distribute it to the Slave.
On the other side, on the SPEC running in Slave mode you can connect LEMO no.1 to get 1-PPS output signal aligned to 1-PPS from Master/GrandMaster. There is no output for the 10 MHz signal (we believe the signal driver used on the DIO is not suited for it).
Q: Is it possible to generate a 10MHz reference from a SPEC FMC-DIO card synchronized to a white rabbit switch?
You cannot use the two oscillators on the SPEC to generate this clock as they are needed for WR synchronization. One of them is the local 125MHz clock being synchronized to WR Master, and the other is DMTD offset clock needed for phase measurements. If you need to generate 10MHz from your node you need to use Fine Delay mezzanine instead of the FMC-DIO.
Q: How can I synthesize WRPC gateware to run on SPEC in standalone mode?
A:
- copy the wrc.ram file generated during the wrpc software compilation to syn/spec_1_1/wr_core_demo/.
- set the generic parameter of xwr_core component in
top/spec_1_1/wr_core_demo/spec_top.vhd
g_dpram_initf => "wrc.ram" - build new gateware
Q: Is alpha parameter a fixed constant in the WRPC-code somewhere or is it somehow estimated from measurement?
A: alpha is a fixed constant for a given type of fiber. WRPC stores its value together with constant delays in EEPROM (check commands: sfp show ; sfp erase ; sfp add). For unknown fiber alpha can be calculated based on measurements as described in the WR Calibration document (https://www.ohwr.org/project/white-rabbit/wikis/Documents/White-Rabbit-calibration-procedure).
Q: Synthesis without wrc.ram takes less time, how can I embed WR PTP Core software binary into the synthesized bitstream ?
A: You can use data2mem to combine the configuration file (.bit with "empty" RAM) with an "elf" file. As a result you will get a ".bit" file with RAM blocks filled with your code. You'll need the placement of the RAM blocks that were generated by the place&route tool of Xilinx. This is the "-bd.bmm" file. An example CMD script do_elf.cmd is attached (Note that this example uses two different instances of the lm32).
Another tricky thing may be to create a ".bmm" file. The Xilinx tools do not provide a template (although the tool has got all the knowledge) so you should prepare one manually. Luckily this is a one time operation (as long as the memory size is not changed). Attached You'll find a sample ".bmm" file (fpga_xst.bmm for the same example design). The tricky bit is that the synthesis tool creates the final names of the BRAMs. The topology of the memory (how the BRAMs are composed to build the complete memory array) also may differ between sysnthesis tools.
These are the steps performed with Xilinx XST design flow:
1. Synthesize design with wr_core parameter g_dpram_initf=""
(empty)
2. Look in design.pcf file (or use PlanAhead) to guess ram instance
name
3. Fix instance prefix in fpga.bmm
4. Add ngdbuild option: -bm fpga.bmm
5. Implement as usual, get bitfile with empty ram
6. Compile wrpc-sw software and get wrc.elf
7. data2mem -bm fpga_bd.bmm -bd wrc.elf -bt work/fvme2tmwr.bit -o b
fvme2tmwr_wrc.bit
8.
Success!
Q: How can I make the LM32 cross compiler work under Scientific Linux 7?
A: If you are using the lm32 cross compiler available from the OHWR
[1] (32bit host version) be aware that it will not work on a SL7 using
the xfs file system. As soon as you use 64 bit inodes the lm32-gcc 32bit
host version will not work any more (for details see [2]). However, if
you create an ext4 partition and store your code and compiler there,
everything works fine.
As an alternative please use 64bit host version of lm32-gcc [3].
[1] https://www.ohwr.org/project/wr-cores/uploads/a2e8eeba448fbc8d580e68004e6f6c7f/lm32.tar.xz
[2] http://www.tcm.phy.cam.ac.uk/sw/inodes64.html
[3]
https://www.ohwr.org/project/wr-cores/uploads/2776ce0ba43503d1486ae205b48fb450/lm32_host_64bit.tar.xz
Q: Why I have problems with synthesizing the WRPC in Vivado?
A: There are two issues in Vivado: textio package procedure READ always returns "true" and Vivado synthesis can not infer the initialized RAM in the way it is described in the WR sources (it is handled correctly by XST and Precision synthesis tools). Both these issues were reported to Xilinx. For more information please check the e-mail thread on our mailing list
Q: After updating the WRPC version to v3.0 I get errors when trying to read/write init script/sfp database/MAC address.
A: Starting from v3.0 we use a simple filesystem SDBFS instead of hardcoded base addresses for Flash/EEPROM storage. This means, before you're able to write information to your non-volatile memory, you should initialize it with the filesystem structure. In the Files section you will find a binary with the filesystem image that should be written to Flash/EEPROM. Then, depending on your setup you should check the release User's Manual instructions in:
- section 3.1 - if you run SPEC in a PCIe slot of your PC
- Appendix C - if you run SPEC in a host-less setup or if you run WRPC on a custom board
If you don't use the WRPC v3.0 reference binaries for SPEC, but you build your custom WRPC, you're still able to use the legacy code for storage based on hardcoded addresses. If that is the case, you have to disable Use SDB to manage storage (instead of legacy eeprom code) option when compiling LM32 software from wrpc-sw repository. However, without SDBFS you won't be able to use Flash for storage - only EEPROM can be then used.
MAC addresses
Q: I'm trying to run WR PTP Core on my personal FPGA boards and the MAC address for each of them is the same.
A: WR PTP Core uses the ID of digital 1-wire thermometer available on SPEC board to generate unique MAC address for each board. If you use some other board instead of SPEC there are high changes that you don't have such thermometer. Then, your MAC address is default, but you can change it manually with "mac set " wrpc shell command.
Q: How is the MAC address set?
A: The wrpc will configure its mac address based on the 1-wire
temperature sensor, the actual mac address = base address + ID of the
1-wire sensor. You can get the mac address by typing the code "mac get"
through the UART.
This is not really a proper method as MAC addresses should be set in
ranges that are made available for specific companies. It is on the list
of the WR team to make this address defined in a different way.
(8/2016)
Q: I would like to check if the communication with 1-wire serial number chip is working properly. It is used for MAC address generation, does it mean that it works well if the link works (it sends frames with source MAC address) or could it just take a default value?
A: The default MAC is 22:33:44:55:66:77. If the frames sent using WR PTP Core (both, PTP frames generated inside the core and any other like frames sent using the core, e.g. Btrain streamers) have source mac address different than the default one, your 1-wire works.
See also this FAQ entry.
Inter-operability
Q: Can I use WR PTP Core as a regular IEEE1588 Slave?
A: Yes, it is possible starting with version v2.1
of the WR PTP Core which uses PPSI as a default WR PTP engine. See also
WR switch
FAQ
Not now, but we are working on it. Currently the WR PTP Core can be
the Master for non-WR, regular IEEE1588 device, but cannot synchronize
itself to a non-WR Master. However, next release of WRPC software will
allow to configure also regular IEEE1588 Slave and Master.
Q: Can the WR Core receive data from a standard Ethernet Switch?
A: Yes. If your application does not require timing, you may use a standard Gigabit Ethernet switch; the WR core will simply pass packets through. The WR Core will not need to wait for any synchronization packets before start receiving data. The data is independent of synchronization. See also can i use copper sfps in wr network for connecting nodes that don't require wr-timing.
Q: Can we connect a WR node to a standard PC?
Sure. Connecting standard PC to WR switch/node is not a problem. We usually connect WR node to WR switch and then WR switch to PC. In any case, one think to keep in mind is that you need to connect your WR node/switch to 1 Gbps interface as WR switch/node does not support autonegotation and other speeds than 1Gbps (e.g. 100Mbps). See also can i use copper sfps in wr network for connecting nodes that don't require wr-timing.
WR PTP core internals
Q: Why is the PLL locking the local clock to the physical link clock implemented with the help of an LM32 processor and is it not just implemented in hardware?
A: Three reasons:
- Because control algorithms for PLLs are not so simple
- Because HDL is difficult to debug
- Because SoftPLL takes much less space
Read the full story in the document Why the SoftPLL is not a HardPLL...
Q: What about all those oscillators?
- Must the clk_dmtd_i and clk_ref_i/clk_sys_i inputs to the WRPC be independently controlled oscillators, or could they be generated by the same oscillator?
- Are the DAC outputs to the VCXOs controlling the phase of the clock signal?
- Is the 125MHz input to the WRPC also required to be the refclk to the MGT?
A: In short: 1 yes, 2 yes, 3 yes.
The two clocks (20MHz and 125MHz in the SPEC reference design) need to
be independently controlled as one of them is used to generate our DMTD
offset frequency for phase measurement. The other one is our main
system&ref clock. Then the two other questions are also true. We need
to do the PTP calculations (with phase measurements) and then shift the
local refclock so that it is phase aligned with the Master refclock.
Q: Could I use less external oscillators?
A: Li hongming from Tsinghua University showed in January 2018 that it is possible to use only a single external oscillator instead of two by using an internal PLL of the FPGA to create the helper clock of DDMTD. The synchronization result is even slightly improved compared to current method which uses the softPLL based on LFVCXO026156. For now it is not implemented in a supported Release of the WRPC. (Jan.2018)
Q: I'm trying to figure out how to add timestamps to samples from an ADC (25Ms/s) on a standalone Specv4 board.
A: Just latch the values of the WRPC outputs tm_tai
and tm_cycles
.
tm_tai
is the current number of TAI seconds, tm_cycles
is the number
of 8 ns ticks since the beginning of the current second. When
tm_cycles
reaches 124999999, tm_tai
counter is incremented and
tm_cycles
goes back to 0.
Note that these counters run in the WR reference clock domain
(clk_ref
). If the signal to latch the timestamp comes from a different
clock domain, you can use the gc_pulse_synchronizer
module from the
general-cores library to pass it to clk_ref
domain.
Q: How can I phase align a non-125 MHz clock locked to WR 125 MHz reference?
_Original question: I'm integrating White Rabbit in a distributed DAQ
system runing at
41.667 MHz. There are clock signals originating from WR 125 MHz main
clock and divided by 3 using sy89229 with async reset. The phase of
divided clock is synchronized by resetting the divider after WR is
locked using 1-pps signal (and tm_tai % 3 == 0). The drawback is an
interruption of divided clock while reset.
Does anyone has an idea how to synchronize derived clock phase with
1-pps using softpll adjustment?_
A: This feature is planned for 2015 to be included in the official WR PTP core release and the upcoming release of FmcAdc100M with WR support.
The idea is to produce a PPS signal from the 41.667 MHz clock and sample it with the WR PPS signal. it's a trick that we already use in recent SoftPLL versions for locking to a 10 MHz external reference. The phase of the aux 125 MHz clock is swept (either by linear sweep or by divide-and-conquer) until a transition is observed in the PPS signal - this will mean that both PPSes are in phase (+- the jitter). The extra benefit is capability of fine phase tuning of the divided clock (of arbitrary frequency).
Q: Which clocks I need to generate for WRPC on a Kintex7?
Q: For Kintex7 (in eg. xwrc_platform_xilinx.vhd) the clock
clk_62m5_sys_o is generated from clk_125m_pllref which is the
single ended version of clk_125m_gtp_i. But for Spartan6 it is
created from clk_125m_pllref_i which in this case is not the single
ended version of clk_125m_gtp_i. The clk_125m_pllref_i is
generated from the VCXO controlled by a DAC via the signal
dac_dpll_data_o.
In the Kintex7 case, is it the clock clk_125m_gtp that shall be
controlled from the DAC and the VCXO?
A: Correct.
In the Kintex7 case the internal clock is named "clk_125m_gtx_buf"
which originates from "clk_125m_gtp_p/n_i" which is connected to the
main DAC/VCXO controlled 125 MHz. Internal "clk_125m_gtx_buf" is
forwarded to the PLL that generates
"clk_62m5_sys_o".
Q: What is the purpose of the different clocks of the platform wrapper in WR PTP Core reference designs?
A: The WR PTP Core (WRPC) module requires just the system clock (clk_sys_i), helper dmtd clock (clk_dmtd_i) and the reference clock (clk_ref_i):
- clk_ref_i: this is the main clock driving all the timing-related modules. Depending on the FPGA family (and the type of transceiver) used it is either a 62.5MHz or 125MHz clock.
- clk_sys_i: this one is always a “slow” clock (62.5MHz in our current designs) for all the non-time critical modules. E.g. the LM32 soft-core processor, which is inside of WRPC, cannot be synthesized with a correct timing closure above ~100MHz.
- clk_dmtd_i: the helper/offset DMTD clock used for phase measurements
Now, depending on the board that you use, and the oscillators that are available on that board, internal FPGA PLLs are instantiated in the board wrappers (board/* in wr-cores repository) to generate the aforementioned clocks. Therefore:
- clk_20m_vcxo_i: is the input clock of 20MHz oscillator that you can find e.g. on our SPEC board which is then multiplied inside the FPGA to 62.5MHz and provided to the WRPC as clk_dmtd_i.
- clk_125m_pllref_p/n: is the input clock of 125MHz oscillator that is fed to an internal FPGA PLL to generate clk_sys_i and clk_ref_i for the WR PTP Core
Finally clk_125m_gtp_p/n is another copy of 125MHz clock arriving to different FPGA pins that has to be fed to Xilinx transceiver.
Regarding the output clocks (clk_sys_62m5_o, clk_ref_62m5_o, clk_ref_125m_o) these are just the WRPC system and reference clocks provided to clock your own logic to ensure you operate in the same (WR-synchronized) clock domain and therefore do not need to put any synchronizers to interface your logic with the WRPC. The system clock is always 62.5MHz while the reference clock is either 62.5MHz or 125MHz (depending on the FPGA family and transceiver used).
Q: Any other helpful documentation on how to implement the WRPC?
A: Yes, users may have written down their own experience, which we try to collect here:
- Examples of “Instantiating WRPC in your own HDL design", W. Hennig, September 2019, whennig@xia.com (from wr-dev forum)
2 June 2021