This tutorial has been tested and verified with Ubuntu LTS 16.04.6 (kernel v.4.15.9) and Ubuntu LTS 18.04.2 (kernel v.4.18.0-17), this mean that **standard support will only be given for these releases**.
However, the `spec-sw` driver should work with other releases, distributions and architectures.
> ***Notes:*** Ubuntu 18.04 LTS is supported but some versions (i.e, 18.04.3 with kernel v5.0) does not work correctly unless you [force to used a previous kernel version](#Set-Previous-Kernel-Version).
> ***Warning:*** Automatic update of Ubuntu LTS might also upgrade the kernel version to a newer one and thus break the compatibility of the WR-Starting-Kit drivers. You should [fix to the current working kernel version](#Set-Previous-Kernel-Version) or disable automatic update in order to avoid this problem.
> ***Notes:*** Ubuntu 18.04 LTS is supported but some versions (i.e, 18.04.3 with kernel v5.0) does not work correctly unless you [force to used a previous kernel version](#set-specific-kernel-version).
> ***Warning:*** Automatic update of Ubuntu LTS might also upgrade the kernel version to a newer one and thus break the compatibility of the WR-Starting-Kit drivers. You should [fix to the current working kernel version](#set-specific-kernel-version) or disable automatic update in order to avoid this problem.
> ***Warning:*** Ubuntu LTS 14.04 is not supported.
...
...
@@ -384,7 +383,8 @@ Then you need to install all the tools that you will need:
You can also run this command[^debian] for minimal setup:
> ***Note:*** please note that the `pulse` command activates the [FMCDIO] mode (without changing the termination)
> ***Note:*** Please note that the `pulse` command activates the [FMCDIO] mode
> (without changing the termination)
> ***Warning:*** You must ensure that WR date is synchronized with system one.
> See setsec command in [Generating pulse](#generating-pulse) section.
### Time stamping output
...
...
@@ -981,8 +985,8 @@ ch 4, 534.500000000
ch 4, 542.500000000
~~~~~~~~
> ***Notes:*** If you see other timestampings messages coming from channel 2 this mean you don't
have disabled the PPS mode.
> ***Notes:*** If you see other timestampings messages coming from channel 2
> this mean you have not disabled the PPS mode.
### Time stamping input & output.
...
...
@@ -1022,7 +1026,7 @@ ch 2, 186790.000000000
ch 4, 186792.999999992
>tornado:$ sudo wr-dio-cmd /dev/fmc-dio-1:0 stamp
ch 3, 186792.999999992
ch 3, 186792.000000000
~~~~~~~~~~~~~~~~~
We can observe that the difference between the output and input timestamps is the same for the "local" and "remote" connection (i.e. "local": output=>ch1@spusa, input=>ch2@spusa, "remote": output=>ch4@spusa, input=>ch2@tornado). The resolution of the timestamps is 8 nanoseconds, so if the lenghts (delays) of the cables are below 8ns, we can observe the minimum difference between the timestamps due it its resolution, i.e. 8ns. This is a very simple example to show how the two [SPEC]s are synchronized.
...
...
@@ -1135,13 +1139,13 @@ tools/stamp-frame: Using interface wri1, with all timestamp options active
## On another terminal (maybe on another host) run
>tornado:$ sudo stamp-frame wri1
tools/stamp-frame: Using interface wr1, with all timestamp options active
timestamp T1: 1891.948736656
timestamp T2: 1892.038390176
timestamp T3: 1892.048625152
timestamp T4: 1891.959080320
round trip time: 0.000108688
forward time: 0.089653520
backward time: -0.089544832
timestamp T1: 1576494173.573300256
timestamp T2: 1576494173.573325576
timestamp T3: 1576494173.583573592
timestamp T4: 1576494173.583598920
round trip time: 0.000050648
forward time: 0.000025320
backward time: 0.000025328
~~~~~~~~~~~~
This is done four times: departure and arrival of the forward frame, followed
...
...
@@ -1214,7 +1218,7 @@ The figure below illustrates how the connection must be done between the differe
> ***Note:*** The cables between DIO outputs and the scope shoud be of the same lenght.
> ***Note:*** The forwarding of pulse is fully done in software, thus has been developed as a proof of concept with limited bandwith (~100Hz) and reliability (it is not recommanded to be used in production)
> ***Note:*** The forwarding of pulse is fully done in software, this has been developed as a proof of concept with limited bandwith (~100Hz) and reliability (it is not recommanded to be used in production)
### How does it work?
...
...
@@ -1342,7 +1346,7 @@ Or compare the signals[^3cables] on the oscilloscope as in the next figure:
> ***Notes:*** To stop the experiment (or restart it), you can use the following commands:
~~~~~{.sh}
>spusa:$ sudo kill wr-dio-ruler
>spusa:$ sudo killall wr-dio-ruler
>tornado:$ sudo killall wr-dio-agent
~~~~~~~~~~~~~~~~
...
...
@@ -1394,7 +1398,7 @@ channel 0.
ch 4, 388.001000000
## To stop the agent and ruler
>spusa:$ sudo kill wr-dio-ruler
>spusa:$ sudo killall wr-dio-ruler
>tornado:$ sudo killall wr-dio-agent
~~~~~~~~~~~~
...
...
@@ -1443,21 +1447,27 @@ When you compare the PPS of two [SPEC+FMCDIO] board you will find that they
are almost always synchronized without the need to calibrate the
gateware. However if you compare these PPS with the one of the switch
and you do not obtain a delay similar as the one in the WRS figure above
you might need to calibrate the [WRS] and the [SPEC+FMCDIO].
you might need to calibrate the [WRS] and/or the [SPEC+FMCDIO].
The value that you need to use can be found at the wiki page:
The wr-starting-kit-v3.0 release has been calibrated for you for the SFPs
provided to you. The values of the calibration parameters that you need to use
with the wr-starting-kit-v3.0 can be found in the wiki page:
If you use a custom gateware or SFPs that are not listed in the wiki page
you should run yourself the calibration in order to obtain the correct values
for your setup.
If you are using a release version of the WRS, it has been also calibrated of
the provided SFPs. On the other hand, if you use a custom gateware or SFPs that
are not listed in the wiki page you should run yourself the [wr-calibration]
process in order to obtain the correct values for your setup.
> ***Warning:*** Cable length/type will add around ~5ns/meter to the delay
between the moment when the PPS was out of the board and when it was
captured by the oscilloscope.
> ***Note:*** Please check that you are using the values that correspond
to your gateware release. We reference the gateware corresponding to this version of the document as wr-starting-kit-v3.0. You can check the version of the gateware you are running with the command ver".
to your gateware release and the hardware version of SPEC/FMCDIO. We reference
the gateware corresponding to this version of the document as
wr-starting-kit-v3.0. Specific hardware versions are 4.1 for SPEC and 2.0 for
FMCDIO. You can check the version of the gateware you are running with the command ver".
~~~~~{.sh}
wrc# ver
...
...
@@ -1488,7 +1498,7 @@ AXGE-1254-0531 or AXGE-3454-0531
Do not forget to also add the proper calibration values to your other
WR equipments (i.e, [WRS]) as explained in the wiki page.
You might also check the [wrpc.pdf] for extended explanations on how to run
You might also check the [wr-calibration] manual for extended explanations on how to run
yourself the calibration procedure.
SPEC+DIO as grandmaster
...
...
@@ -1524,11 +1534,12 @@ input 10MHz signal can be square or sinusoidal.
When the GrandMaster is a [SPEC+FMCDIO]`PPS_in` pulse serves to *mark* which rising edge of the 10MHz signal is considered the first one valid for a new second: the `PPS_out` will be fixed on the `10MHz_in` and not the `PPS_in`. `PPS_in` should therefore arrives at least 8ns before the `10MHz_in` clock.
> ***Note:*** In the case when PPS_in and 10MHz_in are perfectly aligned at your source, you can use different lenghts of cables to provide proper skew between the two signals. As 1m is around 5ns of delay, you can use a cable of 1m legnth (5ns) to connect PPS_in and a cable of 3m lenght (15ns) to connect 10MHz.
> ***Note:*** In the case when PPS_in and 10MHz_in are perfectly aligned at your source, you can use different lenghts of cables to provide proper skew between the two signals. As 1m is around 5ns of delay, you can use a cable of 1m length (5ns) to connect PPS_in and a cable of 3m lenght (15ns) to connect 10MHz.
The following figure shows what kind of signals need to be provided to plug to the GM [SPEC+FMCDIO], and with which fixed delay the `PPS_out` is produced (~8ns).
### Locking GM PLL to external 10MHz/PPS
...
...
@@ -1637,14 +1648,14 @@ If you need a more complete guide on how to create your own HDL for the [SPEC],
or you want to improve the structure of a project using the [OHWR] platform we recommend you
to look at the [spec-getting-started.pdf] guide.
The [WR Starting Kit][wr-starting-kit] is based on various project:
The [WR Starting Kit][wr-starting-kit] is based on various projects:
*[spec-sw], The project that contains the PCIe driver and tools to access to the SPEC FPGA
*[spec-golden], A simple gateware in order to access to the EEPROM of the [FMC](WB-I2C)
*[wr-nic], gateware (FPGA HDL) that includes the NIC capabilities and its corresponding kernel drivers
*[fmc-dio-5chttla], gateware (FPGA HDL) that includes the DIO capabilities (with NIC) and its corresponding kernel drivers
*[wrpc-sw], LM32 software in the White Rabbit PTP core for the synchronization
*[coht-vic], Kernel driver to handles interruption from FPGA such as the one from SPEC
*[coht-vic], Kernel driver to handle interruption from FPGA such as the one from SPEC
> ***Note:*** Next sections describe manual procedures to generate software and
> FPGA firmware binary files. All relative path in their code listings are refered
...
...
@@ -1845,11 +1856,11 @@ However, you will find a resume of the most asked questions below:
### I can't generate a PPS signal or any point in section 5 is not working
In order to generate any signals from section 5 you need to endure that the main PLL has been locked. That means your card has to be either in the a valid Free-running Master, Grand Master or Slave Mode.
In order to generate any signals from section 5 you need to ensure that the main PLL has been locked. That means your card has to be either in the a valid Free-running Master, Grand Master or Slave Mode.
* When you have only one card (or some but unconnected) it should be working as Free Running Master, and you should always be locked to the main PLL.
* If you have two of them (or more), one of them has to work as master/grandmaster, the others as Slave and must be locked (TRACK_PHASE) to its WR master as explained en section [4.3](#configure-in-slave-master-mode).
* If you have configured your card in Grand Master Mode you need to be sure taha you provide a valid external reference (10MHz/PPS).
* If you have configured your card in Grand Master Mode you need to be sure that you provide a valid external reference (10MHz/PPS).
If you want to check that the main PLL has been locked you should connect to the UART/VUART and run the following command
...
...
@@ -1938,7 +1949,7 @@ References
*[spec-sw.pdf] main documentation of the spec-sw
*[wrpc.pdf] White Rabbit Core documentation
* SFPs information <http://www.ohwr.org/projects/white-rabbit/wiki/SFP>
*[wr_external_reference.pdf] How to use the grandmaster mode in the switch.
*[spec-getting-started.pdf] A tutorial to get ready to work with the (SPEC), including hardware deployment instructions, full required toolchain setup and a collection of step-by-step demonstrative tutorials.
