Skip to content

W
White Rabbit Network Interface Card
Commit 6541e534 authored by Miguel Jimenez Lopez's avatar Miguel Jimenez Lopez
Browse files
Options

doc: Project document improved. 

	- wr-nic.md improved
	- Move the dia diagram file to the imgsrc folder
	- Unnecessary files deleted
	- dio_core_pps image improved
parent 71019926
Hide whitespace changes
Inline Side-by-side
Showing with 1158 additions and 473 deletions
doc/img/diocore_pps.png
View replaced file @ 71019926 View file @ 6541e534
doc/img/diocore_pps.png

118 KB | W: | H:

doc/img/diocore_pps.png

137 KB | W: | H:

doc/img/diocore_pps.png
doc/img/diocore_pps.png
doc/img/diocore_pps.png
doc/img/diocore_pps.png
  • 2-up
  • Swipe
  • Onion skin
doc/imgsrc/dio_core_pps.svg
View file @ 6541e534
This source diff could not be displayed because it is too large. You can view the blob instead.
doc/imgsrc/dio_core_pps2.svg deleted 100644 → 0
View file @ 71019926
This source diff could not be displayed because it is too large. You can view the blob instead.
doc/wr-nic.md
View file @ 6541e534
% White-Rabbit NIC Gateware
% Javier Díaz & Miguel Jiménez Univ. of Granada, Rafael Rodriguez & Benoit Rat Seven Solutions
% 27 Jan. 2014
% 14 Feb. 2014
What is new in release v2.0?
What is new in wr-nic-v2.0?
=========================
In this section, we detail what is new in `release-v2.0` comparing with older versions:
1. Etherbone core has been added in WR NIC project design to allow remote configuration over the network using TCP/UDP packets.
2. First DIO channel output is reserved for 1-PPS signal of WRPC. It has been done because we detected an offset between 1-PPS input signal and 1-PPS output signal for Grandmaster configuration.
3. Calibration process was performed to fullfil clock delay restriction (less than 1 ns).
4. DIO register structure has been changed. A number of bits has been increased in order to define new modes for DIO channels.
In this section, we detail what is new in `wr-nic-v2.0` comparing with older versions:
1. The Etherbone core has been added in the WR NIC project design to allow remote configuration over the network using TCP/UDP packets.
2. The first DIO channel output is reserved for the 1-PPS signal of WRPC. It has been done because we detected an offset between 1-PPS input signal and 1-PPS output signal for the Grandmaster configuration.
3. The calibration process was performed to fullfil the clock delay restriction of White Rabbit technology (less than 1 ns).
4. The DIO register structure has been changed. A number of bits has been increased in order to define new modes for DIO channels.
5. The WRPC support has been changed to incorporate the following features (see [wrpc-v2.1-changes]):
- WRPC-SW:
- The WRPC packet filter has been configured with new filter rules in order to allow the Etherbone packets reach the Etherbone core and to avoid discarding the incomming packets. A new Kconfig parameter has been defined (`NIC_PFILTER`) not to interfere with the other packet filter configurations.
- A new WRPC command has been added (`refresh`). This command allows to set the update period of gui and stat commands.
- The Grand-Master lock mechanism has been implemented.
- The t24p calibration is executed every time the WRPC is set to Slave mode.
- A new commands have been added to set time manually.
- The MAC address is set by default if it can not get a value from 1-wire device.
- PPSi is default configuration instead of ptp-noposix now. It adds compatibility with more PTP devices.
- WRPC:
- A new generic parameter has been added to the WRPC interface. the `g_vuart_fifo_size` allows to set the UART fifo size of WRPC.
- A separate 1-PPS output has been added for driving LED.
- The reset module has been improved.
- A new interface has been added for new t24p procedure.
- The SoftPLL mode has been updated (multichannel and selectable phase detector mode).
The complete list of new WRPC features can be found in [wrpc-v2.1-changes].
Introduction
=========================
The White-Rabbit Network Interface Card (WR-NIC) project is concerned with the development of gateware to make the combination of a SPEC card and a DIO mezzanine behave as a Network Interface Card (NIC) under Linux.A basic demo is available at the Starting-kit project example that uses two SPEC boards, one configured as Grandmaster and one as slave. Different simple use cases are provided as basic examples and are available at [starting-kit].
This document focuses on the description of the project gateware. This manual is part of the associated hardware project, hosted at [wr-nic-repo], which git repository hosts the latest version. You can find more information about SPEC card at [spec]/[spec-sw] and about FMC DIO card in [fmc-dio-5ch].
The White-Rabbit Network Interface Card (WR-NIC) project is concerned with the development of gateware to make the combination of a SPEC card and a DIO mezzanine behave as a Network Interface Card (NIC) under Linux. A basic demo is available at the Starting-kit project example that uses two SPEC boards, one configured as Grandmaster and one as slave. Different simple use cases are provided as basic examples and are available at [starting-kit].
This document focuses on the description of the project gateware. This manual is part of the associated hardware project, hosted at [wr-nic-repository], which git repository hosts the latest version. You can find more information about SPEC card at [spec]/[spec-sw] and about FMC DIO card in [fmc-dio-5ch].
![Basic wr-nic project elements](./img/wrnic_components.png)
\begin{figure}[H]
\centering
\Oldincludegraphics[width=16cm]{img/wrnic_components.png}
\caption{Basic wr-nic project elements}
\end{figure}
Note that the WR-NIC project inherits many codes and working methodology of many other projects. Specially important to highly the following ones:
1. White-Rabbit core collection (d85255b): [wr-cores]. Look at the WR PTP Core. It is a black-box standalone WR-PTP protocol core, incorporating a CPU (LatticeMicro 32, LM32), WR MAC and PLLs. It is also convenient to look at TxTSU and NIC project. For further details, search for its related wbgen2 files (extension .wb).
2. Software for White-Rabbit PTP core (9f3bcfa): [wrpc-sw] (a subproject of the previous one)
3. Gennum GN4124 core (3844fe5): [GN4124]. This project shows how to configure and control the gn4124 chip in order to comunicate the SPEC carrier through PCIe interface.
4. The platform independent core collection (76f2e5d): [general-cores]. This project contains several useful IP cores such as memories, synchronizer circuits, uart modules, Wishbone crossbar, etc.
5. Etherbone core (08b55da): [etherbone]. This gateware module allows to access Wishbone peripheral memory map using the Ethernet interface. This make possible to read device data as well as read/write configuration registers across the network.
1. The White-Rabbit core collection (commit d85255b): [wr-cores]. Look at the WR PTP Core. It is a black-box standalone WR-PTP protocol core, incorporating a CPU (LatticeMicro 32, LM32), WR MAC and PLLs. It is also convenient to look at TxTSU and NIC project. For further details, search for its related wbgen2 files (extension .wb).
2. Software for White-Rabbit PTP core (commit 9f3bcfa): [wrpc-sw] (a subproject of the previous one)
3. The Gennum GN4124 core (commit 3844fe5): [GN4124]. This project shows how to configure and control the gn4124 chip in order to comunicate the SPEC carrier through PCIe interface.
4. The platform independent core collection (commit 76f2e5d): [general-cores]. This project contains several useful IP cores such as memories, synchronizer circuits, uart modules, Wishbone crossbar, etc.
5. The Etherbone core (commit 08b55da): [etherbone]. This gateware module allows to access Wishbone peripheral memory map using the Ethernet interface. This make possible to read device data as well as read/write configuration registers across the network.
In addition to these projects, software support is provided from the project:
......@@ -37,7 +58,11 @@ Gateware elements
=====================
The main block modules of the architecture are described on next figure.
![Main HDL blocks of the wr-nic gateware](./img/wr_nic_arch.png)
\begin{figure}[H]
\centering
\Oldincludegraphics[width=18cm]{img/wr_nic_arch.png}
\caption{Main HDL blocks of the wr-nic gateware}
\end{figure}
Here is a quick description of each block:
......@@ -47,9 +72,9 @@ Here is a quick description of each block:
* The `GN4124 core` is a bridge between the GN4124 PCIe interface chip and the internal Wishbone bus, allowing communication with the host and interrupts (pipelined). It is a Wishbone master and therefore able to write system registers.
* The `WRPC (White Rabbit PTP Core)` communicates with the outside world through the SFP socket in the SPEC, typically using fiber optics. It deals with the WR PTP using an internal LM32 CPU running a portable PTP stack. It forwards/receives non-PTP frames to/from the NIC block, using two pipelined Wishbone interfaces (master and slave for forwarding and receiving respectively). It also provides time information to other cores (not represented in the diagram), and time-tags for transmitted and received frames that can be read through Wishbone for diagnostics purposes. The WR-NIC project uses the latest version including the PPSi library for PTPv2 protocol support.
* The `NIC core` ensures communication between the host and the WRPC. More precisely, it interrupts the host and provides a descriptor that the host can use to fetch incoming frames. For outgoing frames, it receives a descriptor from the host, fetches the frame using PCIe via the GN4124 core and sends it to the WRPC using a pipelined Wishbone interface.
* The `TxTSU module` collect timestamps with associated Ethernet frame identifiers and puts them in a shared FIFO (port identifier is also included although not required for the SPEC card because only one Ethernet port is available but it is included to provide a common descriptor with the switch data). A IRQ is triggered when FIFO is not empty so drivers could read TX timestamps and frame/port identifiers.
* The `Etherbone core` is an IP core developped by GSI. It allows remote configuration of device memory map with UDP/TCP network packets.
* The `MUX module` is a multiplexer used to route packets from Endpoint to its destination core. Latter is different depending on the type of packet. Each packet is classified by one configurable packet processor located into WRPC. This core has a filter rule table and allows to define up to 8 different packet classes. For wr-nic project, three classes have been defined: Class 7 is for NIC core, Class 5 for Etherbone one and Class 0 for LM32.
* The `TxTSU module` collects timestamps with associated Ethernet frame identifiers and puts them in a shared FIFO (port identifier is also included although not required for the SPEC card because only one Ethernet port is available but it is included to provide a common descriptor with the switch data). A IRQ is triggered when FIFO is not empty so drivers could read TX timestamps and frame/port identifiers.
* The `Etherbone core` is an IP core developped by GSI that allows remote configuration of device memory map with UDP/TCP network packets.
* The `MUX module` is a multiplexer used to route packets from Endpoint to its destination core. Latter is different depending on the packet type. Each packet is classified by one configurable packet processor located in WRPC. This core has a filter rule table and allows to define up to 8 different packet classes. For WR-NIC project, three classes have been defined: Class 7 is for NIC core, Class 5 for Etherbone one and Class 0 for LM32.
In the next sections we provide a little more information about `DIO core` and the `WRPC (White Rabbit PTP Core)` in order to understand better how the complete system works.
Finally, it is important to know that current HDL code contains commented code to activate on-chip logic analyzer circuitry for debugging based on Chipscope of Xilinx. Top file as well as different peripherals include the signals TRIG0 - TRIG3 to help on this purpose. Nevertheless, by default they are commented to avoid wasting unnecessary resources (in fact it could be required to reduce blockram utilization, for instance of NIC or wr\_core module in order to use Chipscope on the project, otherwise design could be overmapped).
......@@ -58,41 +83,41 @@ WRPC (White Rabbit PTP Core)
----------------------------
The `WRPC (White Rabbit PTP Core)` block is the HDL block that makes possible the White-Rabbit timing functionalities on the White-Rabbit nodes. It is a black-box standalone WR-PTP protocol core, incorporating a CPU LM32, WR MAC and PLLs. It could be configure to work on 3 different operations modes:
* `Grandmaster`: WRPC locks to an external 10 MHz and PPS signal provided for instance from a Cesium clock / GPS. This signals are provided from the channel 4 (PPS) and channel 5 (10 MHz) lemo connectors of the DIO card.
* `Grandmaster`: The WRPC locks to an external 10 MHz and PPS signal provided for instance from a Cesium clock / GPS. This signals are provided from the channel 4 (PPS) and channel 5 (10 MHz) lemo connectors of the DIO card.
* `Master`: the systems uses the VCO oscillators of the SPEC board basically on a free running modes.
* `Master`: The systems uses the VCO oscillators of the SPEC board basically on a free running modes.
* `Slave`: The clock information is recovered from the Ethernet connections and the local clock uses the DDMTD method to follow the external clock.
In this project, WRPC provides the timing information used for accurate output generation and input time stamping of the DIO signals. Note that this data is provided with an accuracy of 16 ns.
In this project, the WRPC provides the timing information used for accurate output generation and input time stamping of the DIO signals. Note that this data is provided with an accuracy of 16 ns.
Please note that the current gateware contains the LM32 binary firmware (as part of the FPGA). Therefore no additional LM32 upload programming is required if default code is appropiate for your project. The embedded binary corresponds to the `wrpc-v2.1-for-wrnic` release available at: [wrpc-sw-repo].
Please note that the current gateware contains the LM32 binary firmware (as part of the FPGA). Therefore no additional LM32 upload programming is required if default code is appropiate for your project. The embedded binary corresponds to the `wrpc-v2.1-for-wrnic` release available at: [wrpc-sw-repository].
It is important to remark that for this release the I2C bus of the FMC-DIO card is connected to WRPC. This is needed because current implementation of WRPC store configuration data on the FMC-DIO card EEPROM. Please be aware that for future releases this could change.
The whole description of the core goes beyond the scope of this documentation but the additional information is available at: [wrpc] and in:
> G. Daniluk, White Rabbit PTP Core the sub-nanosecond time synchronization over Ethernet, 2012.
M.Sc thesis describing the development and implementation of the first standalone HDL module handling the sub-nanosecond synchronization over a regular Ethernet - the White Rabbit PTP Core.
Available at: `http://www.ohwr.org/attachments/1368/GD_mgr.pdf`
The whole description of the core goes beyond the scope of this documentation but the additional information is available at: [wrpc] and in: [GD-mgr].
DIO core
----------
The `DIO Core` block is the HDL block that access to the fmc-dio-5chnls mezzanine card. The core goes beyond a standard GPIO core to include advanced functionalities related with timing. Their main elements are shown on the next figure.
The `DIO Core` block is the HDL block that access to the [fmc-dio-5ch] mezzanine card. The core goes beyond a standard GPIO core to include advanced functionalities related with timing. Their main elements are shown on the next figure.
![Main DIO Core block elements](./img/diocore2.png)
\begin{figure}[H]
\centering
\Oldincludegraphics[width=18cm]{img/diocore2.png}
\caption{Main DIO Core block elements}
\end{figure}
> ***Notes***: Channel 0 output is reserved to work as 1-PPS signal from WRPC. It is necessary to reduce gateware delay beetwen input PPS signal and output one in Grandmaster configuration. So, `P mode` has been removed from configuration options.
> ***Notes***: In the previous release we define the DIO channel `P mode` as a DIO channel mode that allows to generate a 1-PPS WRPC signal output. This was possible for all DIO channels. In the current design, channel 0 output is reserved to work on this mode and no other channels can generate this output. For this reason, the `P mode` has been removed and it is no longer used.
The different submodules description are:
The different submodule descriptions are:
* `GPIO:` GPIO (General Purpose Input/Output) core allows you to use a number of pins as digital input/output signal. It allows you to configure a pin as input/output with/without tri-state buffer and write/read to/from it.
* `GPIO:` GPIO (General Purpose Input/Output) core allows to use a number of pins as digital input/output signal. A GPIO pin can be configured as input/output with/without tri-state buffer and write/read to/from it.
* `I2C:` It allows to set the threshold of the ADCMP604 fast LVDS comparator and to access to write/read data to the EEPROM memory (24AA64). Please note that already commented, current release do not allow to access I2C signals of the FMC-DIO card from this HDL block. This is assigned to WRPC and therefore it should be handled inside its memory maps (concretely, inside WRPC `Syscon` device). This could change for future releases.
* `Onewire:` It is used for temperature acquisition.
* Modules to generate or stamping pulses:
* Modules to generate/stamp pulses:
1. `Pulse generator:` It produces a programmable ticks-long pulse in its output when the time passed to it through a vector equals a pre-programmed time.
2. `Pulse stamper:` It associates a time-tag with an asynchronous input pulse.
......@@ -105,16 +130,23 @@ The different submodules description are:
4. Monostable register, which generates a single clock cycle-long.
5. Update the mode of the drivers: the termination resistors and if it is connected to GPIO, DIO core or WRPC.
As we told in earlier section, we changed connections for channel 0 of DIO core. For its output, we connected 1-PPS output signal from WRPC. In addition, you can configure this channel in order to use it as
user input channel but output is reserved to PPS signal.
As we told in earlier section, we changed connections for the channel 0 of the DIO core. For its output, we connected the 1-PPS output signal from the WRPC. In addition, you can configure this channel in order to use it as user input channel but its output is reserved to PPS signal.
![PPS signal output for channel 0 of DIO](./img/diocore_pps2.png)
\begin{figure}[H]
\centering
\Oldincludegraphics[width=18cm]{img/diocore_pps.png}
\caption{PPS signal output for channel 0 of DIO}
\end{figure}
DIO Configuration & Control
==========================
Accessing to the different system elements is as simple as doing a read/write memory access. The memory map of the different elements of the board are shown in the next figure.
![Memory map](./img/memorymap.png)
\begin{figure}[H]
\centering
\Oldincludegraphics[width=18cm]{img/memorymap.png}
\caption{Memory map}
\end{figure}
Any address within this memory space may be addressed by the PC to configure corresponding module (WRPTP, NIC, TxTSU, etc...). Many of them are already well described in related project so we skip their descriptions to focus on new additions. In next paragraphs we just focus on the DIO core which is the new block added to the OHWR and required for this project.
......@@ -132,24 +164,24 @@ In order to use input/output channels as previously described, the following act
+ [3]\: 50 Ohm termination enable.
1. 0x0 (PPS Output without termination)
#. 0x0 (GPIO out without termination)
#. 0x0 (...)
#. 0x4 (Input without termination)
#. 0x6: Clock[^clkinput] input without termination.
2. 0x0 (GPIO out without termination)
3. 0x0 (...)
4. 0x4 (Input without termination)
5. 0x6: Clock[^clkinput] input without termination.
[^clkinput]: The Clock input is similar the DIO input excepting that no interrupt will be generated to the PCIe core. It is used in Grandmaster mode.
[^clkinput]: The Clock input is similar the DIO input excepting that no interrupt will be generated to the PCIe core. It is used in Grandmaster mode as 10 MHz input channel.
* Time-programmable pulse generation: Generate a programmed output at any time at channel X (X between 1 and 4 identifies the requested channel, channel 0 is reserved for PPS output). For this purpose you need to perform the following actions:
* **Time-programmable pulse generation:** Generate a programmed output at any time at channel X (X between 1 and 4 identifies the requested channel, the channel 0 is reserved for PPS output). For this purpose you need to perform the following actions:
* Set the required time. This means to provide the 40 bits for the time value and the number of cycles (28 bits). This requires to write the registers `dio_trigX_seconds, dio_trighX_seconds` (high part of the time value) and `dio_cyc0_cyc`.
* Checking if the board is ready for accepting new triggers. This can be done by reading a `1` found at each bit of `dio_trig_rdy` register. The EIC bits 9 to 5 have associated interrupts (active means system is ready to accept new trigger values but please check you have properly configured EIC interrupt mask before). Both methods are possible to check the status. Nevertheless note that the non-ready periods are very shorts (13 cycles of a 62.5 MHz clock, 208 ns) so systems is almost always ready for new trigger values.
* Arming the trigger. You need to write a `1` at the corresponding bit of the `dio_latch_time_chX` bit field.
After these operations, an output with the programmed tick-length will be presented on the desired channel at the requested time. It is not necessary to do software reset to the register.
* Immediate pulse generation: An immediate pulse is generated at the output of each of the card channels just by writing a corresponding `1` at the bit field dio\_pulse\_imm\_X when output mode is set to programmable outputs. No reset is required.
* **Immediate pulse generation:** An immediate pulse is generated at the output of each of the card channels just by writing a corresponding `1` at the bit field dio\_pulse\_imm\_X when output mode is set to programmable outputs. No reset is required.
* Variable pulse length: both output modes, time-programmable as well as immediate allows to configure the output length. By writing the value of the registers dio\_progX\_pulse\_length the width of the output pulse could be controlled. The register uses the 28 low significant bits and allow a time duration equal to register\_value x 16 ns.
* **Variable pulse length:** both output modes, time-programmable as well as immediate allows to configure the output length. By writing the value of the registers dio\_progX\_pulse\_length the width of the output pulse could be controlled. The register uses the 28 low significant bits and allow a time duration equal to register\_value x 16 ns.
* Input time-tagging: for each of the 5 inputs, if a `1` is detected at this channel, a precise time information is stored on logic FIFOs including the 40 bits time counters and 28 bits more for the cycles (fifo depth is 256 each one). Currently this information is collected even for pins configured in output mode, GPIO, immediate or time-programmable configurations but it is straightforward to change this at the HDL code. For accessing this information you need to read `dio_tsfX_tag_seconds` (32 low bits), `dio_tsfX_tag_secondsh` (high bits), `dio_tsfX_tag_cycles`. Each time the time tag of any channel is stored, the `fifo not empty` flag generates an interruption to the PC. In the next section we describe these mechanisms.
* **Input time-tagging:** for each of the 5 inputs, if a `1` is detected at this channel, a precise time information is stored on logic FIFOs including the 40 bits time counters and 28 bits more for the cycles (fifo depth is 256 each one). Currently this information is collected even for pins configured in output mode, GPIO, immediate or time-programmable configurations but it is straightforward to change this at the HDL code. For accessing this information you need to read `dio_tsfX_tag_seconds` (32 low bits), `dio_tsfX_tag_secondsh` (high bits), `dio_tsfX_tag_cycles`. Each time the time tag of any channel is stored, the `fifo not empty` flag generates an interruption to the PC. In the next section we describe these mechanisms.
A detailed information about the memory maps and related registers names are available by generating html documentation of the different wishbone slaves. Download the related .wb file and generate the HTML documentation using wbgen2 tool (for instance wbgen2 -D diocore.htm wr\_nic.wb).
......@@ -176,7 +208,7 @@ For instance for the DIO core, please check the status of interrupt registers by
How to synthetize it?
==================
Featching source files from the repositories, synthetize and download into the FPGA are the target actions. Additionally, in the WR-NIC project you will find a `wrc.ram` file which corresponds to the *LM32* firmware that we are going to embedded into our *HDL* gateware. It is not needed to recompile it, however if you want to modify the lm32 behaviour you can look at the [wrpc-sw-lm32-firmware](#wrpc-sw-lm32-firmware) section to check how to build it.
Fetching source files from the repositories, synthetize and download into the FPGA are the target actions. Additionally, in the WR-NIC project you will find a `wrc.ram` file which corresponds to the *LM32* firmware that we are going to embedded into our *HDL* gateware. It is not needed to recompile it, however if you want to modify the lm32 behaviour you can look at the [wrpc-sw-lm32-firmware](#wrpc-sw-lm32-firmware) section to check how to build it.
HDLMake
-------------
......@@ -184,7 +216,7 @@ HDLMake
In order to synthetize the HDL for wr-nic you need to have installed the `hdlmake` tool.
For more information on how to install and use it, you can look at the [hdlmake]
To generate the wr-nic bitstream we have used latest commit in the isyp branch.
To generate the wr-nic bitstream we have used latest commit in the `isyp` branch.
~~~~~~{.bash}
cd <path-hdl-make>
......@@ -197,7 +229,7 @@ git checkout 6b7aa78
WR-NIC (HDL-gateware)
----------------------
This step show you how to prepare the WR-NIC bitstream (SPEC+FMC DIO) with the wrpc-sw (`wrc.ram` file) embeded inside.
This step shows how to prepare the WR-NIC bitstream (SPEC+FMC DIO) with the wrpc-sw (`wrc.ram` file) embeded inside.
~~~~~~{.bash}
## Checkout the code
......@@ -223,9 +255,9 @@ make
WRPC-SW (LM32 firmware)
-----------------------
You can download it from file section of [wr-nic] or you can try to compile it following the instructions below:
This step is not necessary because a precompiled `wrc.ram` is already provided (in file section of [wr-nic]). However, if you want to recompile it, you should follow instructions below:
You should first installed the **lm32** compiler as suggested in wrpc-v2.0.pdf documentation located at attachment section in [wrpc], then you can compile executing the following commands:
You should first installed the **lm32** compiler as suggested in `wrpc-v2.0.pdf` documentation located at attachment section in [wrpc], then you can compile executing the following commands:
~~~~~~{.bash}
#Set up CROSS_COMPILE variable for this terminal
......@@ -247,7 +279,7 @@ $ make wrnic_defconfig
$ make
~~~~~~~~~~
You should obtain several files named wrc.bin, wrc.elf, wrc.vhd, wrc.ram
You should obtain several files named wrc.bin, wrc.elf, wrc.vhd, wrc.ram.
You can therefore try to override it and go back to section [wr-nic-hdl-gateware](#wr-nic-hdl-gateware).
......@@ -257,7 +289,7 @@ cp wrc.ram <wr_root_folder>/wr-nic/syn/spec
~~~~~~~~~~~
> ***Notes***: These steps are a simple resume on how to compile the
> ***Notes***: These steps are a simple summary on how to compile the
firware specifically for the wr-nic, you should also look at the wrpc-v2.0.pdf documentation located at attachments section in [wrpc] to understand how to use it and how to compile for other configurations.
......@@ -277,16 +309,16 @@ This project could be used as starting demo with White-Rabbit technology, illust
Both elements are described in the software manual of the WR-NIC project (Starting kit) and it is out of the scope of current document to describe them with further details [starting-kit]. Please read that document in order to have a complete understanding of the NIC project.
New features: remote configuration based on Etherbone module
New features: Remote configuration based on Etherbone module
=================================
The Etherbone core is an IP core developed by GSI that allows to read/write in device memory map remotely. To archive this goal, the Etherbone project provides a software network library to send TCP/UDP configuration packets. The Etherbone core receives these packets and performs needed configuration actions. If you want to get more information of the Etherbone project, visit [etherbone].
WR NIC project has been designed to work in a PCI co-processor configured as a standalone device. If you use it as a co-processor, there is no problem in configuring it because you can use the kernel drivers developped (see in spec-sw). However, if you want to use it in standalone mode, it can not be accessed directly by PCI bridge since the spec-sw kernel drivers do not solve this problem. You need an access based on network packets.The Etherbone project provides this mechanism. That is why the Etherbone core has been added in the new release of the WR NIC project.
WR NIC project has been designed to work in a PCI co-processor configuration and as a standalone device. If you use it as a co-processor, there is no problem in configuring it because you can use the kernel drivers developped (see in [spec-sw]). However, if you want to use it in standalone mode, it can not be accessed directly by PCI bridge since the spec-sw kernel drivers do not solve this problem. You need an access based on network packets. The Etherbone project provides this mechanism. That is why the Etherbone core has been added in the new release of the WR NIC project.
Two things are needed in order to the packets reach the Etherbone core:
Two things are needed in order to the incomming packets reach the Etherbone core:
* WRPC has an filter packet processor (IP core) to decide which packets are accepted and which are discarded. Packets are classified in one class choosen between eight different classes. This class stores information about to where a packet must be delivered. So, we have to modify filter rules (see dev/ep\_pfilter.c in wrpc-sw-2.1 release) in order to accept Etherbone traffic. We have assigned class 5 to distinguish it from another type of traffic. Moreover, DROP instruction has been removed in order to avoid any packet to be discarded (because a conventional NIC receives all type of packages). We have used a WRNIC macro not to interfere with configurations needed for other projects.
* WRPC has an filter packet processor (IP core) to decide which packets are accepted and which are discarded. Packets are classified in one class choosen between eight different classes. This class stores information about to where a packet must be delivered. So, we have to modify filter rules (see dev/ep\_pfilter.c in [wrpc-sw-repository]) in order to accept Etherbone traffic. We have assigned class 5 to distinguish it from another type of traffic. Moreover, DROP instruction has been removed in order to avoid any packet to be discarded (because a conventional NIC receives all type of packages). We have used a NIC\_PFILTER macro not to interfere with configurations needed for other projects.
~~~~~{.c}
#define R_CLASS(x) (24 + x)
......@@ -351,17 +383,17 @@ pfilter_load();
}
~~~~~~~~~~~
* A route mechanism to reach Etherbone core must be added. It is implemented with a MUX IP core component that can re-direct one packet depending on its class. We have placed one MUX between WRPC (fabric interface) and NIC. So, packets from WRPC fabric interface can reach the NIC or the Etherbone core.
* A route mechanism to reach Etherbone core must be added. It is implemented with a MUX IP core component that can route one packet depending on its class. We have placed one MUX between WRPC (fabric interface) and NIC. So, packets from WRPC fabric interface can reach the NIC or the Etherbone core.
Thanks to the Etherbone core and its software library, we can access the device using network packets.
One interesting application is to allow access in the same way we connect to it via the USB serial port. University of Granada has developped a high-level library based on Etherbone that allows you to define devices configuration files with this associated operations. This files can be loaded with CALoE methods and user can evoke device operations as C functions (using CALoE API of course). This library is a beta version and you can find it in starting kit repository (ugr branch).
One interesting application is to allow access in the same way we connect to it via the USB serial port. University of Granada has developped a high-level library based on Etherbone that allows you to define devices configuration files with this associated operations. This files can be loaded with CALoE methods and user can evoke device operations as C functions (using CALoE API of course). This library is a alpha version and you can find it in [starting-kit] repository (ugr branch).
WR NIC calibration
==================
The WR NIC calibration process is necessary to fullfil clock delay restriction of White Rabbit technology. It tries to find temporal parameters in order to compensate transmission/reception delays of gateware and asymmetry of fiber link.
the calibration process has two steps. The first of them consists in pre-calibrating a device to use it as reference for all other devices in your White Rabbit network. The pre-calibration process requires two identical WR devices (board, gateware and software) to make the assumption that the time delays are equal and thus simplifying the procedure.
The calibration process has two steps. The first of them consists in pre-calibrating a device to use it as reference for all other devices in your White Rabbit network. The pre-calibration process requires two identical WR devices (board, gateware and software) to make the assumption that the time delays are equal and thus simplifying the procedure.
Second step is the calibration of all other devices in your network. To do it, you have to use the pre-calibrator of previous step. The WR calibration process is described in detail in [wr-calibration].
......@@ -372,6 +404,7 @@ We have needed the following equipment in order to calibrate:
+ A WR Switch (18 ports, HW: 3.3, GW: 3.3). You can find the WR Switch v3.3 calibration parameters used by us in [wr-calibration].
+ Two LEMO links (1 meter, they should be same length or you must be know their time delays to compensate the error they introduce).
+ A LEMO-BNC adapter.
+ A SMC-BNC wire.
+ Two optical fiber links. One of then with 2 meters and another with 5 kilometers.
+ Two SFPs, red (AXGE-3454-0531) and blue (AXGE-1254-0531).
+ A LC-LC adapter.
......@@ -429,11 +462,14 @@ Further information will be provided in future releases.
Future work
===============
* Nowadays, WR NIC maximum bandwidth is about 11 Mbps because DMA module is not included in design. As future work, we will develop a DMA module in order to increase wr-nic performance.
* Nowadays, the WR NIC maximum bandwidth is about 11 Mbps because DMA module is not included in the design. As future work, we will develop a DMA module in order to increase the wr-nic performance.
References
============
* G. Daniluk, White Rabbit PTP Core the sub-nanosecond time synchronization over Ethernet, 2012.
M.Sc thesis describing the development and implementation of the first standalone HDL module handling the sub-nanosecond synchronization over a regular Ethernet - the White Rabbit PTP Core.
Available at: [GD-mgr]
* Open Hardware Repository website at [ohwr]
* White Rabbit NIC project at [wr-nic]
* Starting kit project at [starting-kit]
......@@ -448,15 +484,16 @@ References
* FMC DIO card of 5 channels at [fmc-dio-5ch]
* White Rabbit procedure calibration at [wr-calibration]
* HDLmake tool project at [hdlmake]
* WRPC release 2.1 changes at [wrpc-v2.1-changes]
[ohwr]: http://www.ohwr.org/
[wr-nic]: http://www.ohwr.org/projects/wr-nic/
[wr-nic-repo]: http://www.ohwr.org/projects/wr-nic/repository
[wr-nic-repository]: http://www.ohwr.org/projects/wr-nic/repository
[starting-kit]: http://www.ohwr.org/projects/wr-starting-kit
[wr-cores]: http://www.ohwr.org/projects/wr-cores
[wrpc]: http://www.ohwr.org/projects/wr-cores/wiki/Wrpc_core
[wrpc-sw]: http://www.ohwr.org/projects/wrpc-sw
[wrpc-sw-repo]: http://www.ohwr.org/projects/wrpc-sw/repository
[wrpc-sw-repository]: http://www.ohwr.org/projects/wrpc-sw/repository
[GN4124]: http://www.ohwr.org/projects/gn4124-core
[general-cores]: http://www.ohwr.org/projects/general-cores
[etherbone]: http://www.ohwr.org/projects/etherbone-core
......@@ -466,3 +503,5 @@ References
[fmc-dio-5ch]: http://www.ohwr.org/projects/fmc-dio-5chttla
[wr-calibration]: http://www.ohwr.org/projects/white-rabbit/wiki/Calibration
[hdlmake]: http://www.ohwr.org/projects/hdl-make
[GD-mgr]: http://www.ohwr.org/attachments/1368
[wrpc-v2.1-changes]: http://www.ohwr.org/projects/wr-cores/wiki/Wrpc_release-v21
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment