Skip to content

S
Simple PCIe FMC carrier SPEC
Commit 9358d3a1 authored by Federico Vaga's avatar Federico Vaga
Browse files
Options

update documentation 

Signed-off-by: Federico Vaga's avatarFederico Vaga <federico.vaga@cern.ch>
parent 646a9e9a
Hide whitespace changes
Inline Side-by-side
Showing with 304 additions and 100 deletions
doc/conf.py
View file @ 9358d3a1
......@@ -20,11 +20,11 @@
# -- Project information -----------------------------------------------------
project = 'SPEC'
copyright = '2019, Federico Vaga <federico.vaga@cern.ch>, Tristan Gingold <tristan.gingold@cern.ch>, Dimitris Lampridis <dimitrios.lampridis@cern.ch>'
copyright = '2019-2020, Federico Vaga <federico.vaga@cern.ch>, Tristan Gingold <tristan.gingold@cern.ch>, Dimitris Lampridis <dimitrios.lampridis@cern.ch>'
author = 'Federico Vaga <federico.vaga@cern.ch>, Tristan Gingold <tristan.gingold@cern.ch>, Dimitris Lampridis <dimitrios.lampridis@cern.ch>'
# The short X.Y version
version = ''
version = '1.4'
# The full version, including alpha/beta/rc tags
release = 'v1.4'
......
doc/hdl-spec-base.rst
View file @ 9358d3a1
..
SPDX-License-Identifier: CC-BY-SA-4.0
SPDX-FileCopyrightText: 2019-2020 CERN
.. _spec_hdl_spec_base:
SPEC Base HDL Component
=======================
The ``SPEC base`` HDL component provides the basic support for the SPEC card
and it strongly recommended for any SPEC based application. The VHDL code for
this component is part of the `SPEC project`_ source code as well as the
necessary Linux drivers.
Interface Rules
---------------
The ``SPEC base`` is an :ref:`FPGA device <device-structure>` that contains
all the necessary logic to use the SPEC carrier's features.
Rule
The ``SPEC base`` design must follow the FPGA design guide lines
Rule
The ``SPEC base`` instance must be present in any SPEC based
design.
Rule
The ``SPEC base`` metadata table must contain the following
constant values
========== ========== ================== ============
Offset Size (bit) Name Default (LE)
0x00000000 32 Vendor ID 0x000010DC
0x00000004 32 Device ID 0x53504543
0x00000008 32 Version <variable>
0x0000000C 32 Byte Order Mark 0xFFFE0000
0x00000010 128 Source ID <variable>
0x00000020 32 Capability Mask <variable>
0x00000030 128 Vendor UUID 0x00000000
========== ========== ================== ============
Observation
The ``SPEC base`` typically is instantiated in a *top level* design
next to an ``Application Device``.
Rule
The ``SPEC base`` must have a 32bit register containing the offset
to the ``Application Device``. If there is no application, then the content
of this register must be ``0x00000000``.
Observation
The ``Application Device`` offset is design specific and it must be
declared in the ``Application Access`` register
Version 1.4
~~~~~~~~~~~
and it is strongly recommended for any SPEC based design, even though it
is not mandatory. This component groups together a set of ip-cores which
are required to drive hardware chips and FPGA ip-cores that are handy to
develop SPEC based designs.
Rule
The ``SPEC base`` metadata table must contain the following
constant values for this version.
The ``SPEC base`` is compliant with the `FPGA device identification`_ rules.
========== ========== ================== ============
Offset Size (bit) Name Default (LE)
0x00000000 32 Vendor ID 0x000010DC
0x00000004 32 Device ID 0x53504543
0x00000008 32 Version 0x0104xxxx
0x0000000C 32 Byte Order Mark 0xFFFE0000
0x00000010 128 Source ID <variable>
0x00000020 32 Capability Mask 0x0000000x
0x00000030 128 Vendor UUID 0x00000000
========== ========== ================== ============
Components
----------
Rule
The ``SPEC base`` is made of the following components
The following table summarizes the ``SPEC base`` components and after that
you have a brief description of each of them. We do not expect to add or
remove components in the future so this should be an exhaustive list.
=================== ============ ========== =============
Component Start End Cap. Mask Bit
CSR 0x00000040 0x0000005F (Mandatory)
Therm. & ID 0x00000070 0x0000007F 1
Gen-Core I2C Ocore 0x00000080 0x0000009F (Mandatory)
Gen-Core SPI 0x000000A0 0x000000BF 2
Gen-Core SPI Ocore 0x000000A0 0x000000BF 2
DMA for DDR 0x000000C0 0x000000FF 5
Gen-Core VIC 0x00000100 0x000001FF 0
Build info 0x00000200 0x000002FF 4
White-Rabbit 0x00001000 0x00001FFF 3
=================== ============ ========== =============
Observation
The capability mask value ``0x3F`` means that all optional components
are instantiated.
.. note::
The *Capability Mask Bit* (Cap. Mask Bit) refers to the bit in the
capability mask described in the `FPGA device identification`_
rules.
CSR
Control and Status register for the ``SPEC base`` device.
Therm. & ID
A onewire interface from `general cores`_ that accesses the SPEC
thermometer to get temperature and serial number.
General Cores I2C OpenCore
An I2C controller from `general cores`_ which bus is wired to the FMC
connector to access the I2C EEPROM on the FMC module.
General Cores SPI OpenCore
An SPI controller from `general cores`_ which bus is wired to the SPI
flash memory on which we store FPGA configurations.
DMA for DDR
A DMA engine from `GN4124 core`_.
General Cores VIC
An interrupt controller from `general cores`_ that routes FPGA
interrupts to PCIe bridge (``GPIO 8`` on the GN4124 chip). The interrupt
lines from 0 to 5 are reserved for internal use as described in the
following table. All other lines are available for users.
============== ===================
Interrupt Line Component
0 Gen-Core I2C Ocore
1 Gen-Core SPI Ocore
2 DMA for DDR - DONE
3 (reserved)
4 (reserved)
5 (reserved)
============== ===================
Build Info
Free format information (ASCII) about the FPGA synthesis.
White-Rabbit
The `White-Rabbit core`_.
.. note::
If the `White-Rabbit core`_ is instantiated then the components
*Therm. & ID* and *General Cores SPI OpenCore* get disabled because
they are incompatible. This because the `White-Rabbit core`_ needs
the OneWire bus and the SPI bus for internal use, therefore those
resources can't be used.
Usage
-----
The ``SPEC base`` component is in ``hdl/rtl/spec_base_wr.vhd`` and
examples of its usage are available in ``hdl/top/``.
Remember that the Linux driver expects the ``SPEC base`` at offset
``0x00000000``.
Meta-Data ROM
-------------
Fixed Part
~~~~~~~~~~
========== ========== ================== ============
Offset Size (bit) Name Default (LE)
0x00000000 32 Vendor ID 0x000010DC
0x00000004 32 Device ID 0x53504543
0x00000008 32 Version <variable>
0x0000000C 32 Byte Order Mark 0xFFFE0000
0x00000010 128 Source ID <variable>
0x00000020 32 Capability Mask <variable>
0x00000030 128 Vendor UUID 0x00000000
========== ========== ================== ============
Rule
The ``SPEC base`` must connect the VIC IRQ output to the ``GPIO 8`` on
the GN4124 chip
Observation
The GN4124 ``GPIO 9`` can be used for interrupts by the application.
Rule
The ``SPEC base`` reserves the first 6 interrupt lines of
the internal interrupt controller (``VIC``) for the following purposes:
Version 1.4
~~~~~~~~~~~
============== ===================
Interrupt Line Component
0 Gen-Core I2C Ocore
1 Gen-Core SPI
2 Gen-Core Gennum DMA DONE
3 (reserved)
4 (reserved)
5 (reserved)
============== ===================
========== ========== ================== ============
Offset Size (bit) Name Default (LE)
0x00000000 32 Vendor ID 0x000010DC
0x00000004 32 Device ID 0x53504543
0x00000008 32 Version 0x0104xxxx
0x0000000C 32 Byte Order Mark 0xFFFE0000
0x00000010 128 Source ID <variable>
0x00000020 32 Capability Mask 0x0000000x
0x00000030 128 Vendor UUID 0x00000000
========== ========== ================== ============
.. _`SPEC project`: https://ohwr.org/project/spec
.. _`FPGA device identification`: https://www.ohwr.org/project/fpga-dev-id/
.. _`general cores`: https://www.ohwr.org/projects/general-cores
.. _`GN4124 core`: https://www.ohwr.org/project/gn4124-core/
.. _`White-Rabbit core`: https://ohwr.org/project/wr-cores
doc/index.rst
View file @ 9358d3a1
..
SPDX-License-Identifier: CC-BY-SA-4.0
SPDX-FileCopyrightText: 2019-2020 CERN
================================
Welcome to SPEC's documentation!
================================
......@@ -13,6 +17,10 @@ fully use the card.
The `SPEC project`_ is hosted on the `Open HardWare Repository`_
You can clone the GIT project with the following command::
git clone https://ohwr.org/project/spec.git
.. toctree::
:maxdepth: 2
:caption: Contents:
......@@ -20,12 +28,5 @@ The `SPEC project`_ is hosted on the `Open HardWare Repository`_
hdl-spec-base
sw-driver
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
.. _`Open HardWare Repository`: https://ohwr.org/
.. _`SPEC project`: https://ohwr.org/project/spec
doc/sw-driver.rst
View file @ 9358d3a1
..
SPDX-License-Identifier: CC-BY-SA-4.0
SPDX-FileCopyrightText: 2019-2020 CERN
.. _spec_driver:
SPEC Driver(s)
==============
Driver(s) Structure
-------------------
There are drivers for the GN4124 chip and there are drivers for the
:ref:`SPEC base<spec_hdl_spec_base>` component. All these drivers are
managed by:
:ref:`SPEC base<spec_hdl_spec_base>` component.
.. _spec_fmc_carrier:
SPEC FMC Carrier
This is the driver that wrap up all the physical components and the
......@@ -14,11 +25,15 @@ The driver for the GN4124 chip are always present and distributed as
part of the SPEC driver. They must work no matter what FPGA design has
been loaded on FPGA.
.. _gn4124_gpio:
GN4124 GPIO
This driver provides support for the GN4124 GPIOs. It uses the standard
Linux `GPIO interface`_ and it export a dedicated IRQ domain.
Gn4124 FCL
.. _gn4124_fcl:
GN4124 FCL
This driver provides support for the GN4124 FCL (FPGA Configuration Loader).
It uses the `FPGA manager interface`_ to program the FPGA at runtime.
......@@ -27,25 +42,188 @@ base<spec_hdl_spec_base>` component then it can profit from the
following driver. They are not all mandatory, it depends on the
application, and most of them are distributed separately:
.. _gn4124_fpga_dma:
SPEC GN412x DMA
This driver provides for DMA transfers to/from the SPEC DDR. It uses
the standard Linux `DMA Engine`_. It is part of the `SPEC project`_
.. _i2c_ocore:
I2C OCORE
This is the driver for the I2C OCORE IP-core. It is used to communicate with
the standard FMC EEPROM available what on FMC modules. The driver is
available in Linux.
available in Linux but also (as a backport) in `general cores`_.
.. _spi_ocore:
SPI OCORE
This is the driver for the SPI OCORE IP-core. It is used to communicate with
the M25P32 FLASH memory where FPGA bitstreams are stored. The driver is
distributed separately.
distributed separately in `general cores`_.
.. _vic:
VIC
The driver for the VIC interrupt controller IP-core. The driver is
distributed separately.
distributed separately in `general cores`_.
.. _`OHWR`: https://ohwr.org
.. _`SPEC project`: https://ohwr.org/project/spec
.. _`FMC`: https://www.ohwr.org/projects/fmc-sw
.. _`GPIO interface`: https://www.kernel.org/doc/html/latest/driver-api/gpio/index.html
.. _`FPGA manager interface`: https://www.kernel.org/doc/html/latest/driver-api/fpga/index.html
.. _`DMA Engine`: https://www.kernel.org/doc/html/latest/driver-api/dmaengine/index.html~
.. _`DMA Engine`: https://www.kernel.org/doc/html/latest/driver-api/dmaengine/index.html
.. _`general cores`: https://www.ohwr.org/projects/general-cores
Drivers(s) Build and Install
----------------------------
From the project top level directory, you can find the driver(s) source files
under ``software/kernel``.
The SPEC software uses plain ``Makefile`` to build drivers. Therefore, you can
build the driver by executing ``make``. To successfully build the SPEC driver
you need to install the `cheby`_ tool that will generate on fly part of the
code for the :ref:`SPEC base<spec_hdl_spec_base>`. If you do not want to
install `cheby`_ you can define the path to it with the environment
variable ``CHEBY``. Following an example on how to build the driver(s).::
# define CHEBY only if it is not installed
export CHEBY=/path/to/cheby/proto/cheby.py
cd /path/to/spec/
make -C software/kernel modules
make -C software/kernel modules_install
This will build and install 4 drivers:
- :ref:`spec-fmc-carrier.ko<spec_fmc_carrier>`,
- :ref:`gn412x-gpio.ko<gn4124_gpio>`,
- :ref:`gn412x-fcl.ko<gn4124_fcl>`,
- :ref:`spec-gn412x-dma.ko <gn4124_fpga_dma>`.
::
find software -name "*.ko"
software/kernel/gn412x-fcl.ko
software/kernel/gn412x-gpio.ko
software/kernel/spec-fmc-carrier.ko
software/kernel/spec-gn412x-dma.ko
Please note that this will not install all soft dependencies which are
distributed separately (:ref:`I2C OpenCore<i2c_ocore>`,
:ref:`SPI OpenCore<spi_ocore>`, :ref:`HT Vector Interrupt Controller<vic>`,
`FMC`_).
.. _`cheby`: https://gitlab.cern.ch/cohtdrivers/cheby
Driver(s) Loading
-----------------
When the card is plugged and the driver(s) installed, the Linux kernel will
load automatically all necessary drivers.
If you need to manually install/remove the driver and its dependencies, you
can use `modprobe(8)`_.::
sudo modprobe spec-fmc-carrier
If you did not install the drivers you can use `insmod(8)`_ and `rmmod(8)`_.
In this case is useful to know what drivers to load (dependencies) and their
(un)loading order.::
insmod /path/to/fmc-sw/drivers/fmc/fmc.ko
insmod /path/to/general-cores/software/htvic/drivers/htvic.ko
insmod /path/to/general-cores/software/i2c-ocores/drivers/i2c/busses/i2c-ocores.ko
insmod /path/to/general-cores/software/spi-ocores/drivers/spi/spi-ocores.ko
insmod /path/to/spec/software/kernel/gn412x-fcl.ko
insmod /path/to/spec/software/kernel/gn412x-gpio.ko
insmod /path/to/spec/software/kernel/spec-gn412x-dma.ko
# Actually the order above does not really matter, what matters
# it is that spec-fmc-carrier.ko is loaded as last
insmod /path/to/spec/software/kernel/spec-fmc-carrier.ko
.. _`modprobe(8)`: https://linux.die.net/man/8/modprobe
.. _`insmod(8)`: https://linux.die.net/man/8/insmod
.. _`rmmod(8)`: https://linux.die.net/man/8/rmmod
Attributes From *sysfs*
-----------------------
In addition to standard *sysfs* attributes for PCI, `DMA Engine`_,
`FPGA manager`_, `GPIO`_, and `FMC`_ there more SPEC specific *sysfs*
attributes. Here we focus only on those.
At PCI device top-level we can see the `DMA Engine`_ interface and the
GN412x sub-devices for :ref:`GPIO<gn4124_gpio>` and :ref:`FCL<gn4124_fcl>`.
Still at the PCI device top-level there is the directory ``fpga-options``
that contains additional attributes to control the FPGA.
``fpga-options/bootselect`` [R/W]
It selects (returns) the FPGA access mode. Possible values are:
- fpga-flash: (default) the FPGA has access to the SPI flash, it uses it
to load the pre-programmed FPGA configuration;
- gn4124-fpga: the FPGA is accessible from the PCI bridge, it is used to
dynamically load an FPGA configuration;
- gn4124-flash: the SPI flash is accessible form the PCI bridge, it is used
to load an FPGA configuration on the SPI flash
``fpga-options/load_golden_fpga`` [W]
It loads the SPEC golden FPGA (if installed). Just write '1' to this file.
If the FPGA is correctly programmed (an FPGA configuration that uses the
:ref:`SPEC base<spec_hdl_spec_base>`) then there will be a directory named
``spec-<pci-id>`` that contains the reference to all FPGA sub-devices and the
following *sysfs* attributes.
``spec-<pci-id>/application_offset`` [R]
It shows the relative offset (from FPGA base address - resource0) to the
user application loaded.
``spec-<pci-id>/pcb_rev`` [R]
It shows the SPEC carrier PCB revision number.
``spec-<pci-id>/reset_app`` [R/W]
It puts in *reset* (1) or *unreset* (0) the user application.
.. _`GPIO`: https://www.kernel.org/doc/html/latest/driver-api/gpio/index.html
.. _`FPGA manager`: https://www.kernel.org/doc/html/latest/driver-api/fpga/index.html
Attributes From *debugfs*
-------------------------
In addition to standard *debugfs* attributes for PCI, `DMA Engine`_,
`FPGA manager`_, `GPIO`_, and `FMC`_ there more SPEC specific *debugfs*
attributes. Here we focus only on those.
``gn412x-gpio.<ID>.auto/regs`` [R]
It dumps the GN412X registers controlling the GPIO module.
``gn412x-fcl.<ID>.auto/regs`` [R]
It dumps the GN412X registers controlling the FCL module.
``spec-gn412x-dma.<ID>.auto/regs`` [R]
It dumps the GN412X DMA FPGA registers controlling the DMA ip-core.
``<pci-id>/fpga_device_metadata`` [R]
It dumps the FPGA device metadata information for the
:ref:`SPEC base<spec_hdl_spec_base>` and, when it exists, the user
application one.
``<pci-id>/info`` [R]
Miscellaneous information about the card status: IRQ mapping.
``<pci-id>/fpga_firmware`` [W]
It configure the FPGA with a bitstream which name is provided as input.
Remember that firmwares are installed in ``/lib/firmware`` and alternatively
you can provide your own path by setting it in
``/sys/module/firmware_class/parameters/path``.
``<pci-id>/csr_regs`` [R]
It dumps the Control/Status register for
the :ref:`SPEC base<spec_hdl_spec_base>`
``<pci-id>/build_info`` [R]
It shows the FPGA configuration synthesis information
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment