adding calibration document sources with figures

1ebe9110 · Grzegorz Daniluk · 1ebfc52c · 1ebe9110 · 1ebe9110 · 1ebe9110
Commit 1ebe9110 authored Sep 01, 2014 by Grzegorz Daniluk
25 changed files
--- a/documents/calibration/Makefile
+++ b/documents/calibration/Makefile
+all : wrcalib.pdf
+
+.PHONY : all clean
+
+wrcalib.pdf : wrcalib.tex
+	latex $^
+	latex $^
+	dvips wrcalib
+	ps2pdf -dPDFSETTINGS=/prepress -dSubsetFonts=true -dEmbedAllFonts=true -dMaxSubsetPct=100 -dCompatibilityLevel=1.4 wrcalib.ps
+
+clean :
+	rm -f *.eps *.pdf *.dat *.log *.out *.aux *.dvi *.ps *.toc
+
--- a/documents/calibration/appendix.tex
+++ b/documents/calibration/appendix.tex
--- a/documents/calibration/definitions.tex
+++ b/documents/calibration/definitions.tex
+Before going further into the calibration procedure, some initial definitions
+have to be clarified:
+
+\begin{itemize}
+
+	\item {\bf WR Device}: any device that supports White Rabbit
+		synchronization over an optical link. This can be e.g. the White Rabbit
+		Switch, the SPEC board, or any custom hardware made for a specific
+		application. A WR Device can run as WR Master or/and WR Slave.
+
+	\item {\bf WR Calibrator}: any White Rabbit Device selected from the 
+		available set that is used as a reference to calibrate the 
+		rest of the White Rabbit Devices used in a synchronization system.
+
+	\item {\bf WR Device TX latency} ($\Delta_{TX}$): WR PTP packet delay 
+		from the moment it is timestamped in the transmission circuit
+		inside the FPGA (WR Endpoint module) to the moment when the 
+		transmitted packet reaches a fiber link. It is the sum of: 
+		transmission delay inside FPGA chip; propagation delay of 
+		electronic components and PCB traces; SFP transmitter delay.
+
+	\item {\bf WR Device RX latency} ($\Delta_{RX}$): WR PTP packet delay 
+		from the moment it reaches the receiving SFP module, to the 
+		moment it is timestamped in the reception circuit inside the FPGA (WR
+		Endpoint module). It is the sum of: SFP receiver delay; 
+		propagation delay of electronic components and PCB traces; 
+		reception delay inside FPGA chip.
+
+	\item {\bf WR Device asymmetry} (2$\beta$): the difference between the TX 
+		latency $\Delta_{TX}$ and the RX latency $\Delta_{RX}$.
+
+	\item {\bf Fiber round-trip latency} ($\delta$): the total round-trip 
+		delay of a fiber link (without the Tx/Rx latencies of the WR Devices). 
+		It is the sum of the one-way propagation delays: Master-to-Slave 
+		($\delta_{MS}$) and Slave-to-Master ($\delta_{SM}$).
+
+	\item {\bf Relative delay coefficient} ($\alpha$): the parameter describing 
+		the asymmetry (the difference between $\delta_{MS}$ and 
+		$\delta_{SM}$) of the fiber used in a White Rabbit network. It 
+		is the relative difference of Master-to-Slave and Slave-to-Master 
+		propagation delays ($\alpha = \frac{\delta_{MS} - \delta_{SM}}
+		{\delta_{SM}}$).
+
+	\item {\bf Round trip delay} ($delay_{MM}$): the total packet delay on 
+		a White Rabbit link. It is the sum of communicating parties (Master and
+		Slave) fixed delays ($\Delta_{TXM}$, $\Delta_{RXM}$, $\Delta_{TXS}$,
+		$\Delta_{RXS}$), bitslide values ($\epsilon_M$,
+    $\epsilon_S$) and fiber round-trip latency ($\delta$).
+
+\end{itemize}
--- a/documents/calibration/distant_tricks.tex
+++ b/documents/calibration/distant_tricks.tex
+\section{Measuring already deployed fiber}
+
+Most of the methods described in section \ref{sec:calib_proc} require making a
+measurement of 1-PPS signals produced by WR Devices. It's fairly easy to compare
+them when you have a roll of fiber and the devices being calibrated lying on a
+desk in the laboratory. However, sometimes the fiber infrastructure which
+will be used in a WR network is already installed. WR Devices can still be
+collected in a lab for calibration, but measuring the $\alpha$ parameter of a
+buried fiber is not as straightforward as in section \ref{subsec:fiasym}. This
+section proposes a way of measuring 1-PPS skew when the fiber cable is
+already installed and you don't have both ends next to each other. The
+measurements made this way can be used as a substitute for the simple
+oscilloscope measurements in section \ref{sec:calib_proc}.
+
+\subsection{Measurement with a loop-back fiber}
+\label{subsec:loopback}
+This method is based on a fiber delay calibration procedure used in the CERN
+Neutrinos to Gran Sasso project \cite{cngs}. It requires:
+\begin{itemize}
+	\item an additional fiber which will be used to transmit the 1-PPS signal
+		produced by a distant WR Device to the place where it can be compared
+		locally with the other WR Device using an oscilloscope
+	\item two oscilloscope cables of the same type - having equal latency or a
+    known latency difference which can be taken into account and compensated
+	\item one pair of optical transmitter and receiver which will be used to
+		convert the 1-PPS electric signal into a light impulse sent through the
+		loop-back fiber. The transmitter and receiver must have constant
+		transmission/reception delays that don't vary on each power cycle.
+\end{itemize}
+
+\begin{figure}[ht]
+	\begin{center}
+	\includegraphics[width=.8\textwidth]{../../figures/calibration/loopback_fibre.ps}
+	\caption{Measuring 1-PPS offset using a loop-back fiber}
+	\label{fig:loopback}
+	\end{center}
+\end{figure}
+
+Measurement procedure:
+\begin{enumerate}
+	\item When a link is established between the WR Master and the WR Slave,
+		connect the optical transmitter to the 1-PPS signal produced by the WR Slave
+		and the loop-back fiber. On the other side of the fiber connect the optical
+		receiver (fig.\ref{fig:loopback}, step 1).
+	\item Measure the 1-PPS skew ($skew_{PPS1}$) between the Slave (1-PPS
+		transmitted through the loop-back fiber) and the Master (1-PPS directly from
+		the node) using an oscilloscope.
+	\item Swap the optical transmitter and receiver so that you'll now transfer
+		the 1-PPS signal generated from the WR Master using the same loop-back fiber
+		to the WR Slave side (fig.\ref{fig:loopback}, step 2).
+	\item Do the same 1-PPS skew measurement between the Slave and Master, but
+		this time on the WR Slave side (the 1-PPS comes directly from the WR Slave
+		node and the one from the WR Master is transmitted through the loop-back
+		fiber). This way you will obtain the value $skew_{PPS2}$.
+	\item Calculate the actual skew between the WR Master and WR Slave using the
+		equation:
+	\begin{equation}
+	skew_{PPS} = \frac{1}{2} (skew_{PPS1} + skew_{PPS2})
+	\end{equation}
+\end{enumerate}
+
+The skew value calculated that way can be used in any equation from section
+\ref{sec:calib_proc}. However, please remember that you will also need the
+latency of the fiber ($\delta_1$ in figure \ref{fig:loopback}). That means, you
+would have to start the calibration with two WR Devices on your desk, connected
+with a short (few meters long) link (procedure \ref{subsec:refiber}) and later
+use these two WR Devices with the already-installed link.
--- a/documents/calibration/errors.tex
+++ b/documents/calibration/errors.tex
--- a/documents/calibration/intro.tex
+++ b/documents/calibration/intro.tex
+\section{Introduction}
+
+This document describes the calibration procedure for White Rabbit devices. 
+
+A White Rabbit link established between two devices is characterized by the
+hardware delays and fiber propagation latencies presented in figure
+\ref{fig:intro:link-model}.
+
+\begin{figure}[ht]
+	\begin{center}
+		\includegraphics[width=\textwidth]{../../figures/calibration/link-model.ps}
+		\caption{White Rabbit link model}
+		\label{fig:intro:link-model}
+	\end{center}
+\end{figure}
+
+Each WR Master and WR Slave has some constant transmission and reception delays
+($\Delta_{TXM}$, $\Delta_{RXM}$, $\Delta_{TXS}$, $\Delta_{RXS}$). They are the
+summed result of SFP transceiver, PCB trace and electronic component
+delays as well as the delays inside the FPGA chip. Additional reception delay is
+also caused on both sides by aligning the recovered clock signal to the inter-symbol
+boundaries of the data stream. This is called the bitslide value and is marked in figure
+\ref{fig:intro:link-model} as $\epsilon_M$ and $\epsilon_S$.
+
+In addition to hardware delays, packets transmitted in fiber are affected with
+propagation latencies in both directions ($\delta_{MS}$, $\delta_{SM}$). These
+fiber propagation latencies are not equal since different light wavelengths are 
+used to communicate simultaneously in both directions using a single fiber.
+
+Based on those parameters the round-trip delay ($delay_{MM}$) is defined as the
+sum of all delay factors described above:
+\begin{equation}
+	delay_{MM} = \Delta_{TXM} + \Delta_{RXS} + \epsilon_S + \Delta_{TXS} +
+	\Delta_{RXM} + \epsilon_M + \delta_{MS} + \delta_{SM}
+\end{equation}
+
+However, since the bitslide values can be easily obtained from WR software, 
+the calibration procedure described in this document use a modified round-trip delay, 
+without $\epsilon_M$ and $\epsilon_S$:
+\begin{equation}
+	delay_{MM}' = delay_{MM} - \epsilon_M - \epsilon_S
+\end{equation}
+
+In the synchronization process, the PTP daemon tries to calculate the current
+offset between Slave and Master clock. For that it needs a precise estimation of
+the one-way link delay which, using the described link model, is the sum of four
+factors:
+\begin{equation}
+	delay_{MS} = \Delta_{TXM} + \delta_{MS} + \Delta_{RXS} + \epsilon_S
+\end{equation}
+
+The calibration procedure described in this document is required since the
+values of the hardware transmission and reception delays are not known
+($\Delta_{TXM}$, $\Delta_{RXM}$, $\Delta_{TXS}$, $\Delta_{RXS}$). Moreover, 
+fiber cable and each device has unknown asymmetry that prevents from estimating 
+$delay_{MS}$ as half of the round-trip $delay_{MM}$.
--- a/documents/calibration/procedure.tex
+++ b/documents/calibration/procedure.tex
--- a/documents/calibration/python/u_pps.py
+++ b/documents/calibration/python/u_pps.py
+#!/usr/bin/python
+import sys
+import math
+import matplotlib as mpl
+import matplotlib.pyplot as plt
+from matplotlib.ticker import ScalarFormatter
+
+POINTS = 10000
+INIT_LAT = 49970
+STEP = 100
+ALPHA = 2.57302194425528e-4
+
+lat = []
+for i in range(0, POINTS-1):
+	lat.append(INIT_LAT+i*STEP)
+
+#calculate pps skew for latencies
+skew = []
+for i in range(0, POINTS-1):
+	skew.append(lat[i]*ALPHA/2/(2+ALPHA))
+
+#calculate pps factor
+dalds = []
+for i in range(0, POINTS-1):
+	dalds.append( (lat[i]/(0.5*lat[i]-skew[i])**2)**2 )
+
+#draw graph of dalds
+fig = plt.figure(0)
+plt.title("dalds(latency)")
+ax1 = fig.add_subplot(111)
+ax1.plot(range(0, POINTS-1), dalds, marker='.', linestyle='None', c='b', scaley=True, scalex=False)
+ax1.set_xlim([0, POINTS-1])
+plt.show()
+
+print "dalds[0]=" + str(dalds[0])
--- a/documents/calibration/recover_calibrator.tex
+++ b/documents/calibration/recover_calibrator.tex
+\section{Recovering the calibrator}
+
+In a real life, when a WR network operates for a few years, the WR Device selected
+to be a calibrator might become broken. The lack of a WR Calibrator makes it
+impossible to connect new devices without recalibrating the whole WR network.
+This section presents a method for obtaining the $\Delta_{TX}$, $\Delta_{RX}$
+delays for a WR Device so that it can be used as the new WR Calibrator for an
+already-deployed network.
+
+\begin{figure}[ht]
+	\begin{center}
+	\includegraphics[width=.5\textwidth]{../../figures/calibration/recover_calibrator.ps}
+	\caption{Calibrating a new WR Calibrator for an already-existing WR network}
+	\label{fig:recover_calibrator}
+	\end{center}
+\end{figure}
+
+\begin{enumerate}
+	\item Initially please set the transmission and reception delays
+		($\Delta_{TXS}$, $\Delta_{RXS}$) of the new WR Calibrator to 0, but $\alpha$ has
+		to be set to the correct value for fiber $f_1$.
+	\item Connect your new WR Calibrator to one of the WR Devices originally
+		calibrated to the primary WR Calibrator
+		(fig.\ref{fig:recover_calibrator}). Use a short (few meters) fiber of known
+		latency and $\alpha$ coefficient or measure it first according to the
+		instructions in sections \ref{subsec:refiber} and \ref{subsec:fiasym}. Set
+		the WR Calibrator to Slave, and the WR Device to Master mode.
+	\item Run the monitoring software on the Slave node (\emph{wr\_mon} for the WR
+		Switch or \emph{gui} for the WR PTP Core). Write down the round-trip
+		delay and fixed delays of both Master and Slave. The transmission delay of
+		the Slave ($\Delta_{TXS}$) should be 0, but its reception delay
+		($\Delta_{RXS}$) will be equal to the RX bitslide value $\epsilon_S$. The
+		reception delay of the Master ($\Delta_{RXM}$) already includes the bitslide
+		value for this device.
+	\item Calculate the average (coarse) transmission and reception delay for the
+		new Calibrator using the values read from the monitoring software and the
+		latency of the fiber ($\delta_1$):
+	\begin{equation}
+		\Delta'_{TXS} = \Delta'_{RXS} = \frac{1}{2}\Delta_S = \frac{1}{2}(delay_{MM} - \Delta_{TXM} - \Delta_{RXM} - \Delta_{RXS} - \delta_1)
+	\end{equation}
+	\item Write the $\Delta_{TXS}$ and $\Delta_{RXS}$ delays to the configuration
+		of the new WR Calibrator, restart the device, and let it synchronize again
+		with the new values.
+	\item Measure the Slave to Master offset by comparing 1-PPS signals with an
+    oscilloscope. The 1-PPS skew ($skew_{PPS}$) is used as
+    a correction value for the coarse transmission/reception delays:
+	\begin{align}
+		\Delta_{TXS} = \frac{1}{2}\Delta_S - skew_{PPS}\\
+		\Delta_{RXS} = \frac{1}{2}\Delta_S + skew_{PPS}
+	\end{align}
+	\item Update the configuration of the new WR Calibrator to replace the coarse
+		delays with $\Delta_{TXS}$, $\Delta_{RXS}$.
+	\item The new WR Calibrator can be used to calibrate any other WR Device that is
+		supposed to be connected to the existing WR network.\\[12pt]
+	{\bf Note:} Please be aware that the measurement errors accumulate. It might
+	become an issue if the new calibrator uses values recovered with a WR Device
+	that was already calibrated to a recovered calibrator. The longer this chain
+	is, the more inaccuracy you should expect from the calibration procedure. To
+	minimize this effect, it's better to recover the values for the new WR
+	Calibrator using only devices that were calibrated to the primary WR
+	Calibrator for a given network.
+\end{enumerate}
--- a/documents/calibration/references.bib
+++ b/documents/calibration/references.bib
+@article{cngs,
+	author = {J.Serrano P.Alvarez J.Lewis D.Autiero},
+	title = {Inter-Laboratory Synchronization For The CNGS Project},
+	journal = {10th European Particle Accelerator Conference, Edinburgh, UK, 26 - 30 Jun 2006, pp.3092}
+}
+
+@manual{bipm_errors,
+	title = {Evaluation of measurement data - Guide to the expression of uncertainty in measurement (JCGM 100:2008)},
+	organization = {Working Group 1 of the Joint Committee for Guides in Metrology(JCGM/WG 1)},
+	month = {September},
+	year = {2008}
+}
--- a/documents/calibration/requirements.tex
+++ b/documents/calibration/requirements.tex
+\section{Equipment requirements}
+
+The calibration procedure described in this document makes no assumption on what 
+kind of WR device will be used as a WR Calibrator or what kind of fiber (length, 
+producer) is placed in between WR Devices. However, there are a few general 
+requirements listed below. The first two of them are essential for the
+Calibrator pre-calibration steps described in section
+\ref{sec:procedure:calibrator}.\\
+
+WR Calibration needs:
+\begin{itemize}
+	\item two fiber cables having different lengths, one of them is a few
+		kilometers long (to make $\alpha$ coefficient measurement precise), while
+		the other is a few meters long.
+	\item two pieces of a WR Device selected to be the Calibrators: this means two
+		exactly the same WR Devices, having the same PCB design, the same bitstream
+		downloaded into the FPGA and using complementary SFP transceivers (the same
+		producer, matching Tx/Rx WDM wavelengths\footnote{Check the list of
+		supported SFP transceivers http://www.ohwr.org/projects/white-rabbit/wiki/SFP});
+	\item all WR Devices (i.e. the Calibrator and devices under calibration)
+		capable of producing 1-PPS output from their local clock 
+		and providing it through an on-board connector or test point;
+	\item two oscilloscope cables of the same delay (i.e. the same type and
+		length) or different but known delays so that the correction
+		could be made for all oscilloscope measurements described in
+		the document;
+	\item a device of any producer that is able to measure time
+		spans of less than 1ns for measuring 1-PPS skew. This can be an oscilloscope or timer/counter device e.g. Pendulum CNT-91.
+\end{itemize}
--- a/documents/calibration/wrcalib.bbl
+++ b/documents/calibration/wrcalib.bbl
+\begin{thebibliography}{1}
+
+\bibitem{cngs}
+J.Serrano P.Alvarez~J.Lewis D.Autiero.
+\newblock Inter-laboratory synchronization for the cngs project.
+\newblock {\em 10th European Particle Accelerator Conference, Edinburgh, UK, 26
+  - 30 Jun 2006, pp.3092}.
+
+\bibitem{bipm_errors}
+Working Group 1 of the Joint Committee for Guides in Metrology(JCGM/WG 1).
+\newblock {\em Evaluation of measurement data - Guide to the expression of
+  uncertainty in measurement (JCGM 100:2008)}, September 2008.
+
+\end{thebibliography}
--- a/documents/calibration/wrcalib.blg
+++ b/documents/calibration/wrcalib.blg
+This is BibTeX, Version 0.99c (TeX Live 2009/Debian)
+The top-level auxiliary file: wrcalib.aux
+The style file: unsrt.bst
+Database file #1: references.bib
+Warning--empty year in cngs
+You've used 2 entries,
+            1791 wiz_defined-function locations,
+            453 strings with 3893 characters,
+and the built_in function-call counts, 241 in all, are:
+= -- 18
+> -- 5
+< -- 0
+ -- 3
+- -- 1
+* -- 12
+:= -- 43
+add.period$ -- 5
+call.type$ -- 2
+change.case$ -- 1
+chr.to.int$ -- 0
+cite$ -- 3
+duplicate$ -- 14
+empty$ -- 28
+format.name$ -- 1
+if$ -- 50
+int.to.chr$ -- 0
+int.to.str$ -- 2
+missing$ -- 1
+newline$ -- 12
+num.names$ -- 1
+pop$ -- 7
+preamble$ -- 1
+purify$ -- 0
+quote$ -- 0
+skip$ -- 5
+stack$ -- 0
+substring$ -- 0
+swap$ -- 2
+text.length$ -- 0
+text.prefix$ -- 0
+top$ -- 0
+type$ -- 0
+warning$ -- 1
+while$ -- 1
+width$ -- 3
+write$ -- 19
+(There was 1 warning)
--- a/documents/calibration/wrcalib.tex
+++ b/documents/calibration/wrcalib.tex
+\def\us{\char`\_}
+
+\documentclass[a4paper, 12pt]{article}
+%\documentclass{article}
+
+\usepackage{fullpage}
+\usepackage{pgf}
+\usepackage{tikz}
+\usetikzlibrary{arrows,automata,shapes}
+\usepackage{multirow}
+\usepackage{color}
+\usepackage[latin1]{inputenc}
+\usepackage{verbatim}
+\usepackage{amsmath}
+\usepackage{times,mathptmx}
+\usepackage{chngcntr}
+\usepackage{hyperref}
+%\usepackage{draftwatermark}
+
+\usepackage{listings}
+\usepackage{cancel}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+% creating subsubsubsection notation
+% src: http://www.latex-community.org/forum/viewtopic.php?f=5&t=791
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\setcounter{secnumdepth}{6}
+\renewcommand\theparagraph{\Alph{paragraph}}
+
+\makeatletter
+\renewcommand\paragraph{\@startsection{paragraph}{4}{\z@}%
+                                     {-3.25ex\@plus -1ex \@minus -.2ex}%
+                                     {0.0001pt \@plus .2ex}%
+                                     {\normalfont\normalsize\bfseries}}
+\renewcommand\subparagraph{\@startsection{subparagraph}{5}{\z@}%
+                                     {-3.25ex\@plus -1ex \@minus -.2ex}%
+                                     {0.0001pt \@plus .2ex}%
+                                     {\normalfont\normalsize\bfseries}}
+\counterwithin{paragraph}{subsubsection}
+\counterwithin{subparagraph}{paragraph}
+\makeatother
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\newcommand{\eqoffset}[1]{%
+  {\ensuremath{%
+      {\text{offset}}_{#1}}%
+  }%
+}
+\newcommand{\eqdelay}[1]{{\text{delay}}_{#1}}
+\newcommand{\eqasymm}{{\text{asymmetry}}}
+
+\begin{document}
+
+\title{White Rabbit calibration procedure\\[0.5cm]
+\large {version 1.0}}
+\author{Grzegorz Daniluk\\ CERN BE-CO-HT}
+
+\maketitle
+\thispagestyle{empty}
+
+\begin{figure}[ht!]
+  \centering
+  \vspace{1.3cm}
+  \includegraphics[width=0.50\textwidth]{../../figures/logo/WRlogo.ps}
+  \label{fig:wr_logo}
+\end{figure}
+
+\newpage
+
+\newpage
+
+\newpage
+
+\tableofcontents
+
+\newpage
+\input{intro.tex}
+
+\newpage
+\section{Definitions}
+\input{definitions.tex}
+
+\newpage
+\input{requirements.tex}
+
+\newpage
+\section{Calibration procedures}
+\label{sec:calib_proc}
+\input{procedure.tex}
+
+\newpage
+\input{distant_tricks.tex}
+
+\newpage
+\input{recover_calibrator.tex}
+
+\appendix
+\newpage
+\input{appendix.tex}
+\newpage
+\input{errors.tex}
+
+\newpage
+\bibliographystyle{unsrt}
+\bibliography{references}
+
+\end{document}
--- a/figures/calibration/calibrator.pdf
+++ b/figures/calibration/calibrator.pdf
--- a/figures/calibration/fiber_1.pdf
+++ b/figures/calibration/fiber_1.pdf
--- a/figures/calibration/fiber_2.pdf
+++ b/figures/calibration/fiber_2.pdf
--- a/figures/calibration/lecroy_7300_jitter.png
+++ b/figures/calibration/lecroy_7300_jitter.png
--- a/figures/calibration/link-model.pdf
+++ b/figures/calibration/link-model.pdf
--- a/figures/calibration/loopback_fibre.pdf
+++ b/figures/calibration/loopback_fibre.pdf
--- a/figures/calibration/oscil_meas.pdf
+++ b/figures/calibration/oscil_meas.pdf
--- a/figures/calibration/recover_calibrator.pdf
+++ b/figures/calibration/recover_calibrator.pdf
--- a/figures/calibration/rtt_long.png
+++ b/figures/calibration/rtt_long.png
--- a/figures/calibration/rtt_short.png
+++ b/figures/calibration/rtt_short.png
--- a/figures/calibration/wr_device.pdf
+++ b/figures/calibration/wr_device.pdf