Skip to content

W
White Rabbit Standardization
Commit 9216f614 authored by Maciej Lipinski's avatar Maciej Lipinski
Browse files
Options

presentations: WR presentation for IEEE AVB Gen2 standardization meeting in York

parent 90572f77
Hide whitespace changes
Inline Side-by-side
Showing with 616943 additions and 0 deletions
presentations/WR_IEEE-AVB-York2012/Makefile 0 → 100644
View file @ 9216f614
all : WhiteRabbit.pdf
.PHONY : all clean
WhiteRabbit.pdf : WhiteRabbit.tex figs
latex $^
latex $^
dvips WhiteRabbit
ps2pdf WhiteRabbit.ps
figs:
$(MAKE) -C fig
clean :
# $(MAKE) clean -C fig
rm -f *.eps *.pdf *.dat *.log *.out *.aux *.dvi *.ps *.toc *.snm *.nav *~
presentations/WR_IEEE-AVB-York2012/WhiteRabbit.ptp 0 → 100644
View file @ 9216f614
# PCTeX Project File, Version 1.1
name: WhiteRabbit
texformat: latex
typesetmode: DVI
spellwordlist: american
WhiteRabbit.tex
wndpos: 3 -1 -1 -8 -30 0 0 527 554
zorder: 0
edit: 0
WhiteRabbit.dvi
presentations/WR_IEEE-AVB-York2012/WhiteRabbit.tex 0 → 100644
View file @ 9216f614
\documentclass[compress,red]{beamer}
\mode<presentation>
\setbeamertemplate{navigation symbols}{}
\usepackage{pgfpages}
\setbeameroption{show notes}
\setbeameroption{show notes on second screen=right}
\usetheme{Warsaw}
%\hypersetup{pdfpagemode=FullScreen} % makes your presentation go automatically to full screen
% define your own colors:
\definecolor{Red}{rgb}{1,0,0}
\definecolor{Blue}{rgb}{0,0,1}
\definecolor{Green}{rgb}{0,1,0}
\definecolor{magenta}{rgb}{1,0,.6}
\definecolor{lightblue}{rgb}{0,.5,1}
\definecolor{lightpurple}{rgb}{.6,.4,1}
\definecolor{gold}{rgb}{.6,.5,0}
\definecolor{orange}{rgb}{1,0.4,0}
\definecolor{hotpink}{rgb}{1,0,0.5}
\definecolor{newcolor2}{rgb}{.5,.3,.5}
\definecolor{newcolor}{rgb}{0,.3,1}
\definecolor{newcolor3}{rgb}{1,0,.35}
\definecolor{darkgreen1}{rgb}{0, .35, 0}
\definecolor{darkgreen}{rgb}{0, .6, 0}
\definecolor{darkred}{rgb}{.75,0,0}
\xdefinecolor{olive}{cmyk}{0.64,0,0.95,0.4}
\xdefinecolor{purpleish}{cmyk}{0.75,0.75,0,0}
\useoutertheme[subsection=false]{smoothbars}
% include packages
\usepackage{subfigure}
\usepackage{multicol}
\usepackage{amsmath}
\usepackage{epsfig}
\usepackage{graphicx}
\usepackage[all,knot]{xy}
\xyoption{arc}
\usepackage{url}
\usepackage{multimedia}
\usepackage{hyperref}
\usepackage{helvet}
\usepackage[polish,english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{multirow}
%%%%%%%%%%%%5
%\usepackage{geometry}
%\geometry{verbose,letterpaper}
%\usepackage{movie15}
%\usepackage{hyperref}
%%%%%%%
% greetings, introduce yourself
% \includegraphics[height=5cm]{fig/WRlogo.ps}
\title[White Rabbit \hspace{2em}\insertframenumber/\inserttotalframenumber]
{White Rabbit: a next generation synchronization and control network for large distributed systems}
\institute{
\begin{center}
on behalf of \\
{\bf Maciej Lipi\'{n}ski} \\
Hardware and Timing Section / ~~~ Institute of Electronic Systems \\
~~~~~~~~~~~~~~~~~~~~~CERN ~~~~~~~~~~~~~~~~ / Warsaw University of Technology \\
\end{center}
}
\author{
Rodney Cummings \\
% on behalf of Maciej Lipi\'{n}ski
}
\date{May 2012}
\pgfdeclareimage[height=0.6cm]{wr-logo}{fig/WRlogo.ps}
\logo{\pgfuseimage{wr-logo}}
\AtBeginSection[]
% {
% \begin{frame}<beamer>{Outline}
% \tableofcontents[currentsection]
% \end{frame}
% }
\begin{document}
\frame{\titlepage}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}<beamer>{Outline}
\tableofcontents %[currentsection]
\end{frame}
\note[enumerate]{
\item First very short introducation to the White Rabbit project)
\item Then, some basics on how the accelerators are controlled, to better understand the requirements.
At the end, requirements, which abstract of transportation layer, are presented.
\item Then, explanation of how WR has solved (in case of timing) or is intending to solve (in case of data)
problems.
\item A description how WR is intended to use for controlling accelerators.
\item Comparision of the ideas introduced in WR and in AVB gen 2
\item Summary
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}
\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{What is White Rabbit?}
\begin{columns}[c]
\column{0.75\textwidth}
\begin{itemize}
\item Accelerator's control and timing
\item Based on well-known technologies
\item Open Hardware and Open Software
\item International collaboration
\item Main features:
\begin{itemize}
\item transparent, {\bf high-accuracy} time distribution
\item low-latency, {\bf deterministic} data delivery
\item designed for {\bf high reliability}
\item plug \& play
\end{itemize}
\end{itemize}
\column{0.3\textwidth}
\begin{center}
\includegraphics[width=1.1\textwidth]{fig/rabbit.eps}
\end{center}
\end{columns}
\end{frame}
\note{
White Rabbit is a project which was started to develop next generation control and timing network for CERN.
Later, another accelerator facility (GSI in Germany) joined the project. Currently, the project is a
collaboration of many institutes and companies around the world. The project is both, open hardware and
opens software. Any company can download sources and produce WR gear.
The projects aims at creating an Ethernet-based network with low-latency, deterministic packet delivery and
network-wide, transparent, high-accuracy timing distribution. The White Rabbit Network (WRN) is based on
existing standards, namely Ethernet (IEEE 802.3 [2]), Synchronous Ethernet (SyncE [3]) and PTP [4].
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{CERN Control System}
\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{CERN}
\begin{columns}[c]
\column{0.55\textwidth}
\begin{center}
\begin{itemize}
\item 6 accelerators
\item LHC: 27km perimeter
\item Thousands of devices to be controlled and synchronized
\item A huge real-time distributed system
\end{itemize}
\end{center}
\column{0.6\textwidth}
\begin{center}
\includegraphics[width=6.5cm]{fig/accelerators.eps}
\end{center}
\end{columns}
\note{
CERN is a complex of 6 circular and some linear accellerators which are interconnected. The biggest
accellerator is the Large Hadron Colier which is 27 km long. All the devices which serve the accellerators
(magnets, kickers, etc) need to be precisely synchronized and controlled by a central control system.
}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \section{}
% \subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{A simplified explanation of CERN control system (1)}
\begin{center}
\includegraphics[width=.8\textwidth]{fig/event1.eps}
\end{center}
\begin{itemize}
\item {\bf Events} -- points in time at which actions are triggered
\item Each event is identified by an {\bf ID}
\end{itemize}
\end{frame}
\note[itemize]{
\item Each accelerator ramps up the energy/speed of particles (a beam)
\item Once the this energy is achieved, the beam is injected
to a more powerful accelerator and the process is repeated
\item A process of ramping up energy is done in cycles
\item In each cycle, different devices at different points in time need to perform various
activities
\item This activities are triggered by events
\item Events are identified by ID
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \section{}
% \subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{A simplified explanation of CERN control system (2)}
\begin{center}
\includegraphics[width=.8\textwidth]{fig/event2.eps}
\end{center}
\begin{itemize}
\item Devices are subscribed to events
\item Each device "knows" what to do on particular event reception
\end{itemize}
\end{frame}
\note[itemize]{
\item Many devices (in different accelerators) can subscribe to the same event
\item The action taken by the device on the reception of a particular event is defined in each
device
\item You can see in the figure that on event ID=3:
\begin{itemize}
\item a sensor starts takes measuremetn
\item ramping of magnet energy is stopped in PS
\item ramping of magnet energy is started in SPS
\end{itemize}
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \section{}
% \subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{A simplified explanation of CERN control system (3)}
\begin{center}
\includegraphics[width=.7\textwidth]{fig/event3.eps}
\end{center}
\begin{itemize}
\item Each event (ID) has a trigger time associated
\item A set of events is sent as a single {\bf Control Message (CM)}
\item CM is broadcasted to all the end devices (nodes)
\item CM is sent in advance ({\bf Granuality Window})
\end{itemize}
\end{frame}
\note[itemize]{
\item Each event has the trigger time(stamp) associated with it
\item Many events are accumulated into Control Messages which are sent every Granuality Window (e.g. 1ms)
\item Control Messages are sent by a controller device (so called Data Master).
\item Control Messages shall be sent at least Granuality Window in advance with regards to the trigget time
of the events they contain
\item The arrival time of Control Messages is different in different devices (jitter) but the underlying
network must guarantee it to be within the Granuality Window
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \section{}
% \subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{A simplified explanation of CERN control system (4)}
\begin{columns}[c]
\column{0.55\textwidth}
{\bf Granuality Window: }
\begin{center}
\begin{itemize}
\item Controller input to node output (i.e. pulse)
\item Maximum bound latency {\bf guaranteed } by the system
\item Processing and network latency included
\end{itemize}
\end{center}
\column{0.6\textwidth}
\begin{center}
\begin{center}
\includegraphics[width=.8\textwidth]{fig/gw.eps}
\end{center}
\end{center}
\end{columns}
\end{frame}
\note[itemize]{
\item The Granuality Window includes not only the nework latency, but also the processing and
transmission/reception delays
\item In other words: Granuality Window is the time between the moment a controller "says" send
Control Message, and the time an impulse (physical signal) is triggered by an end device on
the event.
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \section{}
% \subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{A simplified explanation of CERN control system (5)}
\begin{center}
\includegraphics[width=.7\textwidth]{fig/accNetworks.eps}
\end{center}
\begin{itemize}
\item 4 accelerator networks
\item Separate {\bf Data Master (DM)} for each network
\item \textcolor{green!90}{LIC Data Master} communicates with other DMs and control devices in their networks
\item Broadcast of {\bf Control Messages} within network(s)
\end{itemize}
\end{frame}
\note[itemize]{
\item Accelerators at CERN are grouped into 4 accelerator networks which should be as independant
as possible but interaction between them is inherent
\item Devices connected to each accelerator network are controlled (mostly) by it's own Data Master
\item Since LIC provides beam to LHC/AD/ISOLDE, the LIC Data Master sometimes needs to control
devices connected to other networks
\item Communication between Data Masters is necessary
\item All the devices in a network need to receive all the Control Messages (the device decides how/whether
to use events from the Control Message), thus Control Messages are broadcasted within network
\item Some Control Messages need to be send on more then one accelerator networks (the case when LIC
Data Master controls devices in more then one network)
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{Requrements}
%\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Control network requirement}
\small
\begin{table}[ht]
\centering
\begin{tabular}{| c | c | c |} \hline
\textbf{Requirement } & {\bf Case 1} & {\bf Case 2 } \\ \hline
Synchronization accuracy & \multicolumn{2}{|c|}{\bf sub-ns } \\ \hline
Synchronization precision & \multicolumn{2}{|c|}{\bf picoseconds } \\ \hline
Granuality Window (GW) & {\bf 1000us } & {\bf 100us } \\ \hline
Network span & {\bf 10km } & {\bf 2km } \\ \hline
End device number & \multicolumn{2}{|c|}{\bf 2000 } \\ \hline
Control Message size & {\bf 1500-5000 bytes}& {\bf 300-500 bytes } \\ \hline
Control Message frequency & \multicolumn{2}{|c|}{\bf every GW } \\ \hline
Data Master/Stream number & {\bf 4 } & {\bf a few } \\ \hline
Traffic characteristics & \multicolumn{2}{|c|}{\bf one-to-alot } \\ \hline
Number of CM lost per year & \multicolumn{2}{|c|}{\bf 1 } \\ \hline
\end{tabular}
\label{tab:CMspDelay}
\end{table}
\end{frame}
\note{
Two usecases of control and timing system requirements. This is because WR will be used in
two accelerator facilities. The control pricipals in both facilities are similar but the
parameters (i.e. size) are different.
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{White Rabbit}
\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{White Rabbit - Ethernet enhancement}
\begin{columns}[c]
\column{.47\textwidth}
Two separate services (enhancements to Ethernet) provided by WR:
\begin{itemize}
\item \color{blue!90}{High accuracy/precision synchronization}
\item \color{red}{Deterministic, reliable and low-latency Control Data delivery}
\end{itemize}
\column{.6\textwidth}
\begin{center}
\includegraphics[height=1.0\textwidth]{fig/wr_network-new.ps}
\end{center}
\end{columns}
\end{frame}
\note{
WR should be treated as a standard Ethernet with additional (optional) features (characteristics) which
should be used if one needs them.
These two features should be treated separately/indepedantly: (1) high accuracy/precision
synchroniation and (2) deterministic, low latency, reliable data delivery.
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{}
% \subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Reliability in a White Rabbit Network (WRN)}
\begin{columns}[c]
\column{.35\textwidth}
\begin{block}{WRN is functional if ...}
... it provides {\bf all} its services to {\bf all} its clients at {\bf any} time.
\end{block}
\column{.7\textwidth}
\begin{center}
\includegraphics[width=0.9\textwidth]{fig/sub_domains-new.ps}
\end{center}
\end{columns}
\end{frame}
\note{
An accelerator in order to work needs all it's devices to perform appropriate actions (react to events at
appropriate time) and any device could perform action at any event. Therefore, WR Network is condidered
functional only if {\bf all} the devices are synchronized and {\bf all} the devices receive
{\bf each} Control Message within the Granuality window.
The problem of reliability in WR has been devided into subdomains: (1) deterministic data deliery, (2) topology
redundancy, (3) data resilience and (4) synchronization resilience which can be handled independently but
need to interact.
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Time Distribution}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Time Distribution in White Rabbit Network}
\begin{center}
Time Distribution in White Rabbit Network \\
(Working solution, test deployment underway)
\end{center}
\end{frame}
\note{
Timing distribution in WR is a working solution which is being currently deployed in Cern Neutrino to Gran
Sasso (CNGS) experiment.
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\subsection{Time Distribution in White Rabbit Network}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Time Distribution in White Rabbit Network}
\begin{itemize}
\item Synchronization with {\bf sub-ns} accuracy {\bf ps} precision
\item Combination of
\begin{itemize}
\item Precision Time Protocol ({\bf PTP}) synchronization
\item Synchronous Ethernet ({\bf SyncE}) syntonization
\item Digital Dual-Mixer Time Difference ({\bf DDMTD}) phase detection
\end{itemize}
\end{itemize}
\begin{center}
\includegraphics[width=1.0\textwidth]{fig/phase_tracking.eps}
\end{center}
\end{frame}
\note[itemize]{
\item Highlights
\begin{itemize}
\item Auto-calibration of link asymmetry
\item Enhanced precision of HW timestamps (picosecond level)
\item High accuracy only with WRPTP-capable devices
\item Interoperable with (virtually) default PTP Profile - hybrid networsk possible
\item Synchronisation topology independant from data topology
\end{itemize}
\item Extenstion to PTP -- defines as PTP Profile: {\bf WRPTP}
\begin{itemize}
\item delay-request two-step mechanims
\item modified BMC (reliability-oriented)
\item Mapping onto Ethernet
\end{itemize}
\item Figure
\begin{itemize}
\item The recovered by slave frequency (SyncE) is phase-shifted (based on the data from PTP) and
looped back to the master
\item DDMTD is used to evaluate with picosecond precision the round trip delay and
obtion precise timestamps
\end{itemize}
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{Why not standard PTP?}
%\subsection{}
% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{PTP is OK but ...}
\resizebox{11cm}{!}
{
\begin{tabular}{ r c l }
{\bf What are the issues...} & {\bf and} & {\bf ... how we address them} \\
& & \\
PTP-base & \multirow{2}{*}{$\Rightarrow$} & \multirow{2}{*}{SyncE }\\
syntonization & & \\
& & \\
limited &\multirow{2}{*}{$\Rightarrow$} & SyncE \\
precision and resolution & & DDTMD phase detection\\
& & \\
& & SyncE \\
unknown link asymmetry & $\Rightarrow$ & DDTMD phase detection \\
& & WR Link Delay Model \\
& & \\
\multicolumn{3}{c}{WR extension to PTP ({\bf WRPTP}) for } \\
\multicolumn{3}{c}{extra data exchange and logic} \\
\end{tabular}
}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Data Distribution}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Time Distribution in White Rabbit Network}
\begin{center}
Data Distribution in White Rabbit\\
(Solutions in a design or implementation phase)
\end{center}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{}
%\subsection{Data Distribution in White Rabbit}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Control Data}
\begin{itemize}
\item Two types of data transported over White Rabbit Network:
\begin{itemize}
\item {\bf Control Data} (High Priority, HP) - priority 7 and broadcast
\item Standard Data (Best Effort) - all the rest
\end{itemize}
\item Characteristics of Control Data
\begin{itemize}
\item Deterministic and low latency delivery (within GW)
\item Broadcast (one-to-alot)
\item Reliable delivery (one Control Message lost per year)
\end{itemize}
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{}
%\subsection{Data Distribution in White Rabbit}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Data Redundancy: Forward Error Correction (FEC)}
\begin{itemize}
\item Re-transmitted of control data not possible
\item {\bf Forward Error Correction} -- additional transparent layer:
\begin{itemize}
\item One Control Message encoded into N Ethernet frames,
\item Recovery of Control Message from any M (M$<$N) frames received
\end{itemize}
\item FEC can prevent data loss due to:
\begin{itemize}
\item {\bf bit error}
\item {\bf network reconfiguration}
\end{itemize}
\end{itemize}
\begin{center}
\includegraphics[width=.7\textwidth]{fig/FEC.eps}
\end{center}
\end{frame}
\note[itemize]{
\item Broadcast and low latency characteristics of Control Data prevents from using
re-transmission for reliable delivery
\item Example implementation of FEC (possibly used in WR)
\item Example implementation: {\bf 500 bytes} frame encoded into {\bf 4 of 288 bytes} frames,
{\bf any 2 frames} enable recovery of the original message
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{}
%\subsection{Data Distribution in White Rabbit}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Topology Redundancy: eSTP or eLACP}
\begin{itemize}
\item Two ideas:
\begin{itemize}
\item enhanced (Multiple/Rapid) Spanning Tree Protocol (eSTP)
\item enhanced Link Aggregation Control Protocol (eLACP)
\end{itemize}
\item FEC + eRSTP/eLACP = seamless redundancy
\item FEC used in both ideas
\item Redundant data received in end stations in both
\item {\bf Solutions for broadcast traffic (within VLAN)}
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{}
%\subsection{Data Distribution in White Rabbit}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{eLACP (short explanation)}
Control Message encoded into 4 Ethernet Frames (F1,F2,F3,F4). Reception of any two
enables to recover Control Message.
\begin{center}
\includegraphics[width=.8\textwidth]{fig/WR_LA_3.ps}
\end{center}
\end{frame}
\note[itemize]{
\item LACP provides the means to show two networks interface as a single interface
\item When the network interfaces are not in the same physical switch a multipath is created
for a frame
\item Normally the multipath is used for balancing the traffic
\item On the other hand, the FEC distributes the control information in several frames
\item What if we send $X\%$ of this frames in one path and $Y\%$ in the alternative path
\item We could overcome single points of failure in the network: switch breakdown and damaged
fiber optic or ports.
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{}
%\subsection{Data Distribution in White Rabbit}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{eSTP}
\begin{columns}[c]
\column{0.55\textwidth}
\begin{itemize}
\item Speed up STP -- max 2 frames lost on re-configuration
\item A priori information (alternate/backup) used
\item Allowed topologies limited
\item Drop only on reception -- within VLAN, except self-sending
\end{itemize}
%
\column{0.6\textwidth}
\begin{center}
\includegraphics[width=.8\textwidth]{fig/portRoles.v2.eps}
\end{center}
\end{columns}
\end{frame}
\note[itemize]{
\item add sth
\item add sth
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{}
%\subsection{Data Distribution in White Rabbit}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{eSTP + FEC}
\begin{itemize}
\item eSTP+FEC=seamless redundancy $<=>$ max 2 frames
\item 500 bytes message (288byte FEC) -- max re-conf: $\approx$ 2.3us
\item Possible only if information about alternative/backup ports known a priori and
switch-over in hardware
\item {\bf The simplest} case in figure
\end{itemize}
\begin{center}
\includegraphics[width=.8\textwidth]{fig/RSTPsimple.ps}
\end{center}
\end{frame}
\note[itemize]{
\item add sth
\item add sth
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{}
%\subsection{Data Distribution in White Rabbit}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{eSTP: semi-automatic re-configuration}
\begin{itemize}
\item In critical networks alternate/backup paths and root switches should
be known in advance (i.e. in RSTP: through proper priority configuration
and network topology)
\item Two types of re-configuration:
\begin{enumerate}
\item Foreseen: know what to do immediately -- safe
\item Unforeseen: most probably causes "long" data delivery disruption -- unsafe
\end{enumerate}
\item Perform re-configuration of type (1) immediately, simulate results and ask for
management ack for reconfiguration type (2)
\end{itemize}
\end{frame}
\note[itemize]{
\item add sth
\item add sth
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{WR-based Control System}
\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{WR-based Control System}
\begin{center}
WR-based Control System \\
(a concept)
\end{center}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{WR-based Control System}
%\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Accelerator Networks}
\begin{center}
\includegraphics[width=.8\textwidth]{fig/NT-overview.eps}
\end{center}
\end{frame}
\note[itemize]{
\item add sth
\item add sth
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{WR-based Control System}
%\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{VLANs}
\begin{center}
\includegraphics[width=.8\textwidth]{fig/NT-VLANs.eps}
\end{center}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{WR-based Control System}
%\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Multicast for redundant controllers (Data Masters)}
\begin{columns}[c]
\column{0.75\textwidth}
\begin{itemize}
\item Multicast address used for Data Masters (DM-A and DM-B)
\item Seamless switch over between DMs: time-triggered synchronous reconfiguration
of 1-Layer switches
\item Nodes send data to multicast address: both DMs receive data
\item No need for network reconfiguration when switching/changing DMs
\end{itemize}
\column{0.25\textwidth}
\begin{center}
\includegraphics[width=1.0\textwidth]{fig/DM-overview.eps}
\end{center}
\end{columns}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{ABV gen2 vs. WR}
\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Synchronization}
\begin{itemize}
\item Differences
\begin{itemize}
\item WR: physical syntonization is obligatory (using SyncE)
\item AVB: logical syntonization is optional
\item WR: request-response mechanism (no technical problem to align with AVB)
\item AVB: peer-delay mechanism
\end{itemize}
\item Similarities
\begin{itemize}
\item Support of network redundancy and fast re-configuration
\item (Semi-)automatic link asymmetry calibration
\end{itemize}
\item WR is inter-operable with (virtually) default PTP profile
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{ABV gen2 vs. WR}
%\subsection{Streams vs. Control Data}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Streams vs. Control Data}
\begin{itemize}
\item Control Data send by Data Master is as AVB Stream
\item WR statically reserves (single) resource altogether for all Control Data (streams)
\item Static stream (Control Data) reservation in WR
\item Stream in WR is broadcast within a VLAN
\item WR Control Data distribution -- a corner caste of AVB Stream
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{ABV gen2 vs. WR}
%\subsection{Reliability (network/data redundancy}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Reliability (network/data redundancy)}
\begin{itemize}
\item WR requires seamless redundancy -- achieved by adding FEC to (very fast) dynamic
redundancy (eSTP) or multipath solution (eLACP)
\item FEC introduces additional latency
\item Advantage is taken from broadcast traffic characteristics
\item In the current concept of WR-based control network - VLAN and mutlicast addresses
are used - any redundancy solutions need to support both
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \section{ABV gen2 vs. WR}
% \subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Latency}
\begin{itemize}
\item Strict priority output queue scheduling for Control Data is considered
\item Time Aware Shaper (both at end stations and WR Switches) is worth considering for WR,
especially to fulfill "Case 2" from slide 10.
\item Preemption was considered but is currently stated an obsolete idea
\item Cut-through forwarding in WR Switches
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% \section{ABV gen2 vs. WR}
% \subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Fault tolerance / isolation}
\begin{center}
\end{center}
\begin{itemize}
\item VLANs are used to separate logically accelerator networks to limit propagation of fault due to
mis-behaving node/switch
\item FEC header has ID and sequence number - can prevent help in duplication issues
\item Control of throughput from nodes which are not supposed to send too much data
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Summary}
\subsection{}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{WR requirements in AVB terms (Case 1)}
\begin{itemize}
\item maximum latency: below 1000us over $\approx$ 5 bridge hops @ 1Gbps
\item guaranteed latency over tree-like (meshed) topology
\item main network characteristics: ~2000 end stations, links of max 10km, max network span: 10km
\item traffic characteristics: "control data" size (payload): 500-5000 bytes (in FEC scheme:
encoded into 4 or 6 frames of size 300-1500 bytes and sent in burst), $\approx$ 8
"control streams" (defined within separate VLANs) sent every 1000us,
critical stream is one-to-many, "normal data" size (payload): ~1500 bytes
\item Support for VLAN and multicast
\item Seamless redundancy or ultra fast dynamic reconfiguration (<2.3 us)
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{Summary}
%\subsection{WR requirements in AVB terms (Case 1}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{WR requirements in AVB terms (Case 2)}
\begin{itemize}
\item maximum latency: 100us over ~4 bridge hops @ 1Gbps
\item guaranteed latency over tree-like (meshed) topology
\item main network characteristics: ~2000 end stations, links of max 2km, max network span: 2km
\item traffic characteristics: "control data" size (payload): 200-500 bytes (in FEC scheme:
encoded into 4 frames of size 150-300 bytes and sent in burst), sent every 100us,
critical stream is one-to-many, "normal data" size (payload): ~1500 bytes
\item Support for VLAN and multicast
\item Seamless redundancy or ultra fast dynamic reconfiguration (<2.3 us)
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{Summary}
%\subsection{WR requirements in AVB terms (Case 1}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Conclusions}
\begin{itemize}
\item WR is in many regards a corner case of AVB Gen2
\item WR adds to AVB Gen2 such requirements as:
\begin{itemize}
\item Physical (SyncE-based) syntonization
\item VLAN-wide streams and broadcast
\item Static resource configuration
\end{itemize}
\item WR could benefit from AVB gen2 from:
\begin{itemize}
\item Time Aware Shaper
\item Preemption
\item Multipath solution
\end{itemize}
\end{itemize}
\end{frame}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{Summary}
%\subsection{WR requirements in AVB terms (Case 1}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{frame}{Thank you}
\begin{center}
\includegraphics[height=6.0cm]{fig/WRlogo.ps}
\end{center}
\end{frame}
\note[enumerate]{
\item White Rabbit Project homepage \\ http://www.ohwr.org/projects/white-rabbit
\item White Rabbit Specification \\ http://www.ohwr.org/documents/21
\item White Rabbit Standardization wiki \\ ttp://www.ohwr.org/projects/wr-std/wiki/WRinAVBgen2
\item White Rabbit CERN Control and Timing Network \\ http://www.ohwr.org/documents/85
\item White Rabbit – a PTP application for sub-ns synchronization \\ http://www.ohwr.org/documents/155
\item White Rabbit and Robustness \\ http://www.ohwr.org/documents/103
\item Topology Resolution, Redundant Links Handling and Fast Convergence in White Rabbit Network \\
http://www.ohwr.org/documents/103
}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\ No newline at end of file
presentations/WR_IEEE-AVB-York2012/fig/FEC.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/Makefile 0 → 100644
View file @ 9216f614
.SUFFIXES: .fig .mp .eps .ps
.mp.eps:
mpost $*.mp && mv $*.0 $*.eps
all:
clean:
rm -f *.eps *.log *.bak *.mmp *.dvi *.tex *.mpx
presentations/WR_IEEE-AVB-York2012/fig/NT-VLANs.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/RSTPsimple.ps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/WR_LA_3.ps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/WR_funny_network.ps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/WRlogo.ps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/accNetworks.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/accelerators.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/event1.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/event2.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/event3.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/gw.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/portRoles.v2.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/rabbit.eps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/sub_domains-new.ps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
presentations/WR_IEEE-AVB-York2012/fig/wr_network-new.ps 0 → 100644
View file @ 9216f614
This source diff could not be displayed because it is too large. You can view the blob instead.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment