Audio Video Bridging (AVB)
A set of IEEE standards that aim at providing time-synchronized low-latency streaming services through IEEE 802 networks. The currently in use AVB technology is called (here) Gen1 and consists of the following IEEE standards (or standard amendments): 802.1AS, 802.1Qat*, 802.1Qav and 802.1BA. However, ever increasing need and requirement for low-latency time-synchronized communication resulted in an on-going works on Gen2 of AVB. The requirements for Gen2 are no longer shaped only by audio-video industry but also by others (e.g. automotive).
It seems that AVB Gen2 and White Rabbit have some commonalties (in terms of synchronization, low-latency, determinism, reliability). This page is meant to compare the next generation of Audio Video Bridging (AVBg2, or AVB Gen2) standard with White Rabbit, both in terms of requirements and proposed solutions. Firstly, short introduction to the current AVB (Gen 1) is provided. Then requirements and proposed solutions for AVBg2 are summarized. In a similar form to AVB and AVBg2 descriptions, WR description is provided. Finally, AVBg2 and WR are compared. The comparison is based on the author's view of AVB and AVBg2 described below (which is not thorough).
White Rabbit presentation for an interim
meeting of Audio/Video
Bridging Task Group in York:
presentation,
presentation with
notes.
Report by Rodney Cummings from the presentation
can be found here.
AVB Gen1
The Audio Video Bridging (Gen1) technology defines mechanisms (IEEE 802.1BA) which allows to recognizes all the AVB-compatible devices in the network. Basically, AVB works only within a domain of directly connected AVB-compatible network devices (i.e. bridges or endpoints). These devices support precise synchronization (IEEE 802.1AS) and enable reservation of their resources (IEEE 802.1Qat) for a data stream. This way, and using special traffic shaping (IEEE 802.1Qav), deterministic and low latency communication (stream) between AVB-compatible endpoints though AVB-compatible network is achieved.
IEEE 802.1AS: Timing and Synchronization for Time-Sensitive Applications (gPTP):
- very tightly-constrained subset of PTP (which, except for Ethernet, supports wireless and Coordinated Shared Networks)
- it is based on, and includes a profile of, PTP (i.e. IEEE 1588-2008)
- topology/network-functionality-wise: an IEEE 802.1AS bridge acts as an PTP boundary clock, and an IEEE 802.1AS end station acts as an ordinary clock [1]
- synchronization-wise: the manner in which an IEEE 802.1AS bridge transports synchronization is very similar (mathematically equivalent) to the manner in which an PTP peer-to-peer transparent clock (TC) transports synchronization [1]
- each bridge measures frequency offset relative to its neighbors; the accumulated frequency offset relative to the GM and the time difference between the arrival of a Sync message on the slave port and the sending of a subsequent Sync message on a master port are used to construct the synchronized time placed in a Follow_Up message [1]
- IEEE 802.1AS defines TLV which is attached to Follow_UP message, this TLV conveys frequency offset, relative to the grandmaster, it is accumulated (added to by each bridge on the way)
- propagation delays between neighboring bridges and/or end stations are measured using the peer delay mechanism [1]
- the bridge is not required to filter phase as part of transporting synchronization; all phase filtering occurs at endpoints (end stations or bridges that have service interfaces to applications) [1]
- it defines modified BMC (but similar to original)
- synchronization spanning tree may be different from the forwarding spanning tree (i.e. RSTP) [2]
- no synchronization over non 802.1AS bridges, not inter-operable with default PTP
- only peer-delay two-step mechanism allowed, only single domain
- defines application interfaces
- IEEE 802.1AS devices are logically syntonized (time-aware Bridges can correct time interval measurements using the grandmaster frequency ratio conveyed in Follow_UP's TLV, p22 of [2]). Optional features [1]: IEEE 802.1AS bridges and end stations are not required to physically syntonize (using frequency ratio) their frequency to the GM frequency (though they are be allowed to do this).
- performance requirement, Annex B of [2]:
- synchronization over seven or fewer hops to within 1us peak-to-peak of each other during steady-state operation,B.3 of [2]
- jitter and wander defined in B.4 of [2], page 250
- references: 1, 2
IEEE 802.1Qat: Stream Reservation Protocol (SRP),
- it utilizes three signaling protocols, MMRP, MVRP and MSRP to
establish stream reservations (register stream and reserve
resources) across a bridged network between two end stations (talker
and listener)
- Talkers (source) initiates stream by sending an SRP talker
advertise message which include:
- Stream ID (source MAC, talker-specific unique ID and destination MAC)
- QoS requirements (e.g., AVB traffic class and data rate information),
- accumulated worst case latency (recalculated at each bridge)
- Listener (destination) acknowledges by replaying with listener ready message
- Talkers (source) initiates stream by sending an SRP talker
advertise message which include:
- the signaling protocols:
-
Multiple MAC Registration Protocol (MMRP) is optionally used to control the propagation of the Talker's registrations throughout the bridged network (clause 10.9 of [1])
-
Multiple VLAN Registratin Protocol (MVRP) - used by end stations and bridges to declare membership in a VLAN where a stream is being sourced (clause 11 of [1])
-
Multiple Stream Registration Protocol (MSRP) - a signaling protocol that provides end stations (talker and listener) with the ability to reserve network resources (ensure Quality of Service), between end stations (clause 35.1 of [1])
-
- stream can be de-registered by either Talker or Listener
- references 1
IEEE 802.1Qav: Forwarding and Queuing for Time-Sensitive Streams (FQTSS)
- it defines constraints for mapping of priorities into traffic
classes
- AVB traffic is transmitted/forwarded using credit-based shaper algorithm
- AVB traffic has higher priority then traffic supporting strict priority, or other transmission algorithm (e.g. non-AVB traffic)
- it specifies credit-based shaper algorithm
- frames distributed evenly in time (only on an aggregate class basis), prevents "bunching" of frames
- effect of smoothing out the devliery times
- references: 1
IEEE 802.1BA: Audio Video Bridging Systems
- specifies the default configuration of AVB devices in a network
- for Ethernet, the method specified by 802.1BA to determine if its peer is AVB-capable is a combination of 802.3 link capabilities (determined during Ethernet link establishment) and the link delay measurements done by IEEE 802.1AS
AVB Gen2 (AVBg2)
The AVB Gen2 is developed to further enhance AVB's performance. The
requirements for AVB Gen2 are defined not only by Audio/Video industry
but others as well (described below). In order to meet the requirements,
the mechanisms defined in AVB Gen 1 are improved and new solutions are
investigated.
Improvements to AVB Gen 1:
- P802.1ASbt: Timing and Synchronization for Time-Sensitive Applications
- P802.1Qbv : Enhancements for Scheduled Traffic
New ideas:
- P802.1Qbu : Frame Preemption
- Static/dynamic redundancy (potentially included into AVB Gen2)
Requirements for AVB Gen 2
- Network reliability:
- support of quick (must be predictable and pre-calculable)
network recovery (<100ms Toyota,
page 6)
and seamless redundancy with zero time recovery (General
Motors),
this includes
- communication
- synchronization (802.1AS)
- reservation of stream (802.1Qat)
- support of fault isolation/fault tolerance (e.g.: "babbling idiot" and "fault tolerant clock sync", General Motors, pages 36)
- any mechanism that supports reliability must not break predictability/determinism/low latency
- support of quick (must be predictable and pre-calculable)
network recovery (<100ms Toyota,
page 6)
and seamless redundancy with zero time recovery (General
Motors),
this includes
- Ultra-low latency for critical traffic
- Toyota
(page 2)
- maximum latency: 100us over 5 AVB bridge hops @ 100Mb or 1Gbps
- guaranteed and topology independent latency
- main network characteristics: max 32 devices (switches+nodes), links of 24m, max network span: 30m
- traffic characteristics: "control data" size (payload): ~256 bytes, max of 32 "control streams" sent every 500us, "normal data" size (payload): ~1500 bytes
- Siemens
(page 14)
- max latency / hop < 3us
- topology (quite) independent latency (required topology support: Daisy Chain / Comb / Ring)
- main network characteristics: max 512 devices, max hps: 64
- traffic characteristics: "control data" size (payload): typical 10-300 bytes, more possible, max of 4096 streams sent every 31.25us-1ms
- Marvell
(page 2)
- max latency / hop < 5us or <15us
- topology: Daisy Chain
- main network characteristics, max hps: 64
- traffic characteristics: "control data" size (payload): typical <300 bytes, small burst of frames at known regular intervals (e.g. a 40us long burst every 125 us)
- Toyota
(page 2)
- new AVB Gen2 features should be optional General Motors, pages 43)
P802.1ASbt: Timing and Synchronization for Time-Sensitive Applications
The enhancements to current IEEE 802.1AS include details:
- support for Link Aggregation, no details specified yet, see page 5
- support for redundant paths, no details specified yet, see page 9
- improved performance (e.g., improved grandmaster changeover time, longer chains of time-aware systems), no details specified yet, see page 12
- management support for automatic measurement of link delay asymmetry
- proposed to exchange Tx and Rx (manually or automatically), see
P802.1Qbv : Enhancements for Scheduled Traffic - Time Aware Shaper
- goal of improvements: provide lower network delays for time-sensitive data
- defines ways for bridges and end stations to schedule the transmission of frames based on timing derived from IEEE Std 802.1AS
- Time Aware (Blocking/De-blocking) Shaper in bridges and end stations, see pages 30
- creates a time window within which only time-sensitive traffic is sent - enables to ensure that time-sensitive traffic does not interfere with best-effort traffic (the former does not need to wait for the latter to be finished sending)
- it seems that it is most efficient with preemption
(page 15)
- preemption enables to decrease the bandwidth overhead of introducing time-window for critical data (page 11)
P802.1Qbu : Frame Preemption
- goal: provide lower network delays
- allows very time-sensitive packets to interrupt a normal best-effort packet being transmitted on an egress port and then resume it once the time-sensitive packet has been transmitted (pages 30)
- most efficient when used with Time Aware Shaper (would be defined in P802.Qbv), page 15 and page 11
Possible effort on new topology resolution (probably enhancement of IEEE 802.1Qat-2010)
- aims at providing multiple paths for both redundancy and (less importantly) enhanced network throughput
- Network redundancy - actually two different types of redundancy are
required
- static redundancy - redundant paths from the source to the receiver exist and are used at any time - duplication of frames must be handled by the receiver (provides seamless redundancy)
- dynamic redundancy - redundant paths from the source to the receiver exist but only one is used at a time - no duplication at the receiver
- proposed solutions
- General Motors: redundancy of physical or logical (VLAN) network or ring topology (like HSR), see
- Toyota: quick version of RSTP+802.1AS+802.1Qat (< 100ms), page 6
- Broadcom: abandon RSTP, improved MSRP (802.1AS defines SRP, 802.1Q-2011 defines multi-SRP) to provide multi-path support (all the potential paths from the talker to the listener are discovered, one or more paths can be used -- listener can request redundant stream through different paths) here
- Cisco: simultaneous delivery of data through multiple paths
(here),
creating disjoint paths using:
- VLANs: Shortest Path Bridging or Spanning Tree Protocol inside a VLAN, one VLAN per path
- SPB+SRP: use Shortest Path Bridging to identify different paths, then Stream Reservation Protocol
White Rabbit
Requrements
- Ultra-low latency for critical traffic
- CERN
- maximum latency: below 1000us over ~5 bridge hops @ 1Gbps (page 10)
- guaranteed latency over tree-like (meshed) topology (page 19)
- main network characteristics: ~2000 end stations, links of max 10km, max network span: 10km
- traffic characteristics: "control data" size (payload): 500-5000 bytes (in FEC scheme "[1]":https://www.ohwr.org/project/white-rabbit/uploads/ff98688542afbe36e9f11877ba4c5f1a/ICALEPCS2011_poster.pdf "[2]":https://www.ohwr.org/project/wr-cores/wikis/FEC : encoded into 4 or 6 frames of size 300-1500 bytes and sent in burst), ~8 "control streams" (defined within separate VLANs, see page 7) sent every 1000us, critical stream is one-to-many, "normal data" size (payload): ~1500 bytes
- GSI (not necessarily up-to-date)
- maximum latency: 100us over ~4 bridge hops @ 1Gbps
- guaranteed latency over tree-like (meshed) topology
- main network characteristics: ~2000 end stations, links of max 2km, max network span: 2km
- traffic characteristics: "control data" size (payload): 200-500 bytes (in FEC scheme "[1]":https://www.ohwr.org/project/white-rabbit/uploads/ff98688542afbe36e9f11877ba4c5f1a/ICALEPCS2011_poster.pdf "[2]":https://www.ohwr.org/project/wr-cores/wikis/FEC : encoded into 4 frames of size 150-300 bytes and sent in burst), sent every 100us, critical stream is one-to-many, "normal data" size (payload): ~1500 bytes
- CERN
- Synchronization:
- accuracy: sub-nanosecond in the entire network
- precision: order of picoseconds in the entire network
- Reliability: seamless redundancy (synchronization-wise and critical data-wise) which should ensure high availability
Timing and Synchronization
- profile/extension of PTP: WRPTP
- provides sub-nanosecond accuracy only over WRPTP-capable devices,
- interoperability with standard PTP -- hybrid networks possible
- requires physical syntonization using Synchronous Ethernet
- uses phase detection (DDMTD) to enhance HW-timestamping precision to picosecond level
- auto-calibration of link asymmetry (single fiber and DDMTD phase detection)
- delay-request two-step mechanism
- defines modified BMC
- synchronization spanning tree may be different from the forwarding spanning tree (i.e. RSTP)
Reliability
- optional support of network/data redundancy
- prevention of critical data loss due to data corruption (e.g.: bit error) is handled by additional layer: Forward Error Correction (FEC, "[1]":https://www.ohwr.org/project/white-rabbit/uploads/ff98688542afbe36e9f11877ba4c5f1a/ICALEPCS2011_poster.pdf "[2]":https://www.ohwr.org/project/wr-cores/wikis/FEC)
- the seamless redundancy is based on the Forward Error Correction
layer - critical data sent over WR network is encoded into several
frames (i.e. 4), only subset of the frames (i.e. 2) is needed by the
receiver (AVB's Listener) to recover the original critical data.
- this allows to use dynamic network redundancy (e.g. enhanced RSTP) as long as the reconfiguration time is predictable and sufficiently small (i.e. so that maximum 2 frames are lost during reconfiguration)
- two ideas are considered for dynamic network redundancy
mechanims:
- enhanced Spanning Tree Protocol which allows fast enough reconfiguration
- enhanced Link Aggregation which enables to send frames which constitute the critical data through independent paths
- any idea needs to support VLANs
- fault tolerance / isolation
- VLANs will be used to isolate different logical (accelerator) network and limit a propagation of fault due to a mis-behaving node/switch
- FEC header has ID and sequence number - can prevent help in duplication issues
- control of throughput from nodes which are not supposed to send too much data (if any)
Latency and scheduling:
- FEC overhead needs to be taken into account -- the fact that "single critical data" translates into many Ethernet Frames
- preemption was considered but is currently stated an obsolete idea
- strict priority scheduling of output queues (Class of Service) for critical data, static resource reservation for critical data and strict control of critical data transmission are considered sufficient for CERN
- cut-through forwarding in the switches
AVBg2 vs. WR
Both solutions, WR and AVB, work only over WR/AVB-compatible devices but provide interoperability with standard Ethernet.
Synchronization-wise
The main difference (between WR and AVBg2) is the usage of SyncE. AVBg2
bridges are logically syntonized. WR requires all WR-devices to be
syntonized which is an option in AVB. The efforts to support network
redundancy and speed-up reconfiguration seem to be inline in WR and AVB.
There is an apparent need in AVBg2 to find a solution for
(semi)automatic link asymmetry compensation. Although WR provides
solution for this, it does not need to be favored by AVBg2. Unlike AVB
(which uses pDelays), WR uses request-response mechanism - there is
no technical problem (just work overhead) in extending WR onto
peer-delay mechanism.
Critical-data-wise
In short, WR is a corner case of AVBg2 where all the end stations in a logic network (VLAN) register to the same stream, thus a stream in WR is defined by VLAN. WR adds sophisticated Forward Error Correction (FEC), at a cost of latency deterioration, to spare network components (switches) and still provide undisturbed critical data stream in case of component failure (seamless redundancy over dynamically redundant network). This is because WR is foreseen for large scale networks with one-to-(really)-many end stations communication.
WR does not foresee dynamic stream reservation -- it is assumed that there are two kinds of data: (1) critical (so called Control Messages, CMs) and (2) non-critical (Best-effort, BE). It is again, a corner case of AVBg2 where there would be Class A stream (critical data) and best-efford data. All critical data is treated equally and share the same resources which are allocated to it (a very limited number of CM streams is foreseen). If I'm not mistaken, the distribution characteristics of the CMs in WR is different then usual streams in AVB. In WR, the critical data stream is supposed to be broadcast within VLAN, thus a stream is propagated to all the end stations within a VLAN: a particular case of such solution is to define VLAN for a stream between two end stations. Still, separate VLAN-streams do not have separate resources. On the other hand, WR critical data one-to-many stream seems to be a corner case of AVB stream: all the devices in the AVB network register to a single stream.
In WR, the critical data is encoded using Forward Error Correction to
prevent losses due to data corruption (bit error rate/switch
malfunction) and to allow dynamic network redundancy (in the enhanced
Spanning Tree idea) while providing undisturbed critical data stream. In
the enhanced Link Aggregation a kind-of multipath streaming is
established.
In AVBg2 simplicity prevails:
- if a network can allow dynamic network reconfiguration (e.g. requirement of reconfiguration time < 100ms, Toyota, page 6), it means that the end applications can accept some data loss. Therefore, some data loss due to data corruption is acceptable.
- if seamless redundancy is required, static network redundancy (independent paths) is needed. Therefore FEC is not needed.
WR takes advantage of assumed one-to-many characteristics of the
critical data in :
ultra-fast forwarding: for critical data the forwarding database needs
to be verified only against VLAN which is much faster then MAC lookup
- redundancy mechanisms: data is always forwarded everywhere except links which create loops
- no need for sophisticated stream reservation
Using FEC has a negative effect on the data delivery latency (probably not acceptable by the most stringent AVBg2 requirements), however:
- in the idea where enhanced STP (eSTP) and FEC are used to provide seamless critical data stream over dynamically redundant network - FEC and eSTP are decoupled
- the idea where enhanced LA (eLA) and FEC are used (LA and FEC are not decoupled) -- the enhancement to LA is very similar to static redundancy provided by AVBg2 (many different ways proposed: SPB, VLANs). Therefore, instead of eLA, AVBg2's idea could be used and FEC on top of this.
The idea of Time Aware Shaper (both at end stations and WR switches) is worth considering for WR, especially to meet GSI's requirements.
In order to align WR and AVBg2 the following would be needed (to my best assessment and knowledge):
- requirements to be added to AVB:
- syncE + WR asymmetry evaluation mechanism - so that sub-nanosecond synchronization could be achieved
- broadcast stream handling for large scale networks (min 2000 end stations)
- bridge number-efficient seamless redundancy or ultra-fast network reconfiguration (< few microseconds) so the FEC layer can be used on top
- changes to WR:
- support of peer-delay mechanism in WRPTP
- implementation of dynamic stream reservation (very heavy implementation-wise)