1

White PaperNorthforge Innovations Inc.

December 2018

FPGAs in Network Equipment FPGAs as the Focus

2

This is the third and final whitepaper in our series about using FPGAs in the design of Networking Equipment.

In the first paper, we introduced the three types of components that are commonly used to provide the functional elements of the equipment: CPUs, ASICs, and FPGAs. In the second whitepaper, we presented a couple of use cases in which FPGAs are used as peripheral/coprocessor devices for the primary functional elements of the systems. In this whitepaper, we present a couple of examples of systems in which an FPGA is the central design element in the system.

Introduction

Why use an FPGA?

FPGAs have the predictable high performance of an ASIC and the flexibility of CPUs. CPUs are a great solution because software is easy to update and CPUs are widely available in broad ranges of performance, interfaces, and added features. But some applications can’t be addressed by a CPU. For example, functions that require very high performance might not be suitable for a CPU, either because a CPU with the necessary performance might be very expensive, require too much power, or might not exist.

Similarly, CPUs can’t implement functions that require very tight or predictable performance. If a function must happen at a specific time with just a few microseconds of tolerance, a CPU won’t solve the problem. Interrupt latency on most CPUs is more than a few microseconds, and that’s assuming that the interrupt is the highest priority or that no other interrupts are in progress.

In these cases, a hardware solution is a more appropriate choice. A custom ASIC provides the lowest unit cost (ignoring development costs), and the most tailored solution from a functionality, power, and performance point of view. However, ASIC development costs reach millions of dollars. If an ASIC needs to have slightly different functionality or has a bug, it has to be respun, which increases the expense. So, this solution is only appropriate for devices where the demand will be many thousands or tens of thousands of units, and the necessary functionality won’t change.

A commercial off-the-shelf (COTS) ASIC is an alternative if one is available. But these devices are often designed to address a broad set of applications and functions to have a large enough market to justify the development expense. Although they might solve the problem at hand, they can be larger, more power hungry, and often more expensive than is needed for the application.

An FPGA is often a better solution that strikes the right balance between performance, design flexibility, and power consumption. It is a hardware device that can, as with an ASIC, have very high, predictable performance. Unlike an ASIC, the internal design and functionality of the device is determined by loading a “program” into it, so it shares much of the malleability of software running on a CPU. This means that fixing bugs and updating functionality can be done with changes to the program, without having to change the chip. Also, unlike an ASIC, which is designed for a particular application, the FPGA device itself is designed to address a wide variety of customers’ needs. The FPGA producer can therefore spread the development cost of the chip across a large number of units and customers.

One additional benefit of modern FPGAs is that they frequently include CPU cores on-chip. This increases the applicability of the device by supporting both the functionality that is best done in software and functionality that is best done in hardware on a single device!

Below, we present two applications where an FPGA is the core functional element in the design. Northforge engineers have contributed to the development of both.

3

In a Passive Optical Network (PON), the headend distribution device is called an Optical Line Terminal (OLT). Traditionally, OLTs have been large devices located at the Central Office (CO), supporting a large number of PON interfaces, but this approach has a few inherent problems. As with most large devices, scaling is a problem. If a Central Office OLT supports, say 32 PON interfaces, adding the 33rd interface is very expensive. More importantly, PON has distance limitations (20km maximum), and as PON customers expand farther and farther from the CO, these limitations become an issue. For more in-depth perspective on Passive Optical Networks, download our three-part series on PON at https://gonorthforge.com/resources/new_white-papers/.

Figure 1. Typical PON Deployment

Figure 2. Remote OLT Deployment

Remote PON OLT

One solution to both of these issues is the Remote OLT. A Remote OLT implements most of the OLT functionality in a small box that supports a smaller number of PON interfaces (e.g., 4). The Remote OLT can be placed near the customers, on poles or in pedestals, and connected back to the CO using Ethernet over DWDM which can reach up to 80km at 10Gbps. This approach resolves both the scaling and distance problems.

Central Office

Remote OLTs

ServicesEthernetSwitch Router

TrafficMgmt

PONMAC

10 G EthOptics

40 km or more ONU

ONU

ONU

ONU

ONU

ONU

ONU

ONU

WD

M

WD

M 10 G EthOptics

PONOptics

TrafficMgmt

PONMAC

10 G EthOptics

PONOptics

Standard POM Wavelengths 5-10 km

OLT

Passive OpticalSplitter

ONUONUONUONUONU

10km

4X8=32

4X16=64 16 Premises

ONUONUONUONUONU

8 Premises

OLT

20km

1:16Splitter

1:4Splitter

1:8Splitter

4

A Remote OLT comes with its own set of challenges. It is outside of the environmentally controlled environment of the Central Office. Therefore, its power dissipation (i.e., heat) has to be limited. But power not only means heat, it means performance. The OLT has to move several Gbps of data and also implement PON management and control protocols. It has to measure latency to PON endpoints with a high degree of accuracy to implement “ranging” to allocate transmit slots for the clients. All of this requires processing performance and a high level of predictability.

This is an ideal application for a modern FPGA. Consider the Xilinx Zynq Ultrascale+ series of FPGAs1. In addition to the programmable logic that is the hallmark of an FPGA (over half a million programmable logic cells), these devices include:

• An ARM System on a Chip (SOC) (actually, two) that can be used to support management of the device, configuration of the control plane for user data, and implementation of the protocol procedures for PON control.

• On-chip memory for heavily accessed/time critical data-structures (≈32Mb)

• Circuitry to support timers and clocking

• Integrated temperature and voltage monitors

• DDR interfaces to support external memory for CPU use and data buffers

• SPI interfaces for connection to PON MAC/PHY devices such as the Broadcom Maple2

1 https://www.xilinx.com/products/silicon-devices/soc/zynq-ultrascale-mpsoc.html2 https://www.broadcom.com/products/broadband/xpon/bcm68620

43

2

ARM®

CortexTM -A53

32KBI-Cachew/Party

D-Cachew/ECC

MemoryManagement

Unit

EmbeddedTrace

Macrocell

NEONTM

Floating Point Unit

Application Processing Unit

Processing System

Programmable Logic

Graphics Processing UnitARM MaliTM-400 MP2

Memory

1

1MB L2 w/ECCGIC-400 SCU CCI/SMMU

2

ARM®

CortexTM -R5

128KBTCM w/ECC

Vector Floating Point Unit

Memory Protection Unit

Real-Time Processing Unit PlatformManagement Unit

1

GIC

32KB I-Cachew/ECC

32KB D-Cachew/ECC

DDR4/3/3LLPDDR4/3

32/64 bit w/ECC

High-SpeedConnectivity

DisplayPort v1.2a

USB 3.0

General Connectivity

GigE

USB 2.0

CAN

UART

SPI

Quad SPI NOR

NAND

SD/eMMC

SATA 3.1

PCIe® 1.0/2.0

PS-GTR

256KB OCMwith ECC

SystemManagement

GeometryProcessor

Memory Management Unit

64KB L2 Cache

PixelProcessor

PowerManagement

FuntionalSafty

Configuration andSecurity Unit

Config AESDecryption

AuthenticationSecure Boot

TrustZone

Block RAM

Block RAM

Ultra RAM

DSP PCle Gen4

100G EMAC

GTY

GTH

Interlaken

High-Performance HP I/O

High-Density HD I/O

Voltage/TempMonitor

SystemFunctions

MultichannelDMA

TimersWDT, Resets,Clocking and

Debug

12

Storage and Signal Processing

General-Purpose I/O

High-Speed Connectivity

Figure 3. Block Diagram of Xilinx Zynq Ultrascale+ FPGA

5

Remote Cable PHY

This device, along with external memory and network interface components, represents most of the circuitry needed to implement the remote OLT resulting in a design that is simple and has lower power consumption (the only power-hungry, high-speed buses are memory and network interfaces). This series of FPGAs has several performance grades, and the system designer can select the one that best matches the functionality, performance, price, and power requirements of the system.

The Cable TV/Internet infrastructure has a requirement for a similar type of solution to the Remote PON OLT. As with most network implementations, the cable plant has undergone many changes over the years. Until the early 1990s coaxial cable was used exclusively—coax from each neighborhood ran back to the cable headend. This had two primary problems—scalability and distance. As cable TV became more popular, it was available in more and more areas, and having all of those individual cables “home run” back to the headend became a scaling problem. Equally problematic is that signals are attenuated rapidly on copper, limiting the distance that a subscriber could be from the headend.

Hybrid Fiber Coax (HFC) was introduced in the early 1990s to address these issues. With HFC, the RF signals for TV and DOCSIS3 are sent over fiber optic cable to a fiber node. At the fiber node, the signal is split and distributed over several coaxial cables to cover a large neighborhood or several smaller neighborhoods. HFC addressed both the scaling problem and the distance problem (since fiber has much lower signal attenuation than copper, the fiber node could be much farther from the head end).

Analog Video (if used)

Digital Video (Broadcast)

Digital Video (on Demand)

Internet/DOCSIS

Other Services

Optical Transmit and Receive

RF

Hybrid Fiber Coax Transport

(HFC)

Amplitude Modulation

Coax Coax

Cable ModemDOCSIS

Set TopBox

Figure 4. HFC Architecture

The HFC solution distributes RF (analog) signals over the fiber. In today’s world of digital video and digital data, this is a limited solution. Any replacement can’t change the basic Frequency Division Multiplexing (FDM) structure of the coax since that would require replacing every Cable Modem and Set Top Box. But, distributing digital information to the fiber node and modulating the streams onto the appropriate channels at the node can provide higher bandwidth, higher information/signal fidelity, and additional features, in addition to increased distance. This is often referred to as Digital Fiber Coax (or DFC). With DFC, the headend connects to the fiber node with standard digital optical transport techniques such as Ethernet or OTN (ITU-T G.709).

DFC requires more work at the fiber node. Instead of just an optical to electrical conversion, DFC requires the remote device to implement various management and control protocols both for video distribution and for DOCSIS. Also, it is now the fiber node’s job to modulate (and demodulate in the upstream direction) the digital data for distribution over coaxial cable. This device and its related functionality is referred to as a “Remote PHY”.

3 Data Over Cable Service Interface Specification. This is the protocol used to provide Internet access over cable TV infrastructure.

6

Figure 5. DFC Architecture

Figure 6. Remote PHY Block Diagram (simplified)

Remote PHYConverged Cable

Access Platform (CCAP)

DigitalOptics

MAC(Downstream)

MAC(Upstream)

MAC(Downstream)

MAC(Upstream)

Optical Transport(Ethernet, PON, OTN)

DigitalOptics

DAC

ADC

Digital Video (Broadcast)

Digital Video (on Demand)

Internet/DOCSIS

Other Services

Digital Fiber Coax Transport

(DFC)

Coax Coax

Cable ModemDOCSIS

Set TopBox

As with the Remote OLT discussed above, this is an ideal situation for an FPGA-based implementation with many of the same requirements. A CPU is needed for implementation of the protocols for control and management. Programmable logic is needed to implement several of the high-performance data distribution algorithms. High-speed interfaces are needed to connect to the fiber upstream interfaces. This application requires access modules to frequency modulate the digital data and provide the electrical interface for the coaxial cable. As with the remote OLT, this application is in a location that is not environmentally-controlled so it must be small and require little power.

The resulting system block diagram would look something like this:

System Control and Status Interfaces

Ethernet Console

LEDs USB

coax

MemoryPowerSupply

AccessModule(s)

Memory

Optical Interfaces

CPU-related components

Programmable Logic

FPGA (e.g. Xilinx Sync-Series

Northforge Engineers are Experts in Network Device Implementation

Do you need to implement an FPGA as the central design element in your network but need help? That’s where Northforge Innovations comes into the picture. Our engineers are experts in developing software for networking devices, especially in implementing a wide variety of protocols and capabilities including switching, security, routing, mobile networking, and virtualization.

We can assist with FPGA implementations for your networking devices using VHDL and Verilog and work with you to implement complete solutions. We’ve assisted with the implementation of both of the devices described in this whitepaper, a Remote PON OLT, and a Remote Cable PHY, including the FPGA development and the software development in the ARM SOCs.

About Northforge Innovations Inc.

Northforge Innovations is an expert software consulting and development company focused on advancing network communications. We target network security, network infrastructure, and media services, with the mission and passion to meet the industry’s demands in the evolving cloud infrastructure, virtualization and software-defined networking.

With an average of 15 years of experience, our consultants comprise a worldwide resource pool that’s based in North America. Northforge employs top technical and project management talent to give customers the “intellectual capital” they need for their network communications software development. Our developers have extensive technical and domain expertise across a breadth of technologies. With expertise extending beyond software development services, our team tackles our customer’s most demanding challenges and delivers innovative solutions. Our culture stresses innovation at every step, from our ability to understand and address our customers’ needs, our constant exchange of innovative ideas to the continuous value that we create for our customers.

For more information about Northforge Innovations Inc., please visit www.gonorthforge.com.

Copyright 2018 - Northforge Innovations Inc.

NORTHFORGE INNOVATIONS INC.

GATINEAU DEVELOPMENT CENTER(Development Center)72 Laval Street, 3rd LevelGatineau (QC) J8X 3H3

MONTREAL DEVELOPMENT CENTER1130 rue Sherbrooke Ouest, Suite 430Montréal, QC H3A 2M8

NORTHFORGE INNOVATIONS ISRAEL LTD.10 Zarchin St.Ra’anana 4366238PO Box 4318Israel

USA OFFICE(Sales Office)One Boston Place, Suite 2600Boston, MA 02108

General Inquiries 819.776.6066Consulting Inquiries 781.897.1727

info@northforgeinc.comwww.gonorthforge.com