MPEG2 Video Decoder

This page describes an MPEG2 video decoder, written in Verilog and implemented in an FPGA chip.

Introduction

Video decoding is very resource-consuming, and puts hard real-time requirements upon software. Implementing an MPEG2 decoder in an FPGA offloads CPU and memory much like any other hardware decoder, but still allows you to change the design afterwards, much like a software decoder.

FPGA's are typically programmed in Verilog or VHDL. In most programming languages, a program describes an algorithm. A program in Verilog describes a digital circuit. As all gates of a digital circuit work at the same time, all the time, all statements in a Verilog program may be active at the same time, all the time. This parallelism provides high processing speed, even at modest clock frequencies, at the cost of design complexity.

Hardware

Development Board

The MPEG2 decoder has been implemented on the Xilinx ML505 development board with an XC5VLX50T Virtex-5 FPGA.
A PC is connected to the Xilinx ML505 by ethernet. A utility on the PC sends an MPEG2 video elementary stream to the decoder on the ML505. The MPEG2 decoder on the ML505 board receives the MPEG2 stream on the ethernet interface, and outputs the decoded video through the DVI interface. ML505 on-board DRAM is used as frame buffer.

Xilinx ML505 Development Board

The Xilinx ML505 Development board is shown. Connections, clockwise from the left: DVI cable, ethernet cable, USB JTAG cable, power supply, logic analyzer.

Inverse Discrete Cosine Transform

The MPEG2 decompression algorithm uses the inverse discrete cosine transform (IDCT). The IDCT used in the MPEG2 decoder.

Uses 12 18x18 multipliers and two dual-port rams.
Can do streaming.
Has been tested to comply with the former IEEE 1180-1990 standard, the actual ISO/IEC 23002-1.

Streaming Video

To receive the MPEG2 video stream, a minimalist IP protocol stack has been implemented in Verilog. The following services are provided:

ICMP echo request: ping
UDP port 16384: read/write MPEG2 decoder registers
TCP port 16385: decode MPEG2 packetized elementary stream
TCP port 16386: decode MPEG2 elementary stream

The implementation is frugal. The TCP server only accepts one connection at a time, as the MPEG2 decoder only decodes a single video at a time. ARP has not been implemented in the FPGA; a static arp entry on the PC sending MPEG2 video is used instead. The resulting protocol stack is sufficient for testbench use, where sender and receiver are connected to the same LAN.

Code Size

Counting the number of source lines is one way to estimate code complexity. A more accurate measure of code complexity is the number of semicolons, as semicolons are used to separate Verilog statements.

Verilog RTL Code size
Module	Lines	Semicolons	Clock
MPEG2 decoder	22505	9190	75 MHz
DVI output	2047	717	25-90 MHz, variable
Xilinx SGMII ethernet controller	6847	1659	125 MHz
Xilinx MIG DDR2 Memory Controller	5900	1392	200 MHz

The MPEG2 decoder uses approximately 50% of the FPGA; memory, ethernet and DVI controllers take up an additional 15%.

Simulation

Logic simulation encompasses behavioral and timing simulation. Behavioral simulation assumes gates and wires are infinitely fast and do not introduce delay. Behavioral simulation is sufficient to verify algorithms have been correctly implemented. However, behavioral simulation will not be able to tell you what the maximum speed of your circuit is, nor will behavioral simulation find setup or hold time errors.
Timing simulation does take gate and wire delays into account. As such, timing simulation flags setup and hold time errors and shows any glitches in the circuit. The downside is that timing simulation is slower than behavioral simulation.

Behavioral simulation of the MPEG2 decoder can be done using Icarus, an open source Verilog implementation. Simulating one TV-quality frame takes about one hour on a 2.66 GHz Xeon. Simulating one second of TV-quality video takes about one day. Simulation output includes screenshots of the RGB output.

Simulation screenshot of an interlaced TV image.

Both fields of an interlaced TV image are shown. Horizontal sync is displayed as a vertical black stripe, to the right of the image. Vertical sync is displayed as a horizontal black stripe, below the image area. Blanking is displayed in a dark grey.
The screenshot pictures above are scaled for web publishing. The full-size pictures are the raw decoder output in ASCII portable pixmap format.
Icarus Verilog is largely limited to behavioral simulation. For timing simulation of the MPEG2 decoder commercial simulators such as ModelSim are preferable.

Synthesis

Synthesis software such as Xilinx ISE converts the Verilog source code into a bitstream. This bitstream can then be used to configure the FPGA chip. Synthesizing the MPEG2 decoder for the ML505 can take up to about an hour on a 2.66 GHz Xeon, depending upon optimisation options chosen.

Verification and Test

Verification of the MPEG2 core is important. There are three ways to verify correct functioning of the MPEG2 decoder:

Reading the Verilog source and comparing it with the MPEG2 standards.
Simulating or watching conformance test bitstreams.
Comparing Verilog simulation output with a reference MPEG2 decoder.

Note the decoder decodes MPEG2 4:2:0 video elementary streams only.

Standards

The MPEG2 standards are published by two standards institutes: the International Organization for Standardization (ISO) and the International Telecommunication Union (ITU). The ISO MPEG2 standards and the ITU MPEG2 recommendations are identical, except for the cover page. MPEG2 video decoding is described in ISO standard ISO/IEC 13818-2 (MPEG-2 Video), ITU recommendation H.262.

Conformance Tests

ISO/IEC 13818-4 describes conformance testing. The MPEG2 test bitstreams described in Annex C of ISO/IEC 13818-4 can be downloaded from the ISO web site.
Note that, contrary to the standard, the decoder is not MPEG1-compatible. No effort for MPEG1 compatibility has been made, as MPEG1 use seems very much on the decline.

Reference Implementation

The MPEG2 reference implementation is mpeg2codec, written in C. The MPEG2 core can be tested by decoding the bitstream using mpeg2decode with the reference floating-point IDCT implementation and comparing the result with the output of the Verilog simulation of the MPEG2 core.

Demo

A single video is worth a thousand words.

MPEG 2 Decoder Demo. Click on the image for playback.

Conclusion

An MPEG2 video decoder has been developed. The decoder is written in synthesizeable Verilog. A bootable image for the Xilinx ML505 Development Board has been released. Linux and Windows utilities for the MPEG2 decoder are available for download. Video can be streamed to the decoder over TCP/IP using common networking tools.

Download

User manual and documentation

The MPEG-2 Decoder User's Guide explains the decoder in detail.
State transition graph of the variable-length decoding FSM.
MPEG2 decoder dataflow and block diagram.
MPEG2 Decoder demo video.

Binaries for the Xilinx ML505 board

Sysace image for Xilinx ML505

Utilities for linux

Linux tools and sample streams to run the MPEG2 decoder on the Xilinx ML505 board

Utilities for Windows

Windows tools and sample streams to run the MPEG2 decoder on the Xilinx ML505 board
replex for Windows, a Windows command-line tool to convert between MPEG2 transport and program streams

Acknowledgements

Thomas Gebauer for his experiment with mixed cpu/fpga solutions, Michael Hunold for information about trick mode, Claire Billaud for her thesis, Steve Williams for icarus. Klaus Schmidinger for vdr.

Comments welcome.

Last update page: January 14, 2009.

Koen De Vleeschauwer

PROJECTS

LINKS