Low code rate real-time based on NiosII H. 264 video encoder
H. As a new generation video coding standard, the 264 standard is a multi-bitrate video coding standard, also known as the JVT/AVC standard, which can be used for both high-rate HDTV and digital storage systems as well as low-rate real-time communication systems. In the case of the same image quality, H. 264 than H. 263 and MPEG-4 can save 20% to 50% of the code rate. In terms of its basic grade, the complexity of the encoder is H. 10 times about 263. H. 264 Good network affinity and excellent compression performance make it the first choice for video applications, but its huge computational complexity has become the bottleneck of many applications. Based on Nios II, the author designed a coding system with low bit rate real-time application. The system makes full use of the parallel design structure of FPGA, and uses high compression ratio H for video data. 264 standard encoding, can well meet the requirements of low bit rate real-time encoding.
1 H. 264 coding system structure design according to H. The principle and structure of the 264/AVC encoder, while considering the limitations of existing hardware resources and the application requirements of the design, the H shown in Figure 1 is designed. 264/AVC encoding system structure.
This article refers to the address: http://
The video image taken by the camera is first processed by the video capture module, and the image data of the current frame is stored in the SRAM. Then, taking the macroblock MB (macroblock) as a unit, reading the original image from the SRAM, and reading the reference pixels in the reconstructed frame for intra prediction according to the position within the image frame of the MB, and predicting the macroblock and the current The macroblock pixels are poorly correlated to obtain the prediction residual. Next, the residual image is subjected to integer DCT transform or Had-amard transform, and the transform output is quantized. The residual image of the quantized output is generated on the one hand by inverse transform and inverse quantization processing to generate a reconstructed image for intra prediction as a reference, and on the other hand, by reordering and entropy encoding processing to obtain a final image compressed code stream output.
According to H. The 264/AVC standard divides the entire coding system designed into several main parts such as image acquisition, intra prediction, transform quantization, and entropy coding. The processing of each module through the pipeline can effectively improve the execution efficiency of the hardware.
2 Based on Cyclone II FPGA H. The realization system of 264 encoder adopts the design mode of SOPC, which is mainly composed of video acquisition module and Nios II processor system. Altera's DE2 development board is used as a development platform to integrate video capture and Nios II processors into a single SOPC system. The Nios II processor system is responsible for image acquisition control and image H. 264 compression coding work. In order to ensure real-time performance, fully analyze H. After the 264 software algorithm runs, use the custom module pair H. The 264 encoder key algorithm performs hardware acceleration.
2.1 Video Capture Module Video capture is the premise of video image processing and transmission. The quality of digital video images collected will directly affect the video processing results. Figure 2 shows the video capture structure of the image processing system.
Analog Devices' multi-format video decoder chip ADV7181B performs analog-to-digital conversion on the captured video image. The ADV7181B automatically detects baseband video signals such as NTSC, PAL, and SEC0M and converts them into a 16/8-bit compatible CCIR601/CCIR656 format digital video signal based on 4:2:2 sampling; with 6 analog video inputs Port with a single 27 MHz crystal clock input; the user can configure the operating mode of the ADV7181B via a two-wire I2C interface.
When the system is powered up, first configure the internal registers of the ADV7181B using the I2C module. Since the camera outputs a PAL analog video signal, the ADV7181B needs to be configured as a PAL analog video signal input and converted to a CCIR656 format digital video signal. The ADV7181B simultaneously inputs the luminance signal, the chrominance signal (TD_DAT) and the line and field sync signal (TD_HS/VS) of the converted real-time digital video image into the FPGA chip, and extracts the required digital image information through the image acquisition module, and It is dumped into SRAM with 512 KB of storage capacity provided by the AlteraDE2 development board to buffer the image frames to be processed.
The following describes the design and implementation of the image acquisition module.
According to the above analysis of the hardware structure of the video acquisition part, the block diagram of the video acquisition module shown in Figure 3 is designed. It can be seen that the image acquisition module mainly includes image extraction, chroma sampling rate conversion, Y/Cb/Cr image component separation, and image buffer SRAM read/write control.
The image extraction sub-module is in H. Under the control of the video acquisition control information of the 264/AVC encoding module, the required image data is extracted from the PAL digital video image converted and output by the ADV7181B. The actual image size captured by the camera is an interlaced video input signal of 768×576 pixels, in which the base field and the even field are input in time. Since the image size processed by the system is 320×240 pixels, the input digital video needs to be intercepted to meet the processing requirements of the system.
Considering that the difference between the top field and the bottom field data in one frame of image is not large, when the image is intercepted, only 320 consecutive pixels in the middle of the bottom field are extracted for 320 pixels to output 320×240 pixels. Video image data. The specific extraction process is shown in Figure 4.
H. 264/AVC supports the processing of progressive or interlaced digital images with chroma sampling in the 4:2:0 format, so chroma image rate conversion is required for the extracted digital image. By simply averaging the chrominance image components of adjacent odd and even rows, a chrominance sample rate conversion from 4:4:4 to 4:2:0 can be achieved, as shown in FIG.
The image data after the sampling rate conversion needs to be buffered in the SRAM in the SRAM according to the Y/Cb/Cr image type, so as to facilitate the subsequent encoding processing of H_264. Figure 6 shows the effect of the chrominance components of the actual image before and after the sampling rate conversion.
2.2 H. The 264 encoder core module integrates existing hardware resources, real-time performance and implementation difficulty. The design only uses intra prediction. The encoder includes intra prediction module, transform quantization module and CAVLC entropy coding module. In the processing, the macroblock (16×16) is used as the unit, and the luminance and chrominance blocks are respectively subjected to intra prediction, transform quantization and inverse transform inverse quantization, and then CAVLC entropy coding is performed, and the bright color ratio of the image is Y:U:V=4 :2:O.
H. The 264 encoder design was first implemented on a PC using VC++, and later ported to an FPGA using a custom hardware module. The time required for both is listed in Table 1. It can be seen that H is implemented in hardware. 264 compression encoding a frame of image takes only about 16 ms, which is much better than PC implementation, and the hardware module occupies less than 50% of resources, and the cost performance is high.
Since the custom intra prediction hardware module has a larger performance improvement than the software implementation, the hardware structure design of the intra prediction module is mainly analyzed here.
According to H. 264 intra prediction algorithm, the intra prediction module is designed in the non-rate distortion optimization mode. It reads an MB (16×16) luminance and chrominance image data from the SDRAM through the interface module, predicts and predicts the current MB in the luminance and chrominance prediction module, and outputs the prediction residual and the best prediction. Mode; at the same time, the prediction result is added to the residual value after the inverse DCT transform and the inverse quantization, and is reconstructed by the reconstruction module and then written back to the SDRAM. The main structure is shown in Figure 7. The whole module is divided into 4 sub-modules: interface module, brightness prediction, chroma prediction and image reconstruction module.
Four RAMs are designed in the interface module to store the original image read and the reference image data for prediction: RAM0 stores the brightness prediction pixel, depth 32, address 0~15 stores the upper prediction reference pixel, address 16~31 Store the left prediction reference pixel; RAM1 stores the current macroblock luminance original value, the depth is 256; RAM2 stores the chrominance prediction reference pixel, depth 32, address 0 to 7 stores the upper Cb prediction reference pixel, address 8 to 15 stores the left side Cb predicts reference pixels, addresses 16-23 store the upper cr prediction reference pixels, addresses 24-31 store the left Cr prediction reference pixels, and RAM3 stores the current macroblock chrominance original values, with a depth of 128.
The internal structure of the brightness prediction module is shown in Figure 8.
The mode selection module specifies that the current macroblock is predicted in a certain order according to the predicted reference pixel available information (avail) of the current macroblock. If avail=“11†indicates that both the upper side and the left side prediction reference pixel are available, the current macro block is used. The sequence is DC, HOR, VERT, and PLANE. In the residual processing module, two RAM sequences are used to store the prediction residuals of various prediction modes, so the cost function of the current prediction mode and the size of the former prediction mode cost function are compared in the mode selection module. If the cost function of the current prediction mode is small, it indicates that the current prediction mode is superior, and the prediction residual is specified in the residual RAM of the last poor prediction mode when the next mode prediction is performed. After the prediction mode of the current macroblock is predicted, the mode selection module determines the optimal prediction mode according to each mode prediction cost function, and indicates that the prediction mode corresponds to the RAM stored in the residual processing module, and the corresponding residual is Enter the integer transform module.
The prediction module includes implementation entities of four prediction modes: DC, HOR, VERT, and PLANE. The prediction mode determined by the mode selection module reads the predicted reference pixel and the original pixel value from the interface module, and the residual is output to the residual processing after prediction. The module, the predicted value output to the compensation reconstruction module is saved.
The residual processing module uses two RAMs for storing residuals. Each macroblock can perform two kinds of predictions in parallel, and the residuals are saved in two RAMs respectively. The better prediction mode is selected, and the next prediction mode is performed. Compared with the previously selected preferred prediction mode, the best prediction mode is selected until all prediction modes are completed.
4 The prediction cost module calculates the prediction cost of each prediction mode, performs hadamard transformation in units of 4×4 blocks, and performs a hadamard transformation on each 4×4 block DC coefficients after transformation, and performs absolute values ​​on all transformation results. Accumulation is the corresponding predicted cost.
The chroma prediction module structure is basically the same as the luma prediction, except that the chrominance has two components, Cb and Cr, and the residuals are stored slightly in RAM; the chrominance prediction and luminance prediction of the same macroblock are performed in parallel. Since the chrominance data to be processed is less than half of the brightness, the author uses the method of processing the chromaticity first and then processing the brightness in the subsequent integer transformation, which makes the water flow more compact, reduces the waiting time, and improves the running speed of the entire module.
3 Conclusions <br> I designed Nios II based low bit rate real-time H. 264 video coding system, when the system clock frequency is 100 MHz, it takes 16.283 ms to compress a frame of 320×240 color images. When the quantization parameter is selected 30, the image compression ratio reaches 2%, and the real-time monitoring image frame rate is 25 frames/ s. The system has the characteristics of less resource occupation, low cost, low bit rate and high-definition video quality, and has a good development prospect.
Figure 9 shows the resource occupancy of the system after integrated simulation in an integrated development environment.
According to the frequency, the Adapter has low frequency power adapter and high frequency Power Supply. The linear adapter is low frequency adapter.
Linear dc power supply is working through the transformer to low the voltage of AC current, and then obtain DC current through the rectifier circuit, and finally obtain a small ripple voltage through filter.
Yidashun can offer different specifications` linear psu such as 15V 200mA, 22V 200mA etc, and Yidashun's linear power adapter has the advantages of high stability, small ripples, and less interference and noise and so on. OEM & ODM linear adapters are accepted by yidashun.
Linear Adapter,Linear Power Supply,Linear Universal Adapter,Linear Power Adapter
Shenzhen Yidashun Technology Co., Ltd. , https://www.ydsadapter.com