Introduction to H.264: (2) SODB vs RBSP vs EBSP
Pages: 1 2
Here is a summary of the relationship between SODB, RBSP, EBSP, NALU and H.264 Byte Stream.
SODB: String of Data Bits
RBSP: Raw Byte Sequence Payload
EBSP: Encapsulate Byte Sequence Payload
SODB + RBSP Stop bit + 0 bit(s) => RBSP
RBSP part 1 + 0x03 + RBSP part 2 + 0x03 + … + RBSP part n => EBSP
In Byte Stream Format (Annex B),
Start Code + NALU + … + Start Code + NALU => H.264 Byte Stream
String of Data Bits (SODB)
Definition:
3.149 string of data bits (SODB): A sequence of some number of bits representing syntax elements present within a raw byte sequence payload prior to the raw byte sequence payload stop bit. Within an SODB, the left-most bit is considered to be the first and most significant bit, and the right-most bit is considered to be the last and least significant bit.
To form a NALU, we need the raw data and we call it String of Data Bits. It is a bitstream of coded raw H.264 data stream and defined by MPEG Part 10 (Free Download). So, it does not include the 3-byte / 4-byte start code, 0x000001 or 0x00000001 and not include the 1-byte NALU header [1] which contains some basic information of the NALU such as NAL unit type.
Raw Byte Sequence Payload (RBSP)
Definition:
3.118 raw byte sequence payload (RBSP): A syntax structure containing an integer number of bytes that is encapsulated in a NAL unit. An RBSP is either empty or has the form of a SODB containing syntax elements followed by an RBSP stop bit and followed by 0 or more subsequent bits equal to 0.
3.119 RBSP stop bit: A bit equal to 1 present within a RBSP after a SODB. The location of the end of the SODB within an RBSP can be identified by searching from the end of the RBSP for the RBSP stop bit, which is the last non-zero bit in the RBSP.
After getting the SODB, a rbsp trailing bit equals to 1 is added to indicate the end of the SODB so the bit is called stop bit (rbsp_stop_one_bit). Also, RBSP is always byte aligned and some 0 bits are added after the stop bit to make it 8-bit aligned.
In short, two things are added to the end of SODB in order to form a RBSP:
- rbsp_stop_one_bit = 1
- rbsp_alignment_zero_bit(s) = 0(s) [[Must make the SODB byte-aligned]]
RBSP
Encapsulate Byte Sequence Payload (EBSP)
I mentioned that we need to prefix the NAL Units with the start code 0x000001 / 0x00000001 in byte stream format in order to identify their boundaries [1]. Yet, there may exist the pattern 0x00000X, where X is 0, 1, 2 or 3, in the RBSP and have emulation of start code. To ease the decoding process, a emulation_prevention_three_byte (0x03) is added after 0x0000 and it becomes 0x0000030X:
- 0x000000 => 0x00000300
- 0x000001 => 0x00000301
- 0x000002 => 0x00000302
- 0x000003 => 0x00000303
Since emulation_prevention_three_byte doesn’t affect the decoding result, the decoder should ignored it.
Form NAL Unit from SODB, RBSP & EBSP
7.4.1.1 Encapsulation of an SODB within an RBSP (informative)
The form of encapsulation of an SODB within an RBSP and the use of the emulation_prevention_three_byte for encapsulation of an RBSP within a NAL unit is specified for the following purposes:
– to prevent the emulation of start codes within NAL units while allowing any arbitrary SODB to be represented within a NAL unit,
– to enable identification of the end of the SODB within the NAL unit by searching the RBSP for the rbsp_stop_one_bit starting at the end of the RBSP,
– to enable a NAL unit to have a size larger than that of the SODB under some circumstances (using one or more cabac_zero_word).
The encoder can produce a NAL unit from an RBSP by the following procedure:
1. The RBSP data is searched for byte-aligned bits of the following binary patterns:
‘00000000 00000000 000000xx’ (where xx represents any 2 bit pattern: 00, 01, 10, or 11), and a byte equal to 0x03 is inserted to replace these bit patterns with the patterns:
‘00000000 00000000 00000011 000000xx’, and finally, when the last byte of the RBSP data is equal to 0x00 (which can only occur when the RBSP ends in a cabac_zero_word), a final byte equal to 0x03 is appended to the end of the data. The last zero byte of a byte-aligned three-byte sequence 0x000000 in the RBSP (which is replaced bythe four-byte sequence 0x00000300) is taken into account when searching the RBSP data for the next occurrence of byte-aligned bits with the binary patterns specified above.2. The resulting sequence of bytes is then prefixed as follows:
– If nal_unit_type is not equal to 14 or 20, the sequence of bytes is prefixed with the first byte of the NAL unit containing the syntax elements forbidden_zero_bit, nal_ref_idc, and nal_unit_type, where nal_unit_type indicates the type of RBSP data structure the NAL unit contains.
– Otherwise (nal_unit_type is equal to 14 or 20), the sequence of bytes is prefixed with the first four bytes of the NAL unit, where the first byte contains the syntax elements forbidden_zero_bit, nal_ref_idc, and nal_unit_type and the following three bytes contain the syntax structure nal_unit_header_svc_extension( ).
The syntax element nal_unit_type in the first byte indicates the presence of the syntax structure nal_unit_header_svc_extension( ) in the following three bytes and the type of RBSP data structure the NAL unit contains.
The process specified above results in the construction of the entire NAL unit.
This process can allow any SODB to be represented in a NAL unit while ensuring that
– no byte-aligned start code prefix is emulated within the NAL unit,
– no sequence of 8 zero-valued bits followed by a start code prefix, regardless of byte-alignment, is emulated within the NAL unit.
Remark
[1] Refer to last article: “Introduction to H.264: (1) NAL Unit”
http://yumichan.net/video-processing/video-compression/introduction-to-h264-nal-unit/
[2] Reference from 14496-10 on Page 2
http://yumichan.net/video-processing/video-compression/introduction-to-h264-2-sodb-vs-rbsp-vs-ebsp/2/
Comments
Pages: 1 2
Hi Yumi,
I’m having issues with some x264 streams that I’ve created. When analyzing them, I get some RBSP error “Invalid rbsp_trailing_bits() in slice_layer_without_partitioning_rbsp()”
To be honest, I don’t have near the knowledge that you seem to have regarding H264 structure 🙂 and I was hoping that you could help me with this. Are these error critical? What can be causing this and is there a way to fix it? Finally, what’s the use of rbsp trailing bits?
Kindly regards,
Frederico
Hi Frederico, thank you for visiting my blog. Could you explain more about how you create the h264 streams?
rbsp_trailing_bits are just some bits append at the end of a SODB (bitstream of coded raw H.264 data) to make it byte-aligned. And these bits start with a bit of 1, followed by some 0s, which are the red and yellow parts of the image on my post above.
Regards,
Yumi
Hi Yumi,
thanks for the reply,
basically, we’re using a x264 build running under Simple x264 GUI to produce some raw h264 file that we then mix in a Transport Stream. The whole purpose is to play those files in a MediaRoom system. The files seems to play fine but my client recently raised this issue where my files where analyzing the files, he got some “Invalid rbsp_trailing_bits() in slice_layer_without_partitioning_rbsp()” message.
After some tests, I check the mix process and noticed that it was not the cause of the issue (the issue seems to come from the raw x264 stream). Now, like I said, this does’t really seem to affect the playback but my client is wondering why my files seems to have invalid tbsp (and so do I :-)) And I was wondering if there is a way to make sure that RBSP are correct.
Once again thanks a lot
Fred
Hi,
I am sorry that I haven’t encountered such problem but I will try to help. You said you mixed the generated raw h264 files in a transport stream. And you said it seems to be the raw h264 streams’ problem. So, could you ensure the raw h264 files are the same on the client side? Are you two using the same decoder? If not, can you try to use the same decoder? Can you make sure that whether the problem comes from the raw stream / data or there is some other causes?
If you ensure it comes from the raw streams, have you checked what the current rbsp_trailing_bits are?
Yumi
Thank you for your Blog.
Do SODB have some end mark bits? it is said in blog that “it does not include the 3-byte / 4-byte start code” . so, what is the mark bits in SODB ? thanks!
Hi,
SODB does not contain any starting / ending bits. It is just the raw data of h.264.
If you add a stopping bit 1 at the end (and some 0s to align), it is called RBSP.
If you also add the NALU header & start code, it is called NAL unit.
The start code you mentioned is used to differentiate a NAL unit from another NAL unit.
got it. thank you. I’m studying h264 due to work needs.
So how are these NAL Units encapsulated in a sample that is found in the “mdat” box inside an MP4 file? Is the first NALU prefixed with the Start Code as well? Or just when we want to start the next NALU after the first one inside the same sample?
I see some samples starting with 0x0000 with the next byte not being any of these “starting” bytes (0x00, 0x01, 0x02, 0x03). How should I interpret those?
the question is about slice header. how to get the slice_type from bit streams ? thank you !
Sorry for late reply!
Do you mean byte stream? If yes, there are lots of slices inside a byte stream. You have to identify the NAL Units (while 1 NAL unit contains 1 slice) in a byte stream first. This article shows you how to identify a nal unit.
After finding a nal unit, you may check this article: http://yumichan.net/video-processing/video-compression/introduction-to-h264-nal-unit/
it tells you how to find the nal_unit_type which also stands for the slice type in that nal unit.
Hi,
I’m trying to construct an annex B stream from what I think is an avcC stream. At the start of the stream I get the codec info and a NAL unit length of 3 along with the SPS and PPS data.
From then on all the packets are the raw video data frames. To convert this to an annexB stream I thought it would be just [0001]+[SPS]+[0001]+[PPS]+dataByte[] but I’m missing the NAL so clearly this isn’t right. Could you point me in the right direction?
Hi,
Regarding the emulation_prevention_three_byte, I recently encounter an edge case where the trailing byte of PPS is 0x00 followed by the start code 0x00000001 of the next IDR, giving four consecutive zeros. Is this is allowed or actually something wrong on the encoder side?
FFMPEG is fine with decoding the bitstream though.
According to the spec, an RBSP sequence cannot end with 0x00 byte. If that’s the case, a 0x03 byte should be appended.
If your encoder doesn’t do that, then it probably does something wrong.
However, in MP4 files, NAL Unit lengths are encoded externally, and only one NAL Unit is allowed per sample, which also means that Start Code prefixes are not used. My guess is that from the same reasons, a sequence ending in 0x00 also shouldn’t cause any problems, because it could only cause a problem if it got conflated with the Start Code prefix of the next NALU (which doesn’t exist in this case).