Introduction to H.264: (2) SODB vs RBSP vs EBSP

Here is a summary of the relationship between SODB, RBSP, EBSP, NALU and H.264 Byte Stream.
SODB: String of Data Bits
RBSP: Raw Byte Sequence Payload
EBSP: Encapsulate Byte Sequence Payload

SODB + RBSP Stop bit + 0 bit(s) => RBSP

RBSP part 1 + 0x03 + RBSP part 2 + 0x03 + … + RBSP part n => EBSP

NALU Header + EBSP => NALU

In Byte Stream Format (Annex B),

Start CodeNALU + … + Start CodeNALU => H.264 Byte Stream

String of Data Bits (SODB)

Definition:

3.149  string of data bits (SODB): A sequence of some number of bits representing syntax elements present within a raw byte sequence payload prior to the raw byte sequence payload stop bit. Within an SODB, the left-most bit is considered to be the first and most significant bit, and the right-most bit is considered to be the last and least significant bit.

To form a NALU, we need the raw data and we call it String of Data Bits. It is a bitstream of coded raw H.264 data stream and defined by MPEG Part 10 (Free Download). So, it does not include the 3-byte / 4-byte start code, 0x000001 or 0x00000001 and not include the 1-byte NALU header [1] which contains some basic information of the NALU such as NAL unit type.

Raw Byte Sequence Payload (RBSP)

Definition:

3.118 raw byte sequence payload (RBSP): A syntax structure containing an integer number of bytes that is encapsulated in a NAL unit. An RBSP is either empty or has the form of a SODB containing syntax elements followed by an RBSP stop bit and followed by 0 or more subsequent bits equal to 0.

3.119 RBSP stop bit: A bit equal to 1 present within a RBSP after a SODB. The location of the end of the SODB within an RBSP can be identified by searching from the end of the RBSP for the RBSP stop bit, which is the last non-zero bit in the RBSP.

After getting the SODB, a rbsp trailing bit equals to 1 is added to indicate the end of the SODB so the bit is called stop bit (rbsp_stop_one_bit). Also, RBSP is always byte aligned and some 0 bits are added after the stop bit to make it 8-bit aligned.
In short, two things are added to the end of SODB in order to form a RBSP:

  1. rbsp_stop_one_bit = 1
  2. rbsp_alignment_zero_bit(s) = 0(s) [[Must make the SODB byte-aligned]]
RBSP

RBSP

Encapsulate Byte Sequence Payload (EBSP)

I mentioned that we need to prefix the NAL Units with the start code 0x000001 / 0x00000001 in byte stream format in order to identify their boundaries [1]. Yet, there may exist the pattern 0x00000X, where X is 0, 1, 2 or 3, in the RBSP and have emulation of start code. To ease the decoding process, a emulation_prevention_three_byte (0x03) is added after 0x0000 and it becomes 0x0000030X:

  • 0x000000 => 0x00000300
  • 0x000001 => 0x00000301
  • 0x000002 => 0x00000302
  • 0x000003 => 0x00000303

Since emulation_prevention_three_byte doesn’t affect the decoding result, the decoder should ignored it.

Form NAL Unit from SODB, RBSP & EBSP

7.4.1.1  Encapsulation of an SODB within an RBSP (informative)

The form of encapsulation of an SODB within an RBSP and the use of the emulation_prevention_three_byte for encapsulation of an RBSP within a NAL unit is specified for the following purposes:
–  to prevent the emulation of start codes within NAL units while allowing any arbitrary SODB to be represented within a NAL unit,
–  to enable identification of the end of the SODB within the NAL unit by searching the RBSP for the rbsp_stop_one_bit starting at the end of the RBSP,
–  to enable a NAL unit to have a size larger than that of the SODB under some circumstances (using one or more cabac_zero_word).
The encoder can produce a NAL unit from an RBSP by the following procedure:
1.  The RBSP data is searched for byte-aligned bits of the following binary patterns:
‘00000000 00000000 000000xx’ (where xx represents any 2 bit pattern: 00, 01, 10, or 11), and a byte equal to 0x03 is inserted to replace these bit patterns with the patterns:
‘00000000 00000000 00000011 000000xx’, and finally, when the last byte of the RBSP data is equal to 0x00 (which can only occur when the RBSP ends in a cabac_zero_word), a final byte equal to 0x03 is appended to the end of the data. The last zero byte of a byte-aligned three-byte sequence 0x000000 in the RBSP (which is replaced bythe four-byte sequence 0x00000300) is taken into account when searching the RBSP data for the next occurrence of byte-aligned bits with the binary patterns specified above.

2.  The resulting sequence of bytes is then prefixed as follows:
–  If nal_unit_type is not equal to 14 or 20, the sequence of bytes is prefixed with the first byte of the NAL unit containing the syntax elements forbidden_zero_bit, nal_ref_idc, and nal_unit_type, where nal_unit_type indicates the type of RBSP data structure the NAL unit contains.
–  Otherwise (nal_unit_type is equal to 14 or 20), the sequence of bytes is prefixed with the first four bytes of the NAL unit, where the first byte contains the syntax elements forbidden_zero_bit, nal_ref_idc, and nal_unit_type and the following three bytes contain the syntax structure nal_unit_header_svc_extension( ).
The syntax element nal_unit_type in the first byte indicates the presence of the syntax structure nal_unit_header_svc_extension( ) in the following three bytes and the type of RBSP data structure the NAL unit contains.
The process specified above results in the construction of the entire NAL unit.
This process can allow any SODB to be represented in a NAL unit while ensuring that
–  no byte-aligned start code prefix is emulated within the NAL unit,
–  no sequence of 8 zero-valued bits followed by a start code prefix, regardless of byte-alignment, is emulated within the NAL unit.

Remark

[1] Refer to last article: “Introduction to H.264: (1) NAL Unit
http://yumichan.net/video-processing/video-compression/introduction-to-h264-nal-unit/

[2] Reference from 14496-10 on Page 2
http://yumichan.net/video-processing/video-compression/introduction-to-h264-2-sodb-vs-rbsp-vs-ebsp/2/

Comments

comments

6 Comments

  1. Frederico Cengarle

    Hi Yumi,

    I’m having issues with some x264 streams that I’ve created. When analyzing them, I get some RBSP error “Invalid rbsp_trailing_bits() in slice_layer_without_partitioning_rbsp()”

    To be honest, I don’t have near the knowledge that you seem to have regarding H264 structure :-) and I was hoping that you could help me with this. Are these error critical? What can be causing this and is there a way to fix it? Finally, what’s the use of rbsp trailing bits?
    Kindly regards,
    Frederico

    Reply
    1. Yumi Chan (Post author)

      Hi Frederico, thank you for visiting my blog. Could you explain more about how you create the h264 streams?

      rbsp_trailing_bits are just some bits append at the end of a SODB (bitstream of coded raw H.264 data) to make it byte-aligned. And these bits start with a bit of 1, followed by some 0s, which are the red and yellow parts of the image on my post above.

      Regards,
      Yumi

      Reply
      1. Frederico Cengarle

        Hi Yumi,

        thanks for the reply,

        basically, we’re using a x264 build running under Simple x264 GUI to produce some raw h264 file that we then mix in a Transport Stream. The whole purpose is to play those files in a MediaRoom system. The files seems to play fine but my client recently raised this issue where my files where analyzing the files, he got some “Invalid rbsp_trailing_bits() in slice_layer_without_partitioning_rbsp()” message.

        After some tests, I check the mix process and noticed that it was not the cause of the issue (the issue seems to come from the raw x264 stream). Now, like I said, this does’t really seem to affect the playback but my client is wondering why my files seems to have invalid tbsp (and so do I :-)) And I was wondering if there is a way to make sure that RBSP are correct.

        Once again thanks a lot
        Fred

        Reply
        1. Yumi Chan (Post author)

          Hi,

          I am sorry that I haven’t encountered such problem but I will try to help. You said you mixed the generated raw h264 files in a transport stream. And you said it seems to be the raw h264 streams’ problem. So, could you ensure the raw h264 files are the same on the client side? Are you two using the same decoder? If not, can you try to use the same decoder? Can you make sure that whether the problem comes from the raw stream / data or there is some other causes?

          If you ensure it comes from the raw streams, have you checked what the current rbsp_trailing_bits are?

          Yumi

          Reply
  2. cheng

    Thank you for your Blog.
    Do SODB have some end mark bits? it is said in blog that “it does not include the 3-byte / 4-byte start code” . so, what is the mark bits in SODB ? thanks!

    Reply
    1. Yumi Chan (Post author)

      Hi,
      SODB does not contain any starting / ending bits. It is just the raw data of h.264.
      If you add a stopping bit 1 at the end (and some 0s to align), it is called RBSP.
      If you also add the NALU header & start code, it is called NAL unit.
      The start code you mentioned is used to differentiate a NAL unit from another NAL unit.

      Reply

Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">