位于libavutil下frame.h文件中
AV系列结构体之AVFrame - 图1

  • 这个结构体用来描述解码出音视频数据。
  • AVFrame必须使用av_frame_alloc分配()。
  • AVFrame必须与av_frame_free释放()。
  • AVFrame通常分配一次,然后重复使用多次，不同的数据（如一个AVFrame持有来自解码器的frames。)在再次使用时,av_frame_unref()将自由持有的任何之前的帧引用并重置它变成初始态。
  • 一个AVFrame所描述的数据通常是通过参考AVBuffer API计算。内部的buffer引用存储在AVFrame.buf /AVFrame.extended_buf。
  • AVFrame将用于引用计数,当至少一个引用被set时,如果AVFrame.buf[0] != NULL, 每个单个数据至少包含一个AVFrame.buf /AVFrame.extended_buf.可能会有一个缓冲的数据,或一个单独的缓冲对每个plane, 或介于两者之间的任何东西。

  • sizeof(AVFrame)不是一个public的API,因此新的成员将被添加到末尾。同样字段标记为只访问av_opt_ptr()可以重新排序 ```cpp typedef struct AVFrame {

    define AV_NUM_DATA_POINTERS 8

    /**

    • pointer to the picture/channel planes.
    • This might be different from the first allocated byte *
    • Some decoders access areas outside 0,0 - width,height, please
    • see avcodec_align_dimensions2(). Some filters and swscale can read
    • up to 16 bytes beyond the planes, if these filters are to be used,
    • then 16 extra bytes must be allocated. *
    • NOTE: Except for hwaccel formats, pointers not needed by the format

    • MUST be set to NULL. / uint8_t data[AV_NUM_DATA_POINTERS];

      /**

    • For video, size in bytes of each picture line.
    • For audio, size in bytes of each plane. *
    • For audio, only linesize[0] may be set. For planar audio, each channel
    • plane must be the same size. *
    • For video the linesizes should be multiples of the CPUs alignment
    • preference, this is 16 or 32 for modern desktop CPUs.
    • Some code requires such alignment other code can be slower without
    • correct alignment, for yet other it makes no difference. *
    • @note The linesize may be larger than the size of usable data — there

    • may be extra padding present for performance reasons. */ int linesize[AV_NUM_DATA_POINTERS];

      /**

    • pointers to the data planes/channels. *
    • For video, this should simply point to data[]. *
    • For planar audio, each channel has a separate data pointer, and
    • linesize[0] contains the size of each channel buffer.
    • For packed audio, there is just one data pointer, and linesize[0]
    • contains the total size of the buffer for all channels. *
    • Note: Both data and extended_data should always be set in a valid frame,
    • but for planar audio with more channels that can fit in data,

    • extended_data must be used in order to access all channels. / uint8_t *extended_data;

      /**

    • width and height of the video frame */ int width, height;

      /**

    • number of audio samples (per channel) described by this frame */ int nb_samples;

      /**

    • format of the frame, -1 if unknown or unset
    • Values correspond to enum AVPixelFormat for video frames,

    • enum AVSampleFormat for audio) */ int format;

      /**

    • 1 -> keyframe, 0-> not */ int key_frame;

      /**

    • Picture type of the frame. */ enum AVPictureType pict_type;

      /**

    • Sample aspect ratio for the video frame, 0/1 if unknown/unspecified. */ AVRational sample_aspect_ratio;

      /**

    • Presentation timestamp in time_base units (time when frame should be shown to user). */ int64_t pts;

      /**

    • PTS copied from the AVPacket that was decoded to produce this frame. */ int64_t pkt_pts;

      /**

    • DTS copied from the AVPacket that triggered returning this frame. (if frame threading isn’t used)
    • This is also the Presentation time of this AVFrame calculated from

    • only AVPacket.dts values without pts values. */ int64_t pkt_dts;

      /**

    • picture number in bitstream order / int coded_picture_number; /*

    • picture number in display order */ int display_picture_number;

      /**

    • quality (between 1 (good) and FF_LAMBDA_MAX (bad)) */ int quality;

      /**

    • for some private data of the user / void opaque;

if FF_API_ERROR_FRAME

  1. /**
  2.  * @deprecated unused
  3.  */
  4. attribute_deprecated
  5. uint64_t error[AV_NUM_DATA_POINTERS];

endif

  1. /**
  2.  * When decoding, this signals how much the picture must be delayed.
  3.  * extra_delay = repeat_pict / (2*fps)
  4.  */
  5. int repeat_pict;
  7. /**
  8.  * The content of the picture is interlaced.
  9.  */
  10. int interlaced_frame;
  12. /**
  13.  * If the content is interlaced, is top field displayed first.
  14.  */
  15. int top_field_first;
  17. /**
  18.  * Tell user application that palette has changed from previous frame.
  19.  */
  20. int palette_has_changed;
  22. /**
  23.  * reordered opaque 64 bits (generally an integer or a double precision float
  24.  * PTS but can be anything).
  25.  * The user sets AVCodecContext.reordered_opaque to represent the input at
  26.  * that time,
  27.  * the decoder reorders values as needed and sets AVFrame.reordered_opaque
  28.  * to exactly one of the values provided by the user through AVCodecContext.reordered_opaque
  29.  * @deprecated in favor of pkt_pts
  30.  */
  31. int64_t reordered_opaque;
  33. /**
  34.  * Sample rate of the audio data.
  35.  */
  36. int sample_rate;
  38. /**
  39.  * Channel layout of the audio data.
  40.  */
  41. uint64_t channel_layout;
  43. /**
  44.  * AVBuffer references backing the data for this frame. If all elements of
  45.  * this array are NULL, then this frame is not reference counted. This array
  46.  * must be filled contiguously -- if buf[i] is non-NULL then buf[j] must
  47.  * also be non-NULL for all j < i.
  48.  *
  49.  * There may be at most one AVBuffer per data plane, so for video this array
  50.  * always contains all the references. For planar audio with more than
  51.  * AV_NUM_DATA_POINTERS channels, there may be more buffers than can fit in
  52.  * this array. Then the extra AVBufferRef pointers are stored in the
  53.  * extended_buf array.
  54.  */
  55. AVBufferRef *buf[AV_NUM_DATA_POINTERS];
  57. /**
  58.  * For planar audio which requires more than AV_NUM_DATA_POINTERS
  59.  * AVBufferRef pointers, this array will hold all the references which
  60.  * cannot fit into AVFrame.buf.
  61.  *
  62.  * Note that this is different from AVFrame.extended_data, which always
  63.  * contains all the pointers. This array only contains the extra pointers,
  64.  * which cannot fit into AVFrame.buf.
  65.  *
  66.  * This array is always allocated using av_malloc() by whoever constructs
  67.  * the frame. It is freed in av_frame_unref().
  68.  */
  69. AVBufferRef **extended_buf;
  70. /**
  71.  * Number of elements in extended_buf.
  72.  */
  73. int        nb_extended_buf;
  75. AVFrameSideData **side_data;
  76. int            nb_side_data;

/**

  • @defgroup lavu_frame_flags AV_FRAME_FLAGS
  • Flags describing additional frame properties. *
  • @{ */

/**

  • The frame data may be corrupted, e.g. due to decoding errors. */

    define AV_FRAME_FLAG_CORRUPT (1 << 0)

    /**

  • @} */

    /**

    • Frame flags, a combination of @ref lavu_frame_flags */ int flags;

      /**

    • MPEG vs JPEG YUV range.
    • It must be accessed using av_frame_get_color_range() and
    • av_frame_set_color_range().
      • encoding: Set by user
      • decoding: Set by libavcodec */ enum AVColorRange color_range;

      enum AVColorPrimaries color_primaries;

      enum AVColorTransferCharacteristic color_trc;

      /**

    • YUV colorspace type.
    • It must be accessed using av_frame_get_colorspace() and
    • av_frame_set_colorspace().
      • encoding: Set by user
      • decoding: Set by libavcodec */ enum AVColorSpace colorspace;

      enum AVChromaLocation chroma_location;

      /**

    • frame timestamp estimated using various heuristics, in stream time base
    • Code outside libavutil should access this field using:
    • av_frame_get_best_effort_timestamp(frame)
      • encoding: unused
      • decoding: set by libavcodec, read by user. */ int64_t best_effort_timestamp;

      /**

    • reordered pos from the last AVPacket that has been input into the decoder
    • Code outside libavutil should access this field using:
    • av_frame_get_pkt_pos(frame)
      • encoding: unused
      • decoding: Read by user. */ int64_t pkt_pos;

      /**

    • duration of the corresponding packet, expressed in
    • AVStream->time_base units, 0 if unknown.
    • Code outside libavutil should access this field using:
    • av_frame_get_pkt_duration(frame)
      • encoding: unused
      • decoding: Read by user. */ int64_t pkt_duration;

      /**

    • metadata.
    • Code outside libavutil should access this field using:
    • av_frame_get_metadata(frame)
      • encoding: Set by user.
      • decoding: Set by libavcodec. / AVDictionary metadata;

      /**

    • decode error flags of the frame, set to a combination of
    • FF_DECODE_ERROR_xxx flags if the decoder produced a frame, but there
    • were errors during the decoding.
    • Code outside libavutil should access this field using:
    • av_frame_get_decode_error_flags(frame)
      • encoding: unused
      • decoding: set by libavcodec, read by user. */ int decode_error_flags;

        define FF_DECODE_ERROR_INVALID_BITSTREAM 1

        define FF_DECODE_ERROR_MISSING_REFERENCE 2

      /**

    • number of audio channels, only used for audio.
    • Code outside libavutil should access this field using:
    • av_frame_get_channels(frame)
      • encoding: unused
      • decoding: Read by user. */ int channels;

      /**

    • size of the corresponding packet containing the compressed
    • frame. It must be accessed using av_frame_get_pkt_size() and
    • av_frame_set_pkt_size().
    • It is set to a negative value if unknown.
      • encoding: unused
      • decoding: set by libavcodec, read by user. */ int pkt_size;

if FF_API_FRAME_QP

  1. /**
  2.  * QP table
  3.  * Not to be accessed directly from outside libavutil
  4.  */
  5. attribute_deprecated
  6. int8_t *qscale_table;
  7. /**
  8.  * QP store stride
  9.  * Not to be accessed directly from outside libavutil
  10.  */
  11. attribute_deprecated
  12. int qstride;
  14. attribute_deprecated
  15. int qscale_type;
  17. /**
  18.  * Not to be accessed directly from outside libavutil
  19.  */
  20. AVBufferRef *qp_table_buf;

endif

  1. /**
  2.  * For hwaccel-format frames, this should be a reference to the
  3.  * AVHWFramesContext describing the frame.
  4.  */
  5. AVBufferRef *hw_frames_ctx;

} AVFrame;

```

  • uint8_t *data[AV_NUM_DATA_POINTERS]：指针数组，存放YUV数据的地方。如图所示，一般占用前3个指针，分别指向Y，U，V数据。
  • 对于packed格式的数据（例如RGB24），会存到data[0]里面。
  • 对于planar格式的数据（例如YUV420P），则会分开成data[0]，data[1]，data[2]…（YUV420P中data[0]存Y，data[1]存U，data[2]存V）
  • int linesize[AV_NUM_DATA_POINTERS]：图像各个分量数据在此结构体中的的宽度。注意这并不是图像的宽度。在此例子中图像的尺寸为672X272，而亮度分量的宽度为704，应该是图像宽度经过64对齐后的结果。
  • uint8_t **extended_data：指向了图像数据。
  • int width, height：图像的宽高。
  • int nb_samples：此帧音频的点数。
  • int format：像素类型（视频），样点类型（音频）
  • int key_frame：是否关键帧，此例中为视频的第一帧，当然是关键帧了。
  • enum AVPictureType pict_type：图像类型，I，P，B等，同样，第一帧是I帧。
  • AVRational sample_aspect_ratio：像素的宽高比，注意不是图像的。
  • int64_t pts，pkt_pts，pkt_dts：和时间戳有关的变量，以后会详细介绍。
  • int coded_picture_number：编码顺序的图像num。
  • int display_picture_number：播放顺序的图像num。
  • int interlaced_frame：图像是否是隔行的。
  • int top_field_first：图像的top field first变量。
  • int64_t pkt_duration：对应packet的显示时长。
  • int pkt_size：对应packet的尺寸。
  • int8_t *qscale_table：据推测是存放qp（量化参数）的数组
  • AVBufferRef *qp_table_buf：成员data指向qscale_table。

如约智惠.png