Reference frame (video)
Reference frames are frames of a compressed video that are used to define future frames. As such, they are only used in inter-frame compression techniques. In older video encoding standards, such as MPEG-2, only one reference frame – the previous frame – was used for P-frames. Two reference frames (one past and one future) were used for B-frames.
Multiple reference frames
Some modern video encoding standards, such as H.264/AVC, allow the use of multiple reference frames. This allows the video encoder to choose among more than one previously decoded frame on which to base each macroblock in the next frame. While the best frame for this purpose is usually the previous frame, the extra reference frames can improve compression efficiency and/or video quality. Note that different reference frames can be chosen for different macroblocks in the same frame. The maximum number of concurrent reference frames supported by H.264 is 16. Different reference frames can be chosen for each 8x8 partition of a macroblock. Another video format that supports multiple reference frames is Snow, which can handle up to eight. The Theora codec provides a limited form of multiple reference frames, allowing references to both the preceding frame and the most recent intra frame.
Encoding
Multiple reference frames can considerably increase encoding time because many of the decisions, such as motion estimation, that are ordinarily carried out only on one reference frame have to be repeated on all of the reference frames. Heuristics can be used to reduce this speed cost at the cost of quality. Very high numbers of reference frames are rarely useful in terms of quality for live-action material because frames from farther back in time generally have less and less correlation with the current frame. This is not as true for animated sources, where repetitive motion can make high numbers of reference frames more useful.
Decoding
When decoding, reference frames must be stored in memory until they are no longer needed for further decoding. This can considerably raise the memory usage of the decoder for videos with large numbers of reference frames. The use of several reference frames also decreases locality of reference, which might cause a speed impact. Stand-alone players that can play AVC/MKV files from recorded DVDs or USB sticks may not be able to handle the full 16 REF frames.
References
- E. G. Richardson, Iain (2003). H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia. Chichester: John Wiley & Sons Ltd.