gstvideo.c.types

Location of a @GstAncillaryMeta.

An enumeration indicating whether an element implements color balancing operations in software or in dedicated hardware. In general, dedicated hardware implementations (such as those provided by xvimagesink) are preferred.

A set of commands that may be issued to an element providing the #GstNavigation interface. The available commands can be queried via the [gstvideo.navigation.Navigation.queryNewCommands] query.

For convenience in handling DVD navigation, the MENU commands are aliased as: GST_NAVIGATION_COMMAND_DVD_MENU = @GST_NAVIGATION_COMMAND_MENU1 GST_NAVIGATION_COMMAND_DVD_TITLE_MENU = @GST_NAVIGATION_COMMAND_MENU2 GST_NAVIGATION_COMMAND_DVD_ROOT_MENU = @GST_NAVIGATION_COMMAND_MENU3 GST_NAVIGATION_COMMAND_DVD_SUBPICTURE_MENU = @GST_NAVIGATION_COMMAND_MENU4 GST_NAVIGATION_COMMAND_DVD_AUDIO_MENU = @GST_NAVIGATION_COMMAND_MENU5 GST_NAVIGATION_COMMAND_DVD_ANGLE_MENU = @GST_NAVIGATION_COMMAND_MENU6 GST_NAVIGATION_COMMAND_DVD_CHAPTER_MENU = @GST_NAVIGATION_COMMAND_MENU7

Enum values for the various events that an element implementing the GstNavigation interface might send up the pipeline. Touch events have been inspired by the libinput API, and have the same meaning here.

A set of notifications that may be received on the bus when navigation related status changes.

Flags to indicate the state of modifier keys and mouse buttons in events.

Typical modifier keys are Shift, Control, Meta, Super, Hyper, Alt, Compose, Apple, CapsLock or ShiftLock.

Types of navigation interface queries.

Enumeration of the different standards that may apply to AFD data:

0) ETSI/DVB:

1) ATSC A/53:

2) SMPTE ST2016-1:

Enumeration of the various values for Active Format Description (AFD)

AFD should be included in video user data whenever the rectangular picture area containing useful information does not extend to the full height or width of the coded frame. AFD data may also be included in user data when the rectangular picture area containing useful information extends to the full height and width of the coded frame.

For details, see Table 6.14 Active Format in:

ATSC Digital Television Standard: Part 4 – MPEG-2 Video System Characteristics

and Active Format Description in Complete list of AFD codes

and SMPTE ST2016-1

Notes:

1) AFD 0 is undefined for ATSC and SMPTE ST2016-1, indicating that AFD data is not available: If Bar Data is not present, AFD '0000' indicates that exact information is not available and the active image should be assumed to be the same as the coded frame. AFD '0000'. AFD '0000' accompanied by Bar Data signals that the active image’s aspect ratio is narrower than 16:9, but is not 4:3 or 14:9. As the exact aspect ratio cannot be conveyed by AFD alone, wherever possible, AFD ‘0000’ should be accompanied by Bar Data to define the exact vertical or horizontal extent of the active image. 2) AFD 0 is reserved for DVB/ETSI 3) values 1, 5, 6, 7, and 12 are reserved for both ATSC and DVB/ETSI 4) values 2 and 3 are not recommended for ATSC, but are valid for DVB/ETSI

Different alpha modes.

Some know types of Ancillary Data identifiers.

Additional video buffer flags. These flags can potentially be used on any buffers carrying closed caption data, or video data - even encoded data.

Note that these are only valid for #GstCaps of type: video/... and caption/... They can conflict with other extended buffer flags.

The various known types of Closed Caption (CC).

Extra flags that influence the result from [gstvideo.video_chroma_resample.VideoChromaResample.new_].

Different subsampling and upsampling methods

Different chroma downsampling and upsampling modes

Various Chroma sitings.

Flags for #GstVideoCodecFrame

The color matrix is used to convert between Y'PbPr and non-linear RGB (R'G'B')

The color primaries define the how to transform linear RGB values to and from the CIE XYZ colorspace.

Possible color range values. These constants are defined for 8 bit color values and can be scaled for other bit depths.

Flags to be used in combination with [gstvideo.video_decoder.VideoDecoder.requestSyncPoint]. See the function documentation for more details.

Extra flags that influence the result from [gstvideo.video_chroma_resample.VideoChromaResample.new_].

Different dithering methods to use.

Field order of interlaced content. This is only valid for interlace-mode=interleaved and not interlace-mode=mixed. In the case of mixed or GST_VIDEO_FIELD_ORDER_UNKOWN, the field order is signalled via buffer flags.

Extra video flags

Enum value describing the most common video formats.

See the GStreamer raw video format design document for details about the layout and packing of these formats in memory.

The different video flags that a format info can have.

Extra video frame flags

Additional mapping flags for [gstvideo.video_frame.VideoFrame.map].

The orientation of the GL texture.

The GL texture type.

The possible values of the #GstVideoInterlaceMode describing the interlace mode of the stream.

Different color matrix conversion modes

GstVideoMultiviewFlags are used to indicate extra properties of a stereo/multiview stream beyond the frame layout and buffer mapping that is conveyed in the #GstVideoMultiviewMode.

#GstVideoMultiviewFramePacking represents the subset of #GstVideoMultiviewMode values that can be applied to any video frame without needing extra metadata. It can be used by elements that provide a property to override the multiview interpretation of a video stream when the video doesn't contain any markers.

This enum is used (for example) on playbin, to re-interpret a played video stream as a stereoscopic video. The individual enum values are equivalent to and have the same value as the matching #GstVideoMultiviewMode.

All possible stereoscopic 3D and multiview representations. In conjunction with #GstVideoMultiviewFlags, describes how multiview content is being transported in the stream.

The different video orientation methods.

Overlay format flags.

The different flags that can be used when packing and unpacking.

Different primaries conversion modes

Different resampler flags.

Different subsampling and upsampling methods

Different scale flags.

Enum value describing the available tiling modes.

Enum value describing the most common tiling types.

Flags related to the time code information. For drop frame, only 30000/1001 and 60000/1001 frame rates are supported.

The video transfer function defines the formula for converting between non-linear RGB (R'G'B') and linear RGB

Return values for #GstVideoVBIParser

#GstMeta for carrying SMPTE-291M Ancillary data. Note that all the ADF fields (@DID to @checksum) are 10bit values with parity/non-parity high-bits set.

This interface is implemented by elements which can perform some color balance operation on video frames they process. For example, modifying the brightness, contrast, hue or saturation.

Example elements are 'xvimagesink' and 'colorbalance'

The #GstColorBalanceChannel object represents a parameter for modifying the color balance implemented by an element providing the #GstColorBalance interface. For example, Hue or Saturation.

Color-balance channel class.

Color-balance interface.

The Navigation interface is used for creating and injecting navigation related events such as mouse button presses, cursor motion and key presses. The associated library also provides methods for parsing received events, and for sending and receiving navigation related bus events. One main usecase is DVD menu navigation.

The main parts of the API are:

• The GstNavigation interface, implemented by elements which provide an

application with the ability to create and inject navigation events into the pipeline.

• GstNavigation event handling API. GstNavigation events are created in

response to calls on a GstNavigation interface implementation, and sent in the pipeline. Upstream elements can use the navigation event API functions to parse the contents of received messages.

• GstNavigation message handling API. GstNavigation messages may be sent on

the message bus to inform applications of navigation related changes in the pipeline, such as the mouse moving over a clickable region, or the set of available angles changing.

The GstNavigation message functions provide functions for creating and parsing custom bus messages for signaling GstNavigation changes.

Navigation interface.

Active Format Description (AFD)

For details, see Table 6.14 Active Format in:

ATSC Digital Television Standard: Part 4 – MPEG-2 Video System Characteristics

and Active Format Description in Complete list of AFD codes

and SMPTE ST2016-1

Extra buffer metadata for performing an affine transformation using a 4x4 matrix. The transformation matrix can be composed with [gstvideo.video_affine_transformation_meta.VideoAffineTransformationMeta.applyMatrix].

The vertices operated on are all in the range 0 to 1, not in Normalized Device Coordinates (-1 to +1). Transforming points in this space are assumed to have an origin at (0.5, 0.5, 0.5) in a left-handed coordinate system with the x-axis moving horizontally (positive values to the right), the y-axis moving vertically (positive values up the screen) and the z-axis perpendicular to the screen (positive values into the screen).

VideoAggregator can accept AYUV, ARGB and BGRA video streams. For each of the requested sink pads it will compare the incoming geometry and framerate to define the output parameters. Indeed output video frames will have the geometry of the biggest incoming video stream and the framerate of the fastest incoming one.

VideoAggregator will do colorspace conversion.

Zorder for each input stream can be configured on the #GstVideoAggregatorPad.

An implementation of GstPad that can be used with #GstVideoAggregator.

See #GstVideoAggregator for more details.

An implementation of GstPad that can be used with #GstVideoAggregator.

See #GstVideoAggregator for more details.

Extra alignment parameters for the memory of video buffers. This structure is usually used to configure the bufferpool if it supports the #GST_BUFFER_POOL_OPTION_VIDEO_ALIGNMENT.

Video Ancillary data, according to SMPTE-291M specification.

Note that the contents of the data are always stored as 8bit data (i.e. do not contain the parity check bits).

Bar data should be included in video user data whenever the rectangular picture area containing useful information does not extend to the full height or width of the coded frame and AFD alone is insufficient to describe the extent of the image.

Extra buffer metadata providing Closed Caption.

This meta is primarily for internal use in GStreamer elements to support VP8/VP9 transparent video stored into WebM or Matroska containers, or transparent static AV1 images. Nothing prevents you from using this meta for custom purposes, but it generally can't be used to easily to add support for alpha channels to CODECs or formats that don't support that out of the box.

A #GstVideoCodecFrame represents a video frame both in raw and encoded form.

Structure representing the state of an incoming or outgoing video stream for encoders and decoders.

Decoders and encoders will receive such a state through their respective @set_format vmethods.

Decoders and encoders can set the downstream state, by using the [gstvideo.video_decoder.VideoDecoder.setOutputState] or [gstvideo.video_encoder.VideoEncoder.setOutputState] methods.

Structure describing the chromaticity coordinates of an RGB system. These values can be used to construct a matrix to transform RGB to and from the XYZ colorspace.

Structure describing the color info.

Content light level information specified in CEA-861.3, Appendix A.

Extra buffer metadata describing image cropping.

This base class is for video decoders turning encoded data into raw video frames.

The GstVideoDecoder base class and derived subclasses should cooperate as follows:

Configuration

• Initially, GstVideoDecoder calls @start when the decoder element

is activated, which allows the subclass to perform any global setup.

• GstVideoDecoder calls @set_format to inform the subclass of caps

describing input video data that it is about to receive, including possibly configuration data. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.

• Incoming data buffers are processed as needed, described in Data

Processing below.

• GstVideoDecoder calls @stop at end of all processing.

Data processing

• The base class gathers input data, and optionally allows subclass

to parse this into subsequently manageable chunks, typically corresponding to and referred to as 'frames'.

• Each input frame is provided in turn to the subclass' @handle_frame

callback.

• When the subclass enables the subframe mode with [gstvideo.video_decoder.VideoDecoder.setSubframeMode],

the base class will provide to the subclass the same input frame with different input buffers to the subclass @handle_frame callback. During this call, the subclass needs to take ownership of the input_buffer as @GstVideoCodecFrame.input_buffer will have been changed before the next subframe buffer is received. The subclass will call [gstvideo.video_decoder.VideoDecoder.haveLastSubframe] when a new input frame can be created by the base class. Every subframe will share the same @GstVideoCodecFrame.output_buffer to write the decoding result. The subclass is responsible to protect its access.

• If codec processing results in decoded data, the subclass should call

@gst_video_decoder_finish_frame to have decoded data pushed downstream. In subframe mode the subclass should call @gst_video_decoder_finish_subframe until the last subframe where it should call @gst_video_decoder_finish_frame. The subclass can detect the last subframe using GST_VIDEO_BUFFER_FLAG_MARKER on buffers or using its own logic to collect the subframes. In case of decoding failure, the subclass must call @gst_video_decoder_drop_frame or @gst_video_decoder_drop_subframe, to allow the base class to do timestamp and offset tracking, and possibly to requeue the frame for a later attempt in the case of reverse playback.

Shutdown phase

• The GstVideoDecoder class calls @stop to inform the subclass that data

parsing will be stopped.

Additional Notes

• Seeking/Flushing

• When the pipeline is seeked or otherwise flushed, the subclass is

informed via a call to its @reset callback, with the hard parameter set to true. This indicates the subclass should drop any internal data queues and timestamps and prepare for a fresh set of buffers to arrive for parsing and decoding.

• End Of Stream

• At end-of-stream, the subclass @parse function may be called some final

times with the at_eos parameter set to true, indicating that the element should not expect any more data to be arriving, and it should parse and remaining frames and call [gstvideo.video_decoder.VideoDecoder.haveFrame] if possible.

The subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to provide information about the output caps, when they are known. This may be when the base class calls the subclass' @set_format function, though it might be during decoding, before calling @gst_video_decoder_finish_frame. This is done via @gst_video_decoder_set_output_state

The subclass is also responsible for providing (presentation) timestamps (likely based on corresponding input ones). If that is not applicable or possible, the base class provides limited framerate based interpolation.

Similarly, the base class provides some limited (legacy) seeking support if specifically requested by the subclass, as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates. To enable it, a subclass should call @gst_video_decoder_set_estimate_rate to enable handling of incoming byte-streams.

The base class provides some support for reverse playback, in particular in case incoming data is not packetized or upstream does not provide fragments on keyframe boundaries. However, the subclass should then be prepared for the parsing and frame processing stage to occur separately (in normal forward processing, the latter immediately follows the former), The subclass also needs to ensure the parsing stage properly marks keyframes, unless it knows the upstream elements will do so properly for incoming data.

The bare minimum that a functional subclass needs to implement is:

• Provide pad templates
• Inform the base class of output caps via

@gst_video_decoder_set_output_state

• Parse input data, if it is not considered packetized from upstream

Data will be provided to @parse which should invoke @gst_video_decoder_add_to_frame and @gst_video_decoder_have_frame to separate the data belonging to each video frame.

• Accept data in @handle_frame and provide decoded results to

@gst_video_decoder_finish_frame, or call @gst_video_decoder_drop_frame.

Subclasses can override any of the available virtual methods or not, as needed. At minimum @handle_frame needs to be overridden, and @set_format and likely as well. If non-packetized input is supported or expected, @parse needs to be overridden as well.

The interface allows unified access to control flipping and rotation operations of video-sources or operators.

#GstVideoDirectionInterface interface.

GstVideoDither provides implementations of several dithering algorithms that can be applied to lines of video pixels to quantize and dither them.

This base class is for video encoders turning raw video into encoded video data.

GstVideoEncoder and subclass should cooperate as follows.

Configuration

• Initially, GstVideoEncoder calls @start when the encoder element

is activated, which allows subclass to perform any global setup.

• GstVideoEncoder calls @set_format to inform subclass of the format

of input video data that it is about to receive. Subclass should setup for encoding and configure base class as appropriate (e.g. latency). While unlikely, it might be called more than once, if changing input parameters require reconfiguration. Baseclass will ensure that processing of current configuration is finished.

• GstVideoEncoder calls @stop at end of all processing.

Data processing

• Base class collects input data and metadata into a frame and hands

this to subclass' @handle_frame.

• If codec processing results in encoded data, subclass should call

@gst_video_encoder_finish_frame to have encoded data pushed downstream.

• If implemented, baseclass calls subclass @pre_push just prior to

pushing to allow subclasses to modify some metadata on the buffer. If it returns GST_FLOW_OK, the buffer is pushed downstream.

• GstVideoEncoderClass will handle both srcpad and sinkpad events.

Sink events will be passed to subclass if @event callback has been provided.

Shutdown phase

• GstVideoEncoder class calls @stop to inform the subclass that data

parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It should also be able to provide fixed src pad caps in @getcaps by the time it calls @gst_video_encoder_finish_frame.

Things that subclass need to take care of:

• Provide pad templates
• Provide source pad caps before pushing the first buffer
• Accept data in @handle_frame and provide encoded results to

@gst_video_encoder_finish_frame.

The #GstVideoEncoder:qos property will enable the Quality-of-Service features of the encoder which gather statistics about the real-time performance of the downstream elements. If enabled, subclasses can use [gstvideo.video_encoder.VideoEncoder.getMaxEncodeTime] to check if input frames are already late and drop them right away to give a chance to the pipeline to catch up.

Subclasses can override any of the available virtual methods or not, as needed. At minimum @handle_frame needs to be overridden, and @set_format and @get_caps are likely needed as well.

Provides useful functions and a base class for video filters.

The videofilter will by default enable QoS on the parent GstBaseTransform to implement frame dropping.

The video filter class structure.

Information for a video format.

A video frame obtained from [gstvideo.video_frame.VideoFrame.map]

Extra buffer metadata for uploading a buffer to an OpenGL texture ID. The caller of [gstvideo.video_gltexture_upload_meta.VideoGLTextureUploadMeta.upload] must have OpenGL set up and call this from a thread where it is valid to upload something to an OpenGL texture.

Information describing image properties. This information can be filled in from GstCaps with [gstvideo.video_info.VideoInfo.fromCaps]. The information is also used to store the specific video info when mapping a video frame with [gstvideo.video_frame.VideoFrame.map].

Use the provided macros to access the info in this structure.

Information describing a DMABuf image properties. It wraps #GstVideoInfo and adds DRM information such as drm-fourcc and drm-modifier, required for negotiation and mapping.

Mastering display color volume information defined by SMPTE ST 2086 (a.k.a static HDR metadata).

Used to represent display_primaries and white_point of #GstVideoMasteringDisplayInfo struct. See #GstVideoMasteringDisplayInfo

Extra buffer metadata describing image properties

This meta can also be used by downstream elements to specifiy their buffer layout requirements for upstream. Upstream should try to fit those requirements, if possible, in order to prevent buffer copies.

This is done by passing a custom #GstStructure to [gst.query.Query.addAllocationMeta] when handling the ALLOCATION query. This structure should be named 'video-meta' and can have the following fields:

• padding-top (uint): extra pixels on the top
• padding-bottom (uint): extra pixels on the bottom
• padding-left (uint): extra pixels on the left side
• padding-right (uint): extra pixels on the right side

The padding fields have the same semantic as #GstVideoMeta.alignment and so represent the paddings requested on produced video buffers.

Since 1.24 it can be serialized using [gst.meta.Meta.serialize] and [gst.meta.Meta.deserialize].

Extra data passed to a video transform #GstMetaTransformFunction such as: "gst-video-scale".

See #GstVideoMultiviewFlags.

The interface allows unified access to control flipping and autocenter operation of video-sources or operators.

#GstVideoOrientationInterface interface.

The #GstVideoOverlay interface is used for 2 main purposes :

• To get a grab on the Window where the video sink element is going to render.

This is achieved by either being informed about the Window identifier that the video sink element generated, or by forcing the video sink element to use a specific Window identifier for rendering.

• To force a redrawing of the latest video frame the video sink element

displayed on the Window. Indeed if the #GstPipeline is in #GST_STATE_PAUSED state, moving the Window around will damage its content. Application developers will want to handle the Expose events themselves and force the video sink element to refresh the Window's content.

Using the Window created by the video sink is probably the simplest scenario, in some cases, though, it might not be flexible enough for application developers if they need to catch events such as mouse moves and button clicks.

Setting a specific Window identifier on the video sink element is the most flexible solution but it has some issues. Indeed the application needs to set its Window identifier at the right time to avoid internal Window creation from the video sink element. To solve this issue a #GstMessage is posted on the bus to inform the application that it should set the Window identifier immediately. Here is an example on how to do that correctly:

static GstBusSyncReply
create_window (GstBus * bus, GstMessage * message, GstPipeline * pipeline)
{
// ignore anything but 'prepare-window-handle' element messages
if (!gst_is_video_overlay_prepare_window_handle_message (message))
  return GST_BUS_PASS;

win = XCreateSimpleWindow (disp, root, 0, 0, 320, 240, 0, 0, 0);

XSetWindowBackgroundPixmap (disp, win, None);

XMapRaised (disp, win);

XSync (disp, FALSE);

gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (GST_MESSAGE_SRC (message)),
    win);

gst_message_unref (message);

return GST_BUS_DROP;
}
...
int
main (int argc, char **argv)
{
...
bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
gst_bus_set_sync_handler (bus, (GstBusSyncHandler) create_window, pipeline,
       NULL);
...
}

Two basic usage scenarios

There are two basic usage scenarios: in the simplest case, the application uses #playbin or #playsink or knows exactly what particular element is used for video output, which is usually the case when the application creates the videosink to use (e.g. #xvimagesink, #ximagesink, etc.) itself; in this case, the application can just create the videosink element, create and realize the window to render the video on and then call [gstvideo.video_overlay.VideoOverlay.setWindowHandle] directly with the XID or native window handle, before starting up the pipeline. As #playbin and #playsink implement the video overlay interface and proxy it transparently to the actual video sink even if it is created later, this case also applies when using these elements.

In the other and more common case, the application does not know in advance what GStreamer video sink element will be used for video output. This is usually the case when an element such as #autovideosink is used. In this case, the video sink element itself is created asynchronously from a GStreamer streaming thread some time after the pipeline has been started up. When that happens, however, the video sink will need to know right then whether to render onto an already existing application window or whether to create its own window. This is when it posts a prepare-window-handle message, and that is also why this message needs to be handled in a sync bus handler which will be called from the streaming thread directly (because the video sink will need an answer right then).

As response to the prepare-window-handle element message in the bus sync handler, the application may use [gstvideo.video_overlay.VideoOverlay.setWindowHandle] to tell the video sink to render onto an existing window surface. At this point the application should already have obtained the window handle / XID, so it just needs to set it. It is generally not advisable to call any GUI toolkit functions or window system functions from the streaming thread in which the prepare-window-handle message is handled, because most GUI toolkits and windowing systems are not thread-safe at all and a lot of care would be required to co-ordinate the toolkit and window system calls of the different threads (Gtk+ users please note: prior to Gtk+ 2.18 GDK_WINDOW_XID was just a simple structure access, so generally fine to do within the bus sync handler; this macro was changed to a function call in Gtk+ 2.18 and later, which is likely to cause problems when called from a sync handler; see below for a better approach without GDK_WINDOW_XID used in the callback).

GstVideoOverlay and Gtk+

#include <gst/video/videooverlay.h>
#include <gtk/gtk.h>
#ifdef GDK_WINDOWING_X11
#include <gdk/gdkx.h>  // for GDK_WINDOW_XID
#endif
#ifdef GDK_WINDOWING_WIN32
#include <gdk/gdkwin32.h>  // for GDK_WINDOW_HWND
#endif
...
static guintptr video_window_handle = 0;
...
static GstBusSyncReply
bus_sync_handler (GstBus * bus, GstMessage * message, gpointer user_data)
{
// ignore anything but 'prepare-window-handle' element messages
if (!gst_is_video_overlay_prepare_window_handle_message (message))
  return GST_BUS_PASS;

if (video_window_handle != 0) {
  GstVideoOverlay *overlay;

  // GST_MESSAGE_SRC (message) will be the video sink element
  overlay = GST_VIDEO_OVERLAY (GST_MESSAGE_SRC (message));
  gst_video_overlay_set_window_handle (overlay, video_window_handle);
} else {
  g_warning ("Should have obtained video_window_handle by now!");
}

gst_message_unref (message);
return GST_BUS_DROP;
}
...
static void
video_widget_realize_cb (GtkWidget * widget, gpointer data)
{
#if GTK_CHECK_VERSION(2,18,0)
 // Tell Gtk+/Gdk to create a native window for this widget instead of
 // drawing onto the parent widget.
 // This is here just for pedagogical purposes, GDK_WINDOW_XID will call
 // it as well in newer Gtk versions
 if (!gdk_window_ensure_native (widget->window))
   g_error ("Couldn't create native window needed for GstVideoOverlay!");
#endif

#ifdef GDK_WINDOWING_X11
 {
   gulong xid = GDK_WINDOW_XID (gtk_widget_get_window (video_window));
   video_window_handle = xid;
 }
#endif
#ifdef GDK_WINDOWING_WIN32
 {
   HWND wnd = GDK_WINDOW_HWND (gtk_widget_get_window (video_window));
   video_window_handle = (guintptr) wnd;
 }
#endif
}
...
int
main (int argc, char **argv)
{
 GtkWidget *video_window;
 GtkWidget *app_window;
 ...
 app_window = gtk_window_new (GTK_WINDOW_TOPLEVEL);
 ...
 video_window = gtk_drawing_area_new ();
 g_signal_connect (video_window, "realize",
     G_CALLBACK (video_widget_realize_cb), NULL);
 gtk_widget_set_double_buffered (video_window, FALSE);
 ...
 // usually the video_window will not be directly embedded into the
 // application window like this, but there will be many other widgets
 // and the video window will be embedded in one of them instead
 gtk_container_add (GTK_CONTAINER (ap_window), video_window);
 ...
 // show the GUI
 gtk_widget_show_all (app_window);

 // realize window now so that the video window gets created and we can
 // obtain its XID/HWND before the pipeline is started up and the videosink
 // asks for the XID/HWND of the window to render onto
 gtk_widget_realize (video_window);

 // we should have the XID/HWND now
 g_assert (video_window_handle != 0);
 ...
 // set up sync handler for setting the xid once the pipeline is started
 bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
 gst_bus_set_sync_handler (bus, (GstBusSyncHandler) bus_sync_handler, NULL,
     NULL);
 gst_object_unref (bus);
 ...
 gst_element_set_state (pipeline, GST_STATE_PLAYING);
 ...
}

GstVideoOverlay and Qt

#include <glib.h>;
#include <gst/gst.h>;
#include <gst/video/videooverlay.h>;

#include <QApplication>;
#include <QTimer>;
#include <QWidget>;

int main(int argc, char *argv[])
{
 if (!g_thread_supported ())
   g_thread_init (NULL);

 gst_init (&argc, &argv);
 QApplication app(argc, argv);
 app.connect(&app, SIGNAL(lastWindowClosed()), &app, SLOT(quit ()));

 // prepare the pipeline

 GstElement *pipeline = gst_pipeline_new ("xvoverlay");
 GstElement *src = gst_element_factory_make ("videotestsrc", NULL);
 GstElement *sink = gst_element_factory_make ("xvimagesink", NULL);
 gst_bin_add_many (GST_BIN (pipeline), src, sink, NULL);
 gst_element_link (src, sink);

 // prepare the ui

 QWidget window;
 window.resize(320, 240);
 window.show();

 WId xwinid = window.winId();
 gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (sink), xwinid);

 // run the pipeline

 GstStateChangeReturn sret = gst_element_set_state (pipeline,
     GST_STATE_PLAYING);
 if (sret == GST_STATE_CHANGE_FAILURE) {
   gst_element_set_state (pipeline, GST_STATE_NULL);
   gst_object_unref (pipeline);
   // Exit application
   QTimer::singleShot(0, QApplication::activeWindow(), SLOT(quit()));
 }

 int ret = app.exec();

 window.hide();
 gst_element_set_state (pipeline, GST_STATE_NULL);
 gst_object_unref (pipeline);

 return ret;
}

Functions to create and handle overlay compositions on video buffers.

An overlay composition describes one or more overlay rectangles to be blended on top of a video buffer.

This API serves two main purposes:

• it can be used to attach overlay information (subtitles or logos)

to non-raw video buffers such as GL/VAAPI/VDPAU surfaces. The actual blending of the overlay can then be done by e.g. the video sink that processes these non-raw buffers.

• it can also be used to blend overlay rectangles on top of raw video

buffers, thus consolidating blending functionality for raw video in one place.

Together, this allows existing overlay elements to easily handle raw and non-raw video as input in without major changes (once the overlays have been put into a #GstVideoOverlayComposition object anyway) - for raw video the overlay can just use the blending function to blend the data on top of the video, and for surface buffers it can just attach them to the buffer and let the sink render the overlays.

Extra buffer metadata describing image overlay data.

#GstVideoOverlay interface

An opaque video overlay rectangle object. A rectangle contains a single overlay rectangle which can be added to a composition.

Helper structure representing a rectangular area.

Extra buffer metadata describing an image region of interest

#GstVideoResampler is a structure which holds the information required to perform various kinds of resampling filtering.

H.264 H.265 metadata from SEI User Data Unregistered messages

#GstVideoScaler is a utility object for rescaling and resampling video frames using various interpolation / sampling methods.

Provides useful functions and a base class for video sinks.

GstVideoSink will configure the default base sink to drop frames that arrive later than 20ms as this is considered the default threshold for observing out-of-sync frames.

The video sink class structure. Derived classes should override the @show_frame virtual function.

Description of a tile. This structure allow to describe arbitrary tile dimensions and sizes.

@field_count must be 0 for progressive video and 1 or 2 for interlaced.

A representation of a SMPTE time code.

@hours must be positive and less than 24. Will wrap around otherwise. @minutes and @seconds must be positive and less than 60. @frames must be less than or equal to @config.fps_n / @config.fps_d These values are NOT automatically normalized.

Supported frame rates: 30000/1001, 60000/1001 (both with and without drop frame), and integer frame rates e.g. 25/1, 30/1, 50/1, 60/1.

The configuration of the time code.

A representation of a difference between two #GstVideoTimeCode instances. Will not necessarily correspond to a real timecode (e.g. 00:00:10;00)

Extra buffer metadata describing the GstVideoTimeCode of the frame.

Each frame is assumed to have its own timecode, i.e. they are not automatically incremented/interpolated.

An encoder for writing ancillary data to the Vertical Blanking Interval lines of component signals.

A parser for detecting and extracting @GstVideoAncillary data from Vertical Blanking Interval lines of component signals.