gstaudio.c.types

Different possible reasons for discontinuities. This enum is useful for the custom slave method.

Different possible clock slaving algorithms used when the internal audio clock is not selected as the pipeline master clock.

Different possible clock slaving algorithms when the internal audio clock was not selected as the pipeline clock.

Mode in which the CD audio source operates. Influences timestamping, EOS handling and seeking.

Flags passed to [gstaudio.audio_channel_mixer.AudioChannelMixer.new_]

Audio channel positions.

These are the channels defined in SMPTE 2036-2-2008 Table 1 for 22.2 audio systems with the Surround and Wide channels from DTS Coherent Acoustics (v.1.3.1) and 10.2 and 7.1 layouts. In the caps the actual channel layout is expressed with a channel count and a channel mask, which describes the existing channels. The positions in the bit mask correspond to the enum values. For negotiation it is allowed to have more bits set in the channel mask than the number of channels to specify the allowed channel positions but this is not allowed in negotiated caps. It is not allowed in any situation other than the one mentioned below to have less bits set in the channel mask than the number of channels.

@GST_AUDIO_CHANNEL_POSITION_MONO can only be used with a single mono channel that has no direction information and would be mixed into all directional channels. This is expressed in caps by having a single channel and no channel mask.

@GST_AUDIO_CHANNEL_POSITION_NONE can only be used if all channels have this position. This is expressed in caps by having a channel mask with no bits set.

As another special case it is allowed to have two channels without a channel mask. This implicitly means that this is a stereo stream with a front left and front right channel.

Extra flags passed to [gstaudio.audio_converter.AudioConverter.new_] and [gstaudio.audio_converter.AudioConverter.samples].

Set of available dithering methods.

Extra audio flags

Enum value describing the most common audio formats.

The different audio flags that a format info can have.

Layout of the audio samples for the different channels.

Set of available noise shaping methods

The different flags that can be used when packing and unpacking.

Extra flags that can be passed to [gstaudio.audio_quantize.AudioQuantize.new_]

The different filter interpolation methods.

Select for the filter tables should be set up.

Different resampler flags.

Different subsampling and upsampling methods

The format of the samples in the ringbuffer.

The state of the ringbuffer.

Enum value describing how DSD bits are grouped.

Different representations of a stream volume. [gstaudio.stream_volume.StreamVolume.convertVolume] allows to convert between the different representations.

Formulas to convert from a linear to a cubic or dB volume are cbrt(val) and 20 * log10 (val).

Subclasses must use (a subclass of) #GstAudioAggregatorPad for both their source and sink pads, [gst.element_class.ElementClass.addStaticPadTemplateWithGtype] is a convenient helper.

#GstAudioAggregator can perform conversion on the data arriving on its sink pads, based on the format expected downstream: in order to enable that behaviour, the GType of the sink pads must either be a (subclass of) #GstAudioAggregatorConvertPad to use the default #GstAudioConverter implementation, or a subclass of #GstAudioAggregatorPad implementing #GstAudioAggregatorPadClass.convert_buffer.

To allow for the output caps to change, the mechanism is the same as above, with the GType of the source pad.

See #GstAudioMixer for an example.

When conversion is enabled, #GstAudioAggregator will accept any type of raw audio caps and perform conversion on the data arriving on its sink pads, with whatever downstream expects as the target format.

In case downstream caps are not fully fixated, it will use the first configured sink pad to finish fixating its source pad caps.

A notable exception for now is the sample rate, sink pads must have the same sample rate as either the downstream requirement, or the first configured pad, or a combination of both (when downstream specifies a range or a set of acceptable rates).

The #GstAggregator::samples-selected signal is provided with some additional information about the output buffer:

• "offset" G_TYPE_UINT64 Offset in samples since segment start

for the position that is next to be filled in the output buffer.

• "frames" G_TYPE_UINT Number of frames per output buffer.

In addition the [gstbase.aggregator.Aggregator.peekNextSample] function returns additional information in the info #GstStructure of the returned sample:

• "output-offset" G_TYPE_UINT64 Sample offset in output segment relative to

the output segment's start where the current position of this input buffer would be placed

• "position" G_TYPE_UINT current position in the input buffer in samples
• "size" G_TYPE_UINT size of the input buffer in samples

An implementation of GstPad that can be used with #GstAudioAggregator.

See #GstAudioAggregator for more details.

The default implementation of GstPad used with #GstAudioAggregator

This is the base class for audio sinks. Subclasses need to implement the ::create_ringbuffer vmethod. This base class will then take care of writing samples to the ringbuffer, synchronisation, clipping and flushing.

#GstAudioBaseSink class. Override the vmethod to implement functionality.

This is the base class for audio sources. Subclasses need to implement the ::create_ringbuffer vmethod. This base class will then take care of reading samples from the ringbuffer, synchronisation and flushing.

#GstAudioBaseSrc class. Override the vmethod to implement functionality.

A structure containing the result of an audio buffer map operation, which is executed with [gstaudio.audio_buffer.AudioBuffer.map]. For non-interleaved (planar) buffers, the beginning of each channel in the buffer has its own pointer in the @planes array. For interleaved buffers, the @planes array only contains one item, which is the pointer to the beginning of the buffer, and @n_planes equals 1.

The different channels in @planes are always in the GStreamer channel order.

Provides a base class for CD digital audio (CDDA) sources, which handles things like seeking, querying, discid calculation, tags, and buffer timestamping.

Using GstAudioCdSrc-based elements in applications

GstAudioCdSrc registers two #GstFormat<!-- -->s of its own, namely the "track" format and the "sector" format. Applications will usually only find the "track" format interesting. You can retrieve that #GstFormat for use in seek events or queries with gst_format_get_by_nick("track").

In order to query the number of tracks, for example, an application would set the CDDA source element to READY or PAUSED state and then query the the number of tracks via [gst.element.Element.queryDuration] using the track format acquired above. Applications can query the currently playing track in the same way.

Alternatively, applications may retrieve the currently playing track and the total number of tracks from the taglist that will posted on the bus whenever the CD is opened or the currently playing track changes. The taglist will contain GST_TAG_TRACK_NUMBER and GST_TAG_TRACK_COUNT tags.

Applications playing back CD audio using playbin and cdda://n URIs should issue a seek command in track format to change between tracks, rather than setting a new cdda://n+1 URI on playbin (as setting a new URI on playbin involves closing and re-opening the CD device, which is much much slower).

Tags and meta-information

CDDA sources will automatically emit a number of tags, details about which can be found in the libgsttag documentation. Those tags are: #GST_TAG_CDDA_CDDB_DISCID, #GST_TAG_CDDA_CDDB_DISCID_FULL, #GST_TAG_CDDA_MUSICBRAINZ_DISCID, #GST_TAG_CDDA_MUSICBRAINZ_DISCID_FULL, among others.

Tracks and Table of Contents (TOC)

Applications will be informed of the available tracks via a TOC message on the pipeline's #GstBus. The #GstToc will contain a #GstTocEntry for each track, with information about each track. The duration for each track can be retrieved via the #GST_TAG_DURATION tag from each entry's tag list, or calculated via [gst.toc_entry.TocEntry.getStartStopTimes]. The track entries in the TOC will be sorted by track number.

Audio CD source base class.

CD track abstraction to communicate TOC entries to the base class.

This structure is only for use by sub-classed in connection with [gstaudio.audio_cd_src.AudioCdSrc.addTrack].

Applications will be informed of the available tracks via a TOC message on the pipeline's #GstBus instead.

Extra buffer metadata describing how much audio has to be clipped from the start or end of a buffer. This is used for compressed formats, where the first frame usually has some additional samples due to encoder and decoder delays, and the last frame usually has some additional samples to be able to fill the complete last frame.

This is used to ensure that decoded data in the end has the same amount of samples, and multiply decoded streams can be gaplessly concatenated.

#GstAudioClock makes it easy for elements to implement a #GstClock, they simply need to provide a function that returns the current clock time.

This object is internally used to implement the clock in #GstAudioBaseSink.

This object is used to convert audio samples from one format to another. The object can perform conversion of:

• audio format with optional dithering and noise shaping

• audio samplerate

• audio channels and channel layout

This base class is for audio decoders turning encoded data into raw audio samples.

GstAudioDecoder and subclass should cooperate as follows.

Configuration

• Initially, GstAudioDecoder calls @start when the decoder element

is activated, which allows subclass to perform any global setup. Base class (context) parameters can already be set according to subclass capabilities (or possibly upon receive more information in subsequent @set_format).

• GstAudioDecoder calls @set_format to inform subclass of the format

of input audio data that it is about to receive. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.

• GstAudioDecoder calls @stop at end of all processing.

As of configuration stage, and throughout processing, GstAudioDecoder provides various (context) parameters, e.g. describing the format of output audio data (valid when output caps have been set) or current parsing state. Conversely, subclass can and should configure context to inform base class of its expectation w.r.t. buffer handling.

Data processing

• Base class gathers input data, and optionally allows subclass

to parse this into subsequently manageable (as defined by subclass) chunks. Such chunks are subsequently referred to as 'frames', though they may or may not correspond to 1 (or more) audio format frame.

• Input frame is provided to subclass' @handle_frame.
• If codec processing results in decoded data, subclass should call

@gst_audio_decoder_finish_frame to have decoded data pushed downstream.

• Just prior to actually pushing a buffer downstream,

it is passed to @pre_push. Subclass should either use this callback to arrange for additional downstream pushing or otherwise ensure such custom pushing occurs after at least a method call has finished since setting src pad caps.

• During the parsing process GstAudioDecoderClass will handle both

srcpad and sinkpad events. Sink events will be passed to subclass if @event callback has been provided.

Shutdown phase

• GstAudioDecoder class calls @stop to inform the subclass that data

parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured. This is typically when base class calls subclass' @set_format function, though it might be delayed until calling @gst_audio_decoder_finish_frame.

In summary, above process should have subclass concentrating on codec data processing while leaving other matters to base class, such as most notably timestamp handling. While it may exert more control in this area (see e.g. @pre_push), it is very much not recommended.

In particular, base class will try to arrange for perfect output timestamps as much as possible while tracking upstream timestamps. To this end, if deviation between the next ideal expected perfect timestamp and upstream exceeds #GstAudioDecoder:tolerance, then resync to upstream occurs (which would happen always if the tolerance mechanism is disabled).

In non-live pipelines, baseclass can also (configurably) arrange for output buffer aggregation which may help to redue large(r) numbers of small(er) buffers being pushed and processed downstream. Note that this feature is only available if the buffer layout is interleaved. For planar buffers, the decoder implementation is fully responsible for the output buffer size.

On the other hand, it should be noted that baseclass only provides limited seeking support (upon explicit subclass request), as full-fledged support should rather be left to upstream demuxer, parser or alike. This simple approach caters for seeking and duration reporting using estimated input bitrates.

Things that subclass need to take care of:

• Provide pad templates
• Set source pad caps when appropriate
• Set user-configurable properties to sane defaults for format and

implementing codec at hand, and convey some subclass capabilities and expectations in context.

• Accept data in @handle_frame and provide encoded results to

@gst_audio_decoder_finish_frame. If it is prepared to perform PLC, it should also accept NULL data in @handle_frame and provide for data for indicated duration.

Subclasses can override any of the available virtual methods or not, as needed. At minimum @handle_frame (and likely @set_format) needs to be overridden.

Extra buffer metadata describing audio downmixing matrix. This metadata is attached to audio buffers and contains a matrix to downmix the buffer number of channels to @channels.

@matrix is an two-dimensional array of @to_channels times @from_channels coefficients, i.e. the i-th output channels is constructed by multiplicating the input channels with the coefficients in @matrix[i] and taking the sum of the results.

This base class is for audio encoders turning raw audio samples into encoded audio data.

GstAudioEncoder and subclass should cooperate as follows.

Configuration

• Initially, GstAudioEncoder calls @start when the encoder element

is activated, which allows subclass to perform any global setup.

• GstAudioEncoder calls @set_format to inform subclass of the format

of input audio data that it is about to receive. Subclass should setup for encoding and configure various base class parameters appropriately, notably those directing desired input data handling. While unlikely, it might be called more than once, if changing input parameters require reconfiguration.

• GstAudioEncoder calls @stop at end of all processing.

As of configuration stage, and throughout processing, GstAudioEncoder maintains various parameters that provide required context, e.g. describing the format of input audio data. Conversely, subclass can and should configure these context parameters to inform base class of its expectation w.r.t. buffer handling.

Data processing

• Base class gathers input sample data (as directed by the context's

frame_samples and frame_max) and provides this to subclass' @handle_frame.

• If codec processing results in encoded data, subclass should call

[gstaudio.audio_encoder.AudioEncoder.finishFrame] to have encoded data pushed downstream. Alternatively, it might also call [gstaudio.audio_encoder.AudioEncoder.finishFrame] (with a NULL buffer and some number of dropped samples) to indicate dropped (non-encoded) samples.

• Just prior to actually pushing a buffer downstream,

it is passed to @pre_push.

• During the parsing process GstAudioEncoderClass will handle both

srcpad and sinkpad events. Sink events will be passed to subclass if @event callback has been provided.

Shutdown phase

• GstAudioEncoder class calls @stop to inform the subclass that data

parsing will be stopped.

Subclass is responsible for providing pad template caps for source and sink pads. The pads need to be named "sink" and "src". It also needs to set the fixed caps on srcpad, when the format is ensured. This is typically when base class calls subclass' @set_format function, though it might be delayed until calling @gst_audio_encoder_finish_frame.

In summary, above process should have subclass concentrating on codec data processing while leaving other matters to base class, such as most notably timestamp handling. While it may exert more control in this area (see e.g. @pre_push), it is very much not recommended.

In particular, base class will either favor tracking upstream timestamps (at the possible expense of jitter) or aim to arrange for a perfect stream of output timestamps, depending on #GstAudioEncoder:perfect-timestamp. However, in the latter case, the input may not be so perfect or ideal, which is handled as follows. An input timestamp is compared with the expected timestamp as dictated by input sample stream and if the deviation is less than #GstAudioEncoder:tolerance, the deviation is discarded. Otherwise, it is considered a discontuinity and subsequent output timestamp is resynced to the new position after performing configured discontinuity processing. In the non-perfect-timestamp case, an upstream variation exceeding tolerance only leads to marking DISCONT on subsequent outgoing (while timestamps are adjusted to upstream regardless of variation). While DISCONT is also marked in the perfect-timestamp case, this one optionally (see #GstAudioEncoder:hard-resync) performs some additional steps, such as clipping of (early) input samples or draining all currently remaining input data, depending on the direction of the discontuinity.

If perfect timestamps are arranged, it is also possible to request baseclass (usually set by subclass) to provide additional buffer metadata (in OFFSET and OFFSET_END) fields according to granule defined semantics currently needed by oggmux. Specifically, OFFSET is set to granulepos (= sample count including buffer) and OFFSET_END to corresponding timestamp (as determined by same sample count and sample rate).

Things that subclass need to take care of:

• Provide pad templates
• Set source pad caps when appropriate
• Inform base class of buffer processing needs using context's

frame_samples and frame_bytes.

• Set user-configurable properties to sane defaults for format and

implementing codec at hand, e.g. those controlling timestamp behaviour and discontinuity processing.

• Accept data in @handle_frame and provide encoded results to

[gstaudio.audio_encoder.AudioEncoder.finishFrame].

Subclasses can override any of the available virtual methods or not, as needed. At minimum @set_format and @handle_frame needs to be overridden.

#GstAudioFilter is a #GstBaseTransform<!-- -->-derived base class for simple audio filters, ie. those that output the same format that they get as input.

#GstAudioFilter will parse the input format for you (with error checking) before calling your setup function. Also, elements deriving from #GstAudioFilter may use [gstaudio.audio_filter_class.AudioFilterClass.addPadTemplates] from their class_init function to easily configure the set of caps/formats that the element is able to handle.

Derived classes should override the #GstAudioFilterClass.setup() and #GstBaseTransformClass.transform_ip() and/or #GstBaseTransformClass.transform() virtual functions in their class_init function.

In addition to the @setup virtual function, you should also override the GstBaseTransform::transform and/or GstBaseTransform::transform_ip virtual function.

Information for an audio format.

Information describing audio properties. This information can be filled in from GstCaps with [gstaudio.audio_info.AudioInfo.fromCaps].

Use the provided macros to access the info in this structure.

Meta containing Audio Level Indication: https://tools.ietf.org/html/rfc6464

#GstAudioDownmixMeta defines an audio downmix matrix to be send along with audio buffers. These functions in this module help to create and attach the meta as well as extracting it.

#GstAudioResampler is a structure which holds the information required to perform various kinds of resampling filtering.

This object is the base class for audio ringbuffers used by the base audio source and sink classes.

The ringbuffer abstracts a circular buffer of data. One reader and one writer can operate on the data from different threads in a lockfree manner. The base class is sufficiently flexible to be used as an abstraction for DMA based ringbuffers as well as a pure software implementations.

The vmethods that subclasses can override to implement the ringbuffer.

The structure containing the format specification of the ringbuffer.

When @type is GST_AUDIO_RING_BUFFER_FORMAT_TYPE_DSD, the @dsd_format is valid (otherwise it is unused). Also, when DSD is the sample type, only the rate, channels, position, and bpf fields in @info are populated.

This is the most simple base class for audio sinks that only requires subclasses to implement a set of simple functions:

• open() :Open the device.

• prepare() :Configure the device with the specified format.

• write() :Write samples to the device.

• reset() :Unblock writes and flush the device.

• delay() :Get the number of samples written but not yet played

by the device.

• unprepare() :Undo operations done by prepare.

• close() :Close the device.

All scheduling of samples and timestamps is done in this base class together with #GstAudioBaseSink using a default implementation of a #GstAudioRingBuffer that uses threads.

This is the most simple base class for audio sources that only requires subclasses to implement a set of simple functions:

• open() :Open the device.
• prepare() :Configure the device with the specified format.
• read() :Read samples from the device.
• reset() :Unblock reads and flush the device.
• delay() :Get the number of samples in the device but not yet read.
• unprepare() :Undo operations done by prepare.
• close() :Close the device.

All scheduling of samples and timestamps is done in this base class together with #GstAudioBaseSrc using a default implementation of a #GstAudioRingBuffer that uses threads.

#GstAudioSrc class. Override the vmethod to implement functionality.

#GstAudioStreamAlign provides a helper object that helps tracking audio stream alignment and discontinuities, and detects discontinuities if possible.

See [gstaudio.audio_stream_align.AudioStreamAlign.new_] for a description of its parameters and [gstaudio.audio_stream_align.AudioStreamAlign.process] for the details of the processing.

Information describing DSD audio properties.

In DSD, the "sample format" is the bit. Unlike PCM, there are no further "sample formats" in DSD. However, in software, DSD bits are grouped into bytes (since dealing with individual bits is impractical), and these bytes in turn are grouped into words. This becomes relevant when interleaving channels and transmitting DSD data through audio APIs. The different types of grouping DSD bytes are referred to as the "DSD grouping forma" or just "DSD format". #GstDsdFormat has a list of valid ways of grouping DSD bytes into words.

DSD rates are equivalent to PCM sample rates, except that they specify how many DSD bytes are consumed per second. This refers to the bytes per second _per channel_; the rate does not change when the number of channel changes. (Strictly speaking, it would be more correct to measure the

more practical to use bytes.) In DSD, bit rates are always an integer multiple of the CD audio rate (44100) or the DAT rate (48000). DSD64-44x is 44100 * 64 = 2822400 bits per second, or 352800 bytes per second (the latter would be used in this info structure). DSD64-48x is 48000 * 64 = 3072000 bits per second, or 384000 bytes per second. #GST_DSD_MAKE_DSD_RATE_44x can be used for specifying DSD-44x rates, *and #GST_DSD_MAKE_DSD_RATE_48x can be used for specifying DSD-48x ones. Also, since DSD-48x is less well known, when the multiplier is given without the 44x/48x specifier, 44x is typically implied.

It is important to know that in DSD, different format widths correspond to different playtimes. That is, a word with 32 DSD bits covers two times as much playtime as a word with 16 DSD bits. This is in contrast to PCM, where one word (= one PCM sample) always covers a time period of 1/samplerate, no matter how many bits a PCM sample is made of. For this reason, DSD and PCM widths and strides cannot be used the same way.

Multiple channels are arranged in DSD data either interleaved or non- interleaved. This is similar to PCM. Interleaved layouts rotate between channels and words. First, word 0 of channel 0 is present. Then word 0 of channel 1 follows. Then word 0 of channel 2 etc. until all channels are through, then comes word 1 of channel 0 etc.

Non-interleaved data is planar. First, all words of channel 0 are present, then all words of channel 1 etc. Unlike interleaved data, non-interleaved data can be sparse, that is, there can be space in between the planes. the @positions array specifies the plane offsets.

In uncommon cases, the DSD bits in the data bytes can be stored in reverse order. For example, normally, in DSDU8, the first byte contains DSD bits 0 to 7, and the most significant bit of that byte is DSD bit 0. If this order is reversed, then bit 7 is the first one instead. In that ase, @reversed_bytes is set to TRUE.

Use the provided macros to access the info in this structure.

Buffer metadata describing planar DSD contents in the buffer. This is not needed for interleaved DSD data, and is required for non-interleaved (= planar) data.

The different channels in @offsets are always in the GStreamer channel order. Zero-copy channel reordering can be implemented by swapping the values in @offsets.

It is not allowed for channels to overlap in memory, i.e. for each i in [0, channels), the range [@offsets[i], @offsets[i] + @num_bytes_per_channel) must not overlap with any other such range.

It is, however, allowed to have parts of the buffer memory unused, by using @offsets and @num_bytes_per_channel in such a way that leave gaps on it. This is used to implement zero-copy clipping in non-interleaved buffers.

Obviously, due to the above, it is not safe to infer the number of valid bytes from the size of the buffer. You should always use the @num_bytes_per_channel variable of this metadata.

This interface is implemented by elements that provide a stream volume. Examples for such elements are #volume and #playbin.

Applications can use this interface to get or set the current stream volume. For this the "volume" #GObject property can be used or the helper functions [gstaudio.stream_volume.StreamVolume.setVolume] and [gstaudio.stream_volume.StreamVolume.getVolume]. This volume is always a linear factor, i.e. 0.0 is muted 1.0 is 100%. For showing the volume in a GUI it might make sense to convert it to a different format by using [gstaudio.stream_volume.StreamVolume.convertVolume]. Volume sliders should usually use a cubic volume.

Separate from the volume the stream can also be muted by the "mute" #GObject property or [gstaudio.stream_volume.StreamVolume.setMute] and [gstaudio.stream_volume.StreamVolume.getMute].

Elements that provide some kind of stream volume should implement the "volume" and "mute" #GObject properties and handle setting and getting of them properly. The volume property is defined to be a linear volume factor.