Edit

Share via

Video Processor MFT

The video processor MFT is a Microsoft Media Foundation transform (MFT) that performs colorspace conversion, tone mapping, video resizing, deinterlacing, frame rate conversion, rotation, cropping, spatial left and right view unpacking, mirroring, renderer effect processing, and sphere mapping.

CLSID

CLSID_VideoProcessorMFT

Interfaces

Input Formats

The following formats are supported values for the MF_MT_SUBTYPE value of the media type set via IMFTransform::SetInputType

  • MFVideoFormat_ABGR32
  • MFVideoFormat_ARGB32
  • MFVideoFormat_AYUV
  • MFVideoFormat_I420
  • MFVideoFormat_I422
  • MFVideoFormat_I444
  • MFVideoFormat_IYUV
  • MFVideoFormat_NV11
  • MFVideoFormat_NV12
  • MFVideoFormat_RGB24
  • MFVideoFormat_RGB32
  • MFVideoFormat_RGB555
  • MFVideoFormat_RGB565
  • MFVideoFormat_RGB8
  • MFVideoFormat_UYVY
  • MFVideoFormat_v410
  • MFVideoFormat_Y216
  • MFVideoFormat_Y41P
  • MFVideoFormat_Y41T
  • MFVideoFormat_Y42T
  • MFVideoFormat_YUY2
  • MFVideoFormat_YV12
  • MFVideoFormat_YVYU
  • MFVideoFormat_P010
  • MFVideoFormat_P016
  • MFVideoFormat_A16B16G16R16F
  • MFVideoFormat_A2R10G10B10
  • MFVideoFormat_Y210
  • MFVideoFormat_Y410
  • MFVideoFormat_Y416
  • MFVideoFormat_L8
  • MFVideoFormat_L16
  • MFVideoFormat_D16

Output Formats

The following formats are supported values for the MF_MT_SUBTYPE value of the media type set via IMFTransform::SetOutputType

  • MFVideoFormat_ARGB32
  • MFVideoFormat_AYUV
  • MFVideoFormat_I420
  • MFVideoFormat_I422
  • MFVideoFormat_I444
  • MFVideoFormat_IYUV
  • MFVideoFormat_NV12
  • MFVideoFormat_RGB24
  • MFVideoFormat_RGB32
  • MFVideoFormat_RGB555
  • MFVideoFormat_RGB565
  • MFVideoFormat_UYVY
  • MFVideoFormat_Y216
  • MFVideoFormat_YUY2
  • MFVideoFormat_YV12
  • MFVideoFormat_P010
  • MFVideoFormat_P016
  • MFVideoFormat_A16B16G16R16F
  • MFVideoFormat_A2R10G10B10
  • MFVideoFormat_Y210
  • MFVideoFormat_Y410
  • MFVideoFormat_Y416

Not every combination of input and output formats is supported. To test whether a conversion is supported, set the input type and then call IMFTransform::GetOutputAvailableType.

For more information about these formats, see Video Subtype GUIDs.

Color Space Transforms

The following attributes can be set on either the input or output types to change the color space of the content:

  • MF_MT_VIDEO_PRIMARIES
  • MF_MT_TRANSFER_FUNCTION
  • MF_MT_YUV_MATRIX
  • MF_MT_VIDEO_CHROMA_SITING
  • MF_MT_VIDEO_NOMINAL_RANGE
  • MF_MT_MAX_MASTERING_LUMINANCE
  • MF_MT_MAX_LUMINANCE_LEVEL
  • MF_MT_CUSTOM_VIDEO_PRIMARIES
  • MF_MT_PALETTE

Image Transforms

The following attributes can be set on either the input or output types to perform spatial transforms on the video:

  • MF_MT_PIXEL_ASPECT_RATIO
  • MF_MT_VIDEO_ROTATION
  • MF_MT_PAN_SCAN_APERTURE
  • MF_MT_GEOMETRIC_APERTURE
  • MF_MT_MINIMUM_DISPLAY_APERTURE
  • MF_MT_VIDEO_3D
  • MF_MT_VIDEO_3D_FORMAT

Framerate Transforms

The following attributes can be set on either the input or output types to perform temporal transforms on the video:

  • MF_MT_INTERLACE_MODE
  • MF_MT_FRAME_RATE

Note that if MF_XVP_DISABLE_FRC is set to TRUE then frame rate conversion is disabled, but deinterlacing will still be performed.

Transform Attributes

The following attributes can be set on the transform using IMFTransform::GetAttributes:

  • MF_XVP_DISABLE_FRC
  • MF_XVP_CALLER_ALLOCATES_OUTPUT
  • MF_LOW_LATENCY
  • MF_XVP_SAMPLE_LOCK_TIMEOUT
  • MF_ENABLE_3DVIDEO_OUTPUT
  • MF_VIDEO_PROCESSOR_ALGORITHM

Remarks

An instance of the video processor can be created in one of the following ways:

  • By calling MFTEnumEx. The video processor is registered under the MFT_CATEGORY_VIDEO_PROCESSOR category.
  • By calling the COM function CoCreateInstance passing it the CLSID CLSID_VideoProcessorMFT.

The following remarks pertain to working with source rectangles and destination rectangles in the Video Processor MFT. Source and destination rectangles are set with IMFVideoProcessorControl::SetDestinationRectangle and SetSourceRectangle and sometimes with IMFMediaEngineEx::UpdateVideoStream.

  • The source rectangle should be aligned and rounded to match the requirements of the color format of the frame inputted to the video processor. This is important because formats like 420 and 422 have requirements about the dimensions and offsets that can be created and accessed. For example, a source rectangle of {1, 0, 319, 240} (left, top, right, bottom) will be rounded to {2, 0, 320, 240} when the input format is 420.
  • Both the destination and source rectangle will always be clamped to fit inside their respective frames—the source rectangle to the source frame and the destination rectangle to the destination frame. This means that negative values are not meaningful—they will be always clamped to 0.
  • The source rectangle is in the destination frame's coordinate system, minus any destination rectangle. This means that transformations like rotation are "undone" on the source rectangle. Therefore, you do not need to know if the video was rotated or 3D unpacked. For example, you could draw a rectangle on top of the video tag, take the relative coordinates (relative to the video tag), normalize them (range 0 to 1) and pass them down as the source rectangle and they should work as expected, even if the video is being rotated.

The video processor supports GPU-accelerated video processing, using Microsoft Direct3D 11, or Direct3D 12. For more information, see MF_SA_D3D11_AWARE and MF_SA_D3D12_AWARE.

The video processor is used as the front-end converter for video rendering when using SVR which is exposed by the MediaPlayer and IMFMediaEngine components. Decoders, plugins or sources used by either of these components

Renderer Effects

The Video Processor MFT is the host component for Renderer effects. Renderer effects allow applications to plugin to the normal video transform and rendering process. For typical applications a video plugin (see: IMFMediaEngineEx::InsertVideoEffect) is sufficient. But some scenarios require more information about the exact conditions that a video is being rendered in. Consider a renderer effect for the following situations:

  • The plugin needs to know the resolution that the video is rendered at (e.g. to perform a special scaling operation like super resolution)
  • The plugin needs to know properties of the display that the video is being rendered on (e.g. to perform accurate tone mapping)
  • The plugin needs to know exactly the final output format of the presented video

To load a renderer effect configure the MF_MT_VIDEO_RENDERER_EXTENSION_PROFILE attribute on the input type of the Video Processor MFT.

Stereoscopic Video

The video processor supports the view unpacking operation on 3D video frames:

If the input frame contains two views packed in the same frame, the video processor can split the views into separate buffers, or extract the base view and discard the second view. To enable view unpacking, set the MF_ENABLE_3DVIDEO_OUTPUT attribute to MF3DVideoOutputType_Stereo or MF3DVideoOutputType_BaseView.

Requirements

Requirement Value
Header
Camerauicontrol.h

See also

Digital Signal Processors