[Windows] The built-in Windows Source Resolver for Media Foundation ignores custom Bytestream Handler if OMAF file contains AAC audio stream

[richard42]

Unity version

any (will add specific version later)

Unity editor platform

Windows

AVPro Video edition

Core

AVPro Video version

any (will add specific version later)

Device hardware

Any PC with NVidia GPU

Which Windows version are you using?

10

Graphics API

D3D 11

Video API

Media Foundation

Audio output

System Direct

Any other Media Player component configuration required to reproduce the issue.

Proprietary OMAF byte stream handler / media source must be installed.

Which output component(s) are you using?

No response

Any other component configuration required to reproduce the issue.

No response

The issue

I am working under contract for a group within Panasonic which is building various pieces of VR technology. I have written a number of tools that they are using for encoding and playing omnidirectional video using the MPEG OMAF standard, ISO 23090-2. Another developer has written a player which works with their VR headset using Unity and AVProVideo for rendering. Of course Windows cannot decode OMAF video files on its own, so I wrote a Media Foundation custom Bytestream Handler (as described here.) The Panasonic Unity player uses AVProVideo, which in turn uses Microsoft Media Foundation, which in turn uses my registered custom bytestream handler to open and decode the OMAF file. My custom media source reads multiple video tracks containing compressed HEVC or AVC images, which are decoded using the CUVID/NVDEC GPU decoder API with NVidia GPU(s), and the decoded sub-frames are composited together with CUDA kernels to create the full omnidirectional video frames (10k up to 16k in width), and these full resolution frames are passed through Media Foundation and AVProVideo to the Panasonic Unity player, which determines head orientation and crops and de-warps the video and displays it on the output hardware (either a VR headset or normal window).

It almost works perfectly. But there is one vexing problem that I have encountered. It's not exactly a bug in AVProVideo; I think it's a bug in Windows, but I also don't think Microsoft will fix it any time soon. And a slightly more sophisticated Media Foundation file opening mechanism in AVProVideo should be able to circumvent the problem. If you could help us avoid this problem, it would be much appreciated.

My custom Media Foundation Bytestream Handler object for decoding OMAF files is properly registered with the Windows media foundation source resolver. The file extension ".omaf" is registered in HKEY_LOCAL_MACHINE/Software/Microsoft/Windows Media Foundation/ByteStreamHandlers/.omaf with a key containing the GUID of the byte stream handler (83DE7D63-8969-4972-9EEE-52F0FF4C3592), and the path to the DLL for the bytestream handler is registered in HKEY_CLASSES_ROOT/CLSID/{83DE7D63-8969-4972-9EEE-52F0FF4C3592}/InProcServer32/(Default).

And it works perfectly if the .omaf file doesn't contain an audio track. The Windows Source Resolver selects my bytestream handler for the .omaf file, it loads my DLL, which decodes the file, and everything in the giant software stack works and the playback is great.

However, if the .omaf file contains an audio track, then it doesn't work. The OMAF file format standard (ISO 23090-2) is based on MP4 (aka Quicktime). So the .omaf file is internally a legal ISO MP4 file. The "ftyp" atom's major_brand is "hevi" and the list of compatible_brands is empty. It has "free" and "mdat" atoms, followed by an "moov" atom with one "mvhd" atom followed by a bunch of "trak" atoms, one for each video and audio track. Typically there will be 4-16 video tracks and 0-1 audio tracks. I have found that if the ".omaf" file contains an audio track (AAC, as specified by the 23090-2 standard) then the Windows 10 Source Resolver will completely ignore my registered ".omaf" byte stream handler (it won't load the DLL at all) and it will instead open up the OMAF file with some other byte stream handler, presumably a default Windows MP4 media source. However this media source cannot handle OMAF multi-track video, and it fails to produce any video frames.

Through trial and error, I have discovered two ways to circumvent this problem.

The first is to break the format of the .omaf file, so that it is not a legal ISO MP4 file, by setting the first four bytes of the file to zeros. Normally the first four bytes will be "0,0,0,16", which is the size of the first atom, the "ftyp" atom. When I set these bytes to zero, the file is no longer a legal ISO MP4 file, and the Windows Source Resolver doesn't try to get clever with it, and properly hands the file off to my custom byte stream handler.

However the business group is unhappy with this solution. They would like their Unity VR player to be able to work with properly compliant ISO MP4 files. There is a way to do this, but it requires explicitly selecting my Media Foundation custom Bytestream handler to use for decoding ".omaf" files. I have written a proof-of-concept implementation of this method using the "MFPlayer2" test program which is included in the Microsoftt Windows 7 sample programs (located here).

Normally, the function in the MFPlayer test application which starts a clip playing looks like this:

HRESULT MFPlayer2::OpenURL(const WCHAR *sURL)
{
    HRESULT hr = S_OK;
    if (sURL == NULL)
    {
        return E_POINTER;
    }
    if (m_pPlayer == NULL)
    {
        return E_UNEXPECTED;
    }
    // Create a new media item for this URL.
    hr = m_pPlayer->CreateMediaItemFromURL(sURL, FALSE, 0, NULL);
    // The CreateMediaItemFromURL method completes asynchronously. When it does,
    // MFPlay sends an MFP_EVENT_TYPE_MEDIAITEM_CREATED event.
    return hr;
}

Inside of that CreateMediaItemFromURL function, the media foundation framework runs the Windows source resolver and creates the necessary objects and starts it playing. In order to avoid the bug in the Source Resolver, I modified the MFPlayer2 application to explicitly choose my OMAF Media Source to decode a file. The modified player code looks like this:

HRESULT MFPlayer2::OpenURL(const WCHAR *sURL)
{
    HRESULT hr = S_OK;
    if (sURL == NULL)
    {
        return E_POINTER;
    }
    if (m_pPlayer == NULL)
    {
        return E_UNEXPECTED;
    }
    CLSID CLSID_OMAFReader;
    hr = CLSIDFromString(OLESTR("{83DE7D63-8969-4972-9EEE-52F0FF4C3592}"), &CLSID_OMAFReader);
    IMFByteStreamHandler* pByteStreamHandler = NULL;
    hr = CoCreateInstance(CLSID_OMAFReader, NULL, CLSCTX_INPROC_SERVER, IID_IMFByteStreamHandler, (void**)&pByteStreamHandler);
    IMFByteStream *pByteStream = NULL;
    hr = MFCreateFile(MF_ACCESSMODE_READ, MF_OPENMODE_FAIL_IF_NOT_EXIST, MF_FILEFLAGS_NONE, sURL, &pByteStream);
    IMFAsyncCallback *pCallback = NULL;
    hr = this->QueryInterface(IID_PPV_ARGS(&pCallback));
    hr = pByteStreamHandler->BeginCreateObject(pByteStream, sURL, MF_RESOLUTION_MEDIASOURCE, NULL, NULL, pCallback, pByteStreamHandler);
    return hr;
}

I also needed to inherit the player class in the example application from a new interface (IMFAsyncCallback) and add a new function to handle the callback like this:

HRESULT STDMETHODCALLTYPE MFPlayer2::Invoke(IMFAsyncResult* pAsyncResult)
{
    // get the byte stream handler from the state member in the async result
    IMFByteStreamHandler* pByteStreamHandler = NULL;
    HRESULT hr = pAsyncResult->GetState((IUnknown**) &pByteStreamHandler);
    // get the media source
    IMFMediaSource* pMediaSource = NULL;
    MF_OBJECT_TYPE objectType;
    hr = pByteStreamHandler->EndCreateObject(pAsyncResult, &objectType, (IUnknown**) &pMediaSource);
    // set the media source to the player and start playing
    hr = m_pPlayer->CreateMediaItemFromObject(pMediaSource, FALSE, 0, NULL);
    return hr;
}

Note that this is not production code; it is not checking any return values and the memory handling is not correct. I just wrote it up to test whether or not this method could be used, and it does work. With these changes, the MFPlayer2 application is able to load and play the MP4-compliant OMAF file containing an audio track as well as video.

Could you add a similar mechanism to the MediaFoundation file opening code in AVProVideo so that it can be used with the Panasonic Unity Player to open MP4-compliant OMAF files with our custom decoder?

Media information

I will get approval to share a test file with you and upload it soon.

Log output

[Chris-RH]

Hi @richard42 ,

We've had a chat, and we think that we might be able to do something with this.
If you could send a sample of the video(s) used for testing and your scripting then that would be great. In fact, it might be helpful if you send your test project that you've used as proof of concept, so that we are all on the same page. Please email unitysupport@renderheads.com. Anything sent to us is kept confidential :)

[richard42]

Thanks for being so responsive on this issue. We will continue moving forward with our separate email conversation, but I wanted to add the following information to this issue report:

1.      Which version of Unity are you using?
          2022.3.13f1
2.      Which version of AVPro Video are you using?
          3.2.0
3.      Which edition of AVPro Video are you using?
          Ultra
4.      What audio output does the player use?
          System Direct
5.      Which output component(s) are you using?
         Apply to Material