Problem playing .bik movie files ingame while using d3d8to9

[Radio30]

This in Indiana Jones and the Emporer's Tomb.

All the intro and cut-scene movies are in .bik format and while using d3d8to9 the audio plays during the videos but the screen is black.
Any idea what could be causing this?

[elishacloud]

Yes this issue was introduced with Windows 10 Fall Creators Edition. If you are running any older version of Windows 10 (or other versions of Windows) this issue does not happen. I noticed this a while ago and was planning to file an issue on this, but you saved me the trouble.

I have seen this "black screen" issue on a number of games starting with the Creators Edition. See here for a similar example.

[HerMajestyDrMona]

Just information: I'm using Windows 10 Fall Creators Update (16299.248 currently) and .bik movies work fine in Empire Earth 2. They work with and without d3d8to9.

[elishacloud]

Yes, I believe the issue is specific to Indiana Jones and the Emperor's Tomb and not directly related to .bik movies. In fact even with Indiana Jones and the Emperor's Tomb this only happens with the intro movie.

[wetcoder]

if anyone needs
Indiana Jones and The Emperor's Tomb (2003) Full controller support FIX + Widescreen Fix

Before

After

[elishacloud]

I just found out that Indiana Jones and the Emperor's Tomb uses DirectDraw. I noticed it calling:

• DirectDrawCreate
• SetCooperativeLevel
• CreateSurface

However CreateSurface fails with error 887600E1. This happens with or without d3d8to9.

The game tries to use both DirectDraw and Direct3D8 cooperatively. Since d3d8to9 converts Direct3D8 into Direct3D9, I wonder if Microsoft changed the way DirectDraw and Direct3D9 work together cooperatively?

Note: I also found another game that uses DirectDraw and Direct3D8 cooperatively called Port Royale. This game also has issues on Windows 10, though the issues are different.

[elishacloud]

It looks like this issue is worse with the Windows 10 April Update. Now it's not only the intro video that shows a black screen, the other videos in the game also only show a black screen.

[HerMajestyDrMona]

Just a guess based on the other issue I experienced in EE2 #69
Did you try to "Disable fullscreen optimizations" by the "Properties" -> "Compatibility"?

[elishacloud]

Yes, I tried "Disable fullscreen optimizations" as well as several other compatibility settings. They have no effect.

[mkultimate]

@elishacloud The Disable fullscreen optimizations option is disabled and the problem still continues

[elishacloud]

Yes, this is not a compatibility issue. This seems like an issue with how Windows 10 handles Direct3D8 differently than Direct3D9.

[elishacloud]

I found out that if I set the game to Windowed mode then the videos play just fine with d3d8to9.

Apparently I just need to set:

pPresentationParameters->Windowed = true;

When creating the Direct3D device for the videos to play.

[elishacloud]

Here is a patch with the fix: d3d8.zip

Note: this just plays the game in windowed mode but forces the window to be full screen.

[Radio30]

Thanks @elishacloud ! That got the videos working but it seems to disable re-shade, is it the same on your end?

[elishacloud]

Here is an updated patch. This one will no longer bypass ReShade: d3d8.zip

However it appears that ReShade has a bug. ReShade crashes for me with Indiana Jones and the Emperor's Tomb even when using the released version of d3d8to9. You may want to open a bug in ReShade for this.

Note: the only after market injector I could get to work with this game is injectSMAA.

[Radio30]

I've just tested your new patch and it works perfectly with reshade! I'm using the presets Magic DOF, HDR, Adaptive sharpen and SMAA and it's working great with no problems at all! I'm using Reshade 3.0.7.180

[elishacloud]

Interesting. It could be the version of the game I am using. I am using the GOG version. I am glad it works for you!

BTW: I am not sure what a generic fix for this would be. Switching all games to use windowed fullscreen is probably not the best solution. Adding an option for this would work. However up until now all fixes have been generic so no configuration has been needed.

[Radio30]

No worries, I'm also using the GOG version.
Also I've found for some games, one being blood II: the chosen, having quicktime player allowed the intro movies to play but i'm not sure how this is connected.

[elishacloud]

having quicktime player allowed the intro movies to play but i'm not sure how this is connected.

That is different. That is likely because QuickTime installs some codec that is needed for the game movies.

For Indiana Jones and the Emperor's Tomb the issue could be related to some display format (similar to issue #42). When in windowed mode it converts the format to the current display format. However in fullscreen it tries to use the format directly. I will need to troubleshoot this more when I have some time.

[Radio30]

Hi elishacloud,
I've been playing Indiana Jones and The Emperor's Tomb happily with videos and d3d8to9 and it had been beautiful. However the latest windows update has made it so only the audio plays from the videos again and has also broken some d3d8to9 games I had up and running, It's such a pain.

I don't know if this topic still interests you But your help on here has been amazing and you're the only person I can think of who might know what caused this to happen

[elishacloud]

@Radio30, I am not quite sure exactly why it is just showing a black screen. I believe it is a bug in Windows 10 and can reproduce this with other games using other versions of DirectX. However, I have found that running in fullscreen widowed mode solves the issue.

I had to make some changes to my windowed mode code because of the latest Windows 10 release. Below is a patch that works for now. There is still one issue with this patch where the game will hang if you click the mouse during the opening video. I will fix this later, but for now you can use this as long as you don't click the mouse during the opening video.

This is using the dxwrapper code since it already has support for fullscreen windowed mode. Simply unzip these files into the GameData\bin folder under the game's folder: d3d8.zip

[Radio30]

Thankyou for the reply! I was excited to try this fix and have Indy working as normal again but sadly after launching, the first 2 intro movies play and then it's CTD without touching the mouse, or anything else :(

[elishacloud]

I believe the game does not respond to certain notification messages which cause this issue. Those messages only happen because the game is running in windowed mode. I will probably need to handle those messages myself.

Ideally we would get this fixed in exclusive fullscreen mode, but I am not quite sure how to do that. I believe this is a bug in Windows, similar to what is described in this thread here.

[Radio30]

Ah, cheers for the link, I had a read.
I like to have quite a few old games running with reshade, some using d3d8to9 and it seems each new windows update is a potential threat to the stability of older games running in such a way.

[Radio30]

@elishacloud
Hey, Have you had a chance to look into this any further? I've tried a few things just messing around like converting the bik files to a different format but that was probably stupid as the game is looking for .bik files lol.
I'm so close to having a perfect running indy, it's just those damn movies

[elishacloud]

I did take a few minutes to look at this issue. I found several other games that also show a black screen during the intro video when they are converted to Direct3D9.

Here are the other games:

• American Conquest
• Konung 1
• Konung 2
• SimCopter

These games are using older versions of DirectX, so it's not just converting from Direct3D8 to Direct3D9 that has this issue.

What appears to be happening is that the videos are not being displayed via DirectX at all. They are using GDI to directly to write to the window. Apparently with the latest version of Direct3D9 on Windows 10 it will no longer show data written directly to the window while in exclusive fullscreen mode. At least I have not figured out any way of showing data written directly to the window while in Direct3D9 exclusive fullscreen mode.

The solution I have so far is to run these games in windowed fullscreen mode. That used to work with this game, but with the latest version of Windows 10 a new issue has developed where forcing the game into windowed mode causes the game window to freeze/hang. I believe the issue is that the game cannot handle some window event, which causes the window to hang.

A better way might be to hook GDI and convert the data to Direct3D9. However, that is outside the scope of this project.

Anyways, I was able to build a slightly better way to enable windowed fullscreen mode. This one should work as long as you don't click the mouse button during any of the intro videos. At least it worked for me. Later I will filter out mouse clicks during the intro videos, but this one should work for now.

Here is the update: dxwrapper.zip

[mirh]

Isn't direct forceful rendering to window the same thing @CookiePLMonster faced with SilentPatchGF?

[Radio30]

@elishacloud
Wow!! You've done it again! 👍 Thankyou so much for posting this fix, works perfectly, I'm very happy!!

You're a proper genius!

[CookiePLMonster]

Isn't direct forceful rendering to window the same thing @CookiePLMonster faced with SilentPatchGF?

Seemingly not, as this doesn't seem to involve DirectDraw. It can be similar, however.

[elishacloud]

@CookiePLMonster, you can ignore the DirectDraw calls. They seem useless. I believe they are just used to get video memory and set cooperative mode. If you block all the DirectDraw functions the game continues to work correctly (even the intro videos).

How did you solve the issue with SilentPatchGF?

Edited: I should mention that this same problem happens even on games that use DirectDraw, such as SimCopter and Konung 1, both of which use DirectDraw version 1. Both work fine when rendering in DirectDraw. However, when rendering the games in Direct3D9 they have this problem. This issue seems specific to rendering a game with Direct3D9 and using fullscreen exclusive mode and trying to write directly to the GDI.

The same problem happens with other wrappers also, for example if you use cnc-ddraw and render them in fullscreen exclusive mode you only see a blank screen also. As mentioned before, I believe this is a bug in Windows 10 because it works fine in other versions of Windows, it even used to work with older versions of Windows 10.

[CookiePLMonster]

How did you solve the issue with SilentPatchGF?

I ended up leveraging my RenderWare knowledge and writing a game specific hook for it as described in detail here:
https://cookieplmonster.github.io/2018/05/18/fixing-the-godfather/

[elishacloud]

I see. Nice work on the fix! Unfortunately that appears to be a completely different issue. I don't think that will help with this issue.

BTW: I am working on a generic DirectDraw -> D3D9 wrapper and I plan to support overlays. I will do this by using an emulated surface. The game will write to the emulated surface (probably created from a DC) then I will copy the data from the emulated surface to the real D3D9 surface. I have already done this with 24-bit surfaces, which are supported in DirectDraw but not in D3D9.

[elishacloud]

As I mentioned earlier, the issue here happens because the game uses GDI for the videos and DirectX for the game play. However, in DirectX 9 when you are using exclusive fullscreen mode all GDI turns black. So far the only way I know of to solve this is to put the game into fullscreen windowed mode, rather than exclusive fullscreen mode. Then GDI and DirectX 9 can play nicely together. But with this particular game if you force it into fullscreen windowed mode it causes the game to hang, which I may be able to fix this later.

However, just recently dxvk added support for DirectX 9 conversion to Vulkan. Vulkan is able to play nicely with GDI even in exclusive fullscreen mode. So you can use d3d8to9 to convert the game to DirectX 9 and then dxvk to convert the game to Vulkan. This should solve the black screen issue with the in-game videos.

[mirh]

Vulkan is able to play nicely with GDI even in exclusive fullscreen mode

Couldn't that be the case because it leverages some DXGI/DWM voodoo?

[elishacloud]

Actually I tested dxvk with this game and it has the same issues. It works with other games that have these issues but not this one.

I did note that dgVoodoo2's DirectX8 dll does work with this game. So far it is the only one I know that works.

[mirh]

Supposedly, he's hooking some GDI function and forcing its way out. Not sure if that's anything specific to the MCIAVI(?) library or it could be generalized.

[elishacloud]

Yes, hooking GDI is the same thing that DDrawCompat does. I also mentioned GDI hooking earlier in this thread.

I am considering adding GDI hooking to my dxwrapper project. However, for most other games I tested, simply setting the game to fullscreen windowed mode solves the issue.

In fact, even for Indiana Jones and the Emperor's Tomb fullscreen windowed mode allows the video to be displayed. The issue is that the game does not properly handle some hwnd events during the videos causing the game to hang when setting it to fullscreen windowed mode. These events don't exist in exclusive fullscreen mode so the game never needed to handle them. However, if you wait until video is completed then the game will recover and become playable again.

I am considering hooking the wndproc like what is done in diablo-ddrawwrapper and cnc-ddraw. Then I can simply filter out the events that the game does not handle. That would be much easier than writing a whole GDI wrapper.

[mirh]

And what about surface sharing with GDI interop instead?

[elishacloud]

ddraw and Direct3D8 seem to handle GDI interop ok. But I have no idea how to do GDI interop with Direct3D9. I did add a feature recently to dxwrapper to allow Direct3D9 to Blt to GDI rather than Direct3D9. This helps resolve an issue where ddraw mixes GDI and DirectX on the same screen at the same time.

[mirh]

D3DXLoadSurfaceFromMemory/IDirect3DSurface9::GetDC maybe?
There's a whole subway of interoperability it seems.

[elishacloud]

GetDC is used to allow a game to use GDI functions directly on DirectX. It was added in version 1 and worked all the way to version 9. DirectX 10 uses a different way of handling GDI.

The trouble is that some games (like this one) call the GDI functions directly rather than using DirectX for GDI. To use this, I would have to hook the GDI functions and redirect them to DirectX using GetDC(). But, that is probably the right way to do it.

[mirh]

You might also get fun with D3DSWAPEFFECT_FLIPEX/IDXGISurface1::GetDC/ID2D1GdiInteropRenderTarget for moar speed, if supported by the hardware :p

[elishacloud]

Ok, I put a fix for this in dxwrapper. I just filtered the window events that were causing it to hang. This should be safe for any game.

Here is the update: dxwrapper.zip

Just unzip this into the GameData\bin folder of Indiana Jones and the Emperor's Tomb.

[elishacloud]

I made a few changes to prevent possible sync issues. This one also has anti-aliasing enabled.

Here is the update: dxwrapper.zip

Note: I just realized that it still hangs sometimes if Task Manager is open. Make sure to close the Task Manager before running.

[BehzadScorpion]

I went to ChooseRenderer.exe and changed the renderer to "Software MMX: FullScreen..." and it worked! I also changed the compatibility setting to win98 but I doubt it had anything to do with that cause when I did that without switching renderer it didn't work

[SwiftyOne1]

Hello, Sorry I'm just a user...I have been following this thread as I used the d3d8.dll in Nascar2003 season, it seemed to work although I could not see a lot of differents. I lost d3d8.dll and came here to find the latest build I tried quite a few d3d8.dll but all would crash the game so I didn't try any more until I tried dxwrapper posted by elishacloud over a year ago, I put all three files into the root directory of Nascar 2003 season, the game works fine and no crashing is there anything else that I could to make the game look better? Maybe that would need some work in the dxwrapper.ini file? Thank you for your work and any help you could give an end-user :)

[elishacloud]

The issue with Indiana Jones and the Emporer's Tomb showing a black screen during the in-game videos is because it uses Bink to write directly to the GDI. However, in later versions of Windows 10 then Direct3D9 will overwrite anything written directly to GDI when in exclusive fullscreen mode. The solution for the black screen is to switch to using windowed fullscreen mode.

As far as the game hanging when using windowed fullscreen mode, this is a bug with the version of Bink included in the game. The version included is 1.5.19. The issue is fixed in Bink 1.5.21.

Here is a package that includes all the fixes for this issue, including the updated version of Bink: d3d8.zip

[elishacloud]

I added a fix for this in dxwrapper so that even if you use the broken Bink 1.5.19 it will still not hang.

Here is the updated fix: d3d8.zip

[elishacloud]

I found a real fix for this issue by calling Direct3D9SetSwapEffectUpgradeShim() with parameter of 0. This changes the present mode to "Hardware: Legacy Flip", which is the same mode that DirectDraw and Direct3D8 is using. For more details on the issue see comments here.

Here is the fix for this in dxwrapper: dxwrapper.zip

@crosire, not sure if we want to put a fix in for this. It does effect some other games. Basically any Direct3D8 game that uses GDI to display anything on the screen will break without this fix. Older games that used Bink often used GDI for videos, as this game does. Here is the code fix I put into dxwrapper for this issue.

Edit: updated the build, last one had a bug in Windows 11.

[crosire]

I'm leaning towards this being better off living in dxwrapper and just seeing d3d8to9 as the base component that takes care of the wrapping, but not doing much compatibility fixing beyond that.
(Also wondering how that shim is triggered officially, I couldn't find a compatibility fix with that in the name in Windows' compatibility fixes database.)

[mirh]

A quick and dirty search @WinDLLsExports shows that while SwapEffectUpgradesShim is already shipping in W10 dlls, SwapEffectUpgradeEnable (and DX_DATABASE_QUERY_TYPE_SWAPEFFECTUPGRADE_ENABLE) is only present from W11.

Unsure if this means that in older Windows you can only control it through GameConfigStore then (this is the only thing I could remotely recognize from a very blind run in ghidra), or if that couldn't just be dead code that microsoft is sharing between different versions for lazy reasons.

[elishacloud]

I'm leaning towards this being better off living in dxwrapper and just seeing d3d8to9 as the base component that takes care of the wrapping, but not doing much compatibility fixing beyond that.

Ok, sounds good.

A quick and dirty search @WinDLLsExports shows that while SwapEffectUpgradesShim is already shipping in W10 dlls, SwapEffectUpgradeEnable (and DX_DATABASE_QUERY_TYPE_SWAPEFFECTUPGRADE_ENABLE) is only present from W11.

Sounds like there is a compatibility fix in Windows 11 for this? I know the fix I put in works for both W10 and W11.

[mirh]

It has "shim" in the name, but after entering a colossal rabbit hole I just found out it's not a normal one implemented in the general application compatibility layers..

First of all, SwapEffectUpgradesShim inside of d3d9 and dxgi was added in W10 1709 (which was also about "peak FSO", before they scaled back on the amount of automatic upgrades). Maybe it was a new knob to toggle what they had already been doing for a while, maybe they refined or covered only some specific cases back then, but nonetheless today it is what it is (W10 1803 further added SwapEffectTransitionShim but idk).

Finally Windows 11 added WindowedSwapEffectUpgrade (which of course controls this) and this "DirectX Database query" thing that I spotted. After some creative search, I guessed that refers to DirectXApps.sdb, DirectXApps_FOD.sdb or DirectXApps_OEM.sdb which should have been introduced from W10 1903 onwards (and which only the god knows in functioning: presumably that's associated with a directxdatabaseupdater.exe scheduled task and starting in W11 with directxdatabasehelper.dll too).

Last but not least, after opening them with SDB explorer (because the ACT tools don't seem to parse anything besides the entries names) I found they are used to set some custom value for settings like GpuPreference, AutoHDREnable, SwapEffectUpgradeEnable, DynamicSwitch and DXGIEffects. The absolute lack of recognition of even just the existence of all of this kinda baffles me.

[elishacloud]

Thanks. A lot of good data.

Finally Windows 11 added WindowedSwapEffectUpgrade (which of course controls this)

I just tested the steps here and enabled "flip presentation model" and it did not fix the issue here with a black screen. I am not quite sure what it does, but it is not a fix for this issue.

I found they are used to set some default value for settings like GpuPreference, AutoHDREnable and SwapEffectUpgradeEnable.

I installed the latest Compatibility Manager for Windows 11 and I was unable to find any Compatibility Fix for SwapEffectUpgradeEnable. If you know a way to enable this outside of calling the API I listed above I would be interested in it.