Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discrepancy between hard-coded file & in-memory patching #220

Closed
llm96 opened this issue Sep 11, 2024 · 7 comments
Closed

Discrepancy between hard-coded file & in-memory patching #220

llm96 opened this issue Sep 11, 2024 · 7 comments
Labels
game: iidx not_tools_issue This will not be worked on

Comments

@llm96
Copy link

llm96 commented Sep 11, 2024

Game and version

beatmania IIDX 30 RESIDENT - LDJ-003-2023090500

Version of spice2x

1.0-V-2024-08-24T00:53:46

Laptop

Replicated on both desktop & laptop

Describe the issue

After isolating the differences between the original bm2dx.dll and the bm2dx_omni.dll from Omnimix v1.30.1 into a spice2x patch, I've noticed some texture issues in music select when playing certain songs (e.g. Scripted Connection⇒ long mix)

File Memory
20240911_0 20240911_0
20240911_1 20240911_1
20240911_2 20240911_2

patches/LDJ-64ef0ff5_1037754.json

[
    {
        "type": "memory",
        "name": "Omnimix",
        "description": "",
        "gameCode": "LDJ",
        "patches": [
            {
                "offset": 5602318,
                "dllName": "bm2dx.dll",
                "dataDisabled": "7407",
                "dataEnabled": "9090"
            },
            {
                "offset": 9449065,
                "dllName": "bm2dx.dll",
                "dataDisabled": "743C",
                "dataEnabled": "9090"
            },
            {
                "offset": 9966082,
                "dllName": "bm2dx.dll",
                "dataDisabled": "32C0",
                "dataEnabled": "B001"
            },
            {
                "offset": 10041005,
                "dllName": "bm2dx.dll",
                "dataDisabled": "75",
                "dataEnabled": "EB"
            },
            {
                "offset": 11505163,
                "dllName": "bm2dx.dll",
                "dataDisabled": "0FB644244085C00F84F800000048837C2448000F84EC000000488B4424480FB6400383F8587539488B4424480FB6400483F858752B488B4424480FB6400583F858751D488B442448C640034A488B442448C6400442",
                "dataEnabled": "90909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090909090"
            },
            {
                "offset": 11505256,
                "dllName": "bm2dx.dll",
                "dataDisabled": "41",
                "dataEnabled": "58"
            },
            {
                "offset": 11523201,
                "dllName": "bm2dx.dll",
                "dataDisabled": "0807",
                "dataEnabled": "FFFF"
            },
            {
                "offset": 11523205,
                "dllName": "bm2dx.dll",
                "dataDisabled": "7F0A",
                "dataEnabled": "9090"
            },
            {
                "offset": 11523211,
                "dllName": "bm2dx.dll",
                "dataDisabled": "0807",
                "dataEnabled": "FFFF"
            },
            {
                "offset": 18601455,
                "dllName": "bm2dx.dll",
                "dataDisabled": "61",
                "dataEnabled": "6F"
            },
            {
                "offset": 19351851,
                "dllName": "bm2dx.dll",
                "dataDisabled": "64617461",
                "dataEnabled": "6F6D6E69"
            }
        ]
    }
]

Attached log.txt file, if available

log_file.txt
log_memory.txt

@llm96 llm96 added the bug Something isn't working label Sep 11, 2024
@LupinThidr
Copy link

If the texture problem is from mdata, try running with -avsverbose to see if it's even opening the correct file (the 0x61 -> 0x6F patch should change mdata.ifs to mdato.ifs?)

@sp2xdev sp2xdev closed this as not planned Won't fix, can't repro, duplicate, stale Sep 12, 2024
@sp2xdev
Copy link
Contributor

sp2xdev commented Sep 12, 2024

This doesn't sound like a mem patch issue, more like you missed a patch that is needed to make Omni work, as pointed out above.

If you can narrow it down to a single patch & can prove that it's not being applied (such as via the memory overlay or under a debugger) feel free to respond back.

@sp2xdev sp2xdev added not_tools_issue This will not be worked on game: iidx and removed bug Something isn't working labels Sep 12, 2024
@llm96
Copy link
Author

llm96 commented Sep 13, 2024

If you can narrow it down to a single patch & can prove that it's not being applied (such as via the memory overlay or under a debugger) feel free to respond back.

The patch is the aforementioned 61 to 6F edit, which is already present in my JSON version above. Just to be certain, I reverted that one byte in the already patched bm2dx_omni.dll file and saw the same visual issues.

As for the in-memory version of that patch, it appears that it is being applied:

1

After stepping over the memcpy call, the filename has changed.

2

More confirmation of the altered filename in-game, despite the still present visual issues.

3

If the texture problem is from mdata, try running with -avsverbose to see if it's even opening the correct file (the 0x61 -> 0x6F patch should change mdata.ifs to mdato.ifs?)

With the pre-patched file, it opens mdato.ifs as expected:

M:avshook: avs_fs_lstat: name: /data/graphic//0/mdato.ifs = 0x1
M:avshook: avs_fs_lstat: name: /data/graphic//0/mdato.ifs = 0x1
M:avshook: avs_fs_open: name: /data/graphic//0/mdato.ifs mode: 1 flags: 420 = 0x3a180086

However, with the in-memory patch applied it still opens the original file instead:

M:avshook: avs_fs_lstat: name: /data/graphic//0/mdata.ifs = 0x1
M:avshook: avs_fs_lstat: name: /data/graphic//0/mdata.ifs = 0x1
M:avshook: avs_fs_open: name: /data/graphic//0/mdata.ifs mode: 1 flags: 420 = 0x3a180082

Curiously, further down the log it does open music_omni.bin rather than the original music_data.bin, so it appears to be an issue specifically with that one mdata.ifs file.

Looking in IDA, /0/mdata.ifs is referenced a single time in a function at 0x1800C22C0, where it gets copied into 0x186B5EF38.

...and as far as I can tell, 0x1800C22C0 is a static initializer function that gets called automatically during CRT initialization.

spice2x is definitely applying the patches correctly; it's just an issue of timing. I can't think of any good ways of addressing this as it's already too late as soon as avs::game::load_dll returns. None of the flags for LoadLibraryEx appear to be useful either.

Oh well. At least it's good to know that static patches in a file don't necessarily translate 1:1 when expressed as memory patches. Unfortunately, altering that one patch doesn't work either, as the offset of the copy exceeds the file size.

@LupinThidr
Copy link

LupinThidr commented Sep 14, 2024

Thank you for looking into this deeper.

Maybe it's time to split it off as a hook DLL+layeredfs? Spicetools patching, mempatch-hook and layeredfs have been around for many years now and they don't seem to be in a rush to move from the "use bm2dx_omni.dll with mystery patches and pollute your game data folder" model... while other projects seem to have embraced all of the tool upgrades since 2013.

Bonus:
A few helpful hook DLLs for fixing song select ratings and marking omnimix songs were recently ported to newer versions, it would be nice to have an all-in-one solution, since these are relatively obscure: https://github.com/nixac/hooks/

@NotAkitake
Copy link

NotAkitake commented Oct 12, 2024

Exact same issue happening with people now creating Epolis' omnimix and mdata.ifs. Hard-patching the DLL does work, memory patching doesn't.

@aixxe
Copy link

aixxe commented Oct 13, 2024

Here's a potential workaround that attempts to use LdrRegisterDllNotification to apply changes a bit earlier.

It was introduced in Vista, so I left the original code to apply patches after loading the game DLL alone.

Tested by building with Docker and comparing behaviour against the latest 24-10-12 binaries while using the same patches and configuration. Confirmed the Omnimix texture issue is no longer present. Also lightly tested with a couple of 32-bit IIDX games to ensure it works there as well, although I'm not aware of any patches that exhibit the same issue.

early-apply.patch

log.txt (24-10-12)

[2024/10/13 21:32:08] I:avs-game: loading DLL 'bm2dx.dll'
[2024/10/13 21:32:08] I:avs-game: DLL path: D:\LDJ-012-2024082600\modules\bm2dx.dll
[2024/10/13 21:32:08] I:avs-game: loaded successfully (0x180000000)
[2024/10/13 21:32:08] I:game: attach: Beatmania IIDX
[2024/10/13 21:32:08] W:iidx: This game does not accept SOUND_OUTPUT_DEVICE environment variable; it will be ignored
[2024/10/13 21:32:08] W:iidx: Make sure you applied appropriate patches to use the correct sound device (ASIO or WASAPI)
[2024/10/13 21:32:08] I:devicehook: init
[2024/10/13 21:32:08] I:bi2x_hook: init
[2024/10/13 21:32:08] I:devicehook: init
[2024/10/13 21:32:08] I:stubs: attaching...
[2024/10/13 21:32:08] I:libraryhook: LibraryHook Attach
[2024/10/13 21:32:08] I:stubs: attached
[2024/10/13 21:32:08] I:hooks::lang: initializing
[2024/10/13 21:32:08] I:signal: attaching...
[2024/10/13 21:32:08] I:libraryhook: LibraryHook Attach
[2024/10/13 21:32:08] I:signal: attached
[2024/10/13 21:32:08] I:audio: initializing
[2024/10/13 21:32:08] I:input::dinput8: attaching...
[2024/10/13 21:32:08] I:input::dinput8: attached
[2024/10/13 21:32:08] I:graphics: initializing
[2024/10/13 21:32:08] I:ScreenResize: initializing
[2024/10/13 21:32:08] I:graphics::d3d9: initializing
[2024/10/13 21:32:08] I:debughook: attaching...
[2024/10/13 21:32:08] I:debughook: attached
[2024/10/13 21:32:08] I:easrv: EASRV running on port 8080
[2024/10/13 21:32:08] I:acio: SpiceTools ACIO
[2024/10/13 21:32:08] I:libraryhook: LibraryHook Attach
[2024/10/13 21:32:08] I:acio: module attach: BI2A IAT
[2024/10/13 21:32:08] I:acio: module attach: BMPU IAT
[2024/10/13 21:32:08] I:acio: module attach: Core IAT
[2024/10/13 21:32:08] I:acio: module attach: HBHI IAT
[2024/10/13 21:32:08] I:acio: module attach: HDXS IAT
[2024/10/13 21:32:08] I:acio: module attach: HGTH IAT
[2024/10/13 21:32:08] I:acio: module attach: I36G IAT
[2024/10/13 21:32:08] I:acio: module attach: I36I IAT
[2024/10/13 21:32:08] I:acio: module attach: ICCA IAT
[2024/10/13 21:32:08] I:acio: module attach: J32D IAT
[2024/10/13 21:32:08] I:acio: module attach: KFCA IAT
[2024/10/13 21:32:08] I:acio: module attach: KLPA IAT
[2024/10/13 21:32:08] I:acio: module attach: MDXF IAT
[2024/10/13 21:32:08] I:acio: module attach: NDDB IAT
[2024/10/13 21:32:08] I:acio: module attach: PANB IAT
[2024/10/13 21:32:08] I:acio: module attach: PJEC IAT
[2024/10/13 21:32:08] I:acio: module attach: PJEI IAT
[2024/10/13 21:32:08] I:acio: module attach: LA9A IAT
[2024/10/13 21:32:08] I:device: skipping device module hooks
[2024/10/13 21:32:08] I:sciunit: skipping sciunit hooks
[2024/10/13 21:32:08] I:network: SpiceTools Network
[2024/10/13 21:32:08] I:network: Network preferences: 10.9.0.0
[2024/10/13 21:32:08] I:patchmanager: loading config
[2024/10/13 21:32:08] I:patchmanager: reloading (local) and applying patches
[2024/10/13 21:32:08] I:patchmanager: loaded patches for bm2dx.dll from patches/LDJ-66c58ff1_9b323c.json
[2024/10/13 21:32:08] I:patchmanager: auto apply: Omnimix = ON
[2024/10/13 21:32:08] I:patchmanager: loaded total of 59 patches

log.txt (with patch)

[2024/10/13 21:33:40] I:patchmanager: registered for DLL load notifications
[2024/10/13 21:33:40] I:patchmanager: loading config
[2024/10/13 21:33:40] I:avs-game: loading DLL 'bm2dx.dll'
[2024/10/13 21:33:40] I:avs-game: DLL path: D:\LDJ-012-2024082600\modules\bm2dx.dll
[2024/10/13 21:33:40] I:patchmanager: reloading (local) and applying patches
[2024/10/13 21:33:40] I:patchmanager: loaded patches for bm2dx.dll from patches/LDJ-66c58ff1_9b323c.json
[2024/10/13 21:33:40] I:patchmanager: auto apply: Omnimix = ON
[2024/10/13 21:33:40] I:patchmanager: loaded total of 59 patches
[2024/10/13 21:33:40] I:avs-game: loaded successfully (0x180000000)
[2024/10/13 21:33:40] I:game: attach: Beatmania IIDX

@sp2xdev sp2xdev reopened this Oct 14, 2024
@sp2xdev sp2xdev closed this as completed Oct 14, 2024
@sp2xdev sp2xdev closed this as completed Oct 14, 2024
@sp2xdev
Copy link
Contributor

sp2xdev commented Oct 14, 2024

I've merged aixxe's patch above, will do a release soon.

One tricky situation that we need to explain is that for patches that were previously no-op can now be working - for example, if someone does not actually have omnimix installed but has Omnimix patch checked, the game would have booted previously, but now it will crash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
game: iidx not_tools_issue This will not be worked on
Projects
None yet
Development

No branches or pull requests

5 participants