Add XML definitions for all textures that are auto-generated by ZAPD

[Archez]

This PR adds dedicated texture definitions to all resource files so that we can control the output names that get used in the resulting OTR files. The goal is to align texture names across roms so that texture packs do not have to provide duplicate textures to cover all possible roms.

Without definitions, ZAPD will auto-detect these texture resources when referenced by a DList and assign them a name based on the resource file and the offset value. Since offset values are not guaranteed to be the same across different roms, this resulted in names that aren't the same. Although this doesn't cause extraction issues and running the game works fine, it makes managing and using texture packs more annoying.

To achieve this I have written a python script that will leverage ZAPD's output to add new definitions to the XMLs so that these textures now have a hard coded output name. The hard coded names will be taken from the MQ debug rom definitions as decomp is based on the MQ debug and existing hard coded names follow this pattern.

The script achieves this by doing a few things:

1. Extract all roms so that we can read the (listfiles) from each OTR
2. Set ZAPD to output textures to a folder (I also made a change so that any palette textures will also append the referenced TLUT name to be used later)
3. Filter the listfiles for any textures and TLUTs that do not have existing XML definitions
4. Determine the width/height/format based on the texture output file
5. Add each missing texture to the appropriate File resource, and for palette based textures, also add the TlutOffset information based on the TLUT name from ZAPD
6. Once all XMLs have their definitions, we then compare the actual texture output files against the MQ debug textures
   • Use direct file comparison to find matches, otherwise
   • Use Structural Similarity image recognition to get similarity scores (this works nicely as textures being matched would have to be the same height/width, as needed by SSIM)
   • For any similarity scores that are too low, use a manual verification to select the appropriate matches
7. Look for any texture definitions who's names do not already match the MQ debug name and update them

Essentially only PAL1.1 needed to utilize SSIM recognition due to the gerudo symbol changes, and only about ~20 files needed to be manually verified in those cases. Outside of that, there was about 10 textures that could not have the TLUT information automated (ZAPD bug I guess), so these were also handled manually by looking at the DList in decomp. Technically the TLUT information is only necessary for outputting to files from ZAPD, but I think it is best to set this information if we have it available rather than losing it.

I have confirmed these changes by using retro to compare the manifest between PAL1.1 and MQ debug, and only the expected files (gerudo symbol, texture screen, button prompts, etc) had different hash values, otherwise all scenes were identical. I also prepared a custom OoT-Reloaded pack that only uses the mq version of the texture files (copied over to nonmq) and all textures were changed in game when running the PAL1.1 rom.

Attached is the outputs from the different stages and the final list of replacements for each rom. The replacements are comma delimited where the left value is the old name, and the right value is the new name. These can be used for texture pack makers to automate the renames. But if they have an MQ set of textures, then copying those over to nonmq should be sufficient 😉
outputs.zip

Fixes #3024

Build Artifacts

• soh.otr.zip
• soh-linux-compatibility.zip
• soh-linux-performance.zip
• soh-mac.zip
• soh-switch.zip
• soh-wiiu.zip
• soh-windows.zip

[garrettjoecox]

Absolute mad lad.

Thanks for clearly laying out all of the cases not handled by the script.

Will we need to repeat this process/script with each additional rom supported in the future?

[Archez]

Will we need to repeat this process/script with each additional rom supported in the future?

My thought is for new rom support, it would probably be easiest to remove any texture definitions that do not follow the "gSomeNameTex" pattern, then leverage this script again to add them back in with the appropriate offset values.

Butttt, I am hoping that instead new rom support can be fully scripted 👀 have much to learn to see if that is possible..

Edit: I just had a thought.. Since this is XML, we can technically add dummy attributes. Do we think it would be worth to have some attribute like AddedByScript="true" to help track/differentiate the textures added by this script? That would make "removing" them to re-run the script easier.

[Archez]

I have updated the PR and script to include AddedByScript="true" to all the added definitions so that we can easy track these additions for future needs. I don't anticipate this needing to be there long term, but it doesn't hurt to add it. ZAPD had to be updated to "register" this attribute so that it doesn't complain, but ZAPD will not parse/use the value for anything internally.