Request: Copy only character appearance, inventory, and stats to another save file

[alexandermueller]

First off, this is a really cool tool you guys have created here!

I'm not sure if this is already in the works, but I guess I have a bit of a special use case that many people might find useful.

Whenever I use this to copy over a character from a borked save I made while playing Seamless Coop, it copies the current location and game progress (found locations, npc quests, etc.) over as well.

It would be nice to be able to take just the character appearance, inventory, and stats of one save and place them into another save or a new slot.

Note: I'm using release v1.67

[Ariescyn]

Hey there, unfortunately i have no clue where or how character appearance data is stored. I probably could figure out how to copy a characters entire inventory to another character but to be honest I'm tired of looking at endless numbers and trying to make sense of it all lol.

[alexandermueller]

I totally understand 😅

I think I might have an idea though, if you don't mind me taking a crack at it 🤔

If my hunch is correct, we might even be able to grab slider values for appearances and/or just copy them outright and place them from the source to the donor file. I'll just need to do some digging in the code and then get back to you in a bit 👍

[Ariescyn]

Dude if you wanna help figure it out that would be awesome. How would you approach finding the address locations of slider values? Lets say you made two saves with the only difference being a single appearance value, it would be hard to find the right address since whenever you load into the game, even if you made no changes and exited immediately, massive amounts of data shift around so it makes it really hard to compare differences between files.

[alexandermueller]

Ah, got me there 😅

I never actually looked through the file data before, but I had a thought that maybe we could still do just that and try to find similarities in the files where instead of changing sliders, you would make two separate saves with the exact same characters and progress different amounts into the game to find the sections of data that match exactly.

I'm guessing that unless the file is compiled in a certain way that scatters the data, there's a good chance that the slider data is all in the same block 🤔 Even without all that, it would be a lot easier to ignore the data that differs until you find data that matches.

Once we find those, then it would just be a matter of making a very unique character that has unique values for each slider and then we'd know which sliders are which.

This is all guesswork though, and I have no clue if that works in practice 😅

[Ariescyn]

There very well could be a "Character appearance block" of data since characters are in their own static containers. I've been wondering for a while if data IN a "static container" shifts predictably, like if there's an offset or something. But i haven't found any way to predict the shift in data yet. I did this by tracking the start address of your stats and seeing where it ends up after each time you load into the game.

the exact same characters and progress different amounts into the game to find the sections of data that match exactly.

I tried that but you still get hundreds of thousands of values that change and millions that don't change. But how to distinguish which "matches" are actually matches and not just chance? Many zeros stay the same so you have to toss those out if you want to shrink the pool of matches. I need a better way to visualize the data.

Since you want to start digging into the data, here are some of the static addresses and patterns I've figured out so far

Character addresses

slot1_start = dat[:0x00000310]
slot1_end = dat[0x0028030F + 1 :]

slot2_start = dat[:0x00280320]
slot2_end = dat[0x050031F + 1 :]

slot3_start = dat[:0x500330]
slot3_end = dat[0x78032F + 1 :]

slot4_start = dat[:0x780340]
slot4_end = dat[0xA0033F + 1 :]

slot5_start = dat[:0xA00350]
slot5_end = dat[0xC8034F + 1 :]

slot6_start = dat[:0xC80360]
slot6_end = dat[0xF0035F + 1 :]

slot7_start = dat[:0xF00370]
slot7_end = dat[0x118036F + 1 :]

slot8_start = dat[:0x1180380]
slot8_end = dat[0x140037F + 1 :]

slot9_start = dat[:0x1400390]
slot9_end = dat[0x168038F + 1 :]

slot10_start = dat[:0x16803A0]
slot10_end = dat[0x190039F + 1 :]

This is from the get_inventory function in hexedit.py

        ls.append({
                      "name": name,
                      "item_id": [l_endian(c1[ind:ind+1]), l_endian(c1[ind+1:ind+2])],
                      "uid": [l_endian(c1[ind+2:ind+3]),  l_endian(c1[ind+3:ind+4])],
                      "quantity": l_endian(c1[ind+4:ind+5]),
                      "pad1": [l_endian(c1[ind+5:ind+6]),l_endian(c1[ind+6:ind+7]),l_endian(c1[ind+7:ind+8])],
                      "iter": l_endian(c1[ind+8:ind+9]),
                      "pad2":[ l_endian(c1[ind+9:ind+10]), l_endian(c1[ind+10:ind+11]),l_endian(c1[ind+11:ind+12])],
                      "index": ind
                      })
        ind+= 12

This pattern repeats itself for every item in your inventory.
uid can be either [128,128] or [0,176] depending on the item.
iter is interesting. It will start at 1 and increment with every inventory item, but if items are removed from your inventory in-game, that "entry" will be removed from the file. So you will see the iter value can skip forwards like 1,2,3,6,7,9.

Character stats are sequential with character level 48 bytes ahead from the start of the first stat. So i iterate through a chunk of data summing values ahead of the index. When the sum of those values - 79 (your level is the sum of your stats - 79) equal the value +48 from the current index, then it's found the stats.

There is a checksum for each character block and an overall checksum for the whole file. Theses values are static.

There is a "header" that holds playtime, level, name, etc. However these values are used for the character selection menu in-game. If you change level, your character level won't be affected and you must change the level value in the character block next to your stats. If you change playtime, the game will just recalculate it once you load into the character. Playtime is stored in seconds.

Character names were very confusing, varying greatly in the location and frequency of a name. Some files might only have one of your character names in it, others could have like 12 of the same name. Because i couldn't figure it out, i had to scan through the entire save file and replace ALL occurences of it.

All integers are encoded little endian, and text is utf-16

Let me know if you have any ideas on how to further analyze the file or how we could visualize the data ( pandas would probably work well but would take some effort)

[alexandermueller]

Awesome, thanks for catching me up!

I'll take a look into it further then. Maybe we need to fill all 10 character slots and progress to different areas to get rid of all those extra 0s 🤔

[alexandermueller]

I gave it a quick shot with Sublime and opened a save file with UTF16-LE, and that's just one big mess. You can find the character name embedded in various places, but most of the data is illegible here. I'll give it a shot with panda tomorrow.

[Ariescyn]

So i decided to make some save files, the only difference between them is that i loaded into each copy once. I wanted to extract all of the index locations where values change after loading into the game to gain some additional insight. Here's the code, maybe it will help you figure out what goes on.

def compare(file1, file2):
difs = []
with open(file1, "rb") as f, open(file2, "rb") as ff:
dat1 = f.read()
dat2 = ff.read()
dat1 = dat1[0x00000310 : 0x0028030F + 1]
dat2 = dat2[0x00000310 : 0x0028030F + 1]
same = 0
dif = 0
for ind, i in enumerate(dat1):
if dat1[ind] == dat2[ind]:
same += 1
else:
dif += 1
difs.append(ind)
return difs

def build_dict():
w = compare("base.sl2", "ER0000.sl2")
x = compare("base.sl2", "ER0001.sl2")
y = compare("base.sl2", "ER0002.sl2")

d = defaultdict(int)

for i in w:
    d[i] += 1
for i in x:
    d[i] += 1
for i in y:
    d[i] += 1
lst = [v for k,v in d.items() if v == 3]

[Ariescyn]

Using the dict is like 10000x faster than using logic like:

for i in w:
if i in x:
if i in y:
z.append(i)

[alexandermueller]

Cool, thanks 👍

[alexandermueller]

I've done a bit of testing with some saves I got here, and played around with your code to spit out the parts that match (I sent you an invite to the repo). Let me know if you like this formatting and then we can go from there I guess 🤔 Either way, the matching data seems pretty binary in nature, so maybe this isn't going to help us much.

I recommend viewing the results.txt file in Sublime or a similar editor with word wrapping off.

[Ariescyn]

Your code looks great man! So i was able to find and change a characters hair type so it is possible!

What i did was i kept creating new save files with everything the same. I compared and found all the differences between the save files (These differences can't be character appearance data since i didn't change appearance).

dif1 = compare("ER0000.sl2", "ER0001.sl2")
dif2 = compare("ER0000.sl2", "ER0002.sl2")
dif3 = compare("ER0001.sl2", "ER0002.sl2")
dif4 = compare("ER0000.sl2", "ER0003.sl2")
dif5 = compare("ER0001.sl2", "ER0003.sl2")
dif6 = compare("ER0002.sl2", "ER0003.sl2")
alldifs = dif1 + dif2 + dif3 + dif4 + dif5 + dif6

Then i created another save but the only difference was the haircut. I compared the differences between the first save file i made and the one with a different haircut.

I removed any differences that were in the above alldifs array and removed any values that went from 0 to 255 or vice versa (since a lot of values change from 255-0 or 0-255)

I printed out the values of the filtered set of changed indexes and found something like this:

34: 128 ==== 129
42: 128 ==== 129
50: 128 ==== 129
58: 128 ==== 129
42828: 128 ==== 129
42832: 128 ==== 129
42836: 128 ==== 129
42840: 128 ==== 129
42844: 128 ==== 129
42848: 128 ==== 129
42876: 128 ==== 129
42880: 128 ==== 129
42884: 128 ==== 129
42888: 128 ==== 129
42944: 128 ==== 129
42956: 128 ==== 129
42968: 128 ==== 129
42980: 128 ==== 129
42992: 128 ==== 129
43004: 128 ==== 129
43016: 128 ==== 129
43028: 128 ==== 129
43040: 128 ==== 129
43052: 128 ==== 129
80298: 9 ==== 3
2187717: 244 ==== 243
2286396: 244 ==== 243
2289060: 244 ==== 243
2289068: 244 ==== 243
2294468: 244 ==== 243
2294628: 244 ==== 243
2294644: 244 ==== 243
2294692: 244 ==== 243
2395548: 244 ==== 243
2397604: 244 ==== 243
2397612: 244 ==== 243
2403012: 244 ==== 243

Anything stand out? The 9 that changed to a 3! I modified the value and was able to change haircut.

I bet a lot of the appearance data is around the same index. Unfortunately you can't just automate it yet until we discover patterns to dynamically find the appearance data (since the index will be different across different save files.)

[alexandermueller]

Hey that's awesome! I was working on comparing the entire files against each other instead of just the save slots in case there's some useful data hidden in there, but when it comes to the last stage for digging through and formatting the resulting data, it ends up running between O(nlogn) to O(n^2) and I can't get past 20% after running more than 1 hour 🙄 Good to know that you were able to change appearance inside the slots though, so that's a huge improvement 😅

[alexandermueller]

I tried looking at all the similarities though, but I guess the differences approach is a much better way to go because the files get pretty big in my case 👍

I'll modify the code to show the differences in each file and then when I get home on Sunday I'll try making each save have a unique feature so that it stands out in the diff.

[alexandermueller]

I'm also thinking it might be more efficient for us to dedicate each save file for a specific appearance modifier and then change the appearance value subtly between each slot. That way we can easily see which file has which change by looking at the name.

Assuming save slots are the exact same size, I bet we can do the same diff comparison without too much trouble this way.

[Ariescyn]

Can you elaborate on this? What your saying is to utilize each character slot to minimize how many entire save files we have to make?

I'm also thinking it might be more efficient for us to dedicate each save file for a specific appearance modifier and then change the appearance value subtly between each slot. That way we can easily see which file has which change by looking at the name.

[Ariescyn]

I added the script i used to find the hair value to your repo if you wanna take a look

[alexandermueller]

Exactly 👍 That way we have less files, and better organization. We could always make more files if we need to, but I think 10 save slots should be enough to cover the lowest and highest values, and 8 values in-between.

[alexandermueller]

Just checked out the code and it looks good!

I'll take a swing at refactoring your code into the stuff I've been working on today so that we can have something that can batch process all files within a designated input folder (let's say ./Files/Input) and spit out the results into a designated results folder (let's say ./Files/Results.)

We drop the .sl2 files (eg hairstyle.sl2) into the input folder, run the script, and the results text files (eg hairstyle_results.txt) will appear in the results folder.

If applicable, we could name each slot according to the value that slot was given for the appearance attribute (eg slot1 == ponytail) and then I can plop that out into the results data so when we reference it for later, we know what did what.

Also, instead of generating a bunch of separate save files to use for alldifs, I'm thinking we could make one base save file (eg ./Files/base.sl2) that has the exact same build over all its slots. We can now use that save file to generate alldifs for every analysis we want to run, assuming we started with the exact same build as it and then changed something to create a new .sl2 file. 🤔

[Ariescyn]

we could name each slot according to the value that slot was given for the appearance attribute

For sure, we will need to make note of which value maps to which hairstyle etc. 9 is the default haircut it seems.

I'm thinking we could make one base save file (eg ./Files/base.sl2) that has the exact same build over all its slots.

Sounds great!

[alexandermueller]

Last night I made two save files to test around with, and so far the results look pretty good! I made a base.sl2 and head_5.sl2 file, where the base save is just a default male wretch class for all 10 slots. Each slot name is named base <slot #> and the game was immediately exited upon loading each slot the first time. The other file is a copied base file but the head size values were changed for the first 5 slots. I haven't had time to look into a way to accurately grab the slot count from the data yet, so my stopgap solution is to include the count in the filename.

Using those files, I managed to fine tune the filter so now it only shows the indices that were changed <active slot count> times (with unique values for every change.) That means, if I create 5 slots in head_5.sl2 and gave unique values to each slot for the head size, then only that index should appear, or any indices that might be related to that change that also changed their values, or any other changes that were made alongside it. This also means that we could change 12 (or even more!) sliders at once in the same save file if we really wanted to, so long as the values we change are easily recognisable to that slider (and don't repeat on that slider.)

Here's the head_5_results.txt file as an example for how precise the filter is now:

         | 1 |  2 |  3 |  4 |  5 
---------+---+----+----+----+----
 [80454] | 0 | 12 | 24 | 36 | 48 

(the header row is each slot I changed, the first column is the index in the slot where the change occurred, and the values are the slider values I assigned to that specific slot)

So yeah, I think my PR is pretty much finished now, and is ready to be tested further if you feel like checking it out:
https://github.com/alexandermueller/EldenRingSaveAnalysis/pull/1/files

[Ariescyn]

Nice work! I love the results format it looks really good. This F string is insane lol:

outputLine = [f' { " " * (lengths[dataIndex] + 2) } ' if dataIndex == 0 and lineIndex == 0 else f' [{ "0" * (lengths[dataIndex] - len(str(data)))}{ data }] ' if dataIndex == 0 else f' { " " * (lengths[dataIndex] - len(str(data))) }{ data } ' for dataIndex, data in enumerate(line)]

Hey do you know any SQL? I'm stuck on a problem in HackerRank and my output looks right but i can't F*&$ figure out why it won't accept my solution lol... If you want to help or just want to chat, shoot me an email [email protected]

[Ariescyn]

On your release can you document the main function? (Just the core stuff, not all of the string formatting for printing to console)

[alexandermueller]

Thanks! Yeah, I definitely sacrificed legibility over compactness here 😅

Sorry, I don't really have much SQL experience, but I'll shoot you a message so you have mine too 👍

For sure, I'm going to fine tune it a bit more and then add documentation when it's all finalized