Manipulated PE can't execute

[jymcheong]

I wrote a strip down script to test manipulate.modify_without_breaking

import numpy as np
from gym_malware.envs.utils import interface
from gym_malware.envs.controls import manipulate2 as manipulate

# np.random.seed(322333)
# random_action = lambda bytez: np.random.choice( list(manipulate.ACTION_TABLE.keys()) )
random_action = lambda bytez: 'section_append'
# original 32bit putty.exe in sha256 file name
bytez = interface.fetch_file('1d673f12ddf0e6cc1545a79471fd5dd56bf0ff9ccff49ff91b41b4085b727665')
action = random_action(bytez)
print(action)
bytez_mod = manipulate.modify_without_breaking( bytez, [action] )
with open("putty-mod32.exe", "wb") as new_file:
    new_file.write(bytez_mod)

The result PE was executed within a Windows 7 32bit but not executable/functional.

The only action that produced functional PE was overlay_append which didn't use LIEF. I tested both GUI & console windows PEs with overlay_append, the resultant files have different sha256-sum from the originals and are functional.

Unfortunately, for other actions that involved LIEF are used, the new file is not functional. I am using macOS python3.6 for this test.

When I ran the script in console, I did notice some error message for all actions that involved LIEF:

Any idea what could be the issue? Thanks!

[drhyrum]

@jymcheong We've been looking at this over the last several weeks. This is caused by a parsing error in LIEF (https://lief.quarkslab.com/), the library used for parsing and manipulating PE files. Namely, some PE files parsed and re-written by LIEF are not functional whether or not they have been mutated.

For example (no mutation):

import lief
pe = lief.parse('putty.exe')
builder = lief.PE.Builder(pe)
builder.build_imports(True) 
builder.patch_imports(True)
# if you remove the lines above (rebuild nothing), then the putty32-mod.exe should work
builder.build()
builder.write('putty32-mod.exe')

Results in

$ ./putty32-mod.exe
Segmentation fault

This appears directly related to the LIEF library parsing errors you observed. If the target PE binary is already malformed (as is the case for a lot of malware and some benignware) the PE builder will a have some difficulties rebuilding a valid one, whether or not any functionality-preserving manipulations were performed.

We've opened a dialog with authors of LIEF about this issue, and are hoping to help with a resolution.

In the meantime, an unfortunate workaround is pre-filter PE files for which LIEF doesn't complain (no parsing errors). We're hoping to provide a simple python script for this as a stop-gap until LIEF parser errors are resolved.

[jymcheong]

Thanks @drhyrum. It make senses now. I also chance upon this

I guess we have to pre-filter the PEs for the time being. Btw, are there any resellers/reps of Endgame in asia, specifically Singapore? We are to keen to explore the product.

[jymcheong]

I copied manipulate2.py and modified:

    def __binary_to_bytez(self, binary, dos_stub=False, imports=False, overlay=False, relocations=False, resources=False, tls=False):
        # write the file back as bytez
        builder = lief.PE.Builder(binary)
        if(dos_stub):
            builder.build_dos_stub(dos_stub) # rebuild DOS stub

        if(imports):
            builder.build_imports(imports) # rebuild IAT in another section
            builder.patch_imports(imports) # patch original import table with trampolines to new import table

        if(overlay):
            builder.build_overlay(overlay) # rebuild overlay
        
        if(relocations):
            builder.build_relocations(relocations) # rebuild relocation table in another section
        
        if(resources):
            builder.build_resources(resources) # rebuild resources in another section
        
        if(tls):
            builder.build_tls(tls) # rebuilt TLS object in another section

        builder.build() # perform the build process

     
        return array.array('B', builder.get_build()).tobytes() 

Then I wrote a script that uses the modified manipulate2.py:

import hashlib
import manipulate2 as manipulate
import sys

def printSHA256(filename):
    with open(filename, "rb") as binfile:
        bytez = binfile.read()
    print(filename + ": " + hashlib.sha256(bytez).hexdigest())

if(len(sys.argv) == 0):
    print("please provide a PE filename")

filename = sys.argv[1]
printSHA256(filename)
# filename str instead of bytestr
# lief.PE.parse will read the file instead
mm = manipulate.MalwareManipulator(filename)
bytez = mm.break_optional_header_checksum()
with open(filename.replace(".exe",".zerochecksum.exe"), "wb") as binfile:
    binfile.write(bytez)
printSHA256(filename.replace(".exe",".zerochecksum.exe"))

mm = manipulate.MalwareManipulator(filename)
bytez = mm.section_append()
with open(filename.replace(".exe",".section_append.exe"), "wb") as binfile:
    binfile.write(bytez)
printSHA256(filename.replace(".exe",".section_append.exe"))

mm = manipulate.MalwareManipulator(filename)
bytez = mm.section_rename()
with open(filename.replace(".exe",".section_rename.exe"), "wb") as binfile:
    binfile.write(bytez)
printSHA256(filename.replace(".exe",".section_rename.exe"))

with open(filename, "rb") as binfile:
    bytez = binfile.read()
mm = manipulate.MalwareManipulator(bytez)
bytez = mm.overlay_append()
with open(filename.replace(".exe",".overlay_append.exe"), "wb") as binfile:
    binfile.write(bytez)
printSHA256(filename.replace(".exe",".overlay_append.exe"))

All modified putty files ended up functional. Using a PE viewer, there are respective changes observed:

But if the PEs are read as bytes, then it doesn't parse properly. I tested it on OSX and Ubuntu.

So I force lief.PE.parse to use a filename string instead. This way will break overlay_append since it is appending bytes to string.

[fanmcgrady]

Maybe this is due to the permissions of win10.
I also run this project on my MacOS, it works!

[Aritran]

Hi, I'm still getting similar issues right now. Since this is an old thread, I'm curious if there is a solution to this? I'm using MacOS and python 3.6

[drhyrum]

Those repo is no longer maintained. Can you try moving to this new repo: https://github.com/bfilar/malware_rl

…

On Tue, Jan 26, 2021, 10:58 AM Aritran Piplai ***@***.***> wrote: Hi, I'm still getting similar issues right now. Since this is an old thread, I'm curious if there is a solution to this? I'm using MacOS and python 3.6 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB3UVB24SJK2AVII6LOLBDDS337CPANCNFSM4D5B3P4Q> .