Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Various improvements for readpe #200

Closed
wants to merge 16 commits into from
Closed

Various improvements for readpe #200

wants to merge 16 commits into from

Conversation

pali
Copy link
Contributor

@pali pali commented Oct 9, 2023

  • Add new machines types
  • Add image characteristic IMAGE_FILE_16BIT_MACHINE
  • Add missing image dll characteristics
  • Add missing subsystems
  • Recognize all known section characteristics
  • Add support for parsing loader flags
  • Validate size of PE optional header
  • Add support for parsing long section names
  • Add support for parsing PE object files
  • Add support for parsing PE ROM files
  • Add machine type for old Alpha
  • Add IMAGE_FILE_MACHINE_OMNI
  • Add support for parsing Win32VersionValue
  • Update information about IMAGE_FILE_BYTES_REVERSED
  • Remove NE signature check
  • Show PE signature if present in readpe output

pali added 16 commits June 23, 2023 20:12
@pali
Add new machines types
c728a24 
IMAGE_FILE_MACHINE_POWERPCBE from https://github.com/microsoft/microsoft-pdb/blob/master/cvdump/dumppdb.cpp#L19
IMAGE_FILE_MACHINE_ARM64EC from https://learn.microsoft.com/en-us/windows/arm/arm64ec
IMAGE_FILE_MACHINE_ARM64X from https://learn.microsoft.com/en-us/windows/arm/arm64x-pe
IMAGE_FILE_MACHINE_CHPE_X86 from Windows 10 WDK 10.0.16299.0 ntimage.h
IMAGE_FILE_MACHINE_LOONGARCH32 from https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#machine-types
IMAGE_FILE_MACHINE_LOONGARCH64 from https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#machine-types
IMAGE_FILE_MACHINE_RISCV32 from https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#machine-types
IMAGE_FILE_MACHINE_RISCV64 from https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#machine-types
IMAGE_FILE_MACHINE_RISCV128 from https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#machine-types
@pali
Add image characteristic IMAGE_FILE_16BIT_MACHINE
8bdea95 
It is defined in DDK's ntimage.h file.
@pali
Add missing image dll characteristics
8b5faa4 
IMAGE_LIBRARY_PROCESS_* from PECOFF 4.0 spec (archived at https://bytepointer.com/resources/pecoff_v4.0.htm)
IMAGE_DLLCHARACTERISTICS_HIGH_ENTROPY_VA from https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#dll-characteristics
IMAGE_DLLCHARACTERISTICS_APPCONTAINER from https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#dll-characteristics
IMAGE_DLLCHARACTERISTICS_GUARD_CF from https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#dll-characteristics
IMAGE_DLLCHARACTERISTICS_X86_THUNK from WDK ntimage.h and US6385567 patent (https://patents.google.com/patent/US6385567)

Note that IMAGE_LIBRARY_PROCESS_* and IMAGE_DLLCHARACTERISTICS_X86_THUNK
are only for DLL files; IMAGE_DLLCHARACTERISTICS_APPCONTAINER is for EXE files.

IMAGE_DLLCHARACTERISTICS_X86_THUNK and IMAGE_DLLCHARACTERISTICS_APPCONTAINER
have really same number flags.
@pali
Add missing subsystems
e5a7264 
Subsystem 4 is mentioned WDK ntimage.h as Old Windows CE subsystem
Subsystem 8 is supported by MSVC4 LINK.EXE by /SUBSYSTEM:MMOSA flag and refers to MMOSA Native Win32E (embedded) API (https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-97-18.doc)
Subsystem 17 (IMAGE_SUBSYSTEM_XBOX_CODE_CATALOG) is defined in Windows 10 SDK 1607 winnt.h
@pali
Recognize all known section characteristics
a76a24f 
Information was gathered from various versions of SDK files ntimage.h,
winnt.h and pecoff*.doc documents and also from behavior of MSVC40
CL.EXE (including M68K version), LINK.EXE and DUMPBIN.EXE.

Machine type M68K has its own meaning of some flags (PROTECTED, FARDATA,
SYSHEAP, PURGEABLE, LOCKED, PRELOAD) which are specific for the old classic
Mac OS memory management.

Some of these M68K specific flags have same values as other non-M68K flags,
so parsing of section characteristics is machine type dependent.

Fix also printing section characteristics names, handle IMAGE_SCN_ALIGN_*
enum correctly (it is enum inside of bitfield) and always print all names,
including those which should not be used for particular file type. This
allows to easily inspect generated PE binary if it does not have some
additional characteristics which it should not have.
@pali
Add support for parsing loader flags
87d4b71 
Loader flags for DLL libraries are defined in older PECOFF 4.0 spec (archived at https://bytepointer.com/resources/pecoff_v4.0.htm)
Loader flags for application executables are defined in WDK ntimage.h file

Note that IMAGE_LOADER_FLAGS_BREAK_ON_LOAD and IMAGE_LOADER_FLAGS_COMPLUS have really same value.
@pali
Validate size of PE optional header
79142db 
SizeOfOptionalHeader is size of IMAGE_OPTIONAL_HEADER_32 and size of all
directories entries (which is variable count and can be also zero).
@pali
Add support for parsing long section names
19579d3 
Long section names are stored in string table and name in the section field
contains a slash that is followed by an ASCII representation of a decimal
number that is an offset into the string table.
@pali
Add support for parsing PE object files
369c3b8 
Object file does not have neither MZ header nor PE\0\0 signature nor
optional header.
@pali
Add support for parsing PE ROM files
8588ab2 
PE ROM file does not have neither MZ header nor PE\0\0 signature.

PE ROM file has either MAGIC_ROM optional header (for machine types R3000,
R4000, R10000, ALPHA) or MAGIC_PE32 optional header (for all other machine
types like: I386, MPPC_601, POWERPC, ...).

When MAGIC_ROM optional header is present then PE Characteristics flags
have different meaning.

PE ROM file can be created by older LINK.EXE (from MSVC 4.0 or older) via
/ROM command line switch.

All information was taken from LINK.EXE behavior and DUMPBIN.EXE outputs.
@pali
Add machine type for old Alpha
b7b5edf 
During NT/Alpha development, the magic number definition in Alpha images was incremented by one.

Old Alpha value is described in Peering Inside the PE: A Tour of the Win32 Portable Executable File Format by Matt Pietrek, March 1994:
https://web.archive.org/web/20080905063304/http://msdn.microsoft.com/en-us/magazine/ms809762.aspx
@pali
Add IMAGE_FILE_MACHINE_OMNI
9a070fd 
Machine type 0xace1 is Microsoft OMNI VM (omniprox.dll).
@pali
Add support for parsing Win32VersionValue
88f49e5 
Meaning of Win32VersionValue is described at https://www.geoffchappell.com/studies/windows/km/ntoskrnl/inc/api/pebteb/peb/index.htm
@pali
Update information about IMAGE_FILE_BYTES_REVERSED
70e56a8 
Microsoft PE 32-Bit LINK.EXE Version 1.00 (part of NT 3.1 SDK) always sets
both IMAGE_FILE_BYTES_REVERSED_LO and IMAGE_FILE_BYTES_REVERSED_HI bits but
produced PE executables do not have any word reversed. New Microsoft Linker
versions do not set these bits anymore.

Nowadays these bits are not set by any linker, they are deprecated and so
their original meaning was forgotten. Both bits are in the same bit
position of in each byte of the short word and so any of them can be
tested, so 16-bit test mask is endian independent.

It was designed that both bits are set when bytes of the word are reversed
from CPU defaults (e.g. Big Endian I386 PE binary would have both bits as
the default word ordering for I386 is Little Endian).

But because PE executables produced by the NT 3.1 SDK do not follow it,
these bits are unreliable and mostly useless.
@pali
Remove NE signature check
78f6597 
libpe/readpe does not support parsing 16-bit Windows NE structures, so it
does not make sense to try detect NE signature. Note that detection never
worked because NE signature is just 16-bit "NE", not 32-bit "NE\0\0".
@pali
Show PE signature if present in readpe output
950067c 
PE ROM files and PE object files do not have PE\0\0 signature.
To easily figure out this fact, show PE signature in the readpe output.
@GoGoOtaku GoGoOtaku self-assigned this Oct 10, 2023
@GoGoOtaku GoGoOtaku mentioned this pull request Oct 10, 2023
Open
@pali
Copy link
Contributor Author

pali commented Oct 17, 2023

@GoGoOtaku if you need some help or some explanation of something there, just ask me.

@GoGoOtaku
Copy link
Collaborator

GoGoOtaku commented Oct 18, 2023 •
edited
Loading

First of all: Thank you very much for this.

My biggest issue is that I know very little about PE ROMs and none of my test executables are ROMs.
That said I don't have any problems with the code per se. There is a missing indentation in an if block but I can just fix it on merge.

I didn't like pe_dll_image_dllcharacteristic_name as a function name.
I do understand tho that IMAGE_DLLCHARACTERISTICS_X86_THUNK and IMAGE_DLLCHARACTERISTICS_APPCONTAINER
cause this. Overall I think I just accept the weird name tho so that's fine.

And again thank you very much.

@pali
Copy link
Contributor Author

pali commented Oct 18, 2023

none of my test executables are ROMs

You can generate x86 PE ROM binary very easily by MSVC4 linker.
Use simple test.c file int main() { return 0; } and run CL.EXE test.c /link /ROM /ENTRY:main

Other machine types can be generated by flags:

  • PowerPC little-endian: LINK.EXE /MACHINE:powerpc /ROM
  • PowerPC big-endian: LINK.EXE /MACHINE:MPPC /ROM
  • Alpha 32-bit: LINK.EXE /MACHINE:ALPHA /ROM
  • MIPS: LINK.EXE /MACHINE:MIPSR10 /ROM

But you need either compiler for those machine types or you can use a trick that MSVC linker can take also binary resource file and generate from it PE executable. Flags like /ENTRY:0 or /FORCE can be useful for playing with it.

I didn't like pe_dll_image_dllcharacteristic_name as a function name.

Feel free to invent other / better name and rename the function.

@GoGoOtaku GoGoOtaku closed this Oct 18, 2023
@GoGoOtaku
Copy link
Collaborator

Feel free to invent other / better name and rename the function.
It's ok

This is merged now

@GoGoOtaku
Copy link
Collaborator

I merged manually due to some conflicts and github doesn't like this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@GoGoOtaku GoGoOtaku

Labels
enhancement libpe
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pali @GoGoOtaku @karenetheridge