Skip to content

Commit

Permalink
Extension of bin2llvmir with optimization of X87 FPU stack. (#715)
Browse files Browse the repository at this point in the history 
* capstone2llvmir/x86: Add support of x87 FISTPP instruction and unit tests of new supported instruction.

* capstone2llvmir/x86: Add support of x87 FCMOVEcc instructions and unit test of new supported instructions.

* capstone2llvmir/x86: Add support of x87 FPREM instructions and unit test of new supported instructions.

* capstone2llvmir/x86: Add tests for x87 FPREM instructions.

* capstone2llvmir/x86: Add support of x87 F2XM1, FYL2X, FYL2xP1 instructions and unit test of new supported instructions.

* capstone2llvmir/x86: Add support of x87 FFREE instructions and unit test of new supported instructions.

* capstone2llvmir/x86: Add support of x87 FNSTCW instructions and unit test of new supported instructions.

* bin2llvmir/optimizations/x87_fpu: Add test for x87 fpu analysis and fix register indexing bug.

* bin2llvmir/optimizations/x87_fpu: Fix test for x87 fpu analysis.

* capstone2llvmir/x86.cpp: remove extra new lines

* capstone2llvmir/x86_impl.h: remove extra newline & indent

* capstone2llvmir/x86: Fix indention of new tests + comments.

* bin2llvmir/optimizations/x87_fpu: Create tests for different calling conventions and different architectures of x86.

Tests for architectures x86-16, x86-32, x86-64.
Tests for calling conventions cdecl, pascal, fastcall, stdcall, thiscall.

* capstone2llvmir/x86: Unit tests and translation of all FPU capstone instruction to llvmir.

FBLD, FBSTP, FNCLEX, FLDCW, FLDENV, FRSTOR, FNSAVE, FNSTENV, FNSTCW, FXSAVE, FXSAVE64, FXRSTOR, FXRSTOR64, FPTAN, FPATAN, FSCALE, FXTRACT, FXAM.

* capstone2llvmir/x86: Fix bug in FXRSTOR unit etst.

* config: Include "watcom" calling convention.

* bin2llvmir/optimizations/x87_fpu: Tests for x87 FPU analyze.

Tests for calling conventions:
- x86-16: cdecl, pascal, fastcall, watcom, unknown
- x86-32: cdecl, stdcall, pascal, fastcall, thiscall, watcom, unknown
- x86-64

* bin2llvmir/optimizations/x87_fpu: Analyze of FPU stack for functions with floating point return value (definitions and calls).

Analyzed calling conventions:
- x86-16: cdecl, pascal, fastcall, watcom, unknown
- x86-32: cdecl, stdcall, pascal, fastcall, thiscall, watcom, unknown
- x86-64

* bin2llvmir/optimizations/x87_fpu: Tests for FPU register stack usage with branches and loops.

* bin2llvmir/optimizations/x87_fpu: Analyze of FPU stack for nested blocks (branches, loops).

* bin2llvmir/optimizations/x87_fpu: Fix nested blocks analyze tests.

* bin2llvmir/optimizations/x87_fpu: Implement linear equation solver of basic block TOP tracking. Include OpenCV lib.

* bin2llvmir/optimizations/x87_fpu: FPU TOP optimization implemented for simple and nested BasicBlocks, floating-point return values of functions.

TODO: Import OpenCV library to Cmake build. Implement special calling conventions for 32bit architecture.

* capstone2llvmir/x86: Add support of x87 FPREM instructions and unit test of new supported instructions.

* capstone2llvmir/x86: Add support of x87 FNSTCW instructions and unit test of new supported instructions.

* capstone2llvmir/x86_impl.h: remove extra newline & indent

* capstone2llvmir/x86: Fix bug in FXRSTOR unit etst.

* deps/eigen: Include third party library Eigen to cmake of project. Library is used to math computation with matricies for X86 FPU analyze optimization.

* bin2llvmir/optimization/x87_fpu: Rewrite optimization with new Eigen library. Use matrix implementation of SVD from eigen library.

* tests/bin2llvmir: Refactor CMakeLists.txt

* bin2llvmir/optimization/x87_fpu: Fix bux for X86-16bit calling convention. Optimization now validate value of FPU top at the end of every terminating block and detects unsupported states.

* bin2llvmir/optimizations/x87_fpu: Fix and improve FPU stack optimization testss

* bin2llvmir/optimizations/x87_fpu: Fix major bug of overdetermined linear equation solver and minor bugs of final llvm ir generating.

* capstone2llvmir/x86: Add support of x87 FPREM instructions and unit test of new supported instructions.

* capstone2llvmir/x86: Add support of x87 FNSTCW instructions and unit test of new supported instructions.

* bin2llvmir/optimizations/x87_fpu: Add test for x87 fpu analysis and fix register indexing bug.

* bin2llvmir/optimizations/x87_fpu: Fix test for x87 fpu analysis.

* capstone2llvmir/x86_impl.h: remove extra newline & indent

* bin2llvmir/optimizations/x87_fpu: Create tests for different calling conventions and different architectures of x86.

Tests for architectures x86-16, x86-32, x86-64.
Tests for calling conventions cdecl, pascal, fastcall, stdcall, thiscall.

* capstone2llvmir/x86: Fix bug in FXRSTOR unit etst.

* capstone2llvmir/x86: Add support of x87 FPREM instructions and unit test of new supported instructions.

* capstone2llvmir/x86_impl.h: remove extra newline & indent

* capstone2llvmir/x86: Fix bug in FXRSTOR unit etst.

* bin2llvmir/optimizations/x87_fpu: Tests for x87 FPU analyze.

Tests for calling conventions:
- x86-16: cdecl, pascal, fastcall, watcom, unknown
- x86-32: cdecl, stdcall, pascal, fastcall, thiscall, watcom, unknown
- x86-64

* bin2llvmir/optimizations/x87_fpu: Analyze of FPU stack for functions with floating point return value (definitions and calls).

Analyzed calling conventions:
- x86-16: cdecl, pascal, fastcall, watcom, unknown
- x86-32: cdecl, stdcall, pascal, fastcall, thiscall, watcom, unknown
- x86-64

* bin2llvmir/optimizations/x87_fpu: Tests for FPU register stack usage with branches and loops.

* bin2llvmir/optimizations/x87_fpu: Analyze of FPU stack for nested blocks (branches, loops).

* bin2llvmir/optimizations/x87_fpu: Fix nested blocks analyze tests.

* bin2llvmir/optimizations/x87_fpu: Implement linear equation solver of basic block TOP tracking. Include OpenCV lib.

* bin2llvmir/optimizations/x87_fpu: FPU TOP optimization implemented for simple and nested BasicBlocks, floating-point return values of functions.

TODO: Import OpenCV library to Cmake build. Implement special calling conventions for 32bit architecture.

* deps/eigen: Include third party library Eigen to cmake of project. Library is used to math computation with matricies for X86 FPU analyze optimization.

* bin2llvmir/optimization/x87_fpu: Rewrite optimization with new Eigen library. Use matrix implementation of SVD from eigen library.

* tests/bin2llvmir: Refactor CMakeLists.txt

* bin2llvmir/optimization/x87_fpu: Fix bux for X86-16bit calling convention. Optimization now validate value of FPU top at the end of every terminating block and detects unsupported states.

* bin2llvmir/optimizations/x87_fpu: Fix and improve FPU stack optimization testss

* bin2llvmir/optimizations/x87_fpu: Fix major bug of overdetermined linear equation solver and minor bugs of final llvm ir generating.

* capstone2llvmir/x86: Add support of x87 FPREM instructions and unit test of new supported instructions.

* capstone2llvmir/x86: Add support of x87 FNSTCW instructions and unit test of new supported instructions.

* bin2llvmir/optimizations/x87_fpu: Add test for x87 fpu analysis and fix register indexing bug.

* bin2llvmir/optimizations/x87_fpu: Fix test for x87 fpu analysis.

* capstone2llvmir/x86_impl.h: remove extra newline & indent

* bin2llvmir/optimizations/x87_fpu: Create tests for different calling conventions and different architectures of x86.

Tests for architectures x86-16, x86-32, x86-64.
Tests for calling conventions cdecl, pascal, fastcall, stdcall, thiscall.

* capstone2llvmir/x86: Fix bug in FXRSTOR unit etst.

* capstone2llvmir/x86: Add support of x87 FPREM instructions and unit test of new supported instructions.

* bin2llvmir/optimizations/x87_fpu: Tests for x87 FPU analyze.

Tests for calling conventions:
- x86-16: cdecl, pascal, fastcall, watcom, unknown
- x86-32: cdecl, stdcall, pascal, fastcall, thiscall, watcom, unknown
- x86-64

* bin2llvmir/optimizations/x87_fpu: Analyze of FPU stack for functions with floating point return value (definitions and calls).

Analyzed calling conventions:
- x86-16: cdecl, pascal, fastcall, watcom, unknown
- x86-32: cdecl, stdcall, pascal, fastcall, thiscall, watcom, unknown
- x86-64

* bin2llvmir/optimizations/x87_fpu: Tests for FPU register stack usage with branches and loops.

* bin2llvmir/optimizations/x87_fpu: Analyze of FPU stack for nested blocks (branches, loops).

* bin2llvmir/optimizations/x87_fpu: Fix nested blocks analyze tests.

* bin2llvmir/optimizations/x87_fpu: Implement linear equation solver of basic block TOP tracking. Include OpenCV lib.

* bin2llvmir/optimizations/x87_fpu: FPU TOP optimization implemented for simple and nested BasicBlocks, floating-point return values of functions.

TODO: Import OpenCV library to Cmake build. Implement special calling conventions for 32bit architecture.

* capstone2llvmir/x86: Add support of x87 FPREM instructions and unit test of new supported instructions.

* capstone2llvmir/x86_impl.h: remove extra newline & indent

* capstone2llvmir/x86: Fix bug in FXRSTOR unit etst.

* deps/eigen: Include third party library Eigen to cmake of project. Library is used to math computation with matricies for X86 FPU analyze optimization.

* bin2llvmir/optimization/x87_fpu: Rewrite optimization with new Eigen library. Use matrix implementation of SVD from eigen library.

* tests/bin2llvmir: Refactor CMakeLists.txt

* bin2llvmir/optimization/x87_fpu: Fix bux for X86-16bit calling convention. Optimization now validate value of FPU top at the end of every terminating block and detects unsupported states.

* bin2llvmir/optimizations/x87_fpu: Fix and improve FPU stack optimization testss

* bin2llvmir/optimizations/x87_fpu: Fix major bug of overdetermined linear equation solver and minor bugs of final llvm ir generating.

* capstone2llvmir/x86_impl.h: remove extra newline & indent

* bin2llvmir/optimizations/x87_fpu: Create tests for different calling conventions and different architectures of x86.

Tests for architectures x86-16, x86-32, x86-64.
Tests for calling conventions cdecl, pascal, fastcall, stdcall, thiscall.

* capstone2llvmir/x86: Add support of x87 FPREM instructions and unit test of new supported instructions.

* capstone2llvmir/x86_impl.h: remove extra newline & indent

* bin2llvmir/optimization/x87_fpu: Fix bux for X86-16bit calling convention. Optimization now validate value of FPU top at the end of every terminating block and detects unsupported states.

* bin2llvmir/optimizations/x87_fpu: Fix and improve FPU stack optimization testss

* src/capstone2llvmir/x86: Fix and remove duplicities caused by my incorrect merge commit

* src/bin2llvmir/x87_fpu: Fix bug - remove forgotten declaration.

* src/bin2llvmir/x87_fpu: Fix bug - make analyze more accurate: fix some bugs when TOP is changed but not stored etc.

* tests/bin2llvmir/optimizations/x87_fpu: Rewrite actual and include new tests - fix bugs revealed by integration tests analyze.

* src/bin2llvmir/optimizations/x87_fpu: Fix bugs revealed by integration testing and improve performance of this optimization.

* src/capstoce2llvmir/x86: Fix merge conflict.

* deps/eigen: Remove Eigen library from deps and moves it into external libs.

* tests/bin2llvmir/optimizations/x87_fpu: Remove tests to 16bit architecture because this feature has depreaced due to unacceptable performance lack.

* src/bin2llvmir/optimizations/x87_fpu: Improve and optimize performance of algorithm. Replace SVD decomposition for QR decomosition due to performance increase. Fix includes for eigen library that is now external project. Optimize matrix rank calculation.

* bin2llvmir/optimization/x87_fpu: Set max performance ceil.

* Add External Eigen to cmake/deps.

* deps/eigen: Update CMake build.

* deps/eigen: Update CMake build.

* bin2llvmir/x87_fpu: Update analyze and tests -> remove FPU TAGS because RetDec do not use them anymore.

* deps/eigen: Fix eigen INTERFACE_INCLUDE_DIRECTORIES which was prefixed in the build directory.

Co-authored-by: Peter Matula <[email protected]>
  • Loading branch information
@JurajHolub @PeterMatula
JurajHolub and PeterMatula authored May 28, 2020
1 parent c213750 commit 2183f84
Show file tree
Hide file tree
Showing 14 changed files with 2,601 additions and 206 deletions.
35 changes: 2 additions & 33 deletions LICENSE-THIRD-PARTY
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ RetDec uses the following third-party libraries or other resources:
9) whereami: https://github.com/gpakosz/whereami
10) yara: https://virustotal.github.io/yara/
11) yaramod: https://github.com/avast/yaramod
12) cmake-modules: https://github.com/rpavlik/cmake-modules
12) Eigen: http://eigen.tuxfamily.org/index.php?title=Main_Page
13) cmake-modules: https://github.com/rpavlik/cmake-modules

These third-party libraries or other resources are licensed under the
following licenses:
Expand Down Expand Up @@ -795,35 +796,3 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

===============================================================================
12) cmake-modules
===============================================================================

Copyright Iowa State University 2009-2014, or Copyright Sensics, Inc. 2014-2015, or Copyright Ryan A. Pavlik 2009-2015

Distributed under the Boost Software License, Version 1.0.

Boost Software License - Version 1.0 - August 17th, 2003

Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:

The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
8 changes: 8 additions & 0 deletions cmake/deps.cmake
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,14 @@ set(YARAMOD_ARCHIVE_SHA256
CACHE INTERNAL ""
)

set(EIGEN_URL
"https://gitlab.com/api/v4/projects/libeigen%2Feigen/repository/archive.zip?sha=0dd9643ad547d3dd2e23ded1d3376d0f7bdc8ada"
CACHE INTERNAL "URL of Eigen archive to use."
)
set(EIGEN_ARCHIVE_SHA256
"f56e0551aed4bdcb39cf6c5e51b1267b1221666e6090bddc1a0eaf89e7c3e2a3"
CACHE INTERNAL "")

set(SUPPORT_PKG_URL
"https://github.com/avast/retdec-support/releases/download/2019-03-08/retdec-support_2019-03-08.tar.xz"
CACHE INTERNAL "URL of RetDec support package to use."
Expand Down
3 changes: 3 additions & 0 deletions cmake/options.cmake
View file
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,9 @@ set_if_at_least_one_set(RETDEC_ENABLE_RAPIDJSON
RETDEC_ENABLE_MACHO_EXTRACTORTOOL
RETDEC_ENABLE_SERDES)

set_if_at_least_one_set(RETDEC_ENABLE_EIGEN
RETDEC_ENABLE_BIN2LLVMIR)

set_if_at_least_one_set(RETDEC_ENABLE_TINYXML2
RETDEC_ENABLE_TINYXML2
RETDEC_ENABLE_FILEINFO
Expand Down
1 change: 1 addition & 0 deletions deps/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,4 @@ cond_add_subdirectory(tinyxml2 RETDEC_ENABLE_TINYXML2)
cond_add_subdirectory(whereami RETDEC_ENABLE_WHEREAMI)
cond_add_subdirectory(yara RETDEC_ENABLE_YARA)
cond_add_subdirectory(yaramod RETDEC_ENABLE_YARAMOD)
cond_add_subdirectory(eigen RETDEC_ENABLE_EIGEN)
41 changes: 41 additions & 0 deletions deps/eigen/CMakeLists.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@

include(ExternalProject)

if(CMAKE_C_COMPILER)
set(CMAKE_C_COMPILER_OPTION "-DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}")
endif()
if(CMAKE_CXX_COMPILER)
set(CMAKE_CXX_COMPILER_OPTION "-DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}")
endif()

ExternalProject_Add(eigen_project
URL ${EIGEN_URL}
URL_HASH SHA256=${EIGEN_ARCHIVE_SHA256}
DOWNLOAD_NAME "eigen.zip"
CMAKE_ARGS
"${CMAKE_C_COMPILER_OPTION}"
"${CMAKE_CXX_COMPILER_OPTION}"
# Disable the install step.
INSTALL_COMMAND ""
)

add_library(eigen INTERFACE)
add_library(retdec::deps::eigen ALIAS eigen)
add_dependencies(eigen eigen_project)

ExternalProject_Get_Property(eigen_project source_dir)

set(EIGEN_INCLUDE_DIR "${source_dir}/Eigen")

target_include_directories(eigen INTERFACE $<BUILD_INTERFACE:${EIGEN_INCLUDE_DIR}>)

install(TARGETS eigen
EXPORT eigen-targets
)

# Export targets.
install(EXPORT eigen-targets
FILE "retdec-eigen-targets.cmake"
NAMESPACE retdec::deps::
DESTINATION ${RETDEC_INSTALL_CMAKE_DIR}
)
8 changes: 8 additions & 0 deletions deps/eigen/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
## Eigen

A clone of only the needed sources (headers) of [Eigen](http://eigen.tuxfamily.org/index.php?title=Main_Page).
Upstream commit: `6601abce868e3284b4829a4fbf91eefaa0d704af`

## License

Eigen is [Free Software](http://www.gnu.org/philosophy/free-sw.html). Starting from the 3.1.1 version, it is licensed under the [MPL2](http://www.mozilla.org/MPL/2.0), which is a simple weak copyleft license. Common questions about the MPL2 are answered in the official [MPL2 FAQ](http://www.mozilla.org/MPL/2.0/FAQ.html).
Loading

0 comments on commit 2183f84

Please sign in to comment.