Initial Release

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2020 Markus Gaasedelen
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,103 @@
+# Lucid - An Interactive Hex-Rays Microcode Explorer
+
+<p align="center">
+<img alt="Lucid Plugin" src="screenshots/lucid_demo.gif"/>
+</p>
+
+## Overview
+
+Lucid is a developer-oriented [IDA Pro](https://www.hex-rays.com/products/ida/) plugin for exploring the Hex-Rays microcode. It was designed to provide a seamless, interactive experience for studying microcode transformations in the decompiler pipeline.
+
+This plugin is labeled only as a prototype & code resource for the community. Please note that it is a development aid, not a general purpose reverse engineering tool. 
+
+Special thanks to [genmc](https://github.com/patois/genmc) / [@pat0is](https://twitter.com/pat0is) et al. for the inspiration.
+
+## Releases
+
+* v0.1 -- Initial release
+
+## Installation
+
+Lucid is a cross-platform (Windows, macOS, Linux) Python 2/3 plugin. It takes zero third party dependencies, making the code both portable and easy to install.
+
+1. From your disassembler's python console, run the following command to find its plugin directory:
+   - **IDA Pro**: `os.path.join(idaapi.get_user_idadir(), "plugins")`
+
+2. Copy the contents of this repository's `/plugins/` folder to the listed directory.
+3. Restart your disassembler.
+
+This plugin is only supported for IDA 7.5 and newer.
+
+## Usage
+
+Lucid will automatically load for any architecture with a Hex-Rays decompiler present. Simply right click anywhere in a Pseudocode window and select `View microcode` to open the Lucid Microcode Explorer.
+
+<p align="center">
+<img alt="View microcode" src="screenshots/lucid_view_microcode.gif"/>
+</p>
+
+By default, the Microcode Explorer will synchronize with the active Hex-Rays Pseudocode window.
+
+## Lucid Layers
+
+Lucid makes it effortless to trace microinstructions through the entire decompiler pipeline. Simply select a microinstruction, and *scroll* (or click... if you must) through the microcode maturity layer list.
+
+<p align="center">
+<img alt="Lucid Layer Traversal Demo" src="screenshots/lucid_layers.gif"/>
+</p>
+
+Watch as the explorer stays focused on your selected instruction, while the surrounding microcode landscape melts away. It's basically magic.
+
+## Sub-instruction Granularity
+
+Cursor tracing can operate at a sub-operand / sub-instruction level. Placing your cursor on different parts of the same microinstruction can trace sub-components back to their respective origins.
+
+<p align="center">
+<img alt="Lucid Sub-instruction Granularity Demo" src="screenshots/lucid_granularity.gif"/>
+</p>
+
+If the instructions at the traced address get optimized away, Lucid will attempt to keep your cursor in the same approximate context. It will change the cursor color from green to red to indicate the loss of precision.
+
+## Sub-instruction Trees
+
+As the Hex-Rays microcode increases in maturity, the decompilation pipeline begins to nest microcode as sub-instructions and sub-operands that form tree-based structures. 
+
+<p align="center">
+<img alt="Lucid Sub-instrution Graph Demo" src="screenshots/lucid_subtree.gif"/>
+</p>
+
+You can view these individual trees by right clicking an instruction and selecting `View subtree`.
+
+## Known Bugs
+
+As this is the initial release, there will probably a number of small quirks and bugs. Here are a few known issues at the time of release:
+
+* When opening the Sub-instruction Graph, window/tab focus can change unexpectedly
+* Microcode Explorer does not dock to the top-level far right compartment on Linux?
+* Switching between multiple Pseudocode windows in different functions might cause problems
+* Double clicking an instruction address comment can crash IDA if there is no suitable view to jump to
+* Plugin has not been tested robustly on Mac / Linux
+* ...?
+
+If you encounter any crashes or bad behavior, please file an issue.
+
+## Future Work
+
+Time and motivation permitting, future work may include:
+
+* Clean up the code.......
+* Interactive sub-instruction graph generalization (to pattern_t / rules)
+* Microcode optimizer development workflow?
+* Microcode optimization manager?
+* Ctree explorer (and similar graph generalization stuff...)
+* Microcode hint text?
+* Improve layer translations
+* Improve performance
+* Migrate off IDA codeview?
+* ...?
+
+I welcome external contributions, issues, and feature requests. Please make any pull requests to the `develop` branch of this repository if you would like them to be considered for a future release.
+
+## Authors
+
+* Markus Gaasedelen ([@gaasedelen](https://twitter.com/gaasedelen))

plugins/lucid/__init__.py

@@ -0,0 +1 @@
+from lucid.core import LucidCore

plugins/lucid/core.py

@@ -0,0 +1,231 @@
+import ida_idaapi
+import ida_kernwin
+
+from lucid.util.ida import UIHooks, IDACtxEntry, hexrays_available
+from lucid.ui.explorer import MicrocodeExplorer
+
+#------------------------------------------------------------------------------
+# Lucid Plugin Core
+#------------------------------------------------------------------------------
+#
+#    The plugin core constitutes the traditional 'main' plugin class. It
+#    will host all of the plugin's objects and integrations, taking
+#    responsibility for their initialization/teardown/lifetime. 
+#
+#    This pattern of splitting out the plugin core from the IDA plugin_t stub
+#    is primarily to help separate the plugin functionality from IDA's and
+#    make it easier to 'reload' for development / testing purposes.
+#
+
+class LucidCore(object):
+
+    PLUGIN_NAME = "Lucid"
+    PLUGIN_VERSION = "0.1.0"
+    PLUGIN_AUTHORS = "Markus Gaasedelen"
+    PLUGIN_DATE = "2020"
+
+    def __init__(self, defer_load=False):
+        self.loaded = False
+        self.explorer = None
+
+        #
+        # we can 'defer' the load of the plugin core a little bit. this
+        # ensures that all the other plugins (eg, decompilers) can get loaded
+        # and initialized when opening an idb/bin 
+        #
+
+        class UIHooks(ida_kernwin.UI_Hooks):
+            def ready_to_run(self):
+                pass
+
+        self._startup_hooks = UIHooks()
+        self._startup_hooks.ready_to_run = self.load
+
+        if defer_load:
+            self._startup_hooks.hook()
+            return
+
+        # plugin loading was not deferred (eg, hot reload), load immediately
+        self.load()
+
+    #-------------------------------------------------------------------------
+    # Initialization / Teardown
+    #-------------------------------------------------------------------------
+
+    def load(self):
+        """
+        Load the plugin core.
+        """
+        self._startup_hooks.unhook()
+
+        # the plugin will only load for decompiler-capabale IDB's / installs
+        if not hexrays_available():
+            return
+
+        # print plugin banner
+        print("Loading %s v%s - (c) %s" % (self.PLUGIN_NAME, self.PLUGIN_VERSION, self.PLUGIN_AUTHORS))
+
+        # initialize the the plugin integrations
+        self._init_action_view_microcode()
+        self._install_hexrays_hooks()
+
+        # all done, mark the core as loaded
+        self.loaded = True
+
+    def unload(self, from_ida=False):
+        """
+        Unload the plugin core.
+        """
+
+        # unhook just in-case load() was never actually called...
+        self._startup_hooks.unhook()
+
+        # if the core was never fully loaded, there's nothing else to do
+        if not self.loaded:
+            return
+
+        print("Unloading %s..." % self.PLUGIN_NAME)
+
+        # mark the core as 'unloaded' and teardown its components
+        self.loaded = False
+
+        self._remove_hexrays_hooks()
+        self._del_action_view_microcode()
+
+    #--------------------------------------------------------------------------
+    # UI Actions
+    #--------------------------------------------------------------------------
+
+    def interactive_view_microcode(self, ctx=None):
+        """
+        Open the Microcode Explorer window.
+        """
+        current_address = ida_kernwin.get_screen_ea()
+        if current_address == ida_idaapi.BADADDR:
+            print("Could not open Microcode Explorer (bad cursor address)")
+            return
+
+        #
+        # if the microcode window is open & visible, we should just refresh
+        # it but at the current IDA cursor address
+        #
+
+        if self.explorer and self.explorer.visible():
+            self.explorer.decompile(current_address)
+            return
+
+        # no microcode window in use, create a new one and show it
+        self.explorer = MicrocodeExplorer()
+        self.explorer.show(current_address)
+
+    #--------------------------------------------------------------------------
+    # Action Registration
+    #--------------------------------------------------------------------------
+
+    ACTION_VIEW_MICROCODE  = "lucid:view_microcode"
+
+    def _init_action_view_microcode(self):
+        """
+        Register the 'View microcode' action with IDA.
+        """
+
+        # describe the action
+        action_desc = ida_kernwin.action_desc_t(
+            self.ACTION_VIEW_MICROCODE,                    # The action name
+            "View microcode",                              # The action text
+            IDACtxEntry(self.interactive_view_microcode),  # The action handler
+            "Ctrl-Shift-M",                                # Optional: action shortcut
+            "Open the Lucid Microcode Explorer",           # Optional: tooltip
+            -1                                             # Optional: the action icon
+        )
+
+        # register the action with IDA
+        assert ida_kernwin.register_action(action_desc), "Action registration failed"
+
+    def _del_action_view_microcode(self):
+        """
+        Delete the 'View microcode' action from IDA.
+        """
+        ida_kernwin.unregister_action(self.ACTION_VIEW_MICROCODE)
+
+    #--------------------------------------------------------------------------
+    # Hex-Rays Hooking
+    #--------------------------------------------------------------------------
+
+    def _install_hexrays_hooks(self):
+        """
+        Install the Hex-Rays hooks used by the plugin core.
+        """
+        import ida_hexrays
+
+        class CoreHxeHooks(ida_hexrays.Hexrays_Hooks):
+            def populating_popup(_, *args):
+                self._hxe_popuplating_popup(*args)
+                return 0
+
+        self._hxe_hooks = CoreHxeHooks()
+        self._hxe_hooks.hook()
+
+    def _remove_hexrays_hooks(self):
+        """
+        Remove the Hex-Rays hooks used by the plugin core.
+        """
+        self._hxe_hooks.unhook()
+        self._hxe_hooks = None
+
+    def _hxe_popuplating_popup(self, widget, popup, vdui):
+        """
+        Handle a Hex-Rays popup menu event.
+
+        When the user right clicks within a decompiler window, we use this
+        callback to insert the 'View microcode' menu entry into the ctx menu.
+        """
+        ida_kernwin.attach_action_to_popup(
+            widget,
+            popup,
+            self.ACTION_VIEW_MICROCODE,
+            None,
+            ida_kernwin.SETMENU_APP
+        )
+
+    #--------------------------------------------------------------------------
+    # Plugin Testing
+    #--------------------------------------------------------------------------
+
+    def test(self):
+        """
+        TODO/TESTING: move this to a dedicated module/file
+
+        just some misc stuff for testing the plugin...
+        """
+        import time
+        import idautils
+        from lucid.util.hexrays import get_mmat_levels, get_mmat_name
+
+        for address in list(idautils.Functions()):
+
+            print("0x%08X: DECOMPILING" % address)
+            self.explorer.decompile(address)
+            self.explorer.view.refresh()
+
+            # change the codeview to a starting maturity levels
+            for src_maturity in get_mmat_levels():
+                self.explorer.select_maturity(get_mmat_name(src_maturity))
+
+                # select each line in the current 'starting' maturity context
+                for idx, line in enumerate(self.explorer.model.mtext.lines):
+                    self.explorer.select_pos(idx, 0, 0)
+
+                    # 
+                    maturity_traversal = get_mmat_levels()
+                    maturity_traversal = maturity_traversal[maturity_traversal.index(src_maturity)+1:] + get_mmat_levels()[::-1][1:]
+
+                    # scroll up / down the maturity traversal
+                    for dst_maturity in maturity_traversal:
+                        #print("%-60s -- %s" % ("S_MAT: %s E_MAT: %s IDX: %u" % (get_mmat_name(src_maturity), get_mmat_name(dst_maturity), idx), line.text))
+                        self.explorer.select_maturity(get_mmat_name(dst_maturity))
+                        #ida_kernwin.refresh_idaview_anyway()
+                        #time.sleep(0.05)
+
+                    self.explorer.select_maturity(get_mmat_name(src_maturity))
+