Mixin 0.8.5

If you only care about the headlines then here's the lowdown on this release:

• Adds module imports for the new module names for Gson and Guava which ship with Minecraft 1.18. This fixes an issue where mixin was failing to load because it imported the old module names.
• Fixes an issue with the AP which was preventing generation of mappings for anonymous classes
• The AP now detects IntelliJ IDEA and disables some warnings in the same way it previously did for Eclipse
• All AP message levels (eg. warning, error) are configurable

Configuring AP Messaging

Improving the behaviour of the AP when running in the IDE also exposed some opportunities to give you, as a developer, more control over the AP's behaviour. Every single error, warning, or informational message generated by the AP is now configurable via the MessageType enum. This means that in addition to the site-specific @SuppressWarnings support defined in SuppressedBy it's now possible to globally control the message level for individual messages.

Message levels can be controlled by their type code, but don't worry, it's not necessary to look up the code in the MessageType enum every time, since controlling the message levels is fully supported by MixinGradle:

To find out the message type code for a particular message being generated by your build, simply specify the showMessageTypes option in the mixin closure:

mixin {
   config "mixins.mymod.json"
   add sourceSets.main, "mixins.mymod.refmap.json"
   
   // add this line
   showMessageTypes = true
}

Now when you run your build, each message will be prefixed with the message type in square brackets:

MixinRegionFileStorage.java:40: warning: [MIXIN_SOFT_TARGET_IS_PUBLIC] Mixin target
net/minecraft/world/level/chunk/storage/RegionFileStorage is public and should be specified in value
@Mixin(targets = "net/minecraft/world/level/chunk/storage/RegionFileStorage")
^

Now that you know the message type code (MIXIN_SOFT_TARGET_IS_PUBLIC) you can define a new level for the message in your mixin closure. Specify a closure for messages as follows:

mixin {
   config "mixins.mymod.json"
   add sourceSets.main, "mixins.mymod.refmap.json"
   
   showMessageTypes = true
   messages {
      MIXIN_SOFT_TARGET_IS_PUBLIC = 'error'
   }
}

Now when you run your build, the warning has been upgraded to error:

MixinRegionFileStorage.java:40: error: [MIXIN_SOFT_TARGET_IS_PUBLIC] Mixin target
net/minecraft/world/level/chunk/storage/RegionFileStorage is public and must be specified in value
@Mixin(targets = "net/minecraft/world/level/chunk/storage/RegionFileStorage")
^

Likewise, you can downgrade error to warning or warning to note. You can specify any of the following levels:

• disabled - disables the message entirely
• note - display the message at the note level, essentially an informational message
• warning - set the message as a warning
• error - set the message as an error

Downgrading messages can be useful to suppress errors and warnings globally. Upgrading messages can be useful if you want to have your build treat warnings as errors in order to make your build more robust.

Note that you can comment out the showMessageTypes = true line once you're done tuning message levels.

Mixin 0.8.4

Sometimes new Mixin releases present an opportunity to discuss different aspects of Mixin which are not covered in detail in the docs, or limitations which exist are not well understood by mixin authors, leading them to experience behaviour they aren't anticipating. The topic of the 0.8 series has definitely bounced around the topic of Locals, in particular CallbackInjector's unique ability to capture locals, so I will discuss some of the changes to Locals later in these notes. But first, the headlines:

Features

Minecraft 1.17, ModLauncher 9, Java 16, Oh My!

Minecraft 1.17 represents a significant jump in technology, adopting as it does Java 16, a big jump from the previous Java 8. Those in the modding community who remember the jump to Java 8 from Java 6 will anticipate something of what that means. Mixin has been for some time rooted in that Java 8 land because the primary consumers are using Java 8 and and (in some cases) Java 6, so supporting newer language features hasn't been a high priority.

With the release of Mixin 0.8.4, the core functionality of Mixin now supports up to Java 13 (cheating a little since nothing of note changed between Java 11 and 13), the missing piece of the puzzle being nesting. Mixins applied to inner classes will now be properly added to the existing nest.

The internal version of ASM has been bumped to 9.x to track ModLauncher, which now provides access to class version literals up to Java 18, which have been duly contributed to the CompatibilityLevel enum, along with associated language feature flags. Records and sealed classes are now on the radar for support, but will not be immediately available as things are ahead of them in the feature queue.

Mixin itself is now built as as Java module, though classes in the final jar will still be compatible back as far as Java 8 without modification, Java 6 users will still need to compile from source.

Fixes and Improvements

Besides the internal restructuring to support Java 16 and ModLauncher 9, this release is primarily aimed at addressing some regressions introduced in version 0.8.3. One of my primary goals with Mixin is to make a library with stable, predictable behaviour - at least where this can be reasonably asserted - and some regressions in 0.8.3 I felt breached this threshold of acceptable change.

Version 0.8.4 does still bring some changes to the table, but the scope - and type - of these changes are now much more manageable with respect to 0.8.2.

Mixin 0.8.4 therefore represents the "what 0.8.3 should have been" release, and offers the promised improvements of the previous release, with fewer gotchas. Check out the release notes for 0.8.3 if you haven't already, as they cover the major changes and bugfixes since 0.8.2.

Local Variables in Mixins

This brings us on to the bonus topic of today's release notes: Locals.

Support for interacting with local variables has a long and storied past and has, during its tenure, overcome such obstactles as differing output from ASM in production vs. development. Yielding different results across different versions and platforms for ostensibly the same code. Failing to expose locals which seem intuitively in-scope at the capture point, and exposing others which shouldn't be. Accidentally capturing locals added by Mixin itself. The list goes on. I can say that the 0.8 series of Mixin definitely represents a high point in the general functionality of local capture.

Those who have been on board the Mixin train for a long time have observed the growing pains of locals and treat it with the air of an unexploded bomb which might detonate at any moment, and they are justified in doing so. However I realise that I have been negligent in assuming that this feeling prevails and therefore failing to provide the appropriate caveats and warnings around the use of locals. This is especially true recently since the state of Locals is objectively quite good, and belies the dangers of change over time that still lurk beneath the surface.

So why, if it's in such a good place, does it need to change at all?

I have two main goals for Locals in the long run:

• Reveal locals that are intuitively in scope from the mixin author's perspective
• Provide stable results even across different platforms and versions, for code which appears to change. Eg. reduce sensitivity to binary-level changes
• Do all this with reasonable efficiency, eg. avoid going full-blown analytical decompiler, make an algorithm the works predictably even if it's not perfect

The reason for the first should be obvious, authors of mixins should have a reasonable expectation of being able to capture variables which appear to be in-scope based on reading the decompiled source. This reduces development effort and makes for a better experience.

The second goal is the reason that things are still changing: reducing binary sensitivity is a tricky thing to achieve given that it conflicts with the third goal somewhat, this is where tuning the algorithm comes into play.

The algorithm used by Locals does a relatively simple scan of target methods from top to bottom, weaving knowledge gained from the LVT (either explicit or computed), the stack map frames, and LOAD and STORE opcodes in the method itself together to create a not unreasonable view of the available local variables at any point in the method. However, the weight given to different information sources is where the tunables come in, as the entire algorithm is essentially a hueristic which tries to blend sometimes conflicting accounts of what is available and where.

If you're wondering why on earth different aspects of the method structure would disagree, and why there is no "universal truth" that can be easily reverse engineered from simply reading the bytecode, it's important to understand that no part of the runtime actually needs to know the information we're after, and the different mechanims in the bytecode (the instructions, the LVT and the stack map frames) have entirely different goals. The LVT is the closest thing we have to truth, and it's not even required at runtime since it's purely used for debugging. In fact in older versions of the game the LVT was stripped, and even to this day it is obfuscated.

Where am I going with this?

As I said above, Locals is in quite a good place right now, but tweaks and changes may still occur. This means that using Locals in a defensive manner is still - for the time being - important. The end goal is to achieve a level of repeatability between versions, platforms and binary changes that can provide a sense that captured locals will change when the code changes, in other words converging with the general contract of injectors as a whole.

Mixin 0.8.3 and 0.8.4 represent changes to the locals algorithm which make it less leaky¹, however regressions reported after the release of 0.8.3 hinted that the leakiness was a useful characteristic in some cases, and justifiably so. Mixin 0.8.4 doesn't turn the leakiness all the way back up to pre-0.8.3 levels but now uses a more intelligent algorithm to decide when to bring a variable back into scope which was previously axed, a kind of intelligent leak if you will.

Does this mean all injectors written with 0.8.2 will now work in 0.8.4? No, though fewer should break. Does it provide a good step towards more stable results in future versions of Mixin? I hope so.

¹ Leaky in this case is one of the trade-offs when processing the stack map frames in target code. A stack map frame in the code may indicate that some variables have gone out of scope. However the stack map frames may just indicate this because the method doesn't use them any more, the variable values themselves still exist in the method frame, but the stack map "chops" the variables because the code afterward doesn't use them any more. In our algorithm we can choose to "leak" those variables (not chop them) but this can lead to some unintuitive results where - for example - a variable declared inside a loop seems to be still available after the loop, and this becomes very binary-sensitive if the same variable slot is reused later on for a different variable.

Why Take This Approach?

In short, because stability of results allows Mixin to do more things with locals. I have, to date, been reluctant to expand local capture and interaction with locals (eg. via ModifyVariable). This is because the unstable nature of locals means that interacting with the frame as a whole, or only a single variable, at least provides some safeguards in that if the frame doesn't match what the code is expecting then it breaks quickly and predictably.

The driving force behind these changes is that a huge overhaul of injectors, "Injectors three point oh" if you will, is roadmapped for Mixin 0.10, this will include the ability to interact with local variables at a more granular level (eg. capturing and even modifying specific locals in injectors) and extending locals functionality to all injectors, not just limiting it to capturing the whole frame in @Inject. Achieving more predictable results when determining available locals will make these improvements much more feasible.

What's Next

The next version of Mixin will be 0.8.5, which is specifically aimed at eliminating some of the pain points with selecting lambda methods to target, and limitations of some existing injectors when multiple mixins want to target the same instruction. As always, watch this space.

Join the discussion ...

Mixin 0.8.3

Despite being only a point release, Mixin 0.8.3 packs a decent punch in terms of bundling up a lot of bug fixes for a respectable number of issues, as well as laying the groundwork for some upcoming features and freezing the contract of some internal APIs so they can be safely exposed to mixin authors. Among the fixes are some (yes more) small but significant improvements to the way local variables are computed, which makes them much more robust than ever before.

Also clarified is the general contract of the compatibilityLevel behaviour in Mixin, which has been a sticking point for adopting newer Java versions, see below for more details on that.

Features

Target Selector Improvements

With the introduction of Mixin 0.8, I abstracted much of the interface for the venerable MemberInfo into a new more generalised target selector in order to make the entire system of querying for targets more pluggable. Mixin 0.8.3 sees this almost to completion, finalising and freezing the contract of ITargetSelector, and introducing registration mechanics similar to those for InjectionPoints.

Mixin configs can now specify a dynamicSelectors collection, with the names of classes implementing ITargetSelectorDynamic. Selectors declared in this way should be decorated with @SelectorId similarly to the way that custom injection points are decorated with @AtCode.

Dynamic target selectors are specified in the selector string but have a simple recognisable format:

// Use of a built-in custom selector
target = "@SelectorId(custom,argument,string,in,any,format)"

In order to prevent accidental overlaps, mixin configs should now specify a namespace for their custom selectors and injection points, thus a user-provided dynamic selector is used like this:

// Use of a user-provided custom selector
target = "@Namespace:SelectorId(some arguments)"

Namespaces and selector IDs are case-insensitive.

New Built-In Dynamic Target Selectors

With the introduction of dynamic selectors, two new built-in selectors are now available:

• MemberMatcher (javadoc) introduces a new syntax for matching members via regular expressions. Since these regular expressions are not remapped, this is primarily of use to products mixing into non-obfuscated platforms which change often. However with some creativity they can be leveraged to great effect on obfuscated platforms too, providing a similar mechanic to aliases (eg. /(devName|obfName)/).
• DynamicSelectorDesc (javadoc) introduces @Desc selectors, more on these later once support in the toolchain matures.

User-Defined Classes in Mixin Packages

In 0.8.3 it is now possible to declare custom classes (Injection Points and Target Selectors) inside of mixin packages. This allows one-use custom injection points to be more neatly packaged with the mixin which consumes them for example.

Changes to Service Bootstrap

It was previously necessary for third-party services to camp in one of the mixin packages in order to instatiate the transformer chain. Since the Mixin binaries are digitally signed this was a problem for third-party services since they needed to strip the signer information from the mixin jar in order to inject their service.

In Mixin 0.8.3, service startup has been reworked so that services are now offered a one-use IMixinTransformerFactory via the new internals mechanism. (This approach chosen so that other internal objects can be passed easily in the future).

Enhancements to MemberInfo

It wouldn't be fair for all this new stuff to be happening without the trusty MemberInfo getting some light buffs. Since the new ITargetSelector supports both a minMatchCount and a maxMatchCount, new syntax for MemberInfo now extends the previous * (match all) syntax to support declaring min and max values using a similar format to regex quantifiers. Example quantifiers might be {3} (match exactly 3), {2,} (match at least 2), {1,6} (match 1 to 6 (inclusive)). The javadoc for MemberInfo has more examples.

About compatibilityLevel in Mixins

Mixin 0.8.3 fixes a bug in the handling of compatibility level which has caused confusion because nobody realised it was a bug. Whilst I have spoken about its role in the past, the language isn't particularly clear about what the level should be, other than it needing to be "set to the highest level required by your mixins" without clarifying what "highest level" means.

Much like minVersion, the compatibilityLevel declaration is meant as a safeguard against consuming a mixin which uses certain features of the JVM, that mixin must understand and process, thus just choosing the highest level available simply removes this safeguard. The reason the "must understand and process" qualification is important is that Java as a language introduces new features all the time, but often these don't require any special handling on Mixin's part. For example Java could introduce the elvis operator tomorrow, and it wouldn't require any additional bytecode to do so, it's simply syntactic sugar for a compiler feature.

New features such as private methods in interfaces, dynamic constants and nesting are bytecode-level features that do require support in Mixin, and therefore knowing the compatibility needs to be high enough to support those features is an important distinction, since if those features exist in the mixins being consumed, then Mixin has to process and merge them.

The source of the confusion has stemmed from two places, largely because a lot of this stuff was designed when Sponge was the only project using Mixin and these concepts were understood and never written down explicitly anywhere:

• Firstly, the CompatibilityLevel enum declares java levels above the highest supported by Mixin (previously JAVA_9) with no formal declaration anywhere that these values don't necessarily indicate supported versions.

• Secondly, the check in the code which validated the class version in use came with the implicit assumption that mixin authors would know which versions of Java were supported by mixin, and would set the source compatibility of their mixin SourceSet to the version they needed. This assumption existed for so long that it was largely forgotten, since JAVA_8 has been the de-facto highest version in use for such a long time. The class version check was therefore sufficient and didn't provide any more information when the check was violated, simply advocating raising the level.

In order to address these issues, and make compatibilityLevel meaningful again, the naïve check has now been replaced with a much more intelligent heuristic, designed to check in a useful way whether the language features required are actually supported by the current level, and encourage mixin authors to elevate to the lowest level which supports their required feature set. Since Mixin doesn't fully support features for any version beyond Java 11 at the moment, this has also been encoded so that a warning can be emitted when using compatibility levels higher than this.

As of Mixin 0.8.3, the current compatibility matrix...

Mixin 0.8.1

Hotfix for ModLauncher 7.0 support. Also fixes a critical error with resolving methods in inherited interfaces which was breaking many builds.

Mixin 0.8

Mixin 0.8 is something of a milestone so I'd like to take this opportunity to talk about both important changes in this version, and also draw attention to some more subtle changes addressing minor bugs and improving general quality of life for mixin authors.

Version 0.8 brings a lot of internal structural changes to Mixin but a not insignificant part of the update deals with myriad small issues which both make some Mixin features more reliable and consistent (from a mixin author point of view) and also make 0.8 feel more mature by fixing small corner-cases where functionality was either poorly documented or just plain missing. The result is that Mixin 0.8 presents a noticably more refined end-user experience which I hope will be felt by those who work with mixins on a regular basis, but also by newcomers who could often be caught out by some of Mixin's hidden gotchas.

Headline Features

Obviously headline features for 0.8 are support for ASM 6.2 and cpw's excellent new modding platform ModLauncher. Like all Mixin versions, a focus on backwards compatibility has always been paramount, and it's with some reluctance that I've had to introduce a fairly unpleasant departure from that policy in the 0.8: namely the removal of the shaded-and-relocated ASM library in favour of using stock ASM. To be clear, this only has implications for the following elements:

• Mixin config companion plugins which make use of the preApply and postApply callbacks to inspect or make changes to target classes.
• Custom injection points

In the former case, a shim is provided so that any mixins compiled for 0.7 or earlier and using a config companion plugin will continue to work if they do not make any use of the supplied ClassNode (eg. if they use the callback purely for logging or other similar purposes). For the latter case, an error will be reported.

This change was unfortunately necessary since, under the ModLauncher regime, transformers work directly with a ClassNode created by the transformation pipeline rather than with raw byte arrays. Since mixin works extensively with the ASM tree API itself, the only alternative would be to somehow proxy the incoming ClassNode - an unacceptable performance penalty for such a small backwards compatibility benefit.

The practical upshot of all this is that the vast majority of mods running under 1.12.x and using Mixin will continue to work if they upgrade to Mixin 0.8 in-place, or if a newer mod bundling 0.8 is present. Things will only break if a mod compiled for 0.7.x and using either of the features mentioned above is loaded under a Mixin 0.8 library bundled by another mod. As always, I have adhered to the "fail fast" contract, and Mixin will crash early with a descriptive error if this situation arises.

With ASM 6.2 comes the possibility of supporting Java 9 and partial support for Java 10. Though mixin doesn't specifically provide any functionality for those java versions yet, the compatibility levels are now available. But why - I hear you ask - ASM 6.2 when 7.0 and even 7.2 are available? The primary reason is to maintain parity with ModLauncher, since it now exists as the primary target platform for Mixin, compiling under that version makes the most sense for ensuring stability. A secondary reason is that incremental updates make the process of tracking down version-specific bugs slightly easier, since they can be identified and squashed on an ongoing basis.

In order to preserve backwards compatibility with older platforms, Mixin will still run on ASM 5.x and this is achieved by detecting the ASM version at startup and gating unsupported functionality based on the detected version. This mechanism also provides a means for supporting future versions, so expect to see ASM 7.x support arrive in the not-too-distant future.

Toolchain and Environment Changes

Mixin 0.8 adds support for ForgeGradle 3.0 used for Minecraft 1.13 and 1.14 modding, specifically adding support for the new TSRG mappings format. The old services are still available to maintain backwards compatibility with 1.12 and earlier. However note that for MixinGradle, the companion gradle plugin which makes using the mixin AP much easier, you need to update to MixinGradle 0.7 in order to support ForgeGradle 3.0 and the new mappings type.

Since CoreMods have changed significantly in Forge, Mixin now provides a new bootstrapping mechanism in the form of Mixin Connectors. Mixin Connectors replace the functionality which would have previously been the duty of your CoreMod class, with the exception of initialising mixin itself. Migrating CoreMod code to Mixin Connector for 1.14 requires 2 steps:

• Create a class which implements IMixinConnector and place your initialisation code sans the call to MixinBootstrap.init() in the connect() method.
• Specify the name of your IMixinConnector class as the value for the MixinConnector key in your manifest

Notable Bug Fixes

If you're a long-time user of Mixin, then the more interesting changes are likely to be those with a smaller footprint but which nevertheless address some irritations, minor bugs and omissions with Mixin. Notable changes are as follows:

• Static accessor methods now work properly
  Previously a static @Accessor method in an interface would function only if the class loading order was such that the target class was loaded first. Static accessors now work correctly regardless of the class loading order

• @Invoker can now be used to invoke private constructors
  It's my life mission to make Access Transformers redundant and a new feature in Mixin 0.8 is the ability to create an @Invoker method which acts as a factory, invoking an inaccessible constructor.

• Accessor mixins now work properly inside other mixins
  A small bug with a big impact. Accessor Mixins now work properly when used inside another mixin's code, previously attempting to do this would yield a slightly cryptic error about not finding the accessor mixin in the target's hierarchy.

• Mixin application continues if an optional mixin fails
  Previously any mixin failing during preprocessing or application would lead the mixin applicator shutting down, with optional mixins generating a warning and required mixins raising an exception. From 0.8, failures in optional mixins are collected and reported, but other mixins for the same target will still be applied. The behaviour for required mixins remains unchanged.

• More reliable local variable capture in @Inject
  Local variable capture via callback injector has always been somewhat brittle due to the fickle nature of the LVT and the knock-on effects this has had when passing obfuscated classes through ASM. Previously the local variable detection in Mixin relied heavily on ASM's EXPAND_FRAMES and a combination of heuristics and analysis to walk through a target method to determine available locals at the injection point.

  Since EXPAND_FRAMES is not applied by ModLauncher, it was necessary to improve the local variable detection to handle compressed frames. This has had a side-effect of producing more reliable local variable tables and also revealed some minor bugs in the analyser code which, although unlikely to have had any detectable effect in the real world, nonetheless represent an improvement of the system as a whole.

Improvements and New Features

• Injectors with no targets are now allowed to fail
  The introduction of the expect and require expressions on injectors made it possible for Mixin to determine if failure to inject was an error condition or not. Since injectors might fail because code is changed by another transformer this allowed injectors some degree of lattitude to fail quietly if they weren't important. However an injector which matched no targets was always treated as an error condition. This was problematic when, for example, an extra method might be added by another transformer, since the failing injector would lead to a crash when the mod was not present. From 0.8 onwards, the expect and require are now honoured for zero-target injectors.

• Java SuppressWarnings is now supported by the Annotation Processor
  It is now possible to use the regular java @SuppressWarnings annotation to silence AP warnings which you don't care about. I will add a wiki page with the available options but for now you can check out the source for SuppressedBy which lists the available options. As you'd expect, the warning suppression is available at the element, class or package levels as required.

• Improvements to @Coerce for injector arguments
  Support for the @Coerce annotation has always been a bit inconsistent. Mixin 0.8 overhauls support for @Coerce to add three main improvements:

  • Can now be used to coerce not only to superclasses, but also interfaces including mixed-in interfaces such as acccessors
  • Now works on field, array and constructor redirectors as well as the previously-supported invoke redirectors
  • Can now be used on all redirector and modifyconstant arguments, including captured target arguments and the method return types (to coerce return type, annotate the handler method itself)

• @ModifyVariable injectors can now capture target arguments
  Like its related injectors @Redirect, @ModifyConstant and @Invoke, the @ModifyVariable injector is now able to capture arguments from the injector target. @Coerce can be ...