Support custom threads

[eupp]

No description provided.

[eupp]

The only failing CI configuration is "Integration Test with kotlinx.coroutines".
I believe it is because of #412, and once we merge #413 the problem should be fixed.
I've create a separate test branch where I merged the #413 into my branch, and it seems that there all kotlinx.coroutines tests are passed.

[eupp]

@ndkoval while working on this I discovered a few problems, which are probably should be addressed in separate PRs.

One of the problems is related to the local objects tracking --- the current implementation does not work correctly with custom threads (see example below and the comment in DataStructuresTests::incorrectHashMap test).
So the problem can be illustrated by the following example:

class Box(var x: Int)

fun test(): Int {
    val box = Box() // <- this object is incorrectly classified as a local object
    thread {
        box.x = 42 // the local object tracker does not detect here that the `box` object,
                   // stored in the local variable, escapes into another thread;
                   // thus it will not insert a switch point before accesses to this object fields
    }
    return box.x
}

I would propose that we can address this problem separately in another PR after we merge this one.
The reasons is that adding support of this case would require significant refactoring of local objects tracking algorithm,
and this PR is already big.

Alternatively, we can first perform the necessary refactoring of the local objects tracking algorithm in a separate PR,
and after this rebase and merge custom threads PR.

[eupp]

Another small bug fix on which this PR relies on: #426

[ndkoval]

Can we use a weak hash map instead?

[eupp]

I am afraid no, because we need concurrent identity hash map with weak keys, and there is no such data structure out-of-the-box in the Java standard library.

Please also have a look at the comment before threadDescriptorsMap field declaration --- it explains the reason and the proposed solution.

[ndkoval]

Would using Collections.synchronizedMap(...) solve the issue?

[eupp]

Collections.synchronizedMap(...) would not fully solve the problem. It would make the collection thread-safe, but WeakIdentityHashMap is still not a thing in Java stdlib (remember we need thread-safe weak identity hash-map).

Also, there is a method to lookup for a descriptor of the given thread:

public static ThreadDescriptor getThreadDescriptor(Thread thread)

which uses threadDescriptorsMap.get.

Forcing the descriptor lookup function to acquire a lock on each lookup (instead of using concurrent hash map), may in theory have a performance penalty.

[ndkoval]

Have you checked how custom threads work with the plugin?

[eupp]

No it does not currently work.

I found one problem in the plugin code that I've managed to fix by myself, but it still does not work.
There are some other bugs, and I need help from someone on the plugin side to debug the remaining problems.

[ndkoval]

Do we need to have to Utils.kt files?

[eupp]

The idea is to eventually leave only one file, inside the util subpackage, and move the stuff from the top-level package Utils.kt file either into util subpackage or to other more suitable places.

However, I didn't want to clutter this PR with unnecessary changes, thus I haven't modified the top-level Utils.kt file.

[ndkoval]

Why UNKNOWN? The documentation here and of the UNKNOWN_CODE_LOCATION field says it is the coroutine suspension code location.

[eupp]

It was COROUTINE_SUSPENSION_CODE_LOCATION previously, I renamed it into UNKNOWN_CODE_LOCATION because I re-used it for in a different context, but with a similar purpose (that is to represent an unknown code location).

I fixed the comments in the code so they now mention UNKNOWN_CODE_LOCATION.

[ndkoval]

Then, I need help understanding the logic. Why does codeLocation == UNKNOWN_CODE_LOCATION indicate coroutine suspension?

[eupp]

It doesn't. The code now looks like this:

        // Ignore unknown code locations.
        if (codeLocation == UNKNOWN_CODE_LOCATION) return Decision.Idle

[eupp]

From what I can see, there is no significant changes in the time of CI builds between this branch and develop:

• dynamic-threads branch
• develop branch

The reason is that in this PR I strive to preserve the old behavior whenever possible, by adding special treatment of TestThread class, see an example here:

lincheck/bootstrap/src/sun/nio/ch/lincheck/Injections.java

Line 91 in 3feb8a9

if (thread instanceof TestThread) {

Thus, all the code interacting with TestThread-s (i.e. threads created by Lincheck) should work almost the same as before, and there should be no new performance penalties for it.