-
Notifications
You must be signed in to change notification settings - Fork 327
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
+ Monix Task and BIO support and Cats IO improvements #879
base: master
Are you sure you want to change the base?
Conversation
// - root | ||
// - 1 (value = 1) | ||
// - 2 (value = 2) | ||
// - 3 (value = 3) | ||
val rootSpan = for { | ||
root <- F.delay(Kamon.spanBuilder("root").start()) | ||
_ <- (1L to 3L) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why would the spans be nested in a tree?
I thinks this test just creates unrelated spans.
Maybe try creating the other spans as children of the first one, or using Kamon.runWithSpan
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the test all operations are sequenced as follows:
- a root span is created and started without being finished
- 3 additional spans that are started and finished, executed one after another (
.map(...).sequence
) - the root span is finished
The named
function that is applied to the F.delay(idx)
is actually a call to the operationName
in the Tracing
object that does the following:
- creates and starts a new span with the provided name and tags that sets as parent span the result of
Kamon.currentSpan()
- executes the
F[_]
effect, which in this case isF.delay(idx)
- finishes the span
Assuming the operation sequence defined above, the three effects F.delay(idx)
should be wrapped in spans that use the root
span as a parent, because that's the current span (the started and not finished).
I don't understand what part of this test is "iffy". :) ... but please let me know if I'm missing something.
The assertion at the end is obviously wrong because it also considers the root span for the equality check, but that's not the problem here as I've manually printed the span ids when they are created.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at the kamon-scala-future module, it looks like it doesn't just create a span, but explicitly stores it in the context storage and removes it afterwards. I assumed that simply starting and finishing a span is sufficient. I'll try to do the same.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I've put down some println's and Kamon.currentSpan()
give Span.Empty
both in the span creation code and in the test code.
Adding something like Kamon.runWithSpan(root)
to the test should correctly nest it in the test code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, you figured it out! 🎉
Feel free to use me as a rubber duck, while I try and wrap my head around Monix code 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That was the problem, I needed to store the span in the context storage...
But now I've hit another issue where every now and again the test fails due to the fact that the fibers are interleaved. I think this is a more fundamental issue with my PR, I may have to use TaskLocal and IOLocal to get named spans working correctly. Still investigating.
Hey man, sorry for the delay, I'll go on a deeper dive here soon! P.S. you can configure your editor to add newlines at the end of files, so github stops putting red marks everywhere! |
Hi so this was an issue I hit a brick wall with when developing https://github.com/jatcwang/kamon-cats-effect After staring at it for a while with @ivantopo I think we've found the problem. The gist of the problem is that we're calling scope.close() in a different thread than the original thread where the Scope was created. See this line Kamon/core/kamon-core/src/main/scala/kamon/context/Storage.scala Lines 78 to 86 in f3fbf48
Note how in def close() , we have a reference to the array thread local. This is fine if everything happens in one thread, but if close() is called from a different thread, we have disastrous consequences such as context contamination.
The solution (for this problem) is to resolve the thread local again in
If there are benchmarks we should make sure this doesn't have significant performance impact, since the array trick was originally done for performance reasons. (We should also try removing the array trick altogether if we decide to "fix" this, since at this point it's really not doing anything) |
Thanks for this @jatcwang, I'll have a look. |
Hi @jatcwang!
Please take a look: https://github.com/kamon-io/Kamon/blob/master/core/kamon-core-bench/src/main/scala/kamon/bench/ThreadLocalStorageBenchmark.scala if you want measure the performance impact. |
Ok this one is interesting, the "old" (supposedly) non-optimized version is faster when I run the benchmark. Some unscientific benchmarks I ran with my laptop (Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz, 16GB) On GraalVM:
and on OpenJDK:
Note that "currentThreadLocal" is the naive, non-optimized version. So yeah the optimized version is consistently faster but for my use case (Mainly future/IO instead of akka) I'd say the impact is negligible. |
Great work @jatcwang! |
@SimunKaracic PR up in #961 |
* @return the same effect wrapped within a named span | ||
*/ | ||
def operationName[F[_] : Sync, A](name: String, tags: Map[String, Any] = Map.empty, takeSamplingDecision: Boolean = true)(fa: F[A]): F[A] = { | ||
val F = implicitly[Sync[F]] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we take (implicit F: Sync[F])
like buildSpan?
|
||
implicit def timer: Timer[F] | ||
|
||
private val customExecutionContext = ExecutionContext.fromExecutorService(Executors.newCachedThreadPool()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd go with a fixed ThreadPool to ensure the test are running with multiple threads
val contextTag = F.toIO(contextTagF).unsafeRunSync() | ||
contextTag shouldEqual "value" | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should test cancellation and errors?
Would be good to run multiple IOs in parallel (with shift inserted in between) to ensure everything works.
operationName
function that allows creating named child spansmonix-eval
Task
andmonix-bio
IO
effect typesThe span nesting does not work as evidenced by the test
AbstractCatsEffectInstrumentationSpec.nest spans correctly
that is failing. Any pointers are much appreciated.The purpose of this PR is to add support for Monix as a separate module, while enhancing the Cats Effect support.