Error reporting causes performance issues

[isaacseymour]

We've noticed a couple of situations where the error rate is just high enough (e.g. someone deploys a broken version of a page), that the load caused by the Bugsnag middleware is high enough to cause performance issues on unaffected pages.

Having a dig through the code, it seems like there's a lot of CPU time spent preparing the notification in the main request-handling thread, before the actual HTTP work is offloaded to a background thread.

Would you be open to a PR moving that work to the background thread?

[snmaynard]

I think that the reason that it isn't done there is that we need to block for a certain amount of time to ensure we gather the requisite context from the thread locals etc before moving on. I think that we should raise a ticket internally to look at refactoring some of the library to improve performance.

Are there any areas you see as being especially bad for you? We'd have to look at what we can remove or how we can serialize information before offloading work to the background thread.

[Cawllec]

Hi @isaacseymour, I'm closing this issue for now, but it would be great to hear your feedback if possible. If you do have some I'll reopen this and we'll look into any performance issues.

[snmaynard]

We have also raised an internal ticket to audit the performance and see if there are any quick wins in our test cases.

[gigr]

Has there been any movement on this? We have also been bitten by this issue. It cropped up recently in a case where some code inadvertently sends events to Bugsnag in a loop and can take 10–15 seconds to send a few dozen errors. We see heavy CPU usage during these times. We've started to mitigate this by sending events to Bugsnag in background jobs, but it's not ideal.

[xljones]

Hi @gigr, we haven't identified any performance improvements in this area yet.

Do you know what is causing the slowdown in your case? Have you profiled which paths are the worst offenders for this?

Additionally, if it is possible for you to share a reproduction case, this would help us target performance upgrades.

[imjoehaines]

We've shipped a number of performance improvements in recent releases; let us know if these don't seem to have helped