Bugsnag sends all reports in store regardless of releaseStage settings

[dwmatteson]

Description

If I have reports in the Bugsnag store (~/Library/Caches/KSCrashReports/) that were generated during Unit Testing and not sent due to releaseStage and notifyReleaseStage not matching up the next time I run the app where they do match up, it will send all the reports that are left in the store. Even if they have a releaseStage set on them that does NOT match the current notifyReleaseStage setting.

Issue

Reports are generated during unit testing with a releaseStage setting of "testing" and notifyReleaseStage setting of [ "development", "production" ]. The next time I run my app with the releaseStage set to "development" (or "production") it sends ALL the reports that are sitting in the store (all of which have a releaseStage of "testing", seen in the report files.) Note the reports are properly NOT sent when running unit tests with releaseStage set to "testing", but the files are created and written to the store. The backlog of reports are all sent the next time you run the app with a releaseStage that is valid for sending.

Environment

Library versions:

• Bugsnag version bugsnag-cocoa-5.22.4
• CocoaPods version (if any) (pod -v): N/A
• Carthage version (if any): N/A
• iOS/tvOS/macOS version(s) macOS 10.14.6
• debug mode or production?: Debug
• How are you initializing Bugsnag? Can you share a snippet of your
  AppDelegate where Bugsnag is configured?

import Bugsnag

// Normal initialization
        let bugSnagConfig = BugsnagConfiguration()
        bugSnagConfig.apiKey = Constants.BugSnagKey
        bugSnagConfig.notifyReleaseStages = [ "development", "production" ]
        // tried to do this as a way to circumvent the described behavior, but it doesn't work.
        bugSnagConfig.add(beforeSend: { (rawData, report) -> Bool in
            // note this doesn't work because releaseStage is always nil on the report object....
            if(report.releaseStage == "testing") {
                return false
            }
            
            return true
        })

        Bugsnag.start(with: bugSnagConfig)

// In unit tests' setUp() method
Bugsnag.configuration()?.releaseStage = "testing"

Error messages:

None

[mattdyoung]

@dwmatteson
That does sound like it may be a bug. We'll investigate and confirm whether this needs a fix or whether there's a different way to configure this that works.

[mattdyoung]

Fixed in v5.22.6