[QA] Flaky API tests while counting the number of received emails

[anon-pradip]

Describe the bug

API tests are failing while asserting the number of emails received.

The problem could be with inbucket the latest image is 2 years old. or simply the test code could not perform the cleanup.

Builds:

• https://drone.owncloud.com/owncloud/ocis/43008/20/7
• https://drone.owncloud.com/owncloud/ocis/43036/20/7

Scenario:

Scenario: check share expired in-app and mail notification for Personal space resource      # /drone/src/tests/acceptance/features/apiSettings/notificationSetting.feature:612
    Given user "Alice" has uploaded file with content "hello world" to "testfile.txt"         # FeatureContext::userHasUploadedAFileWithContentTo()
    And user "Alice" has sent the following resource share invitation:                        # SharingNgContext::userHasSentTheFollowingResourceShareInvitation()
      | resource           | testfile.txt         |
      | space              | Personal             |
      | sharee             | Brian                |
      | shareType          | user                 |
      | permissionsRole    | Viewer               |
      | expirationDateTime | 2025-07-15T14:00:00Z |
    When user "Alice" expires the last created share:                                         # SharingNgContext::userExpiresTheLastCreatedShare()
      | space    | Personal     |
      | resource | testfile.txt |
    Then user "Brian" should have "2" emails                                                  # NotificationContext::userShouldHaveEmail()
      Failed asserting that actual size 3 matches expected size 2.

Request Log:

### Then user "Brian" should have "2" emails

		_______________________________________________________________________


		==> REQUEST

		GET /api/v1/mailbox/brian

		X-Request-ID: apiSettings/notificationSetting.feature:612-624


		<== RESPONSE

		200 OK

		Content-Length: 911

		<== RES BODY

		[{"mailbox":"brian","id":"20250130T033007-0002","from":"Alice Hansen via ownCloud \[email protected]\u003e","to":["\[email protected]\u003e"],"subject":"Alice Hansen invited you to join newSpace","date":"2025-01-30T03:30:07.985899043Z","posix-millis":1738207807985,"size":2768,"seen":false},{"mailbox":"brian","id":"20250130T033009-0003","from":"Alice Hansen via ownCloud \[email protected]\u003e","to":["\[email protected]\u003e"],"subject":"Alice Hansen invited you to join new-space","date":"2025-01-30T03:30:09.724633894Z","posix-millis":1738207809724,"size":2771,"seen":false},{"mailbox":"brian","id":"20250130T033011-0004","from":"Alice Hansen via ownCloud \[email protected]\u003e","to":["\[email protected]\u003e"],"subject":"Alice Hansen shared 'testfile.txt' with you","date":"2025-01-30T03:30:11.398038304Z","posix-millis":1738207811398,"size":2599,"seen":false}]

Scenario:

Scenario: disable mail and in-app notification for Added as space member event                  # /drone/src/tests/acceptance/features/apiSettings/notificationSetting.feature:612
    Given using spaces DAV path                                                                   # FeatureContext::usingOldOrNewOrSpacesDavPath()
    And the administrator has assigned the role "Space Admin" to user "Alice" using the Graph API # GraphContext::theAdministratorHasGivenTheRoleUsingTheGraphApi()
    And user "Alice" has created a space "newSpace" with the default quota using the Graph API    # SpacesContext::theUserHasCreatedASpaceByDefaultUsingTheGraphApi()
    When user "Brian" disables notification for the following events using the settings API:      # SettingsContext::userDisablesNotificationForFollowingEventUsingSettingsApi()
      | Added as space member | mail,in-app |
    Then the HTTP status code should be "201"                                                     # FeatureContext::thenTheHTTPStatusCodeShouldBe()
    And the JSON data of the response should match                                                # FeatureContext::theJsonDataOfTheResponseShouldMatch()
      """
      {
        "type": "object",
        "required": ["value"],
        "properties": {
          "value": {
            "type": "object",
            "required": ["identifier","value"],
            "properties": {
              "identifier":{
                "type": "object",
                "required": ["extension","bundle","setting"],
                "properties": {
                  "extension":{
                    "const": "ocis-accounts"
                  },
                  "bundle":{
                    "const": "profile"
                  },
                  "setting":{
                    "const": "event-space-shared-options"
                  }
                }
              },
              "value":{
                "type": "object",
                "required": [
                  "id",
                  "bundleId",
                  "settingId",
                  "accountUuid",
                  "resource",
                  "collectionValue"
                ],
                "properties":{
                  "id":{
                    "pattern":"%user_id_pattern%"
                  },
                  "bundleId":{
                    "pattern":"%user_id_pattern%"
                  },
                  "settingId":{
                    "pattern":"%user_id_pattern%"
                  },
                  "accountUuid":{
                    "pattern":"%user_id_pattern%"
                  },
                  "resource":{
                    "type": "object",
                    "required":["type"],
                    "properties": {
                      "type":{
                        "const": "TYPE_USER"
                      }
                    }
                  },
                  "collectionValue":{
                    "type": "object",
                    "required":["values"],
                    "properties": {
                      "values":{
                        "type": "array",
                        "maxItems": 2,
                        "minItems": 2,
                        "uniqueItems": true,
                        "items": {
                          "oneOf": [
                            {
                              "type": "object",
                              "required": [
                                "key",
                                "boolValue"
                              ],
                              "properties": {
                                "key":{
                                  "const": "mail"
                                },
                                "boolValue":{
                                  "const": false
                                }
                              }
                            },
                            {
                              "type": "object",
                              "required": [
                                "key",
                                "boolValue"
                              ],
                              "properties": {
                                "key":{
                                  "const": "in-app"
                                },
                                "boolValue":{
                                  "const": false
                                }
                              }
                            }
                          ]
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
      """
    And user "Alice" has sent the following space share invitation:                               # SharingNgContext::userHasSentTheFollowingShareShareInvitation()
      | space           | newSpace     |
      | sharee          | Brian        |
      | shareType       | user         |
      | permissionsRole | Space Viewer |
    And user "Brian" should have "0" emails                                                       # NotificationContext::userShouldHaveEmail()
      Failed asserting that actual size 2 matches expected size 0.

Request log:

### And user "Brian" should have "0" emails

		_______________________________________________________________________


		==> REQUEST

		GET /api/v1/mailbox/brian

		X-Request-ID: apiSettings/notificationSetting.feature:612-734


		<== RESPONSE

		200 OK

		Content-Length: 606

		<== RES BODY

		[{"mailbox":"brian","id":"20250128T064915-0002","from":"Alice Hansen via ownCloud \[email protected]\u003e","to":["\[email protected]\u003e"],"subject":"Alice Hansen invited you to join newSpace","date":"2025-01-28T06:49:15.297190034Z","posix-millis":1738046955297,"size":2768,"seen":false},{"mailbox":"brian","id":"20250128T064917-0003","from":"Alice Hansen via ownCloud \[email protected]\u003e","to":["\[email protected]\u003e"],"subject":"Alice Hansen invited you to join new-space","date":"2025-01-28T06:49:17.32811097Z","posix-millis":1738046957328,"size":2771,"seen":false}]

Setup

CI

[anon-pradip]

More failed builds:

1. https://drone.owncloud.com/owncloud/ocis/43047/20/7

• Scenario:

Scenario: disable mail and in-app notification for Added as space member event # /drone/src/tests/acceptance/features/apiSettings/notificationSetting.feature:612
  Given using spaces DAV path # FeatureContext::usingOldOrNewOrSpacesDavPath()
  And the administrator has assigned the role "Space Admin" to user "Alice" using the Graph API # GraphContext::theAdministratorHasGivenTheRoleUsingTheGraphApi()
  And user "Alice" has created a space "newSpace" with the default quota using the Graph API # SpacesContext::theUserHasCreatedASpaceByDefaultUsingTheGraphApi()
  When user "Brian" disables notification for the following events using the settings API: # SettingsContext::userDisablesNotificationForFollowingEventUsingSettingsApi()
    | Added as space member | mail,in-app |
  Then the HTTP status code should be "201" # FeatureContext::thenTheHTTPStatusCodeShouldBe()
  And the JSON data of the response should match # FeatureContext::theJsonDataOfTheResponseShouldMatch()
    """  
    {  
      "type": "object",  
      "required": ["value"],  
      "properties": {  
        "value": {  
          "type": "object",  
          "required": ["identifier","value"],  
          "properties": {  
            "identifier": {  
              "type": "object",  
              "required": ["extension","bundle","setting"],  
              "properties": {  
                "extension": { "const": "ocis-accounts" },  
                "bundle": { "const": "profile" },  
                "setting": { "const": "event-space-shared-options" }  
              }  
            },  
            "value": {  
              "type": "object",  
              "required": ["id", "bundleId", "settingId", "accountUuid", "resource", "collectionValue"],  
              "properties": {  
                "id": { "pattern": "%user_id_pattern%" },  
                "bundleId": { "pattern": "%user_id_pattern%" },  
                "settingId": { "pattern": "%user_id_pattern%" },  
                "accountUuid": { "pattern": "%user_id_pattern%" },  
                "resource": {  
                  "type": "object",  
                  "required": ["type"],  
                  "properties": { "type": { "const": "TYPE_USER" } }  
                },  
                "collectionValue": {  
                  "type": "object",  
                  "required": ["values"],  
                  "properties": {  
                    "values": {  
                      "type": "array",  
                      "maxItems": 2,  
                      "minItems": 2,  
                      "uniqueItems": true,  
                      "items": {  
                        "oneOf": [  
                          {  
                            "type": "object",  
                            "required": ["key", "boolValue"],  
                            "properties": {  
                              "key": { "const": "mail" },  
                              "boolValue": { "const": false }  
                            }  
                          },  
                          {  
                            "type": "object",  
                            "required": ["key", "boolValue"],  
                            "properties": {  
                              "key": { "const": "in-app" },  
                              "boolValue": { "const": false }  
                            }  
                          }  
                        ]  
                      }  
                    }  
                  }  
                }  
              }  
            }  
          }  
        }  
      }  
    }  
    """
  And user "Alice" has sent the following space share invitation: # SharingNgContext::userHasSentTheFollowingShareShareInvitation()
    | space           | newSpace     |
    | sharee          | Brian        |
    | shareType       | user         |
    | permissionsRole | Space Viewer |
  And user "Brian" should have "0" emails # NotificationContext::userShouldHaveEmail()
    │ array(2) {
    │   [0]=>
    │   object(stdClass)#3239 (9) {
    │     ["mailbox"]=> string(5) "brian"
    │     ["id"]=> string(20) "20250130T090314-0002"
    │     ["from"]=> string(47) "Alice Hansen via ownCloud <[email protected]>"
    │     ["to"]=> array(1) { [0]=> string(19) "<[email protected]>" }
    │     ["subject"]=> string(41) "Alice Hansen invited you to join newSpace"
    │     ["date"]=> string(30) "2025-01-30T09:03:14.740316014Z"
    │     ["posix-millis"]=> int(1738227794740)
    │     ["size"]=> int(2768)
    │     ["seen"]=> bool(false)
    │   }
    │   [1]=>
    │   object(stdClass)#3363 (9) {
    │     ["mailbox"]=> string(5) "brian"
    │     ["id"]=> string(20) "20250130T090316-0003"
    │     ["from"]=> string(47) "Alice Hansen via ownCloud <[email protected]>"
    │     ["to"]=> array(1) { [0]=> string(19) "<[email protected]>" }
    │     ["subject"]=> string(42) "Alice Hansen invited you to join new-space"
    │     ["date"]=> string(30) "2025-01-30T09:03:16.963163659Z"
    │     ["posix-millis"]=> int(1738227796963)
    │     ["size"]=> int(2771)
    │     ["seen"]=> bool(false)
    │   }
    │ }
    │ Failed asserting that actual size 2 matches expected size 0.