Price Tiers are not synced

[dfranek]

We are syncing products between two environments, the source product has price tiers. These price tiers are not synced to the target environment.

Prices of source product:

"prices": [
    {
    "value": {
        "type": "centPrecision",
        "currencyCode": "EUR",
        "centAmount": 95000,
        "fractionDigits": 2
    },
    "id": "d67d52a8-9bfc-4655-8739-372d816d16cd",
    "country": "AT",
    "customerGroup": {
        "typeId": "customer-group",
        "id": "bbb93cf7-0379-47ba-8a9b-4629244fe151"
    },
    "tiers": [
        {
        "minimumQuantity": 3,
        "value": {
            "type": "centPrecision",
            "currencyCode": "EUR",
            "centAmount": 93000,
            "fractionDigits": 2
        }
        },
        {
        "minimumQuantity": 5,
        "value": {
            "type": "centPrecision",
            "currencyCode": "EUR",
            "centAmount": 92000,
            "fractionDigits": 2
        }
        },
        {
        "minimumQuantity": 10,
        "value": {
            "type": "centPrecision",
            "currencyCode": "EUR",
            "centAmount": 86100,
            "fractionDigits": 2
        }
        }
    ]
    }
],

This results in this price object in the target:

"prices": [
    {
    "value": {
        "type": "centPrecision",
        "currencyCode": "EUR",
        "centAmount": 95000,
        "fractionDigits": 2
    },
    "id": "1b16fe45-5bdc-4b6d-a9c7-5b55c8e13198",
    "country": "AT",
    "customerGroup": {
        "typeId": "customer-group",
        "id": "768252cf-fe31-450e-bd42-f65de6415e46"
    }
    }
],

In the target, the tiers are missing.

[ahmetoz]

@dfranek Thanks for opening the issue, we will try to take a look at it asap, btw which version are you using? Did you get any logs for those in your application?

[dfranek]

Hi, I was using version 3.12.0.

This is what I got from the call:

➜ docker run \
-e SOURCE_PROJECT_KEY=xxx \
-e SOURCE_CLIENT_ID=xxx \
-e SOURCE_CLIENT_SECRET=xxx \
-e SOURCE_AUTH_URL=https://auth.europe-west1.gcp.commercetools.com \
-e SOURCE_API_URL=https://api.europe-west1.gcp.commercetools.com \
-e TARGET_PROJECT_KEY=yyy \
-e TARGET_CLIENT_ID=xxx \
-e TARGET_CLIENT_SECRET=xxx \
-e TARGET_AUTH_URL=https://auth.europe-west1.gcp.commercetools.com \
-e TARGET_API_URL=https://api.europe-west1.gcp.commercetools.com \
commercetools/commercetools-project-sync:3.12.0 -s all -f
{"timestamp":"2021-04-15T11:40:55.665Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting CustomObjectSync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:40:55.705Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting TaxCategorySync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:40:55.711Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting TypeSync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:40:55.719Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting StateSync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:40:55.726Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting ProductTypeSync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:41:06.615Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":0,\"failed\":0,\"processed\":2,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":117,\"latestBatchHumanReadableProcessingTime\":\"0d, 0h, 0m, 0s, 117ms\",\"reportMessage\":\"Summary: 2 custom objects were processed in total (0 created, 0 updated and 0 failed to sync).\"}"}
{"timestamp":"2021-04-15T11:41:06.696Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":0,\"failed\":0,\"processed\":2,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":108,\"latestBatchHumanReadableProcessingTime\":\"0d, 0h, 0m, 0s, 108ms\",\"reportMessage\":\"Summary: 2 tax categories were processed in total (0 created, 0 updated and 0 failed to sync).\"}"}
{"timestamp":"2021-04-15T11:41:06.746Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":0,\"failed\":0,\"processed\":2,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":126,\"latestBatchHumanReadableProcessingTime\":\"0d, 0h, 0m, 0s, 126ms\",\"productTypeKeysWithMissingParents\":{},\"reportMessage\":\"Summary: 2 product types were processed in total (0 created, 0 updated, 0 failed to sync and 0 product types with at least one NestedType or a Set of NestedType attribute definition(s) referencing a missing product type).\",\"numberOfProductTypesWithMissingNestedProductTypes\":0}"}
{"timestamp":"2021-04-15T11:41:06.778Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":0,\"failed\":0,\"processed\":4,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":169,\"latestBatchHumanReadableProcessingTime\":\"0d, 0h, 0m, 0s, 169ms\",\"reportMessage\":\"Summary: 4 types were processed in total (0 created, 0 updated and 0 failed to sync).\"}"}
{"timestamp":"2021-04-15T11:41:06.868Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":0,\"failed\":0,\"processed\":1,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":297,\"latestBatchHumanReadableProcessingTime\":\"0d, 0h, 0m, 0s, 297ms\",\"reportMessage\":\"Summary: 1 state(s) were processed in total (0 created, 0 updated, 0 failed to sync and 0 state(s) with missing transition(s)).\",\"numberOfStatesWithMissingParents\":0}"}
{"timestamp":"2021-04-15T11:41:06.873Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting CartDiscountSync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:41:06.880Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting CustomerSync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:41:06.888Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting CategorySync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:41:06.908Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting InventorySync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:41:06.975Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":0,\"failed\":0,\"processed\":0,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":0,\"latestBatchHumanReadableProcessingTime\":\"\",\"reportMessage\":\"Summary: 0 inventory entries were processed in total (0 created, 0 updated and 0 failed to sync).\"}"}
{"timestamp":"2021-04-15T11:41:07.009Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":0,\"failed\":0,\"processed\":0,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":0,\"latestBatchHumanReadableProcessingTime\":\"\",\"numberOfCategoriesWithMissingParents\":0,\"reportMessage\":\"Summary: 0 categories were processed in total (0 created, 0 updated, 0 failed to sync and 0 categories with a missing parent).\"}"}
{"timestamp":"2021-04-15T11:41:07.010Z","severity":"ERROR","loggerName":"com.commercetools.project.sync.customer.CustomerSyncer","message":"Error when trying to sync customer. Existing key: <<not present>>. Update actions: []","stackHash":"933a364b","stackTrace":"c.c.s.c.e.SyncException: CustomerDraft with email: johndoe doesn't have a key. Please make sure all customer drafts have keys.\n\tat c.c.s.c.h.BaseBatchValidator.handleError(BaseBatchValidator.java:86)\n\tat c.c.s.c.h.CustomerBatchValidator.isValidCustomerDraft(CustomerBatchValidator.java:107)\n\tat j.u.s.ReferencePipeline$2$1.accept(Unknown Source)\n\tat j.u.ArrayList$SubList$2.forEachRemaining(Unknown Source)\n\tat j.u.s.AbstractPipeline.copyInto(Unknown Source)\n\tat j.u.s.AbstractPipeline.wrapAndCopyInto(Unknown Source)\n\tat j.u.s.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)\n\tat j.u.s.AbstractPipeline.evaluate(Unknown Source)\n\tat j.u.s.ReferencePipeline.collect(Unknown Source)\n\tat c.c.s.c.h.CustomerBatchValidator.validateAndCollectReferencedKeys(CustomerBatchValidator.java:97)\n\tat c.c.s.c.CustomerSync.processBatch(CustomerSync.java:117)\n\tat c.c.s.c.BaseSync.lambda$syncBatches$1(BaseSync.java:93)\n\tat j.u.c.CompletableFuture.uniComposeStage(Unknown Source)\n\tat j.u.c.CompletableFuture.thenCompose(Unknown Source)\n\tat j.u.c.CompletableFuture.thenCompose(Unknown Source)\n\tat c.c.s.c.BaseSync.syncBatches(BaseSync.java:93)\n\tat c.c.s.c.CustomerSync.process(CustomerSync.java:110)\n\tat c.c.s.c.BaseSync.sync(BaseSync.java:48)\n\tat j.u.c.CompletableFuture.uniComposeStage(Unknown Source)\n\tat j.u.c.CompletableFuture.thenCompose(Unknown Source)\n\tat j.u.c.CompletableFuture.thenCompose(Unknown Source)\n\tat c.c.p.sync.Syncer.syncPage(Syncer.java:239)\n\tat c.c.s.c.u.QueryAll.mapOrConsume(QueryAll.java:145)\n\tat c.c.s.c.u.QueryAll.processPageAndGetNext(QueryAll.java:128)\n\tat c.c.s.c.u.QueryAll.lambda$queryNextPages$2(QueryAll.java:107)\n\tat j.u.c.CompletableFuture$UniCompose.tryFire(Unknown Source)\n\tat j.u.c.CompletableFuture.postComplete(Unknown Source)\n\tat j.u.c.CompletableFuture.complete(Unknown Source)\n\tat i.s.s.u.CompletableFutureUtils.lambda$transferResult$0(CompletableFutureUtils.java:84)\n\tat j.u.c.CompletableFuture.uniWhenComplete(Unknown Source)\n\t... 7 frames truncated\n"}
Apr 15, 2021 11:41:07 AM org.javamoney.moneta.DefaultMonetaryContextFactory createMonetaryContextNonNullConfig
INFO: Using custom MathContext: precision=256, roundingMode=HALF_EVEN
{"timestamp":"2021-04-15T11:41:07.155Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":0,\"failed\":1,\"processed\":4,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":146,\"latestBatchHumanReadableProcessingTime\":\"0d, 0h, 0m, 0s, 146ms\",\"reportMessage\":\"Summary: 4 customers were processed in total (0 created, 0 updated and 1 failed to sync).\"}"}
{"timestamp":"2021-04-15T11:41:07.216Z","severity":"ERROR","loggerName":"com.commercetools.project.sync.cartdiscount.CartDiscountSyncer","message":"Error when trying to sync cart discount. Existing key: <<not present>>. Update actions: []","stackHash":"b2508b1f","stackTrace":"i.s.s.c.ErrorResponseException: detailMessage: The referenced object of type 'product' with identifier '14bce74d-101d-4262-b002-1c15282c9f47' was not found. It either doesn't exist, or it can't be accessed from this endpoint (e.g., if the endpoint filters by store or customer account).\nsummary: POST https://api.europe-west1.gcp.commercetools.com/yyy/cart-discounts failed  with response code 400 with X-Correlation-ID `yyy/ae6a6692-ebac-4027-b6d1-f0ecf5500be9` on 2021-04-15T11:41:07.172845Z\nhttp response formatted body: {\n  \"statusCode\" : 400,\n  \"message\" : \"The referenced object of type 'product' with identifier '14bce74d-101d-4262-b002-1c15282c9f47' was not found. It either doesn't exist, or it can't be accessed from this endpoint (e.g., if the endpoint filters by store or customer account).\",\n  \"errors\" : [ {\n    \"code\" : \"ReferencedResourceNotFound\",\n    \"message\" : \"The referenced object of type 'product' with identifier '14bce74d-101d-4262-b002-1c15282c9f47' was not found. It either doesn't exist, or it can't be accessed from this endpoint (e.g., if the endpoint filters by store or customer account).\",\n    \"typeId\" : \"product\",\n    \"id\" : \"14bce74d-101d-4262-b002-1c15282c9f47\"\n  } ]\n}\nhttp response: io.sphere.sdk.http.HttpResponseImpl@65db6202[statusCode=400,headers={date=[Thu, 15 Apr 2021 11:41:07 GMT], content-length=[634], server=[istio-envoy], x-envoy-upstream-service-time=[10], Alt-Svc=[clear], access-control-allow-headers=[Accept, Authorization, Content-Type, Origin, User-Agent, X-Correlation-ID], x-correlation-id=[yyy/ae6a6692-ebac-4027-b6d1-f0ecf5500be9], x-http-status-caused-by-extension=[false], access-control-allow-methods=[GET, POST, DELETE, OPTIONS], Via=[1.1 google], access-control-allow-origin=[*], access-control-max-age=[299], content-type=[application/json; charset=utf-8], server-timing=[projects;dur=9]},associatedRequest=<null>,textInterpretedBody={\"statusCode\":400,\"message\":\"The referenced object of type 'product' with identifier '14bce74d-101d-4262-b002-1c15282c9f4...\n"}
{"timestamp":"2021-04-15T11:41:07.309Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":3,\"failed\":1,\"processed\":4,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":250,\"latestBatchHumanReadableProcessingTime\":\"0d, 0h, 0m, 0s, 250ms\",\"reportMessage\":\"Summary: 4 cart discounts were processed in total (3 created, 0 updated and 1 failed to sync).\"}"}
{"timestamp":"2021-04-15T11:41:07.320Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting ProductSync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:41:08.412Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":4,\"failed\":0,\"processed\":11,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":785,\"latestBatchHumanReadableProcessingTime\":\"0d, 0h, 0m, 0s, 785ms\",\"numberOfProductsWithMissingParents\":0,\"reportMessage\":\"Summary: 11 product(s) were processed in total (4 created, 0 updated, 0 failed to sync and 0 product(s) with missing reference(s)).\"}"}
{"timestamp":"2021-04-15T11:41:08.420Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"Starting ShoppingListSync from CTP project with key 'xxx' to project with key 'yyy'"}
{"timestamp":"2021-04-15T11:41:08.464Z","severity":"INFO","loggerName":"com.commercetools.project.sync.Syncer","message":"{\"updated\":0,\"created\":0,\"failed\":0,\"processed\":0,\"latestBatchProcessingTimeInDays\":0,\"latestBatchProcessingTimeInHours\":0,\"latestBatchProcessingTimeInMinutes\":0,\"latestBatchProcessingTimeInSeconds\":0,\"latestBatchProcessingTimeInMillis\":0,\"latestBatchHumanReadableProcessingTime\":\"\",\"reportMessage\":\"Summary: 0 shopping lists were processed in total (0 created, 0 updated and 0 failed to sync).\"}"}

The two errors in the log are not related to products, so I think they should not be relevant.

[ahmetoz]

I wrote a quick test to reproduce the issue, and yes it's not synced.

I found out this is related to the mapper of PriceDraftBuilder in SDK, it's missing the passing the price tier.

https://github.com/commercetools/commercetools-jvm-sdk/blob/cdbea981982e1c4289c0ff87c3a6ef089aba53fd/commercetools-models/src/main/java/io/sphere/sdk/products/PriceDraftBuilder.java#L178-L188

The issue comes through the commercetools-jvm-sdk and then commercetools-sync-java until the results seen in project-sync. As we can not do much in the project-sync, we would need to fix that issue in commercetools-jvm-sdk and then commercetools-sync-java.

Now, I put the issue to our backlog, also we will communicate with the SDK to fix those issues, but in general, this could take a bit of time to resolve as it requires updates in other libraries, we will try to check also if there is a workaround for this.

Besides that thanks for opening the issue.

[praveenkumarct]

Fixed as part of version 4.0.1