Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(Postgres Node): Allow passing in arrays to JSON columns for insert #12452

Merged
merged 3 commits into from
Jan 8, 2025

Conversation

dana-gill
Copy link
Contributor

@dana-gill dana-gill commented Jan 3, 2025

Summary

An array is technically JSON. Postgres allows for this and we should
enable this for the insert node.

Workflow for testing:

{
  "nodes": [
    {
      "parameters": {
        "assignments": {
          "assignments": [
            {
              "id": "d61ec233-ee6a-4580-9ba2-154546806f72",
              "name": "=obj",
              "value": "={{ [\n{data: [], name: 'mac',},\n{data: { age: 3 }, name: 'greg',},\n{data: [0,1], name: 'pat',},\n{data: [0], name: 'mig',}\n] }}",
              "type": "array"
            }
          ]
        },
        "options": {}
      },
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        -20,
        200
      ],
      "id": "79367664-8958-463c-9b43-65858b5218d1",
      "name": "Edit Fields"
    },
    {
      "parameters": {
        "schema": {
          "__rl": true,
          "mode": "list",
          "value": "public"
        },
        "table": {
          "__rl": true,
          "value": "array_json_table",
          "mode": "list",
          "cachedResultName": "array_json_table"
        },
        "columns": {
          "mappingMode": "defineBelow",
          "value": {
            "data": "={{ $json.obj[2].data }}",
            "name": "={{ $json.obj[0].name}}"
          },
          "matchingColumns": [
            "id"
          ],
          "schema": [
            {
              "id": "id",
              "displayName": "id",
              "required": false,
              "defaultMatch": true,
              "display": true,
              "type": "number",
              "canBeUsedToMatch": true,
              "removed": true
            },
            {
              "id": "name",
              "displayName": "name",
              "required": false,
              "defaultMatch": false,
              "display": true,
              "type": "string",
              "canBeUsedToMatch": true,
              "removed": false
            },
            {
              "id": "data",
              "displayName": "data",
              "required": true,
              "defaultMatch": false,
              "display": true,
              "type": "object",
              "canBeUsedToMatch": true
            }
          ],
          "ignoreTypeMismatchErrors": false,
          "attemptToConvertTypes": false,
          "convertFieldsToString": false
        },
        "options": {}
      },
      "type": "n8n-nodes-base.postgres",
      "typeVersion": 2.5,
      "position": [
        240,
        200
      ],
      "id": "dc57fff2-cf66-495c-9270-97f2b208b11b",
      "name": "Postgres",
      "credentials": {
        "postgres": {
          "id": "UeO14x83grJWzgYI",
          "name": "dgl neon"
        }
      }
    }
  ],
  "connections": {
    "Edit Fields": {
      "main": [
        [
          {
            "node": "Postgres",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "pinData": {}
}

Related Linear tickets, Github issues, and Community forum posts

https://linear.app/n8n/issue/NODE-1705/postgres-node-cant-insert-an-array-into-json-column-in-postgres

Review / Merge checklist

  • PR title and summary are descriptive. (conventions)
  • Docs updated or follow-up ticket created.
  • Tests included.
  • PR Labeled with release/backport (if the PR is an urgent fix that needs to be backported)

@dana-gill dana-gill force-pushed the node-1705-insert-array-json branch from e7498ba to 4005bec Compare January 3, 2025 16:38
Copy link

codecov bot commented Jan 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

@n8n-assistant n8n-assistant bot added n8n team Authored by the n8n team node/improvement New feature or request labels Jan 3, 2025
@dana-gill dana-gill marked this pull request as ready for review January 3, 2025 17:03
Copy link
Contributor

@ShireenMissi ShireenMissi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🎉

Copy link
Contributor

github-actions bot commented Jan 7, 2025

⚠️ Some Cypress E2E specs are failing, please fix them before merging

Copy link

cypress bot commented Jan 7, 2025

n8n    Run #8647

Run Properties:  status check passed Passed #8647  •  git commit d84b1c53ba: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 dana-gill 🗃️ e2e/*
Project n8n
Branch Review node-1705-insert-array-json
Run status status check passed Passed #8647
Run duration 04m 53s
Commit git commit d84b1c53ba: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 dana-gill 🗃️ e2e/*
Committer Dana Lee
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 1
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 489
View all changes introduced in this branch ↗︎

An array is technically JSON. Postgres allows for this and we should
enable this for the insert node.

We will need to get the `column` node parameters first and then call
values on it since the validator in core will not accept arrays to be
passed in for columns with JSON. As a result, we will use `pgp.as.json`
to handle validation. This also gives us the added benefit of converting
the values which are meant to be JSON into a format Postgres can
understand. We also need to enable raw mode since we will use
`pgp.helpers.concat` for the queries in `configureQueryRunners`.
@ShireenMissi ShireenMissi force-pushed the node-1705-insert-array-json branch from 4005bec to d84b1c5 Compare January 8, 2025 09:17
Copy link
Contributor

github-actions bot commented Jan 8, 2025

✅ All Cypress E2E specs passed

@ShireenMissi ShireenMissi merged commit 9dd0686 into master Jan 8, 2025
37 checks passed
@ShireenMissi ShireenMissi deleted the node-1705-insert-array-json branch January 8, 2025 09:41
@github-actions github-actions bot mentioned this pull request Jan 8, 2025
@janober
Copy link
Member

janober commented Jan 9, 2025

Got released with [email protected]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
n8n team Authored by the n8n team node/improvement New feature or request Released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants