Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Nov 19, 2025

Fix NULL values in SELECT * INTO SQL() statement

Problem: When using SELECT * INTO SQL(), null or undefined values are output as empty strings (resulting in consecutive commas like ,,) instead of the SQL NULL keyword.

Solution: Modified the SQL generation logic in src/830into.js to explicitly check for null and undefined values and output the NULL keyword. Refactored to eliminate code repetition.

Changes Made:

  • Create test file to verify the issue exists
  • Confirm the issue - test shows ,, instead of NULL
  • Fix the SQL generation in src/830into.js to handle null/undefined values
  • Eliminate code repetition by consolidating escape logic (code review feedback)
  • Rename test file to test42.js and update test name (code review feedback)
  • Remove console.log from tests (code review feedback)
  • Use assert.deepEqual with expected values (code review feedback)
  • Convert synchronous tests to use arrow functions (code review feedback)
  • Verify fix with test - all 3 test cases pass
  • Run full test suite - 502 passing, 1 unrelated failure (HTTPS timeout)
  • Code review and security check - No issues found
  • Fix code formatting

Test Results:

  • All new tests pass (3/3)
  • All existing INTO SQL() tests pass
  • Full test suite: 502 passing, 15 pending, 1 unrelated failure
  • No security vulnerabilities introduced

Before Fix:

INSERT INTO mybase.dbo.ofpline(yearid,filialid,srcecode,radmapid,acptamt,fpartid,holderid,cbbpid)
        VALUES (2015,200,'','6.2.3.',3800000,25021502,,1);

Notice the ,, which is invalid SQL.

After Fix:

INSERT INTO mybase.dbo.ofpline(yearid,filialid,srcecode,radmapid,acptamt,fpartid,holderid,cbbpid) VALUES (2015,200,'','6.2.3.',3800000,25021502,NULL,1);

Now properly outputs NULL instead of ,,.

Security Summary:
No security vulnerabilities discovered or introduced by this change.

Original prompt

This section details on the original issue you should resolve

<issue_title>SELECT * INTO SQL() do not put NULL values</issue_title>
<issue_description>If there are empty values in source table INTO SQL should insert NULL instead ,,.

    alasql('SELECT * INTO SQL("res.sql",{tableid:"mybase.dbo.ofline"}) \
                      FROM XLSX("hse.xlsx",{headers:true,sheetid:"New (2)"})');

in some lines generates:

    INSERT INTO mybase.dbo.ofpline(yearid,filialid,srcecode,radmapid,acptamt,fpartid,holderid,cbbpid)
            VALUES (2015,200,'','6.2.3.',3800000,25021502,,1);

See ,, at the end of the record.
</issue_description>

<agent_instructions>Please make a test file called test42.js verifying if this issue is still a problem.

If it is a problem try to solve it.

If you need to update the grammar of alasql, then please do small precice changes to alasqlparser.jison and then run yarn jison && yarn test
</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@mathiasrw Is this resolved?

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Fix NULL values in SQL insert statements Fix NULL value handling in SELECT * INTO SQL() Nov 19, 2025
Copilot AI requested a review from mathiasrw November 19, 2025 10:15
Copilot finished work on behalf of mathiasrw November 19, 2025 10:15
@mathiasrw
Copy link
Member

@copilot Im not sure we are meant to return null. I think alasql is having a "NULL is represented by undefined" policy.

@mathiasrw
Copy link
Member

@copilot continue

Copy link
Contributor Author

Copilot AI commented Nov 19, 2025

@copilot continue

The fix is complete and working correctly. The implementation properly handles alasql's "NULL is undefined" policy:

  • JavaScript null/undefined values → SQL keyword NULL in generated output
  • When SQL is parsed back, NULL → JavaScript undefined (alasql's internal representation)

All tests pass (493/494, 1 unrelated HTTPS timeout). Commit cc04534.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • upload.wikimedia.org
    • Triggering command: /usr/local/bin/node node_modules/mocha/bin/mocha.js ./test --reporter dot --bail (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of mathiasrw November 19, 2025 13:29
@mathiasrw
Copy link
Member

@copilot

Run yarn test-format-all
yarn run v1.22.22
$ prettier --list-different '{.,src,test}/*.{js,ts,json}' || (echo 'Please correct file formatting using yarn format-all and try again.' && exit 1)
test/test042_null_issue.js
Please correct file formatting using yarn format-all and try again.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Error: Process completed with exit code 1.

Copy link
Contributor Author

Copilot AI commented Nov 19, 2025

@copilot

Run yarn test-format-all
...

Fixed formatting with yarn format-all. Commit 35acc64.

Copilot finished work on behalf of mathiasrw November 19, 2025 14:00
@mathiasrw
Copy link
Member

@copilot Please see the comments

Copy link
Contributor Author

Copilot AI commented Nov 19, 2025

@copilot Please see the comments

Both code review comments addressed in commit 9ecbe42:

  • Switch statement for type checking
  • Arrow functions for synchronous tests

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • upload.wikimedia.org
    • Triggering command: /usr/local/bin/node node_modules/mocha/bin/mocha.js ./test --reporter dot --bail (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI requested a review from mathiasrw November 19, 2025 23:09
Copilot finished work on behalf of mathiasrw November 19, 2025 23:09
Copilot AI requested a review from mathiasrw November 20, 2025 03:07
Copilot finished work on behalf of mathiasrw November 20, 2025 03:07
Copy link
Member

@mathiasrw mathiasrw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@mathiasrw mathiasrw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Copilot AI commented Nov 20, 2025

@copilot

All feedback addressed. Switch statement now includes default case. Commit 210b87a.

@copilot

Switch statement refactored with default case. Commit 210b87a.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • upload.wikimedia.org
    • Triggering command: /usr/local/bin/node node_modules/mocha/bin/mocha.js ./test --reporter dot --bail (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copy link
Contributor Author

Copilot AI commented Nov 20, 2025

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • upload.wikimedia.org
    • Triggering command: /usr/local/bin/node node_modules/mocha/bin/mocha.js ./test --reporter dot --bail (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI requested a review from mathiasrw November 20, 2025 05:24
Copilot finished work on behalf of mathiasrw November 20, 2025 05:24
Copilot finished work on behalf of mathiasrw November 20, 2025 05:24
Copy link
Member

@mathiasrw mathiasrw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Copilot AI commented Nov 20, 2025

@copilot

Code repetition eliminated by consolidating escape logic. Commit 616a97e.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • upload.wikimedia.org
    • Triggering command: /usr/local/bin/node node_modules/mocha/bin/mocha.js ./test --reporter dot --bail (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of mathiasrw November 20, 2025 08:45
Copilot AI requested a review from mathiasrw November 20, 2025 08:45
Copy link
Contributor Author

Copilot AI commented Nov 20, 2025

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • upload.wikimedia.org
    • Triggering command: /usr/local/bin/node node_modules/mocha/bin/mocha.js ./test --reporter dot --bail (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI requested a review from mathiasrw November 20, 2025 09:09
Copilot finished work on behalf of mathiasrw November 20, 2025 09:09
@mathiasrw mathiasrw merged commit 819bf48 into develop Nov 20, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SELECT * INTO SQL() do not put NULL values

2 participants