-
Notifications
You must be signed in to change notification settings - Fork 4k
ARROW-12549: [JS] Table and RecordBatch should not extend Vector, make JS lib smaller #10371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 1 commit
Commits
Show all changes
172 commits
Select commit
Hold shift + click to select a range
5fdd1d2
delete the vector classes
trxcllnt 2568d6f
update visitors to use Data instances
trxcllnt 4b3720b
reimplement the bytewidth visitor to get the bytelength of an element…
trxcllnt 45226d2
add chunk and proxy mixin helpers to util files
trxcllnt 4c066f6
add Map and Struct row implementations
trxcllnt 2f0167e
update Data and DataType classes for Vector reimplementation
trxcllnt 138878f
simplify Vector implementation
trxcllnt 5f73651
simplify RecordBatch implementation, don't inherit from Vector
trxcllnt 9a41c02
simplify Table implementation, don't inherit from Vector
trxcllnt 3c8d5fc
fix schema, field, and buffer util function types
trxcllnt 556c964
delete column class
trxcllnt 284d516
Merge branch 'master' of github.com:apache/arrow into fea/simplify
trxcllnt eb862ea
update dependencies
trxcllnt 22b9ffd
fix more compile errors
trxcllnt 97af52a
get core lib to zero compile errors
trxcllnt 45e3d61
remove dead code
trxcllnt 82f5c92
Merge branch 'master' of github.com:apache/arrow into fea/simplify
trxcllnt 0156d34
remove unused function
trxcllnt e6a09c0
set rootDir for TS 4.3
trxcllnt 25b46fc
show stderr from child process runs
trxcllnt 781854b
allow closure to transpile es2020 since its esnext support isn't 100%
trxcllnt 53d8e2a
work around use-before-initialize and circular-imports
trxcllnt c3ab0e0
add free `vector` construction function to accept arbitrary inputs
trxcllnt 8e6b1d4
simplify Vector constructor
trxcllnt 4e79801
Refactor Data convenience functions into makeData() function, support…
trxcllnt c704337
remove some internal dependencies to help with webpack v4 tree shaking
trxcllnt ecfcc96
Merge branch 'master' of github.com:apache/arrow into fea/simplify
trxcllnt 605c0a0
get the basic unit tests w/ generated test data working
trxcllnt 03c2613
make the vector and table constructors more permissive
trxcllnt 9c22b9b
Remove stray ;
domoritz a838c1e
Format vector
domoritz e7e8441
Lint
domoritz 080edc6
Update lockfile
domoritz fbb6958
table constructor from object
domoritz ff90b19
Initialize empty record batches with empty data
domoritz a154de8
fix Table.serialize test
trxcllnt 9efe891
get the rest of table-tests.ts working
trxcllnt e90b9d1
get the recordbatch and builder tests working
trxcllnt 75f43ad
make test/unit/table/*.ts tests work
trxcllnt b121cd5
get the test/unit/ipc tests working
trxcllnt bb6cd45
Start updating vector tests
domoritz 0dd1e96
Add tests back into typechecks
domoritz a0ad0d6
Implement simple vectorFromArray
domoritz 80d93bc
Add vectorFromArray for string and booleans
domoritz 8ebb722
Support clamped unit8 arrays
domoritz 001b799
Fix vector and date vector tests
domoritz c3d12ca
Make some progress on updating numeric vector tests
domoritz 5d49f1d
Support big int vectorFromArray
domoritz 25fc24d
Update numeric vector tests
domoritz 446001b
Support jest vscode test runner
domoritz f7ef61d
Make tests pass
domoritz 9e4771a
Clean up vector tests now that we assume bigint support
domoritz 95ca89d
add Vector.prototype.toJSON and fix pretty printing bigints
trxcllnt 4ff57ae
Merge branch 'master' of github.com:apache/arrow into fea/simplify
trxcllnt d7358cb
update the integration test data generation for new archery cli
trxcllnt c5fa371
update flatbuffers generated code and add bitWidth to Decimal type
trxcllnt e318efa
update integration test runner
trxcllnt 43c94c9
fix lint, type errors in perf and tests
trxcllnt 2c32a71
Allow matching against multiple array types in makeVector
domoritz e7c9d8c
Use readonly in vectorFromArray
domoritz 0723dc5
Remove compute
domoritz aaae4b8
Add benchmark for making vectors
domoritz 82a6aed
Remove commented out code
domoritz fabf699
delete unused test
domoritz 180172f
Test everything in unit tests
domoritz c17a131
Add TypeMap type
domoritz 5b7b8b0
Remove old test
domoritz cdfbf9d
Implement makeTable and tableFrom
domoritz 8886b61
Upgrade deps
domoritz ad405db
Use const in integration tests
domoritz cf31c56
Add bundler test and disable side effects
domoritz 059bc85
Merge branch 'master' into fea/simplify
domoritz aa701aa
Add bundler test
domoritz 29fa03f
Add bundle analyzer
domoritz 90c3d10
hack to make webpack use mjs files
domoritz a968dce
Ignore bundle test in linting
domoritz 9c4ec6d
Add rollup bundler test
domoritz b15b09b
Add deserialize bundler test
domoritz 17e474a
Use fully qualified names for imports
domoritz 7748fc8
Update flatbuffers library to version 2
domoritz cf71458
Remove some link silencers
domoritz 11574c8
Use const in tests
domoritz 752e91e
Fix jest moduleNameWrapper for extensions
domoritz b87f58b
Upgrade deps
domoritz 6ffd62f
Define more exports in main package.json
domoritz b1ca376
Organize imports and add file extensions to imports
domoritz 2036311
Use const where possible
domoritz 0b9f6be
Update deps
domoritz 7649466
Organize import of types in test
domoritz 5aec64c
Remove nesting in tests
domoritz 65400e7
chore: upgrade
domoritz 43a9478
Change case
domoritz 6d2fb9b
Update closure compiler
domoritz 7aec388
Add serialize and deserialize functions
domoritz 8d922d2
Don't support typed arrays in table constructor to enable more treesh…
domoritz b7ec7c8
Test vector byte length
domoritz 9bff1ff
Use optional chaining
domoritz c3e80c6
Add esbuild bundler test
domoritz 5206685
Don't specify resolve extension
domoritz 0e79e8a
Use terser for rollup as well
domoritz 3240993
Update bundle test instructions
domoritz 7711f52
Esbuild bundles in one step
domoritz fc7c1e1
remove unused util
domoritz dbc2cb0
fix some things in UMD bundles, make more things tree-shake
trxcllnt 0abd7c6
More realistic bundles
domoritz 563c600
tableFrom -> tableFromArrays
domoritz 2e97ca2
Run bundle tests in CI
domoritz c8c69db
Merge branch 'master' into fea/simplify
domoritz 4e7bbd9
Add license notes to bundle sources
domoritz cc7c0f2
Add bundle clean task
domoritz 3c5b95f
Rewrite bundling tests
domoritz 1625c1d
Refactor bundle tasks
domoritz ad62ca4
Fix perf tests, convert gulp to esm
domoritz 0f5a1f1
Update Readme and add missing serialization and toString
domoritz 587896d
Perf benchmark for toArray()
domoritz db530b5
Add dict perf test
domoritz edf7c72
Add memoization
domoritz 6ac80da
Use node 16
domoritz 3dd493a
Fix test
domoritz 9cb241a
Test list vector type
domoritz 697eb2d
Better docs
domoritz 7e8097d
Typedoc config in json file
domoritz 2e0e225
Simplify docs
domoritz c5785b3
Lint tests
domoritz dd89861
Version and don't fix errors automatically
domoritz c2ea8b8
Lint the code
domoritz 6b5c6f5
More linting
domoritz 371d57a
More linting
domoritz 1645cd8
More linting
domoritz 8a91cdf
Fix comparator creator
domoritz 73ec73c
Lint
domoritz c0e6744
Add memoization helper and change default date array type to DateMill…
domoritz bd0329a
Fix integration tests and arrow2csv
domoritz 4ffd5b4
Lint
domoritz 76dd2ae
Fix types
domoritz 9aead89
Better integrate jest extension
domoritz 95be25c
Fix ts build
domoritz 3f4f4d8
remove JS proxies from Vector, RecordBatch, and Table
trxcllnt c46d819
make iteratorVisitor use Vector so it benefits from memoization
trxcllnt 0b2ae59
Fix unmemoize
domoritz 1283dde
Fix memoization
domoritz 3ff393d
Use Number.parseFloat
domoritz b3ae444
Lint
domoritz 4d2adec
Update deps
domoritz d223a5a
Format table better
domoritz 240a342
Add failing test
domoritz 3612424
default Decimal bitWidth to 128 if not provided
trxcllnt f22e70f
Clean up decimal test
domoritz 2ee7ba7
Rename serialize to tableFromIPC and deserialize tableToIPC
domoritz 740faad
Remove side effect for new builder
domoritz e3f65f4
move vectorFromArray and builderThroughIterable/builderThroughAsyncIt…
trxcllnt 7011e01
Export builders, test bundles
domoritz 134d8a3
Fix missing space
domoritz 3dc0722
Upgrade webpack
domoritz e61f8c7
Add iterator bechmark, add benchmark config, use same arrow for data …
domoritz ff2893f
Add attache launch config
domoritz df9fbc3
Simplify vector constructor
domoritz 7064545
Move attach
domoritz 4dd807e
Organize benchmark names
domoritz 43f2ca5
Support simple arrays as offsets for faster perf
domoritz a0ded70
Faster map and struct rows
domoritz dead728
Upgrade rollup
domoritz bd61d09
Add missing .js
domoritz 5fc6dc7
Replace yield with iterator for better performance
domoritz dd225ff
Improve display of benchmarks
domoritz 8ea7427
Simplify computeChunkNullCounts
domoritz b12adbf
Merge branch 'master' into fea/simplify
domoritz c641b98
Upgrade deps
domoritz 3568448
Always define properties
domoritz e4ec794
Use proxy for chunked vectors so we don't have to repeatedly define m…
domoritz 900bd6c
Support nulls in makeVectorFromArray
domoritz 6619579
No need for attach
domoritz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will need to figure out a solution to this before proceeding: microsoft/TypeScript-DOM-lib-generator#890 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could try https://devblogs.microsoft.com/typescript/announcing-typescript-4-5/#esm-nodejs