Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
df6c1c6
initial working extension type
paleolimbot Feb 18, 2022
d7ea064
simplify implementation
paleolimbot Feb 18, 2022
7d69c71
test subclasses
paleolimbot Feb 18, 2022
2055776
with some array infrastructure
paleolimbot Feb 18, 2022
eec9407
with registration of custom subclasses
paleolimbot Feb 18, 2022
6abc0db
get array and type subclasses working better
paleolimbot Feb 18, 2022
be1992c
clang-format
paleolimbot Feb 18, 2022
45af032
license
paleolimbot Feb 19, 2022
9184a60
Update r/R/extension.R
paleolimbot Mar 14, 2022
88145da
make sure ExtensionEquals can be overridden from R
paleolimbot Mar 14, 2022
f3afad7
start on vctrs extension type + tests
paleolimbot Mar 14, 2022
e50744f
improvement, but test still failing
paleolimbot Mar 15, 2022
f92496e
fix Deserialize from C++
paleolimbot Mar 15, 2022
d0ec080
failing test for chunked array out
paleolimbot Mar 15, 2022
aef8607
shuffle responsibility between ExtensionArray, ChunkedArray, and Exte…
paleolimbot Mar 15, 2022
b91ab8f
test roundtripping through Table, RecordBatch, and ChunkedArray
paleolimbot Mar 15, 2022
4012ac1
clang-format
paleolimbot Mar 16, 2022
536715e
only register extension type if arrow is available
paleolimbot Mar 16, 2022
79dd3a0
start to document, simplify circular logic in methods
paleolimbot Mar 16, 2022
3560bed
more documentation and simplification
paleolimbot Mar 16, 2022
1d5db0f
simplify C++--R6 interaction
paleolimbot Mar 16, 2022
99eda35
more clarifying and simplifying the C++--R6 interaction
paleolimbot Mar 16, 2022
10ed09b
more documenting
paleolimbot Mar 16, 2022
3e4df79
more documentation and renaming methods
paleolimbot Mar 16, 2022
6731c36
documentation for extension types and vctrs extension type
paleolimbot Mar 16, 2022
0542119
make sure new functions are in the pkgdown index
paleolimbot Mar 16, 2022
64c5731
add missing pkgdown reference section
paleolimbot Mar 16, 2022
ca24c76
correct the docs
paleolimbot Mar 17, 2022
f201359
add setfaulting test for dataset
paleolimbot Mar 17, 2022
59cdcf2
with non-segfaulting dataset read!
paleolimbot Mar 17, 2022
ebebdb4
properly fence Dataset test
paleolimbot Mar 17, 2022
68a78e1
Update r/R/arrow-package.R
paleolimbot Mar 21, 2022
e4ec12e
Update r/R/arrow-tabular.R
paleolimbot Mar 21, 2022
151ead3
Update r/R/extension.R
paleolimbot Mar 21, 2022
7eeb637
actually use new_extension_array()!
paleolimbot Mar 22, 2022
8cc73c2
fix roxygen include
paleolimbot Mar 22, 2022
9895d1c
typo in C++ comment
paleolimbot Mar 22, 2022
1fdf638
better default method for .chunked_array_as_vector()
paleolimbot Mar 22, 2022
25a9980
add example, export ExtensionArray, support ExtensionType$create() an…
paleolimbot Mar 22, 2022
7d3e877
allow strings for extension_metadata (but make sure they're UTF-8 enc…
paleolimbot Mar 22, 2022
184f8e8
remove confusing methods
paleolimbot Mar 22, 2022
30172b6
actually test UTF-8 conversion and printing of non-utf-8 metadata
paleolimbot Mar 22, 2022
df25ad1
better example
paleolimbot Mar 22, 2022
26fb363
Less redundant .Deserialize() method
paleolimbot Mar 22, 2022
685e00a
try to fix windows test
paleolimbot Mar 28, 2022
66772d3
move extension type definition to extension.h so that the Convert met…
paleolimbot Mar 28, 2022
f2b2e45
use Converter for extension types
paleolimbot Mar 28, 2022
5711594
add license
paleolimbot Mar 28, 2022
93a0326
reomve unused extension type method
paleolimbot Mar 28, 2022
6ae66f6
.ToString -> ToString()
paleolimbot Mar 28, 2022
b03988a
extension_metadata -> Serialize
paleolimbot Mar 28, 2022
743f657
.Deserialize -> Deserialize
paleolimbot Mar 28, 2022
396b38e
.ExtensionEquals -> ExtensionEquals
paleolimbot Mar 28, 2022
9aa909c
consolidate .array_as_vector() and .chunked_array_as_vector() to as_v…
paleolimbot Mar 28, 2022
68245a6
better testing of array conversion
paleolimbot Mar 28, 2022
ef459ee
test with vctrs extension type
paleolimbot Mar 28, 2022
2d7fa62
undo changes to Table/RecordBatch
paleolimbot Mar 28, 2022
a0d658b
fix typos, remove unused field in Converter_Extension
paleolimbot Mar 28, 2022
1d71a03
move extension type logic to the extension type converter
paleolimbot Mar 29, 2022
374a6ff
Use old ChunedArray$as_vector() method that used the converter API
paleolimbot Mar 29, 2022
b45d181
update comments to remind about the future SafeCallIntoR.
paleolimbot Mar 30, 2022
4130110
Serialize -> extension_metadata
paleolimbot Mar 30, 2022
c105112
Deserialize -> populate_instance
paleolimbot Mar 30, 2022
2e5aa01
populate_instance() -> deserialize_instance()
paleolimbot Mar 30, 2022
46386cb
use SafeCallIntoR()
paleolimbot Apr 7, 2022
5c847cb
punt on SafeCallIntoR for Deserialize()
paleolimbot Apr 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions r/DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ Collate:
'table.R'
'dplyr.R'
'duckdb.R'
'extension.R'
'feather.R'
'field.R'
'filesystem.R'
Expand Down
9 changes: 9 additions & 0 deletions r/NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,8 @@ export(DictionaryArray)
export(DirectoryPartitioning)
export(DirectoryPartitioningFactory)
export(Expression)
export(ExtensionArray)
export(ExtensionType)
export(FeatherReader)
export(Field)
export(FileFormat)
Expand Down Expand Up @@ -267,6 +269,8 @@ export(match_arrow)
export(matches)
export(mmap_create)
export(mmap_open)
export(new_extension_array)
export(new_extension_type)
export(null)
export(num_range)
export(one_of)
Expand All @@ -282,6 +286,8 @@ export(read_parquet)
export(read_schema)
export(read_tsv_arrow)
export(record_batch)
export(register_extension_type)
export(reregister_extension_type)
export(s3_bucket)
export(schema)
export(set_cpu_count)
Expand All @@ -300,8 +306,11 @@ export(uint32)
export(uint64)
export(uint8)
export(unify_schemas)
export(unregister_extension_type)
export(utf8)
export(value_counts)
export(vctrs_extension_array)
export(vctrs_extension_type)
export(write_arrow)
export(write_csv_arrow)
export(write_dataset)
Expand Down
5 changes: 5 additions & 0 deletions r/R/arrow-package.R
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@
}
}

if (arrow_available()) {
# register extension types that we use internally
reregister_extension_type(vctrs_extension_type(vctrs::unspecified()))
}

invisible()
}

Expand Down
36 changes: 36 additions & 0 deletions r/R/arrowExports.R

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading