Releases: apache/orc
Releases · apache/orc
v1.9.1
Milestone
Changelog
Bug
- ORC-1455 Fix build failure on non-x86 with unused macro in
CpuInfoUtil.cc
- ORC-1457 Fix ambiguous overload of
Type::createRowBatch
- ORC-1462 Bump
aircompressor
to 0.25 to fix JDK-8081450
Test
v1.9.0
Milestone
Changelog
New Feature and Notable Changes
- ORC-961 Expose metrics of the reader
- ORC-1167 Support orc.row.batch.size configuration
- ORC-1252 Expose io metrics for write operation
- ORC-1301 Enforce C++17
- ORC-1310 allowlist Support for plugin filter
- ORC-1356 Use Intel AVX-512 instructions to accelerate the Rle-bit-packing decode
- ORC-1385 Support schema evolution from numeric to numeric
- ORC-1386 Support schema evolution from primitive to string group/decimal/timestamp
Improvement
- ORC-827 Utilize Array copyOf
- ORC-1170 Optimize the RowReader::seekToRow function
- ORC-1232 Disable metrics collector by default
- ORC-1278 Update Readme.md cmake to 3.12
- ORC-1279 Update cmake version
- ORC-1286 Replace DataBuffer with BlockBuffer in the BufferedOutputStream
- ORC-1298 Support dedicated ColumnVectorBatch of numeric types
- ORC-1302 Upgrade Github workflow to build on Windows
- ORC-1306 Fixed indented code style for Java modules
- ORC-1307 Add coding style enforcement
- ORC-1314 Remove macros defined before C++11
- ORC-1347 Use make_unique and make_shared when creating unique_ptr and shared_ptr
- ORC-1348 TimezoneImpl constructor should pass std::vector<> & instead of std::vector<>
- ORC-1349 Remove useless bufStream definition
- ORC-1352 Remove ORC_[NOEXCEPT|NULLPTR|OVERRIDE|UNIQUE_PTR] macro usages
- ORC-1355 Writer::addUserMetadata change parameter to reference
- ORC-1373 Add log when DynamicByteArray length overflow
- ORC-1401 Allow writing an intermediate footer
- ORC-1421 Use PyArrow 12.0.0 in document
Bug
- ORC-1225 Bump maven-assembly-plugin to 3.4.2
- ORC-1266 DecimalColumnVector resets the isRepeating flag in the nextVector method
- ORC-1273 Bump opencsv to 5.7.0
- ORC-1297 Bump opencsv to 5.7.1
- ORC-1304 throw ParseError when using SearchArgument with nested struct
- ORC-1315 Byte to integer conversions fail on platforms with unsigned char type
- ORC-1320 Fix build break of C++ code on docker images
- ORC-1363 Upgrade
zookeeper
to 3.8.1 - ORC-1368 Bump commons-csv to 1.10.0
- ORC-1398 Bump
aircompressor
to 0.24 - ORC-1399 Fix boolean type with useTightNumericVector enabled
- ORC-1433 Fix comment in the Vector.hh
- ORC-1447 Fix a bug in CpuInfoUtil.cc to support ARM platform
- ORC-1449 Add
-Wno-unused-macros
for Clang 14.0 - ORC-1450 Stop enforcing override keyword
- ORC-1453 Fix
fall-through
warning cases
Task
- ORC-1164 Setting version to 1.9.0-SNAPSHOT
- ORC-1218 Bump apache pom to 27
- ORC-1219 Remove redundant
toString
- ORC-1237 Remove a wrong image link to
article-footer.png
- ORC-1239 Upgrade maven-shade-plugin to 3.3.0
- ORC-1256 Publish test-jar to maven central
- ORC-1259 Bump
slf4j
to 2.0.0 - ORC-1269 Remove FindBugs
- ORC-1270 Move opencsv dependency to the tools module.
- ORC-1274 Add a checkstyle rule to ban starting LAND and LOR
- ORC-1275 Bump maven-jar-plugin to 3.3.0
- ORC-1276 Bump
slf4j
to 2.0.1 - ORC-1277 Bump maven-shade-plugin to 3.4.0
- ORC-1284 Add permissions to GitHub Action labeler
- ORC-1296 Bump reproducible-build-maven-plugin to 0.16
- ORC-1311 Bump maven-shade-plugin to 3.4.1
- ORC-1316 Bump slf4j.version to 2.0.4
- ORC-1334 Bump slf4j.version to 2.0.6
- ORC-1335 Bump netty-all to 4.1.86.Final
- ORC-1351 Update PR Labeler definition
- ORC-1358 Use spotless to format pom files
- ORC-1371 Remove unsupported SLF4J bindings from classpath
- ORC-1372 Bump
zstd
to v1.5.4 - ORC-1375 Cancel old running ci tasks when a pr has a new commit
- ORC-1377 Enforce override keyword
- ORC-1383 Upgrade
aircompressor
to 0.22 - ORC-1395 Enforce license check
- ORC-1396 Bump
slf4j
to 2.0.7 - ORC-1410 Bump
zstd
to v1.5.5 - ORC-1411 Remove Ubuntu18.04 from docker-based tests
- ORC-1419 Bump
protobuf-java
to 3.22.3 - ORC-1428 Setup GitHub Action CI on
branch-1.9
- ORC-1443 Enforce Java version
- ORC-1444 Enforce JDK Bytecode version
- ORC-1446 Publish snapshot from branch-1.9
Test
- ORC-1231 Update supported OS list in
building.md
- ORC-1233 Bump
junit
to 5.9.0 - ORC-1234 Upgrade
objenesis
to 3.2 in Spark benchmark - ORC-1235 Bump
avro
to 1.11.1 - ORC-1240 Update site README to use apache/orc-dev
- ORC-1241 Use
apache/orc-dev
DockerHub repository in Docker tests - ORC-1250 Bump
mockito
to 4.7.0 - ORC-1254 Add
spotbugs
check - ORC-1258 Bump
byte-buddy
to 1.12.14 - ORC-1262 Bump
maven-checkstyle-plugin
to 3.2.0 - ORC-1265 Upgrade
spotbugs
to 4.7.2 - ORC-1267 Bump
mockito
to 4.8.0 - ORC-1271 Bump
spotbugs-maven-plugin
to 4.7.2.0 - ORC-1272 Bump
byte-buddy
to 1.12.16 - ORC-1300 Update Spark to 3.3.1 and its dependencies
- ORC-1303 Upgrade
GoogleTest
to 1.12.1 - ORC-1318 Upgrade mockito.version to 4.9.0
- ORC-1319 Upgrade byte-buddy to 1.12.19
- ORC-1321 Bump checkstyle to 10.5.0
- ORC-1322 Upgrade centos7 docker image to use gcc9
- ORC-1324 Use Java 19 instead of 18 in GHA
- ORC-1333 Bump
mockito
to 4.10.0 - ORC-1341 Bump
mockito
to 4.11.0 - ORC-1353 Bump
byte-buddy
to 1.12.21 - ORC-1359 Bump
byte-buddy
to 1.12.22 - ORC-1366 Bump
checkstyle
to 10.7.0 - ORC-1367 Bump
maven-enforcer-plugin
to 3.2.1 - ORC-1369 Bump
byte-buddy
to 1.12.23 - ORC-1370 Bump
snappy-java
to 1.1.9.1 - ORC-1374 Update Spark to 3.3.2
- ORC-1378 Add slf4j impl to avoid warning message in example module
- ORC-1379 Upgrade
spotbugs
to 4.7.3.2 - ORC-1380 Upgrade
checkstyle
to 10.8.0 - ORC-1394 Bump
maven-assembly-plugin
to 3.5.0 - ORC-1397 Bump
checkstyle
to 10.9.2 - ORC-1405 Bump
spotbugs-maven-plugin
to 4.7.3.4 - ORC-1406 Bump
maven-enforcer-plugin
to 3.3.0 - ORC-1408 Add
testVectorBatchHasNull
test case and comment - ORC-1415 Add Java 20 to GitHub Action CI
- ORC-1417 Bump
checkstyle
to 10.10.0 - ORC-1418 Bump
junit
to 5.9.3 - ORC-1426 Use Java
21-ea
instead of 20 in GitHub Action - ORC-1435 Bump
maven-checkstyle-plugin
to 3.3.0 - ORC-1436 Bump
snappy-java
to 1.1.10.0 - ORC-1452 Use the latest OS versions in variant tests
v1.8.4
Milestone
Changelog
Bug
ORC-1304: [C++] Fix seeking over empty PRESENT stream
ORC-1400: Use Hadoop 3.3.5 on Java 17+ and benchmark
ORC-1413: Fix for ORC row level filter issue with ACID table
Test:
ORC-1404 Bump parquet to 1.13.0
ORC-1414 Upgrade java bench module to spark3.4
ORC-1416 Upgrade Jackson dependency to 2.14.2 in bench module
ORC-1420 Pin net.bytebuddy package to 1.12.x
Task:
ORC-1395 Enforce license check via github action
v1.7.9
v1.8.3
Milestone
Changelog
Bug
- ORC-1357 Handle missing compression block size
- ORC-1382 Fix secondary config names org.sarg.* to orc.sarg.*
- ORC-1384 Fix ArrayIndexOutOfBoundsException when reading dictionary stream bigger then dictionary
- ORC-1393 Add reset(DiskRangeList input, long length) to InStream impl class
Test
Tasks
- ORC-1358 Use spotless to format pom files
v1.7.8
v1.8.2
Milestone
Changelog
Bug
Improvement
- ORC-1327 Exclude the proto files from the nohive jar
- ORC-1328 Exclude the proto files from the shaded protobuf jar
- ORC-1329 Add
OrcConf.getStringAsList
method - ORC-1338 Set bloom filter fpp to 1%
- ORC-1342 Publish SBOM artifacts
- ORC-1344 Skip SBOM generation during CMake
- ORC-1345 Use
makeBom
and skip snapshot check in GitHub Actionpublish_snapshot
job
Test
- ORC-1323 Make
docker/reinit.sh
support target OS arguments - ORC-1330 Add
TestOrcConf
- ORC-1339 Remove orc.sarg.to.filter default value assumption in test cases
- ORC-1350 Upgrade
setup-java
to v3
Tasks
v1.8.1
Milestone
Changelog
Bug
- ORC-1283 ENABLE_INDEXES does not take effect
- ORC-1288 Invalid memory freeing with ZLIB compression
- ORC-1291 NullPointerException in TypeDescription
Improvement
- ORC-1268 Set CMP0135 policy for CMake 3.24+
- ORC-1282 Add slf4j impl to avoid warning message
- ORC-1294 Build error when skip tests build
- ORC-1295 Improve ORC Spec example (Decoding RLE v2 direct)
- ORC-1299 benchmark can't work for data resource 403
- ORC-1305 Add more orc java examples
- ORC-1308 Avoid star import
Test
Tasks
v1.7.7
v1.8.0
Milestone
Changelog
New Feature and Notable Changes
- ORC-450 Support selecting list indices without materializing list items
- ORC-824 Add column statistics for List and Map
- ORC-1004 Java ORC writer supports the selection vector
- ORC-1075 Support reading ORC files with no column statistics
- ORC-1125 Support decoding decimals in RLE
- ORC-1136 Optimize reads by combining multiple reads without significant separation into a single read
- ORC-1138 Seek vs Read Optimization
- ORC-1172 Add row count limit config for one stripe
- ORC-1212 Upgrade protobuf-java to 3.17.3
- ORC-1220 Set min.hadoop.version to 2.7.3
- ORC-1248 Redefine Hadoop dependency for Apache ORC 1.8.0
- ORC-1256 Publish test-jar to maven central
- ORC-1260 Publish
shaded-protobuf
classifier artifacts
Improvement
- ORC-825 Use Empty Array For Collections toArray
- ORC-826 Do Not Use Collection Contains/Get
- ORC-828 Improve Fetch Data Set Process
- ORC-829 Optimize Serialization percentileBits
- ORC-831 Do Not Copy String When Flushing Dictionary
- ORC-833 RunLengthIntegerReaderV2 Calculate Batch Size Once
- ORC-834 Do Not Convert to String in DecimalFromTimestampTreeReader
- ORC-835 Cache TRUE/FALSE Bytes in StringGroupFromBooleanTreeReader
- ORC-836 StringGroupFromDoubleTreeReader Use Double toString
- ORC-837 Reuse HiveDecimalWritable in ConvertTreeReaderFactory
- ORC-838 Simplify compareTo/equals/putBuffer of ByteBufferAllocatorPool
- ORC-840 Remove Superfluous Array Fill in RecordReaderImpl
- ORC-841 Remove Superfluous Array Fill in StringHashTableDictionary
- ORC-842 Remove newKey from StringHashTableDictionary
- ORC-844 Improve hashCode Methods
- ORC-847 Do Not Create Empty Array in StringGroupFromBinaryTreeReader
- ORC-852 Allow DynamicByteArray to Return a ByteBuffer
- ORC-853 Optimize writeDouble Implementation
- ORC-855 Remove Unused isRepeating from RunLengthIntegerReaderV2
- ORC-865 Bump opencsv from 3.9 to 5.5.1
- ORC-883 Dependency Audit and QA
- ORC-897 optimization loop termination condition in readerIsCompatible method
- ORC-935 Bump commons-csv from 1.8 to 1.9.0
- ORC-937 Replace deprecated method
- ORC-958 Convert command support overwrite option
- ORC-969 Evaluate SearchArguments using file and stripe level stats
- ORC-975 Avoid double counting closestFixedBits in percentileBits method
- ORC-982 Extract checkstyle to a single file, help newcomers check code style
- ORC-988 Bump opencsv from 5.5.1 to 5.5.2
- ORC-992 Reached max repeat length, we can directly decide to use DELTA encoding
- ORC-1005 Make that the java and C++ implementations of determineEncoding in RunLengthIntegerWriterV2 are consistent.
- ORC-1007 Fix a warning from the shade plugin
- ORC-1013 Renaming a parameter in constructors of TreeWriter's derived classes
- ORC-1014 Add details when we get IOExceptions from file system
- ORC-1020 Improve orc::RleDecoderV2::nextDirect
- ORC-1027 Filter processing to allow filter injections that cannot be represented via SArgs
- ORC-1047 Handle quoted field names during string schema parsing
- ORC-1077 Remove commons-codec dependency and use java.util.Base64
- ORC-1099 Extend ReadIntent to support MAP and UNION type
- ORC-1101 Improve malformed STRUCT handling
- ORC-1122 Add buffer to decode the whole run in RleDecoderV2
- ORC-1137 Improve float/double conversion in DoubleColumnReader::next()
- ORC-1149 Bump slf4j.version to 1.7.36
- ORC-1150 Improve RowReaderImpl::computeBatchSize()
- ORC-1152 Support encoding short decimals in RLEv2
- ORC-1156 Update opencsv to 5.6
- ORC-1163 Bump zookeeper from 3.7.0 to 3.8.0
- ORC-1169 Use Hadoop 3.3.2 on Java 17+
- ORC-1178 Use hadoop 3.3.3 on Java 17+
Bug
- ORC-845 Fix NPE in DynamicIntArray toString
- ORC-929 Fix NaN at orc-tools 'meta' command
- ORC-1129 The build of tool-test should depends on cpp tools
- ORC-1159 Crash when the last stripe is skipped
- ORC-1242 Bump threeten-extra to 1.7.1
Test
- ORC-860 Add dependabot
- ORC-864 Bump jackson.version from 2.12.2 to 2.12.4
- ORC-877 Bump junit-vintage-engine from 5.7.0 to 5.7.2
- ORC-888 Bump objenesis from 3.1 to 3.2
- ORC-905 Add an integration test for
example
- ORC-917 Bump mockito-core from 3.7.0 to 3.11.2
- ORC-919 Spark bench objenesis should be the same as Spark.
- ORC-920 Use junit.version and mockito.version property and bump junit to 5.7.2
- ORC-925 Simplify assertions
- ORC-928 Bump checkstyle from 8.44 to 8.45.1
- ORC-932 Bump byte-buddy from 1.10.19 to 1.11.12
- ORC-934 Add integration tests for Java bench
- ORC-940 Use Hadoop 3.3.1 in bench module
- ORC-955 Add Javadoc generation GitHub Action job
- ORC-963 Build
benchmark
module always for integration testing - ORC-966 Bump byte-buddy from 1.11.12 to 1.11.13
- ORC-967 Bump mockito.version from 3.11.2 to 3.12.1
- ORC-986 Bump mockito.version from 3.12.1 to 3.12.4
- ORC-987 Bump jackson.version from 2.12.4 to 2.12.5
- ORC-1001 Bump maven-enforcer-plugin to 3.0.0
- ORC-1019 Remove redundant jackson dependencies
- ORC-1022 Bump byte-buddy from 1.11.13 to 1.11.19
- ORC-1038 Bump mockito.version from 3.12.4 to 4.0.0
- ORC-1074 Bump byte-buddy from 1.11.19 to 1.12.6
- ORC-1079 Add Linux clang GitHub Action job
- ORC-1085 Bump auto-service from 1.0 to 1.0.1
- ORC-1089 Add test cases verifying writers with selected vector
- ORC-1104 Use Spark 3.2.1 in benchmark
- ORC-1107 Fix NPE at benchmark data schema loading
- ORC-1110 Bump mockito.version from 4.0.0 to 4.3.1
- ORC-1126 Bump byte-buddy from 1.12.6 to 1.12.8
- ORC-1139 Benchmark for Seek vs Read
- ORC-1141 Bump mockito.version from 4.3.1 to 4.4.0
- ORC-1145 Add Java 18 to GitHub Action CI.
- ORC-1153 Bump byte-buddy from 1.12.8 to 1.12.9
- ORC-1157 Update guava to 31.1-jre
- ORC-1168 Update byte-buddy to 1.12.10
- ORC-1177 Upgrade mockito.version to 4.5.1
- ORC-1179 Upgrade checkstyle to 10.2 on Java 11+
- ORC-1187 Use main instead of master in merge_orc_pr.py
- ORC-1194 Bump mockito.version to 4.6.0
- ORC-1195 Bump checkstyle to 10.3
- ORC-1196 Add spark benchmark integration tests to GHA
- ORC-1197 Bump mockito.version from 4.6.0 to 4.6.1
- ORC-1201 Remove Debian 9 from Docker Tests
- ORC-1203 Bump maven-enforcer-plugin to 3.1.0
- ORC-1206 Bump netty-all to 4.1.78.Final
- ORC-1207 Upgrade Spark to 3.3.0
- ORC-1208 Bump byte-buddy to 1.12.12
- ORC-1209 Bump checkstyle to 10.3.1
- ORC-1234 Upgrade objenesis to 3.2 in Spark benchmark
- ORC-1236 Bump checkstyle to 10.3.2
- ORC-1243 Bump byte-buddy to 1.12.13
- ORC-1253 Add Fedora 37 docker test
- ORC-1254 Add spotbugs check
Task
- ORC-868 Pin gson to 2.2.4
- ORC-869 Pin jmh 1.20
- ORC-872 Bump kryo-shaded from 3.0.3 to 4.0.2
- ORC-874 Bump zookeeper from 3.6.2 to 3.7.0
- ORC-884 Bump jettison from 1.1 to 1.4.1
- ORC-887 Remove ORC Twitter link from
news
page - ORC-890 Pin minimum support Hadoop version to 2.2.0
- ORC-892 Pin scala-library to 2.12.10
- ORC-898 Bump threeten-extra from 1.5.0 to 1.7.0
- ORC-899 Archive Apache ORC 1.4.x in
releases
page - ORC-900 Update doap_orc.rdf for Apache Projects page
- ORC-908 Use https instead of http for website links in
pom.xml
- ORC-914 Pin maven-dependency-plugin to 3.1.2
- ORC-916 Bump annotations from 17.0.0 to 21.0.1
- ORC-918 Pin protobuf-java to 2.5.0
- ORC-923 Bump apache from 23 to 24
- ORC-946 Unified json library
- ORC-949 Add CustomImportOrder rule
- ORC-956 Bump annotations from 21.0.1 to 22.0.0
- ORC-977 Update webpages and TestVectorOrcFile.java to be more neutral
- ORC-1045 Bump commons-cli to 1.5
- ORC-1056 Bump annotations from 22.0.0 to 23.0.0
- ORC-1103 Use Maven 3.8.4
- ORC-1140 Documentation for Seek vs Read
- ORC-1158 Add notification settings to .asf.yam
- ORC-1162 Fix Apache Project Website Checks Warningl
- ORC-1165 Enable GitHub Action in branch-1.8
- ORC-1166 Enable snapshot publishing in branch-1.8
- ORC-1171 Skip build and test on docker and site updates
- ORC-1173 Pin jodd-core to 3.5.2
- ORC-1176 Upgrade maven-jar-plugin to 3.2.2
- ORC-1185 Add merge_orc_pr.py
- ORC-1210 Upgrade maven to 3.8.6
- ORC-1216 Pin org.jetbrains.annotations dependency to 17.0.0
- ORC-1211 Upgrade maven-assembly-plugin to 3.4.0
- ORC-1214 Bump maven-assembly-plugin to 3.4.1
- ORC-1217 Downgrade org.jetbrains.annotations to 17.0.0
- ORC-1223 Move DirectDecompressWrapper to org.apache.orc.impl
- ORC-1224 Move getDecompressor to HadoopShimsCurrent
- ORC-1226 Add a deprecation warning for Hadoop 2.7.2 and below
- ORC-1229 Move KeyProviderImpl to org.apache.orc.impl
- ORC-1230 Move encryption utility functions to HadoopShimsCurrent
- ORC-1246 Revamp ORC Website
- ORC-1247 Improve Apache ORC website and docs
- ORC-1249 Move site/_docs/releases.md to site/releases/index.md
- ORC-1255 Fix ORC website navbar highlight
- ORC-1257 Publish multi-architecture ORC-dev docker images
- ORC-1261 Rename shaded pattern
com.google.protobuf25
toorg.apache.orc.protobuf
- ORC-1263 Add decimal type to ORC Website
- ORC-1221 Move NullKeyProvider to org.apache.orc.impl