Releases: apache/orc
Releases · apache/orc
v1.9.1
Milestone
Changelog
Bug
- ORC-1455 Fix build failure on non-x86 with unused macro in
CpuInfoUtil.cc - ORC-1457 Fix ambiguous overload of
Type::createRowBatch - ORC-1462 Bump
aircompressorto 0.25 to fix JDK-8081450
Test
v1.9.0
9d43439
This commit was signed with the committer’s verified signature.
dongjoon-hyun
Dongjoon Hyun
Milestone
Changelog
New Feature and Notable Changes
- ORC-961 Expose metrics of the reader
- ORC-1167 Support orc.row.batch.size configuration
- ORC-1252 Expose io metrics for write operation
- ORC-1301 Enforce C++17
- ORC-1310 allowlist Support for plugin filter
- ORC-1356 Use Intel AVX-512 instructions to accelerate the Rle-bit-packing decode
- ORC-1385 Support schema evolution from numeric to numeric
- ORC-1386 Support schema evolution from primitive to string group/decimal/timestamp
Improvement
- ORC-827 Utilize Array copyOf
- ORC-1170 Optimize the RowReader::seekToRow function
- ORC-1232 Disable metrics collector by default
- ORC-1278 Update Readme.md cmake to 3.12
- ORC-1279 Update cmake version
- ORC-1286 Replace DataBuffer with BlockBuffer in the BufferedOutputStream
- ORC-1298 Support dedicated ColumnVectorBatch of numeric types
- ORC-1302 Upgrade Github workflow to build on Windows
- ORC-1306 Fixed indented code style for Java modules
- ORC-1307 Add coding style enforcement
- ORC-1314 Remove macros defined before C++11
- ORC-1347 Use make_unique and make_shared when creating unique_ptr and shared_ptr
- ORC-1348 TimezoneImpl constructor should pass std::vector<> & instead of std::vector<>
- ORC-1349 Remove useless bufStream definition
- ORC-1352 Remove ORC_[NOEXCEPT|NULLPTR|OVERRIDE|UNIQUE_PTR] macro usages
- ORC-1355 Writer::addUserMetadata change parameter to reference
- ORC-1373 Add log when DynamicByteArray length overflow
- ORC-1401 Allow writing an intermediate footer
- ORC-1421 Use PyArrow 12.0.0 in document
Bug
- ORC-1225 Bump maven-assembly-plugin to 3.4.2
- ORC-1266 DecimalColumnVector resets the isRepeating flag in the nextVector method
- ORC-1273 Bump opencsv to 5.7.0
- ORC-1297 Bump opencsv to 5.7.1
- ORC-1304 throw ParseError when using SearchArgument with nested struct
- ORC-1315 Byte to integer conversions fail on platforms with unsigned char type
- ORC-1320 Fix build break of C++ code on docker images
- ORC-1363 Upgrade
zookeeperto 3.8.1 - ORC-1368 Bump commons-csv to 1.10.0
- ORC-1398 Bump
aircompressorto 0.24 - ORC-1399 Fix boolean type with useTightNumericVector enabled
- ORC-1433 Fix comment in the Vector.hh
- ORC-1447 Fix a bug in CpuInfoUtil.cc to support ARM platform
- ORC-1449 Add
-Wno-unused-macrosfor Clang 14.0 - ORC-1450 Stop enforcing override keyword
- ORC-1453 Fix
fall-throughwarning cases
Task
- ORC-1164 Setting version to 1.9.0-SNAPSHOT
- ORC-1218 Bump apache pom to 27
- ORC-1219 Remove redundant
toString - ORC-1237 Remove a wrong image link to
article-footer.png - ORC-1239 Upgrade maven-shade-plugin to 3.3.0
- ORC-1256 Publish test-jar to maven central
- ORC-1259 Bump
slf4jto 2.0.0 - ORC-1269 Remove FindBugs
- ORC-1270 Move opencsv dependency to the tools module.
- ORC-1274 Add a checkstyle rule to ban starting LAND and LOR
- ORC-1275 Bump maven-jar-plugin to 3.3.0
- ORC-1276 Bump
slf4jto 2.0.1 - ORC-1277 Bump maven-shade-plugin to 3.4.0
- ORC-1284 Add permissions to GitHub Action labeler
- ORC-1296 Bump reproducible-build-maven-plugin to 0.16
- ORC-1311 Bump maven-shade-plugin to 3.4.1
- ORC-1316 Bump slf4j.version to 2.0.4
- ORC-1334 Bump slf4j.version to 2.0.6
- ORC-1335 Bump netty-all to 4.1.86.Final
- ORC-1351 Update PR Labeler definition
- ORC-1358 Use spotless to format pom files
- ORC-1371 Remove unsupported SLF4J bindings from classpath
- ORC-1372 Bump
zstdto v1.5.4 - ORC-1375 Cancel old running ci tasks when a pr has a new commit
- ORC-1377 Enforce override keyword
- ORC-1383 Upgrade
aircompressorto 0.22 - ORC-1395 Enforce license check
- ORC-1396 Bump
slf4jto 2.0.7 - ORC-1410 Bump
zstdto v1.5.5 - ORC-1411 Remove Ubuntu18.04 from docker-based tests
- ORC-1419 Bump
protobuf-javato 3.22.3 - ORC-1428 Setup GitHub Action CI on
branch-1.9 - ORC-1443 Enforce Java version
- ORC-1444 Enforce JDK Bytecode version
- ORC-1446 Publish snapshot from branch-1.9
Test
- ORC-1231 Update supported OS list in
building.md - ORC-1233 Bump
junitto 5.9.0 - ORC-1234 Upgrade
objenesisto 3.2 in Spark benchmark - ORC-1235 Bump
avroto 1.11.1 - ORC-1240 Update site README to use apache/orc-dev
- ORC-1241 Use
apache/orc-devDockerHub repository in Docker tests - ORC-1250 Bump
mockitoto 4.7.0 - ORC-1254 Add
spotbugscheck - ORC-1258 Bump
byte-buddyto 1.12.14 - ORC-1262 Bump
maven-checkstyle-pluginto 3.2.0 - ORC-1265 Upgrade
spotbugsto 4.7.2 - ORC-1267 Bump
mockitoto 4.8.0 - ORC-1271 Bump
spotbugs-maven-pluginto 4.7.2.0 - ORC-1272 Bump
byte-buddyto 1.12.16 - ORC-1300 Update Spark to 3.3.1 and its dependencies
- ORC-1303 Upgrade
GoogleTestto 1.12.1 - ORC-1318 Upgrade mockito.version to 4.9.0
- ORC-1319 Upgrade byte-buddy to 1.12.19
- ORC-1321 Bump checkstyle to 10.5.0
- ORC-1322 Upgrade centos7 docker image to use gcc9
- ORC-1324 Use Java 19 instead of 18 in GHA
- ORC-1333 Bump
mockitoto 4.10.0 - ORC-1341 Bump
mockitoto 4.11.0 - ORC-1353 Bump
byte-buddyto 1.12.21 - ORC-1359 Bump
byte-buddyto 1.12.22 - ORC-1366 Bump
checkstyleto 10.7.0 - ORC-1367 Bump
maven-enforcer-pluginto 3.2.1 - ORC-1369 Bump
byte-buddyto 1.12.23 - ORC-1370 Bump
snappy-javato 1.1.9.1 - ORC-1374 Update Spark to 3.3.2
- ORC-1378 Add slf4j impl to avoid warning message in example module
- ORC-1379 Upgrade
spotbugsto 4.7.3.2 - ORC-1380 Upgrade
checkstyleto 10.8.0 - ORC-1394 Bump
maven-assembly-pluginto 3.5.0 - ORC-1397 Bump
checkstyleto 10.9.2 - ORC-1405 Bump
spotbugs-maven-pluginto 4.7.3.4 - ORC-1406 Bump
maven-enforcer-pluginto 3.3.0 - ORC-1408 Add
testVectorBatchHasNulltest case and comment - ORC-1415 Add Java 20 to GitHub Action CI
- ORC-1417 Bump
checkstyleto 10.10.0 - ORC-1418 Bump
junitto 5.9.3 - ORC-1426 Use Java
21-eainstead of 20 in GitHub Action - ORC-1435 Bump
maven-checkstyle-pluginto 3.3.0 - ORC-1436 Bump
snappy-javato 1.1.10.0 - ORC-1452 Use the latest OS versions in variant tests
v1.8.4
Milestone
Changelog
Bug
ORC-1304: [C++] Fix seeking over empty PRESENT stream
ORC-1400: Use Hadoop 3.3.5 on Java 17+ and benchmark
ORC-1413: Fix for ORC row level filter issue with ACID table
Test:
ORC-1404 Bump parquet to 1.13.0
ORC-1414 Upgrade java bench module to spark3.4
ORC-1416 Upgrade Jackson dependency to 2.14.2 in bench module
ORC-1420 Pin net.bytebuddy package to 1.12.x
Task:
ORC-1395 Enforce license check via github action
v1.7.9
v1.8.3
3530d78
This commit was signed with the committer’s verified signature.
dongjoon-hyun
Dongjoon Hyun
Milestone
Changelog
Bug
- ORC-1357 Handle missing compression block size
- ORC-1382 Fix secondary config names org.sarg.* to orc.sarg.*
- ORC-1384 Fix ArrayIndexOutOfBoundsException when reading dictionary stream bigger then dictionary
- ORC-1393 Add reset(DiskRangeList input, long length) to InStream impl class
Test
Tasks
- ORC-1358 Use spotless to format pom files
v1.7.8
7072f2b
This commit was signed with the committer’s verified signature.
williamhyun
William Hyun
v1.8.2
74135da
This commit was signed with the committer’s verified signature.
dongjoon-hyun
Dongjoon Hyun
Milestone
Changelog
Bug
Improvement
- ORC-1327 Exclude the proto files from the nohive jar
- ORC-1328 Exclude the proto files from the shaded protobuf jar
- ORC-1329 Add
OrcConf.getStringAsListmethod - ORC-1338 Set bloom filter fpp to 1%
- ORC-1342 Publish SBOM artifacts
- ORC-1344 Skip SBOM generation during CMake
- ORC-1345 Use
makeBomand skip snapshot check in GitHub Actionpublish_snapshotjob
Test
- ORC-1323 Make
docker/reinit.shsupport target OS arguments - ORC-1330 Add
TestOrcConf - ORC-1339 Remove orc.sarg.to.filter default value assumption in test cases
- ORC-1350 Upgrade
setup-javato v3
Tasks
v1.8.1
1064b86
This commit was signed with the committer’s verified signature.
dongjoon-hyun
Dongjoon Hyun
Milestone
Changelog
Bug
- ORC-1283 ENABLE_INDEXES does not take effect
- ORC-1288 Invalid memory freeing with ZLIB compression
- ORC-1291 NullPointerException in TypeDescription
Improvement
- ORC-1268 Set CMP0135 policy for CMake 3.24+
- ORC-1282 Add slf4j impl to avoid warning message
- ORC-1294 Build error when skip tests build
- ORC-1295 Improve ORC Spec example (Decoding RLE v2 direct)
- ORC-1299 benchmark can't work for data resource 403
- ORC-1305 Add more orc java examples
- ORC-1308 Avoid star import
Test
Tasks
v1.7.7
v1.8.0
f2bb463
This commit was signed with the committer’s verified signature.
williamhyun
William Hyun
Milestone
Changelog
New Feature and Notable Changes
- ORC-450 Support selecting list indices without materializing list items
- ORC-824 Add column statistics for List and Map
- ORC-1004 Java ORC writer supports the selection vector
- ORC-1075 Support reading ORC files with no column statistics
- ORC-1125 Support decoding decimals in RLE
- ORC-1136 Optimize reads by combining multiple reads without significant separation into a single read
- ORC-1138 Seek vs Read Optimization
- ORC-1172 Add row count limit config for one stripe
- ORC-1212 Upgrade protobuf-java to 3.17.3
- ORC-1220 Set min.hadoop.version to 2.7.3
- ORC-1248 Redefine Hadoop dependency for Apache ORC 1.8.0
- ORC-1256 Publish test-jar to maven central
- ORC-1260 Publish
shaded-protobufclassifier artifacts
Improvement
- ORC-825 Use Empty Array For Collections toArray
- ORC-826 Do Not Use Collection Contains/Get
- ORC-828 Improve Fetch Data Set Process
- ORC-829 Optimize Serialization percentileBits
- ORC-831 Do Not Copy String When Flushing Dictionary
- ORC-833 RunLengthIntegerReaderV2 Calculate Batch Size Once
- ORC-834 Do Not Convert to String in DecimalFromTimestampTreeReader
- ORC-835 Cache TRUE/FALSE Bytes in StringGroupFromBooleanTreeReader
- ORC-836 StringGroupFromDoubleTreeReader Use Double toString
- ORC-837 Reuse HiveDecimalWritable in ConvertTreeReaderFactory
- ORC-838 Simplify compareTo/equals/putBuffer of ByteBufferAllocatorPool
- ORC-840 Remove Superfluous Array Fill in RecordReaderImpl
- ORC-841 Remove Superfluous Array Fill in StringHashTableDictionary
- ORC-842 Remove newKey from StringHashTableDictionary
- ORC-844 Improve hashCode Methods
- ORC-847 Do Not Create Empty Array in StringGroupFromBinaryTreeReader
- ORC-852 Allow DynamicByteArray to Return a ByteBuffer
- ORC-853 Optimize writeDouble Implementation
- ORC-855 Remove Unused isRepeating from RunLengthIntegerReaderV2
- ORC-865 Bump opencsv from 3.9 to 5.5.1
- ORC-883 Dependency Audit and QA
- ORC-897 optimization loop termination condition in readerIsCompatible method
- ORC-935 Bump commons-csv from 1.8 to 1.9.0
- ORC-937 Replace deprecated method
- ORC-958 Convert command support overwrite option
- ORC-969 Evaluate SearchArguments using file and stripe level stats
- ORC-975 Avoid double counting closestFixedBits in percentileBits method
- ORC-982 Extract checkstyle to a single file, help newcomers check code style
- ORC-988 Bump opencsv from 5.5.1 to 5.5.2
- ORC-992 Reached max repeat length, we can directly decide to use DELTA encoding
- ORC-1005 Make that the java and C++ implementations of determineEncoding in RunLengthIntegerWriterV2 are consistent.
- ORC-1007 Fix a warning from the shade plugin
- ORC-1013 Renaming a parameter in constructors of TreeWriter's derived classes
- ORC-1014 Add details when we get IOExceptions from file system
- ORC-1020 Improve orc::RleDecoderV2::nextDirect
- ORC-1027 Filter processing to allow filter injections that cannot be represented via SArgs
- ORC-1047 Handle quoted field names during string schema parsing
- ORC-1077 Remove commons-codec dependency and use java.util.Base64
- ORC-1099 Extend ReadIntent to support MAP and UNION type
- ORC-1101 Improve malformed STRUCT handling
- ORC-1122 Add buffer to decode the whole run in RleDecoderV2
- ORC-1137 Improve float/double conversion in DoubleColumnReader::next()
- ORC-1149 Bump slf4j.version to 1.7.36
- ORC-1150 Improve RowReaderImpl::computeBatchSize()
- ORC-1152 Support encoding short decimals in RLEv2
- ORC-1156 Update opencsv to 5.6
- ORC-1163 Bump zookeeper from 3.7.0 to 3.8.0
- ORC-1169 Use Hadoop 3.3.2 on Java 17+
- ORC-1178 Use hadoop 3.3.3 on Java 17+
Bug
- ORC-845 Fix NPE in DynamicIntArray toString
- ORC-929 Fix NaN at orc-tools 'meta' command
- ORC-1129 The build of tool-test should depends on cpp tools
- ORC-1159 Crash when the last stripe is skipped
- ORC-1242 Bump threeten-extra to 1.7.1
Test
- ORC-860 Add dependabot
- ORC-864 Bump jackson.version from 2.12.2 to 2.12.4
- ORC-877 Bump junit-vintage-engine from 5.7.0 to 5.7.2
- ORC-888 Bump objenesis from 3.1 to 3.2
- ORC-905 Add an integration test for
example - ORC-917 Bump mockito-core from 3.7.0 to 3.11.2
- ORC-919 Spark bench objenesis should be the same as Spark.
- ORC-920 Use junit.version and mockito.version property and bump junit to 5.7.2
- ORC-925 Simplify assertions
- ORC-928 Bump checkstyle from 8.44 to 8.45.1
- ORC-932 Bump byte-buddy from 1.10.19 to 1.11.12
- ORC-934 Add integration tests for Java bench
- ORC-940 Use Hadoop 3.3.1 in bench module
- ORC-955 Add Javadoc generation GitHub Action job
- ORC-963 Build
benchmarkmodule always for integration testing - ORC-966 Bump byte-buddy from 1.11.12 to 1.11.13
- ORC-967 Bump mockito.version from 3.11.2 to 3.12.1
- ORC-986 Bump mockito.version from 3.12.1 to 3.12.4
- ORC-987 Bump jackson.version from 2.12.4 to 2.12.5
- ORC-1001 Bump maven-enforcer-plugin to 3.0.0
- ORC-1019 Remove redundant jackson dependencies
- ORC-1022 Bump byte-buddy from 1.11.13 to 1.11.19
- ORC-1038 Bump mockito.version from 3.12.4 to 4.0.0
- ORC-1074 Bump byte-buddy from 1.11.19 to 1.12.6
- ORC-1079 Add Linux clang GitHub Action job
- ORC-1085 Bump auto-service from 1.0 to 1.0.1
- ORC-1089 Add test cases verifying writers with selected vector
- ORC-1104 Use Spark 3.2.1 in benchmark
- ORC-1107 Fix NPE at benchmark data schema loading
- ORC-1110 Bump mockito.version from 4.0.0 to 4.3.1
- ORC-1126 Bump byte-buddy from 1.12.6 to 1.12.8
- ORC-1139 Benchmark for Seek vs Read
- ORC-1141 Bump mockito.version from 4.3.1 to 4.4.0
- ORC-1145 Add Java 18 to GitHub Action CI.
- ORC-1153 Bump byte-buddy from 1.12.8 to 1.12.9
- ORC-1157 Update guava to 31.1-jre
- ORC-1168 Update byte-buddy to 1.12.10
- ORC-1177 Upgrade mockito.version to 4.5.1
- ORC-1179 Upgrade checkstyle to 10.2 on Java 11+
- ORC-1187 Use main instead of master in merge_orc_pr.py
- ORC-1194 Bump mockito.version to 4.6.0
- ORC-1195 Bump checkstyle to 10.3
- ORC-1196 Add spark benchmark integration tests to GHA
- ORC-1197 Bump mockito.version from 4.6.0 to 4.6.1
- ORC-1201 Remove Debian 9 from Docker Tests
- ORC-1203 Bump maven-enforcer-plugin to 3.1.0
- ORC-1206 Bump netty-all to 4.1.78.Final
- ORC-1207 Upgrade Spark to 3.3.0
- ORC-1208 Bump byte-buddy to 1.12.12
- ORC-1209 Bump checkstyle to 10.3.1
- ORC-1234 Upgrade objenesis to 3.2 in Spark benchmark
- ORC-1236 Bump checkstyle to 10.3.2
- ORC-1243 Bump byte-buddy to 1.12.13
- ORC-1253 Add Fedora 37 docker test
- ORC-1254 Add spotbugs check
Task
- ORC-868 Pin gson to 2.2.4
- ORC-869 Pin jmh 1.20
- ORC-872 Bump kryo-shaded from 3.0.3 to 4.0.2
- ORC-874 Bump zookeeper from 3.6.2 to 3.7.0
- ORC-884 Bump jettison from 1.1 to 1.4.1
- ORC-887 Remove ORC Twitter link from
newspage - ORC-890 Pin minimum support Hadoop version to 2.2.0
- ORC-892 Pin scala-library to 2.12.10
- ORC-898 Bump threeten-extra from 1.5.0 to 1.7.0
- ORC-899 Archive Apache ORC 1.4.x in
releasespage - ORC-900 Update doap_orc.rdf for Apache Projects page
- ORC-908 Use https instead of http for website links in
pom.xml - ORC-914 Pin maven-dependency-plugin to 3.1.2
- ORC-916 Bump annotations from 17.0.0 to 21.0.1
- ORC-918 Pin protobuf-java to 2.5.0
- ORC-923 Bump apache from 23 to 24
- ORC-946 Unified json library
- ORC-949 Add CustomImportOrder rule
- ORC-956 Bump annotations from 21.0.1 to 22.0.0
- ORC-977 Update webpages and TestVectorOrcFile.java to be more neutral
- ORC-1045 Bump commons-cli to 1.5
- ORC-1056 Bump annotations from 22.0.0 to 23.0.0
- ORC-1103 Use Maven 3.8.4
- ORC-1140 Documentation for Seek vs Read
- ORC-1158 Add notification settings to .asf.yam
- ORC-1162 Fix Apache Project Website Checks Warningl
- ORC-1165 Enable GitHub Action in branch-1.8
- ORC-1166 Enable snapshot publishing in branch-1.8
- ORC-1171 Skip build and test on docker and site updates
- ORC-1173 Pin jodd-core to 3.5.2
- ORC-1176 Upgrade maven-jar-plugin to 3.2.2
- ORC-1185 Add merge_orc_pr.py
- ORC-1210 Upgrade maven to 3.8.6
- ORC-1216 Pin org.jetbrains.annotations dependency to 17.0.0
- ORC-1211 Upgrade maven-assembly-plugin to 3.4.0
- ORC-1214 Bump maven-assembly-plugin to 3.4.1
- ORC-1217 Downgrade org.jetbrains.annotations to 17.0.0
- ORC-1223 Move DirectDecompressWrapper to org.apache.orc.impl
- ORC-1224 Move getDecompressor to HadoopShimsCurrent
- ORC-1226 Add a deprecation warning for Hadoop 2.7.2 and below
- ORC-1229 Move KeyProviderImpl to org.apache.orc.impl
- ORC-1230 Move encryption utility functions to HadoopShimsCurrent
- ORC-1246 Revamp ORC Website
- ORC-1247 Improve Apache ORC website and docs
- ORC-1249 Move site/_docs/releases.md to site/releases/index.md
- ORC-1255 Fix ORC website navbar highlight
- ORC-1257 Publish multi-architecture ORC-dev docker images
- ORC-1261 Rename shaded pattern
com.google.protobuf25toorg.apache.orc.protobuf - ORC-1263 Add decimal type to ORC Website
- ORC-1221 Move NullKeyProvider to org.apache.orc.impl