Optimize Hive Connector ORC Readers and Filter using Java Vector API by AbhijitKulkarni1 · Pull Request #21670 · prestodb/presto

AbhijitKulkarni1 · 2024-01-10T19:38:17Z

Description

Enable Presto to be compatible with JDK 21.0.3 LTS version. Related to #16268
Optimize Hive connector to harness data parallelism in ORC Readers and Filters using Java Vector API

Motivation and Context

This PR enables Presto to be used with latest features available in JDK 21.0.3 LTS version like Java Vector API and Foreign API updates.

Vectorization of ORC Readers:

Presto Hive Connector for Presto ORC module is optimized with vectorized Reader to harness data parallelism using Java Vector API. Hive Connector is updated to provide hive property "orc-use-vector-filter" to use the optimized Presto ORC to read ORC format data in vectorized way.
Following are the scope of optimizations in this PR to start with:

Optimized Double data type ORC readers to read and filter Uncompressed ORC data in vectorized way. Other data types will follow in future.
Optimization is applicable for Uncompressed, Unencrypted ORC data read and filter. For Compressed and Encrypted data support, logic of processing data in OrcInputStream need to be simplified to decouple uncompressed and compressed data processing.
Optimization is applicable for first level filter where we have contiguous positions available for filtering and is very appropriate for vectorization.

Class Design:
Presto Double Filter:

Class DoubleRange implements the interface TupleDomainFilter having the scalar version of filtering double values for certain range as part of testDouble() method.
DoubleRangeVector class is a newly introduced class for implementation of new interface TupleDomainFilterVector, providing vectorized implementation of filtering double values for certain range as part of testDoubleVector() method.

Presto Double Reader:

DoubleSelectiveStreamReader class is the scalar reading implementation where double values are read one after another and testDouble() method is called for filtering.
As shown in above figure, the vectorization design extends the DoubleSelectiveStreamReader class with a new DoubleSelectiveStreamVectorReader class and further provides interfaces to write vectorized reading through a new DoubleVectorSelectiveReadAndFilter interface.
The implementation of this interface is the DoubleVectorSelectiveReadAndFilterSpecies512 class, which provides the vectorized reading for bulk data needed for vectorized filtering using testDoubleVector() method.

This design is crafted with the solution's extensibility in mind to provide implementations for other AVX bit width too, just by implementing the DoubleVectorSelectiveReadAndFilter interface.

Note the following points about this optimization.

Updates are compatible with JDK 21.0.3 version. So use jdk 21.0.3 for compilation and execution of Presto binary.
The JDK 21.0.3 compilation for vector updates in presto-common, presto-orc modules are handled using maven profiles. The updates are backward compatible with the lower version JDK compilation (JDK 8 and 11) and do not impact at runtime with lower JRE execution.
Note the following points for compilation and launch of Presto using JDK 21.0.3

i. Compilation:

export MAVEN_OPTS="--add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.text=ALL-UNNAMED --add-opens java.base/java.awt.font=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-modules=jdk.incubator.vector --enable-preview"
./mvnw clean install -DskipTests

ii. Runtime in Presto "etc\jvm.config" file set the following jvm options

--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
-Djdk.reflect.useDirectMethodHandle=false
--add-modules=jdk.incubator.vector
--enable-preview

Hive Connector Changes
Addition of hive property "orc-use-vector-filter" to enable/disable the use of vector ORC readers in Hive connector. If set to "true" in hive.properties file (hive.orc-use-vector-filter="true"), new vectorized readers and filters are used for compilation/execution. Default being "false" for current non vectorized flow.

Impact

JDK21 specific maven profiles are added to update the versions of dependencies/plugins as required for Presto to be compatible with JDK 21.0.3 LTS version. These updates do not impact the default compilation with JDK8 or 11 which is officially supported.

No functional impact for execution of Presto with JDK8 or 11 which are officially supported.
Presto execution with JDK 21.0.3, Improved Hive Connector for ORC Readers and Filters with vectorization using Java Vector API for Uncompressed ORC data reading and filtering.
Please find the claims of throughput (queries execution per unit time) improvement as below:

Throughput of Presto execution with JDK 21.0.3 LTS version is ~1.24X higher than the execution of Unmodified Presto with JDK 11.0.22
Throughput of Vector Modified Presto execution with JDK 21.0.3 LTS version is ~1.38X (~1.24X from JDK 21.0.3 LTS version and ~1.12X from vectorization updates) higher than Unmodified Presto execution with JDK 11.0.22.
Observed above data points on SF100 Uncompressed TPCH data in ORC format executed with Presto on one of the socket of m7i.metal-48xl aws bare metal machine.
Query Used for Evaluation is similar to q01 and q06 filter queries of TPCH. Query used to evaluate the stress filtering of double data is:

SELECT COUNT(l.extendedprice) FROM hive.sf100.lineitem AS l WHERE  l.extendedprice < 20000;

Throughput is measured for 100 concurrent executions of the query.

Test Plan

Test all smoke test suites
Added unit tests to verify the vectorization updates and performed test coverage.

Contributor checklist

Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
Documented new properties (with its default value), SQL syntax, functions, or other functionality.
If release notes are required, they follow the release notes guidelines.
Adequate tests were added if applicable.
CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==
General Changes
* Update Presto to be compatible with JDK 21.0.3 LTS version :pr: `21760`

Hive Connector Changes
* Add configuration property  ``orc-use-vector-filter`` to enable or disable the use of vector ORC readers :pr: `21760`

linux-foundation-easycla · 2024-01-10T19:38:21Z

The committers listed above are authorized under a signed CLA.

✅ login: AbhijitKulkarni1 (d50a8c3, 09ca2ff, f5f0a66, 120e88c, 787ceb1, 70f9914, e12162a, b93e744, ee45576, 2228940, d94ea54, 213e86c)

tdcmeehan · 2024-03-18T14:01:18Z

@AbhijitKulkarni1 this would be a very nice and exciting change. I am wondering if you need any help?

steveburnett · 2024-03-18T19:42:27Z

Suggest a change to the current release notes entry: perhaps this draft describes the effect of this PR?

== RELEASE NOTES ==
General Changes
* Improve Presto compatibility with JDK 19.0.2. 

Hive Connector Changes
* Add configuration property  `orc-use-vector-filter` to enable or disable the use of vector ORC readers.

AbhijitKulkarni1 · 2024-03-19T09:00:43Z

@tdcmeehan Thanks for reaching out. We are in process of internal data review and would be able to submit PR for review shortly. Will surely connect with you in case I need any help.

steveburnett · 2024-06-10T19:22:59Z

Two items:

Please add documentation for orc-use-vector-filter - probably in Hive Configuration Properties.
Suggest a change to the current release notes entry: perhaps this draft describes the effect of this PR?

== RELEASE NOTES ==
General Changes
* Improve Presto compatibility with JDK 19.0.2 :pr: `21760`

Hive Connector Changes
* Add configuration property  ``orc-use-vector-filter`` to enable or disable the use of vector ORC readers :pr: `21760`

AbhijitKulkarni1 · 2024-06-11T03:32:01Z

@steveburnett Can you please let me know about the second point? Do you mean to add one liner release notes and move the details in other section?

Suggest a change to the current release notes entry: perhaps this draft describes the effect of this PR?

steveburnett · 2024-06-11T15:11:22Z

Good question, @AbhijitKulkarni1!

I suggest that each release note entry should be a summary of the work done in the PR. The summary version allows a reader of the new Release Notes to quickly identify if an entry is something they need more detail on. If not, they can move on to the next entry. You could examine the recent release notes, for example the 0.287 Release Notes, for examples.

Because each release note entry ends with a link to the PR it describes - for example, if the reader wants to know what Improve Presto compatibility with JDK 19.0.2 means - they can select the link to the PR and find that explanation in the Description part of your PR.

I suggest you move much of the content you have in your release note draft into the Description of your PR.

Each individual item does not have to be only a single line: I attempted to summarize each of yours at that high level, but you should revise my suggested draft and include detail you feel must be present in the release notes.

For PRs that include documentation changes, a link can be added to the release note entry that links directly to the new documentation.
See Formatting in the Release Notes Guidelines for how to code such a link.
For a specific example, see the 0.287 Release Notes and the release note entry:

"* Add configuration property legacy_json_cast whose default value is true. See Legacy Compatible Properties. #21869"

I hope this helps to answer your question, and please ask anything else you would like to know.

AbhijitKulkarni1 · 2024-06-12T03:26:43Z

Thanks @steveburnett for the guidance. I will add the property to Hive Configurations property file hive.rst. For release notes can you please clarify if they are picked up from PR automatically during release preparation or I need to add it somewhere because I didn't find the file release-0.288.rst, Kindly let me know.

steveburnett · 2024-06-12T14:15:50Z

Of course! The release note entries are picked up from the PRs automatically during release preparation - if you have a correctly formatted block in your PR for your release note entries, that's all you need to do.
The automatically collected release note entries are reviewed and revised to create the .rst file.
My suggestions hope to reduce the work of the reviewers preparing the next release notes, which can help make the Presto release process easier and maybe even faster as well.

If you're interested in how the release notes are created for a Presto release, you're welcome to take a look at #22647.

presto-orc/src/main/java/com/facebook/presto/orc/OrcSelectiveRecordReader.java

ZacBlanco · 2024-06-12T18:37:12Z

First off, awesome work. I'm sure this was difficult to get working properly. Upgrading Java versions can be tricky, especially for a large legacy system like Presto that has barely migrated off of 8. One concern I have though is with the choice of Java version. Java 19 is not an LTS release and support for this JVM version has already ended. It will not be receiving any further updates: https://javaalmanac.io/jdk/

It would be really great if this could target an LTS release. 21 is the latest LTS release and EOL is in 2028. I did a quick search and it seems there were some API changes between 19 and 21 in the Vector API. I'm not sure what was behind the decision to target Java 19, but it would be good to know the motivation. Not sure what others think about the Java version, but I think using LTS is desirable.

steveburnett

One formatting error that I fixed locally so I could review the content. The content is good, just fix the table entry and I'm happy to approve.

presto-docs/src/main/sphinx/connector/hive.rst

AbhijitKulkarni1 · 2024-06-13T11:05:25Z

First off, awesome work. I'm sure this was difficult to get working properly. Upgrading Java versions can be tricky, especially for a large legacy system like Presto that has barely migrated off of 8. One concern I have though is with the choice of Java version. Java 19 is not an LTS release and support for this JVM version has already ended. It will not be receiving any further updates: https://javaalmanac.io/jdk/

It would be really great if this could target an LTS release. 21 is the latest LTS release and EOL is in 2028. I did a quick search and it seems there were some API changes between 19 and 21 in the Vector API. I'm not sure what was behind the decision to target Java 19, but it would be good to know the motivation. Not sure what others think about the Java version, but I think using LTS is desirable.

Thanks for the compliments @ZacBlanco . Ensuring Presto worked with the latest Java was indeed a tough task. We started with compatibility explorations and considered Java 17 LTS, 18 for vectorized filtering with shuffle and rearrange Vector APIs to start with but chose Java 19 for "compress" Vector API (https://openjdk.org/jeps/426), which does shuffle and rearrange in one API call and performs better. I'm of the opinion that this solution should be compatible with Java 21 LTS; however, I must confirm its compatibility before moving forward with the submission.

steveburnett

LGTM! (doc)

Pull updated branch, new doc build, looks good!

github-actions · 2024-10-01T07:02:52Z

Codenotify: Notifying subscribers in CODENOTIFY files for diff 1a8c27d...787ceb1.

No notifications.

… vectorization-v1 # Conflicts: # presto-docs/src/main/sphinx/connector/hive.rst

…resto into vectorization-v1 # Conflicts: # presto-docs/src/main/sphinx/connector/hive.rst

…l. Add the hive.orc-use-vector-filter configuration

zhenxiao

Nice work
do we need to update documentation to change the compilation page?

zhenxiao · 2024-10-01T20:59:18Z

presto-common/src/main/java/com/facebook/presto/common/predicate/vector/DoubleRangeVector.java

+import static jdk.incubator.vector.VectorOperators.LT;
+import static jdk.incubator.vector.VectorOperators.NE;
+
+public class DoubleRangeVector


how about BigintRange and FloatRange, etc?

zhenxiao · 2024-10-01T21:03:09Z

hi @AbhijitKulkarni1 nice work! So glad to see latest java version upgrade
could you please make the PR on top of current master, and break it into separate commits, one commit for java upgrade, another one for vector api. And also make corresponding changes in the compilation documentation?
CC @beinan @ChunxuTang

rschlussel

Thanks for this change!

Can you please clean up the commits and break it into 2 commits

changes required for java > 19 support
Orc vector changes

I do feel a bit worried about how this change will affect development, and would love to hear thoughts about it. Will we need to set our java version in intellij to java 19? will it make it hard for people to avoid using features after java8 until we upgrade our required java version for the whole project?

rschlussel · 2024-10-02T16:18:42Z

presto-common/src/main/java/com/facebook/presto/common/predicate/TupleDomainFilter.java

    {
-        private final double lower;
-        private final double upper;
+        protected final double lower;


probably better to keep these fields private and access via getters.

rschlussel · 2024-10-02T16:20:16Z

presto-docs/src/main/sphinx/connector/hive.rst

 ``hive.skip-empty-files``                          Enable skipping empty files. Otherwise, it will produce an   ``false``
                                                   error iterating through empty files.
+
+``hive.orc-use-vector-filter``                     Enable use of vector ORC readers in compilation and execution ``false``


add a note that this requires jdk 19?

rschlussel · 2024-10-02T16:21:25Z

presto-hive/src/main/java/com/facebook/presto/hive/HiveClientConfig.java

    }
+
+    @Config("hive.orc-use-vector-filter")
+    @ConfigDescription("Experimental: enable vector path for orc filters")


note that this requires jdk 19. Also, would probably remove "Experimental". Notes like that tend to just stick around forever. Would be great to add validation that the jdk meets the minimum version and fail with invalid_configuration_property otherwise.

rschlussel · 2024-10-02T16:22:10Z

presto-hive/src/main/java/com/facebook/presto/hive/HiveSessionProperties.java

                        false),
+                booleanProperty(
+                        ORC_USE_VECTOR_FILTER,
+                        "Experimental: enable vector path for orc filters",


same comment about noting the jdk requirement and probably removing "experimental". Would be great to add validation that the jdk meets the minimum version if you set this, and fail with INVALID_SESSION_PROPERTY otherwise.

rschlussel · 2024-10-02T16:28:53Z

presto-orc/src/main/java/com/facebook/presto/orc/reader/SelectiveStreamReaders.java

+                        outputType.isPresent(),
+                        systemMemoryContext.newOrcLocalMemoryContext(SelectiveStreamReaders.class.getSimpleName()));
+            }
+            catch (ClassNotFoundException | NoSuchMethodException | InstantiationException | IllegalAccessException | InvocationTargetException e) {


we should fail loudly here.

Can you please help me know ? And BTW there is old flow available if the incubator classes are not available during runtime.

I mean to say instead of falling back if the classes aren't available, it would be better to fail the query so people know they have misconfigured presto.

rschlussel · 2024-10-02T16:45:06Z

presto-orc/src/main/java/com/facebook/presto/orc/reader/DoubleSelectiveStreamReader.java

-    private final boolean nonDeterministicFilter;
-    private final boolean nullsAllowed;
-    private final boolean outputRequired;
+    protected final TupleDomainFilter filter;


I don't love the idea of exposing all of these internal details to the whole package. I wonder if we can have a "FilterReader function" that gets passed into this class instead and that gets called in readWithFilter, and when the vector reader ins enable it's the VectorFilterReader and a DefaultFilterReader. then we could pass any required fields as function arguments?

rschlussel · 2024-10-02T16:48:02Z

presto-common/src/main/java/com/facebook/presto/common/predicate/vector/DoubleRangeVector.java

+import static jdk.incubator.vector.VectorOperators.NE;
+
+public class DoubleRangeVector
+        extends TupleDomainFilter.DoubleRange


what do we get out of extending DoubleRange? Seems like we don't use any of the DoubleRange functionality. We should add those fields directly to DoubleRangeVector instead and make it independent of DoubleRange.

DoubleRangeVector is an extention of the scalar equivalent and in the vector processing logic for handling the scalar tail part of execution we need the DoubleRange functionality to filter in scalar way.

rschlussel · 2024-10-02T16:49:12Z

presto-common/src/main/java/com/facebook/presto/common/predicate/vector/DoubleRangeVector.java

+        extends TupleDomainFilter.DoubleRange
+        implements TupleDomainFilterVector
+{
+    protected DoubleRangeVector(double lower, boolean lowerUnbounded, boolean lowerExclusive, double upper, boolean upperUnbounded, boolean upperExclusive, boolean nullAllowed)


Can this be private?

rschlussel · 2024-10-02T16:53:01Z

presto-common/pom.xml

+
+    <profiles>
+        <profile>
+            <id>java21PrestoCommon</id>


just curious why the build profiles are named differently for every module vs. all having a profile called java21 and javaLessThan21?

rschlussel · 2024-10-02T16:53:33Z

presto-orc/pom.xml

+            </activation>
+            <dependencies>
+                <dependency>
+                    <groupId>org.mockito</groupId>


We don't use mocks. Please create stubbed out classes instead.

Could you kindly direct me to an example within Presto? That would be very helpful.

you can look at any classes named TestingXXX as an example. We create testing implementations of interfaces that follow the needs of a test. Can be simple or complex as needed. Here are a few examples, and you can see how they are used in tests: https://github.com/prestodb/presto/blob/master/presto-main/src/main/java/com/facebook/presto/testing/TestingMetadata.java, https://github.com/prestodb/presto/blob/master/presto-spi/src/test/java/com/facebook/presto/spi/ttl/TestingNodeTtlFetcher.java, https://github.com/prestodb/presto/blob/master/presto-spi/src/test/java/com/facebook/presto/spi/TestingColumnHandle.java, https://github.com/prestodb/presto/blob/master/presto-main/src/main/java/com/facebook/presto/testing/TestingAccessControlManager.java

AbhijitKulkarni1 · 2024-10-03T04:43:52Z

Thanks for this change!

Can you please clean up the commits and break it into 2 commits

changes required for java > 19 support

Orc vector changes

I do feel a bit worried about how this change will affect development, and would love to hear thoughts about it. Will we need to set our java version in intellij to java 19? will it make it hard for people to avoid using features after java8 until we upgrade our required java version for the whole project?

The updates are handled such a way that it does not affect the compilation and execution for lower java version. This is done with specific maven profiles for JDK 21 during compile time. Also for Hive connector execution, the Vector API code is executed only if the flag 'hive.orc-use-vector-filter' is set to true and we have the Vector API support from Java runtime. So these updates do support backward compatibility.

wanglinsong · 2024-10-03T05:46:17Z

Do we need to update other test workflows besides .github/workflows/hive-tests.yml?
How do we make sure other core functionalities work correctly with jdk21?

tdcmeehan self-assigned this Jan 11, 2024

AbhijitKulkarni1 changed the base branch from master to release-0.283-edge3 January 11, 2024 19:00

AbhijitKulkarni1 changed the base branch from release-0.283-edge3 to master January 11, 2024 19:00

AbhijitKulkarni1 force-pushed the vectorization-v1 branch from e567301 to 0ecca0c Compare June 12, 2024 05:58

AbhijitKulkarni1 changed the title ~~[WIP] Optimize Hive Connector ORC Readers and Filter using Java Vector API~~ Optimize Hive Connector ORC Readers and Filter using Java Vector API Jun 12, 2024

AbhijitKulkarni1 marked this pull request as ready for review June 12, 2024 15:31

AbhijitKulkarni1 requested review from a team, sdruzkin and steveburnett as code owners June 12, 2024 15:31

AbhijitKulkarni1 requested a review from presto-oss June 12, 2024 15:31

sdruzkin reviewed Jun 12, 2024

View reviewed changes

presto-orc/src/main/java/com/facebook/presto/orc/OrcSelectiveRecordReader.java Outdated Show resolved Hide resolved

wanglinsong self-requested a review June 12, 2024 16:41

steveburnett requested changes Jun 12, 2024

View reviewed changes

presto-docs/src/main/sphinx/connector/hive.rst Outdated Show resolved Hide resolved

AbhijitKulkarni1 added 3 commits June 17, 2024 11:58

JDK19 compatibility changes

e12162a

Optimize Hive Connector ORC Readers and Filters using Java Vector API

213e86c

Review comments fix and Support for Java 21 LTS version

ee45576

AbhijitKulkarni1 force-pushed the vectorization-v1 branch from 0ecca0c to ee45576 Compare June 20, 2024 16:38

steveburnett previously approved these changes Jun 20, 2024

View reviewed changes

Merge branch 'prestodb:master' into vectorization-v1

09ca2ff

lucagiac81 mentioned this pull request Jul 2, 2024

Support Java 16 #16268

Open

AbhijitKulkarni1 dismissed steveburnett’s stale review via fe30141 October 1, 2024 07:01

AbhijitKulkarni1 requested review from a team, 7c00, agrawalreetika, elharo, feilong-liu, hantangwangd, jaystarshot, shangxinli, shrinidhijoshi, vinothchandar, vivek-bharathan and yhwang as code owners October 1, 2024 07:01

AbhijitKulkarni1 added 2 commits October 1, 2024 15:09

Merge branch 'master' of https://github.com/intel-staging/presto into…

d94ea54

… vectorization-v1 # Conflicts: # presto-docs/src/main/sphinx/connector/hive.rst

Merge branch 'master' of https://github.com/intel-staging/presto into…

2228940

… vectorization-v1 # Conflicts: # presto-docs/src/main/sphinx/connector/hive.rst

AbhijitKulkarni1 force-pushed the vectorization-v1 branch from fe30141 to f5f0a66 Compare October 1, 2024 10:18

AbhijitKulkarni1 added 3 commits October 1, 2024 16:07

Resolve merge conflict for syncing

70f9914

Merge branch 'vectorization-v1' of https://github.com/intel-staging/p…

d50a8c3

…resto into vectorization-v1 # Conflicts: # presto-docs/src/main/sphinx/connector/hive.rst

Support hive github actions for java 21 and related changes in pom.xm…

120e88c

…l. Add the hive.orc-use-vector-filter configuration

zhenxiao previously approved these changes Oct 1, 2024

View reviewed changes

rschlussel reviewed Oct 2, 2024

View reviewed changes

AbhijitKulkarni1 added 2 commits October 3, 2024 15:45

Merge branch 'prestodb:master' into vectorization-v1

b93e744

Support hive github actions for java 21 and related changes in pom.xml.

787ceb1

AbhijitKulkarni1 dismissed zhenxiao’s stale review via 787ceb1 October 3, 2024 10:28

Conversation

AbhijitKulkarni1 commented Jan 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Impact

Test Plan

Contributor checklist

Release Notes

Uh oh!

linux-foundation-easycla bot commented Jan 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tdcmeehan commented Mar 18, 2024

Uh oh!

steveburnett commented Mar 18, 2024

Uh oh!

AbhijitKulkarni1 commented Mar 19, 2024

Uh oh!

steveburnett commented Jun 10, 2024

Uh oh!

AbhijitKulkarni1 commented Jun 11, 2024

Uh oh!

steveburnett commented Jun 11, 2024

Uh oh!

AbhijitKulkarni1 commented Jun 12, 2024

Uh oh!

steveburnett commented Jun 12, 2024

Uh oh!

Uh oh!

ZacBlanco commented Jun 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steveburnett left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

AbhijitKulkarni1 commented Jun 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steveburnett left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhenxiao left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zhenxiao commented Oct 1, 2024

Uh oh!

rschlussel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AbhijitKulkarni1 commented Jan 10, 2024 •

edited

Loading

linux-foundation-easycla bot commented Jan 10, 2024 •

edited

Loading

ZacBlanco commented Jun 12, 2024 •

edited

Loading

AbhijitKulkarni1 commented Jun 13, 2024 •

edited

Loading

github-actions bot commented Oct 1, 2024 •

edited

Loading