Skip to content

DX-67936 Netty 4.1.96 upgrade for CVE-2023-34462 io.netty:netty-handler 4.1.93.Final #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
lriggs merged 1 commit into from
Jul 28, 2023
Merged

DX-67936 Netty 4.1.96 upgrade for CVE-2023-34462 io.netty:netty-handler 4.1.93.Final #36

lriggs merged 1 commit into from
Jul 28, 2023

Conversation

@lriggs
Copy link

@lriggs lriggs commented Jul 28, 2023

I ran tpcds for this upgrade using arrow jars built from my personal fork. Looks good: https://jenkins.drem.io/job/AWS-EE-tpcds-perf-jdbc-client/1089/

@lriggs lriggs requested a review from xxlaykxx July 28, 2023 16:01
@github-actions
Copy link

github-actions bot commented Jul 28, 2023

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

@whenamanlies
Copy link

whenamanlies commented Jul 28, 2023

https://netty.io/news/2023/07/27/4-1-96-Final.html

We are happy to announce the release of netty 4.1.96.Final. This releases fixes a performance regression introduced in 4.1.95.Final and a small regression in HTTP/2. Because of that we urge everyone to upgrade as soon as possible.

The most important changes are:

Move the PoolThreadCache finalizer to a separate object (#13510)
Fix kevent(..) failed: Invalid argument (#13503)
Revert "Always increment Stream Id on createStream (apache#13485)" to fix b…ug which caused sending multiple RST frames for the same id (1735b4230a4935cbecedb48fd687c67aba950068)

whenamanlies
whenamanlies approved these changes Jul 28, 2023
View reviewed changes
Copy link

@whenamanlies whenamanlies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lriggs lriggs merged commit cf53c1e into dremio:dremio_24.2_12.0 Jul 28, 2023
@lriggs lriggs deleted the lriggs-netty-upgrade branch July 28, 2023 17:38
xxlaykxx added a commit that referenced this pull request Jul 30, 2023
@xxlaykxx @lriggs
DX-61115 cherry picked commits form arrow (#37)
a2e8d49 
* apacheGH-36375: [Java] Added creating MapWriter in ComplexWriter. (apache#36351) (#32)

Added new method rootAsMap() to ComplexWriter and implement it in ComplexWriterImpl for supporting map type.
Previously in dremio side:

When i trying to return map like output ComplexWrite
with this code:

org.apache.arrow.vector.complex.writer.BaseWriter.MapWriter mapWriter = out.rootAsList().map(false);

mapWriter.startMap();
for (java.util.Map.Entry<java.lang.Integer, java.lang.Integer> element : map.entrySet()) {
    mapWriter.startEntry();
    mapWriter.key().integer().writeInt((Integer) element.getKey());
    mapWriter.value().integer().writeInt((Integer) element.getValue());
    mapWriter.endEntry();
}
mapWriter.endMap();
It use UnionMapWriter and generate schema like:
EXPR$0: Map(false)<$data$: Union(Sparse, [1, 39])<struct: Struct<key: Int(32, true) not null, value: Int(32, true) not null> not null, map: Map(false)<entries: Struct<key: Int(32, true) not null, value: Int(32, true)> not null>>>
But in OutputDerivation impl class where i should create output Complete type

List<Field> children = Arrays.asList( CompleteType.INT.toField("key", false), CompleteType.INT.toField("value", false));
return new CompleteType(CompleteType.MAP.getType(), CompleteType.struct(children).toField(MapVector.DATA_VECTOR_NAME, false));
(This is only one valid case, because MapVector.initializeChildrenFromFields())
return
EXPR$0::map<key::int32, value::int32> I found a place where it start using union - PromotableWriter.promoteToUnion.
And in the end i have

SCHEMA_CHANGE ERROR: Schema changed during projection. Schema was 
schema(EXPR$0::map<key::int32, value::int32>)
 but then changed to 
schema(EXPR$0::map<struct::struct<key::int32, value::int32>, map::map<key::int32, value::int32>>)
* Closes: apache#36375

Authored-by: Ivan Chesnov <[email protected]>

Signed-off-by: David Li <[email protected]>

* DX-67936 Upgrade to Netty 4.1.96 for CVE-2023-34462 io.netty:netty-handler 4.1.93.Final (#36)

* Update README_DREMIO for new commit.

---------

Signed-off-by: David Li <[email protected]>
Co-authored-by: lriggs <[email protected]>
Co-authored-by: Logan Riggs <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xxlaykxx xxlaykxx xxlaykxx approved these changes

+1 more reviewer

@whenamanlies whenamanlies whenamanlies approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

awaiting review Component: Java

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@lriggs @whenamanlies @xxlaykxx