Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map #6547

Open
wants to merge 1 commit into
base: branch-2
Choose a base branch
from
Open

HBASE-25839 Bulk Import fails with java.io.IOException: Type mismatch in value from map #6547

wants to merge 1 commit into from

Conversation

srinireddy2020
Copy link
Contributor

  1. CellSortImporter emits values wrapped inside MapReduceExtendedCell
  2. Converted KV to CellUtil.

@srinireddy2020
Copy link
Contributor Author

@anoopsjohn
As per your suggestion #3232 (comment)
I have changed and tested in the branch-2. It is working as expected.

Please review the changes.

@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
pankaj72981
pankaj72981 reviewed Dec 19, 2024
View reviewed changes
hbase-mapreduce/src/main/java/org/apache/hadoop/hbase/mapreduce/Import.java
@@ -413,7 +417,7 @@ public void map(ImmutableBytesWritable row, Result value, Context context) throw
// skip if we filtered it out
if (kv == null) continue;
Cell ret = convertKv(kv, cfRenameMap);
context.write(new CellWritableComparable(ret), ret);
context.write(new CellWritableComparable(ret), new MapReduceExtendedCell(ret));
}
Copy link
Contributor

@pankaj72981 pankaj72981 Dec 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add UT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this issue.
Comment addressed, please verify.

@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache9
Copy link
Contributor

Apache9 commented Dec 29, 2024

We do not have this problem on master branch?

@Apache9
Copy link
Contributor

Apache9 commented Jan 13, 2025

Any updates here?

@srinireddy2020
Copy link
Contributor Author

Any updates here?

Sorry for the Late reply, This issue is exist in the master branch also.

@srinireddy2020
Copy link
Contributor Author

We do not have this problem on master branch?

Raised PR for the Master Branch.
@Apache9 please check and review.

@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase

This comment has been minimized.

Sign in to view
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 0s Docker mode activated.
-1 ❌ docker 0m 4s Docker failed to build run-specific yetus/hbase:tp-30718}.
Subsystem Report/Notes
GITHUB PR #6547
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/console
versions git=2.17.1
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 44s Docker mode activated.
-0 ⚠️ yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 2m 38s branch-2 passed
+1 💚 compile 0m 17s branch-2 passed
+1 💚 javadoc 0m 14s branch-2 passed
+1 💚 shadedjars 5m 17s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 2m 21s the patch passed
+1 💚 compile 0m 18s the patch passed
+1 💚 javac 0m 18s the patch passed
+1 💚 javadoc 0m 12s the patch passed
+1 💚 shadedjars 5m 10s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 11m 26s hbase-mapreduce in the patch passed.
29m 54s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #6547
Optional Tests javac javadoc unit compile shadedjars
uname Linux a704e5d4cfc1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 5908276
Default Java Temurin-1.8.0_412-b08
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/testReport/
Max. process+thread count 1951 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 44s Docker mode activated.
-0 ⚠️ yetus 0m 6s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 3m 10s branch-2 passed
+1 💚 compile 0m 22s branch-2 passed
+1 💚 javadoc 0m 16s branch-2 passed
+1 💚 shadedjars 6m 19s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 2m 53s the patch passed
+1 💚 compile 0m 19s the patch passed
+1 💚 javac 0m 19s the patch passed
+1 💚 javadoc 0m 14s the patch passed
+1 💚 shadedjars 6m 12s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 17m 25s hbase-mapreduce in the patch passed.
39m 35s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #6547
Optional Tests javac javadoc unit compile shadedjars
uname Linux f5366effa844 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 5908276
Default Java Eclipse Adoptium-11.0.23+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/testReport/
Max. process+thread count 2241 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 45s Docker mode activated.
-0 ⚠️ yetus 0m 5s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
_ Prechecks _
_ branch-2 Compile Tests _
+1 💚 mvninstall 4m 19s branch-2 passed
+1 💚 compile 0m 39s branch-2 passed
+1 💚 javadoc 0m 21s branch-2 passed
+1 💚 shadedjars 7m 7s branch has no errors when building our shaded downstream artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 3m 38s the patch passed
+1 💚 compile 0m 24s the patch passed
+1 💚 javac 0m 24s the patch passed
+1 💚 javadoc 0m 15s the patch passed
+1 💚 shadedjars 6m 29s patch has no errors when building our shaded downstream artifacts.
_ Other Tests _
+1 💚 unit 17m 35s hbase-mapreduce in the patch passed.
42m 54s
Subsystem Report/Notes
Docker ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR #6547
Optional Tests javac javadoc unit compile shadedjars
uname Linux aeb70cce9a86 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2 / 5908276
Default Java Eclipse Adoptium-17.0.11+9
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/testReport/
Max. process+thread count 2057 (vs. ulimit of 30000)
modules C: hbase-mapreduce U: hbase-mapreduce
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6547/4/console
versions git=2.34.1 maven=3.9.8
Powered by Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pankaj72981 pankaj72981 pankaj72981 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@srinireddy2020 @Apache-HBase @Apache9 @pankaj72981