Skip to content

[HUDI-2942] add error message log in HoodieCombineHiveInputFormat #4224

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
xushiyan merged 10 commits into from
Dec 8, 2021
Merged

[HUDI-2942] add error message log in HoodieCombineHiveInputFormat #4224

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
81a459a
add error message log in HoodieCombineHiveInputFormat
xuzifu666 Dec 6, 2021
f7eab9f
remove unused fs refence
xuzifu666 Dec 6, 2021
4008025
remove blank row
xuzifu666 Dec 6, 2021
fe8d1bd
restore
xuzifu666 Dec 6, 2021
776d9a6
remove unused fs refence
xuzifu666 Dec 6, 2021
ed6d8f7
restore fs
xuzifu666 Dec 6, 2021
a291420
Update HoodieCombineHiveInputFormat.java
xuzifu666 Dec 6, 2021
131ac59
Update HoodieCombineHiveInputFormat.java
xuzifu666 Dec 6, 2021
187c165
Update HoodieCombineHiveInputFormat.java
xuzifu666 Dec 6, 2021
e06de54
Update HoodieCombineHiveInputFormat.java
xuzifu666 Dec 6, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/hive/HoodieCombineHiveInputFormat.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@
import org.apache.hudi.hadoop.utils.HoodieInputFormatUtils;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileStatus;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.fs.PathFilter;
import org.apache.hadoop.hive.common.StringInternUtils;
Expand Down Expand Up @@ -183,8 +182,8 @@ private InputSplit[] getCombineSplits(JobConf job, int numSplits, Map<Path, Part
deserializerClassName = part.getDeserializer(job).getClass().getName();
} catch (Exception e) {
// ignore
LOG.error("Getting deserializer class name error ", e);
}
FileSystem inpFs = path.getFileSystem(job);

// don't combine if inputformat is a SymlinkTextInputFormat
if (inputFormat instanceof SymlinkTextInputFormat) {
Expand Down