Skip to content

Conversation

@Zouxxyy
Copy link
Contributor

@Zouxxyy Zouxxyy commented Sep 28, 2023

Change Logs

Steps to reproduce

-- spark
set hoodie.schema.on.read.enable=true;

create table hudi_cow_test_tbl (
  id bigint,
  name string,
  ts int,
  dt string,
  hh string
) using hudi
tblproperties (
  type = 'cow',
  primaryKey = 'id',
  preCombineField = 'ts'
)
partitioned by (dt, hh);

insert into hudi_cow_test_tbl values (1, 'a1', 1001, '2021-12-09', '10');
ALTER TABLE hudi_cow_test_tbl ALTER COLUMN ts TYPE bigint;
insert into hudi_cow_test_tbl values (2, 'a2', 1001, '2021-12-09', '11');

-- hive
select id, dt from xinyu_test.hudi_cow_test_tbl;

Failed with exception java.io.IOException:org.apache.hudi.exception.HoodieException: 
The size of hive.io.file.readcolumn.ids: 5,8,9 is not equal to projection columns: id

The core reason is we need to modify the conf first (by #7355) and then execute new SchemaEvolutionContext(split, job).doEvolutionForParquetFormat();

Impact

Fix above

Risk level (write none, low medium or high below)

low

Documentation Update

none

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@danny0405
Copy link
Contributor

@hudi-bot run azure

1 similar comment
@danny0405
Copy link
Contributor

@hudi-bot run azure

@hudi-bot
Copy link
Collaborator

hudi-bot commented Oct 2, 2023

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@Zouxxyy Zouxxyy closed this Oct 2, 2023
@Zouxxyy
Copy link
Contributor Author

Zouxxyy commented Oct 2, 2023

will consider more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants