Skip to content

Broker load could ingest malformed DATETIME value #62758

@GuangzheGao

Description

@GuangzheGao

Steps to reproduce the behavior (Required)

  1. Create a bad parquet with a badDateTimeKey, data like: "57324-01-20 19:28:48+00:00"
  2. Create table with this badDateTimeKey as partition key
CREATE TABLE IF NOT EXISTS database.tablename (
    `someNonImportantKey` STRING,
    `badDateTimeKey` DATETIME
)
ENGINE=OLAP
PARTITION BY date_trunc('day', `badDateTimeKey`)
DISTRIBUTED BY HASH(`someNonImportantKey`) BUCKETS 1
ORDER BY (`someNonImportantKey`, `badDateTimeKey`);
  1. Broker load ingest the parquet file
LOAD LABEL database.`loadLabelName`
(
    DATA INFILE("s3a://...bad.parquet
)
WITH BROKER
(
    ...
)
PROPERTIES
(
    "partial_update" = "FALSE",
    "max_filter_ratio" = "0",
    "timeout" = "72000"
);

Expected behavior (Required)

Fail with reason like: YYYYY format is invalid

Real behavior (Required)

If use the date time field as partition key: we see error: "type:LOAD_RUN_FAIL; msg:txn_id=373425 failed. Text ';389-08-07 00:00:00' could not be parsed at index 0: BE:493858"

If create table withOUT partition key: it will successfully ingest the out of bound datetime as malformed value ';389-08-07 00:00:00' into the table

StarRocks version (Required)

3.4.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions