Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix fe meta error when use merge condition #13337

Merged
merged 3 commits into from
Nov 14, 2022
Merged

[BugFix] Fix fe meta error when use merge condition #13337

merged 3 commits into from
Nov 14, 2022

Conversation

caneGuy
Copy link
Contributor

@caneGuy caneGuy commented Nov 14, 2022

What type of PR is this：

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Which issues of this PR fixes ：

Fixes #

Problem Summary(Required) ：

Related with #12159 we should not add new fields into loadjob

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr will affect users' behaviors
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function

gengjun-git
gengjun-git previously approved these changes Nov 14, 2022
View reviewed changes
@caneGuy
Copy link
Contributor Author

caneGuy commented Nov 14, 2022

run starrocks_fe_unittest

@caneGuy
Copy link
Contributor Author

caneGuy commented Nov 14, 2022

run starrocks_fe_unittest

@sonarcloud
Copy link

sonarcloud bot commented Nov 14, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

@gengjun-git
Copy link
Contributor

run starrocks_admit_test

@Astralidea Astralidea changed the title [Bugfix] Fix fe meta error when use merge condition [BugFix] Fix fe meta error when use merge condition Nov 14, 2022
@Astralidea
Copy link
Contributor

run starrocks_fe_unittest

@Astralidea Astralidea enabled auto-merge (squash) November 14, 2022 12:09
@wanpengfei-git wanpengfei-git added the Approved Ready to merge label Nov 14, 2022
@wanpengfei-git
Copy link
Collaborator

run starrocks_admit_test

@wanpengfei-git
Copy link
Collaborator

[FE PR Coverage Check]

😞 fail : 1 / 4 (25.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/analysis/BrokerDesc.java 0 3 00.00% [79, 80, 82]
🔵 com/starrocks/load/loadv2/BrokerLoadJob.java 1 1 100.00% []

@caneGuy
Copy link
Contributor Author

caneGuy commented Nov 14, 2022

run starrocks_admit_test

@Astralidea Astralidea merged commit 2f9711e into StarRocks:main Nov 14, 2022
@caneGuy caneGuy deleted the zhoukang/fix-condition branch November 14, 2022 14:46
@decster
Copy link
Contributor

decster commented Nov 15, 2022

@mergify backport branch-2.5

@mergify
Copy link
Contributor

mergify bot commented Nov 15, 2022

backport branch-2.5

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Nov 15, 2022
@caneGuy @mergify
[BugFix] Fix fe meta error when use merge condition (#13337)
4f63108 
(cherry picked from commit 2f9711e)
@mergify mergify bot mentioned this pull request Nov 15, 2022
Merged
decster pushed a commit that referenced this pull request Nov 15, 2022
@mergify @caneGuy
[BugFix] Fix fe meta error when use merge condition (#13337) (#13380)
ff02148 
(cherry picked from commit 2f9711e)

Co-authored-by: zhoukang <[email protected]>
murphyatwork added a commit to murphyatwork/starrocks that referenced this pull request Nov 16, 2022
@murphyatwork
Merge main branch
4e9dffc 
Squashed commit of the following:

commit 5506636
Author: Murphy <[email protected]>
Date:   Wed Nov 16 14:52:31 2022 +0800

    [Feature] infer property for mv (StarRocks#12929)

commit e0a5989
Author: eyes_on_me <[email protected]>
Date:   Wed Nov 16 14:28:01 2022 +0800

    [Feature] support big query log (StarRocks#13352)

commit 63792e1
Author: leoyy0316 <[email protected]>
Date:   Wed Nov 16 14:22:39 2022 +0800

    [Feature] Support show create external catalog table (StarRocks#13245)

commit 7795735
Author: rickif <[email protected]>
Date:   Wed Nov 16 14:07:10 2022 +0800

    [Enhancement] Add default group id for routine load (StarRocks#13160)

    When the consumer group id is not set by the user, the routine load coordinator BE would generate a random consumer group id for every routine load task. The kafka would retain these consumer groups until `offsets.retention.minutes` later.
    This PR is to avoid the creation of the consumer group on every task scheduling. It generates a random consumer group id for the routine load job whose group id is not set. All tasks of the job would share the same group id.

    The doc is updated by StarRocks/docs.zh-cn#2748.

commit 7e4bfc5
Author: Murphy <[email protected]>
Date:   Wed Nov 16 12:47:30 2022 +0800

    [BugFix] fix project prune rule  (StarRocks#13278)

commit f18128b
Author: Stephen <[email protected]>
Date:   Wed Nov 16 12:18:33 2022 +0800

    [Tool] automerge label to trigger automerge (StarRocks#13444)

    automerge label to trigger automerge

commit 585566f
Author: packy92 <[email protected]>
Date:   Wed Nov 16 11:07:34 2022 +0800

    [Enhancement] colocate join need consider column equivalent conduction (StarRocks#13344)

commit 91b82e8
Author: Stephen <[email protected]>
Date:   Wed Nov 16 11:06:05 2022 +0800

    [Tool] new label approved to trigger admit test (StarRocks#13442)

    fix pr label trigger admit job bug

commit 0664c10
Author: Gavin <[email protected]>
Date:   Wed Nov 16 11:01:11 2022 +0800

    [BugFix] Fix the `show_be_version.sh` script execute error due to the lack of cachelib dependency. (StarRocks#13411)

commit 2bb88ac
Author: yiming <[email protected]>
Date:   Wed Nov 16 09:57:14 2022 +0800

    [Enhancement] optimize db write lock in report handler (StarRocks#13423)

    Because we need to write bdb with db write lock hold, to avoid block other threads too long,
    we periodically release and acquire the db write lock
    (every MAX_DB_WLOCK_HOLDING_TIME_MS milliseconds).

commit 3769636
Author: yiming <[email protected]>
Date:   Wed Nov 16 09:56:52 2022 +0800

    [BugFix] Disable tablet migration between local disks for pk table (StarRocks#13422)

    Currently, primary key table doesn't support tablet migration between local disks.

commit 76087b9
Author: evelyn.zhaojie <[email protected]>
Date:   Wed Nov 16 09:53:30 2022 +0800

    add 2.2.9 (StarRocks#13429)

commit 7a33ee5
Author: meegoo <[email protected]>
Date:   Wed Nov 16 09:48:11 2022 +0800

    [Refactor] Refactor event base compaction framework (StarRocks#13043)

commit 08d1f60
Author: gengjun-git <[email protected]>
Date:   Wed Nov 16 03:38:19 2022 +0800

    [BugFix] Fix ReportHandler log Error (StarRocks#13428)

commit 1dedad3
Author: Stephen <[email protected]>
Date:   Wed Nov 16 00:53:50 2022 +0800

    Update pr-title-check.yml (StarRocks#13424)

commit f845676
Author: dorianzheng <[email protected]>
Date:   Tue Nov 15 23:22:38 2022 +0800

    [BugFix] Fix duplicate scan timer (StarRocks#13314)

commit 8f979a1
Author: Smith Cruise <[email protected]>
Date:   Tue Nov 15 21:31:22 2022 +0800

    [Enhancement][Support Struct] Support Struct subfield materialize (StarRocks#13401)

    This pr avoids reading redundant data in BE by determining the struct subfield that needs to be materialized in FE.

commit 75ed6c1
Author: dorianzheng <[email protected]>
Date:   Tue Nov 15 20:26:23 2022 +0800

    [BugFix] `SlotDescriptor` might be nullptr (StarRocks#13315)

commit 6a4a0a8
Author: zihe.liu <[email protected]>
Date:   Tue Nov 15 19:50:17 2022 +0800

    [BugFix] Use MockUp to mock SystemInfoService (StarRocks#13388)

commit ac59e92
Author: meegoo <[email protected]>
Date:   Tue Nov 15 19:43:32 2022 +0800

    [Refactor] update the log level of compaction tablet selection (StarRocks#13350)

commit 069c66f
Author: gengjun-git <[email protected]>
Date:   Tue Nov 15 19:23:06 2022 +0800

    [BugFix] Fix the bug of slow execution of show frontends StarRocks#13340

    When executing `show frontends` command, FE will get node info from bdb using ReplicationGroupAdmin. But FE only add/remove the fe host to/from ReplicationGroupAdmin in leader node. This will leave some unreachable host in non-leader node. It will cause the slow problem.
    To fix this bug, add/remove fe node from ReplicationGroupAdmin client in the non-leader FE.

commit 7e753b6
Author: Youngwb <[email protected]>
Date:   Tue Nov 15 19:07:41 2022 +0800

    [BugFix] Fix hive column statistics Min/Max with wrong default value (StarRocks#13371)

commit 22f3619
Author: Stephen <[email protected]>
Date:   Tue Nov 15 19:04:18 2022 +0800

    [Tool] actions info update (StarRocks#13405)

    Standardize action job name

commit a5d7371
Author: packy92 <[email protected]>
Date:   Tue Nov 15 18:24:11 2022 +0800

    [BugFix] fix CompoundPredicateOperator equals method (StarRocks#13271)

commit 0a7fbee
Author: waittting <[email protected]>
Date:   Tue Nov 15 16:48:52 2022 +0800

    [Feature] Support cluster manage statements in new RBAC privilege framework (StarRocks#13122)

    PrivilegeCheckerV2 support cluster manage statements
    1. `show backends`, `show frontends`, `show broker` statements.
    2. `show tablet`, `show transaction`.

    Support upgrading the privilege of system.

commit db1b8ff
Author: Stephen <[email protected]>
Date:   Tue Nov 15 16:36:27 2022 +0800

    [Tool] actions info update for readability (StarRocks#13398)

    actions info update for readability

commit f5513e9
Author: Kevin Li <[email protected]>
Date:   Tue Nov 15 15:32:39 2022 +0800

    [BugFix] fix JoinEliminateNulls ut in ReplyFromDumpTest (StarRocks#13394)

commit a30d34f
Author: Youngwb <[email protected]>
Date:   Tue Nov 15 15:30:20 2022 +0800

    [BugFix] Adjust session variable cbo_cte_reuse_rate to enable tpcds-47 cte reuse (StarRocks#13247)

commit 003c97c
Author: hongli.chang <[email protected]>
Date:   Tue Nov 15 15:25:02 2022 +0800

    [Enhancement] prune empty Range predicate (StarRocks#13323)

commit b55095d
Author: Stephen <[email protected]>
Date:   Tue Nov 15 14:55:03 2022 +0800

    [Tool] actions info update (StarRocks#13361)

    Standardize action naming and support bug fix pr's merge into version hint information

commit bf3534c
Author: Sihui <[email protected]>
Date:   Tue Nov 15 14:36:16 2022 +0800

    Update hive_catalog.md (StarRocks#13389)

commit 8077f66
Author: Binglin Chang <[email protected]>
Date:   Tue Nov 15 13:51:35 2022 +0800

    [Enhancement] load delvec when tablet scanner is openned (StarRocks#13293)

    This PR changes segment_iterator, get delvec when segment_iterator is created, not in _init, so delvecs are got when the query started, preventing delvec not found error when query runs for a very long time(>30min) and delvec got GCed.

commit 880c49e
Author: zombee0 <[email protected]>
Date:   Tue Nov 15 10:35:11 2022 +0800

    [Feature] Support external file table (StarRocks#13064)

    CREATE EXTERNAL TABLE test_0(c1int NULL, c2 array<array> NULL ) ENGINE=FILE PROPERTIES ( "path"="hdfs://172.17.0.3/user/hive/warehouse/array2d_parq", "format"="parquet" );

    the path can be a dir or a file, if it is a path, it must ends with '/', and we just scan the current level not its subdirectory.
    the table name and type must be same with the file in the directory or the file itself.
    support orc and parquet.

    then can query the hive_0:
    select * from hive_0

commit 17b28eb
Author: Kevin Li <[email protected]>
Date:   Tue Nov 15 10:33:14 2022 +0800

    [BugFix] Fix type bugs in Join's Eliminate nulls (StarRocks#13224)

commit e8af424
Author: Xuefen Liu <[email protected]>
Date:   Tue Nov 15 10:26:09 2022 +0800

    [BugFix] update storage_cache_ttl default 30 days (StarRocks#13228)

commit 73d0575
Author: meegoo <[email protected]>
Date:   Tue Nov 15 10:24:03 2022 +0800

    [Enhancement] Add replicated storage blacklist mechanism (StarRocks#12281)

    * [Enhancement] Add replicated storage blacklist mechanism

commit 95cf52e
Author: zombee0 <[email protected]>
Date:   Tue Nov 15 01:27:45 2022 +0800

    [Enhancement] when use desc statement, return the keyType and valueType of MapType (StarRocks#13343)

commit cddfffe
Author: Sihui <[email protected]>
Date:   Tue Nov 15 01:26:14 2022 +0800

    [Doc]Update iceberg_catalog.md (StarRocks#13363)

commit 45b1d33
Author: Youngwb <[email protected]>
Date:   Mon Nov 14 22:07:05 2022 +0800

    [Enhancement] Normalizes the naming of the glue property (StarRocks#13299)

commit e9c8ddb
Author: 絵空事スピリット <[email protected]>
Date:   Mon Nov 14 21:48:38 2022 +0800

    [Doc] Add SHOW PROC (StarRocks#13345)

commit 2f9711e
Author: zhoukang <[email protected]>
Date:   Mon Nov 14 21:35:03 2022 +0800

    [BugFix] Fix fe meta error when use merge condition (StarRocks#13337)

commit 7ab3e6f
Author: meegoo <[email protected]>
Date:   Mon Nov 14 20:54:34 2022 +0800

    [BugFix] Fix broker load crash since merge code problem (StarRocks#13362)

commit 401ced3
Author: gengjun-git <[email protected]>
Date:   Mon Nov 14 19:03:46 2022 +0800

    [BugFix] Fix broker kerberos ticket expired bug (StarRocks#12944)

    Set hadoop.kerberos.keytab.login.autorenewal.enabled to true to enable auth refresh ticket.

commit 6680e90
Author: packy92 <[email protected]>
Date:   Mon Nov 14 18:46:35 2022 +0800

    [UT] ignore unstable case (StarRocks#13318)

    ignore unstable case

commit 106d477
Author: stdpain <[email protected]>
Date:   Mon Nov 14 16:09:00 2022 +0800

    [BugFix] Fix BE crash in streaming distinct aggregate (StarRocks#13269)

commit 8a81ee5
Author: stdpain <[email protected]>
Date:   Mon Nov 14 16:08:39 2022 +0800

    [BugFix] 'Not Found string in global dict: xxx' error in UNIQUE KEY (StarRocks#13263)

commit 4fb8ef9
Author: padmejin <[email protected]>
Date:   Mon Nov 14 15:22:16 2022 +0800

    [Feature] upgrade alter/drop/create/select on table/db/view (StarRocks#13274)

    upgrade alter/drop/create/select on table/db/view

commit bc3e6d2
Author: Stephen <[email protected]>
Date:   Mon Nov 14 14:50:40 2022 +0800

    [Tool] java code style check (StarRocks#13334)

    add java code style check

commit 60c99b2
Author: stdpain <[email protected]>
Date:   Mon Nov 14 14:07:58 2022 +0800

    [Refactor] remove some unused code in FunctionContext (StarRocks#13066)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Astralidea Astralidea Astralidea approved these changes

@gengjun-git gengjun-git gengjun-git approved these changes

Assignees
No one assigned
Labels
2.5 Approved Ready to merge be-build
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@caneGuy @gengjun-git @Astralidea @wanpengfei-git @decster