Skip to content

Conversation

@xudong963
Copy link
Contributor

@xudong963 xudong963 commented Mar 4, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Briefly describe what this PR aims to solve. Include background context that will help reviewers understand the purpose of the PR.

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Mar 4, 2024
@xudong963 xudong963 marked this pull request as draft March 4, 2024 11:20
@xudong963 xudong963 added the ci-cloud Build docker image for cloud test label Mar 4, 2024
@github-actions
Copy link
Contributor

github-actions bot commented Mar 4, 2024

Docker Image for PR

  • tag: pr-14828-6f874bf

note: this image tag is only available for internal use,
please check the internal doc for more details.

@xudong963 xudong963 marked this pull request as ready for review March 4, 2024 15:07
@xudong963 xudong963 requested review from Dousir9 and zhang2014 March 4, 2024 15:07
@xudong963
Copy link
Contributor Author

A large left join test with spilling on databend cloud:

SELECT * FROM lineitem as l1 left JOIN lineitem AS o ON (l1.l_orderkey * 1000  = o.l_orderkey) where o.l_orderkey  + l1.l_orderkey < 100000 or o.l_orderkey IS  NOT DISTINCT FROM null order by o.l_orderkey nulls first limit 10;

SELECT
  *
FROM
  lineitem AS l1
  LEFT JOIN lineitem AS o ON (l1.l_orderkey * 1000 = o.l_orderkey)
WHERE
  o.l_orderkey + l1.l_orderkey < 100000
  OR o.l_orderkey IS NOT DISTINCT
FROM
  NULL
ORDER BY
  o.l_orderkey nulls FIRST
LIMIT
  10

┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ l_orderkey │ l_partkey │ l_suppkey │ l_linenumber │   l_quantity   │ l_extendedprice │   l_discount   │      l_tax     │ l_returnflag │ l_linestatus │ l_shipdate │ l_commitdate │ l_receiptdate │   l_shipinstruct  │ l_shipmode │                  l_comment                  │    l_orderkey   │    l_partkey    │    l_suppkey    │   l_linenumber  │        l_quantity        │      l_extendedprice     │        l_discount        │           l_tax          │   l_returnflag   │   l_linestatus   │   l_shipdate   │  l_commitdate  │  l_receiptdate │  l_shipinstruct  │    l_shipmode    │     l_comment    │
│    Int64   │   Int64   │   Int64   │     Int64    │ Decimal(15, 2) │  Decimal(15, 2) │ Decimal(15, 2) │ Decimal(15, 2) │    String    │    String    │    DateDateDate     │       String      │   String   │                    String                   │ Nullable(Int64) │ Nullable(Int64) │ Nullable(Int64) │ Nullable(Int64) │ Nullable(Decimal(15, 2)) │ Nullable(Decimal(15, 2)) │ Nullable(Decimal(15, 2)) │ Nullable(Decimal(15, 2)) │ Nullable(String) │ Nullable(String) │ Nullable(Date) │ Nullable(Date) │ Nullable(Date) │ Nullable(String) │ Nullable(String) │ Nullable(String) │
├────────────┼───────────┼───────────┼──────────────┼────────────────┼─────────────────┼────────────────┼────────────────┼──────────────┼──────────────┼────────────┼──────────────┼───────────────┼───────────────────┼────────────┼─────────────────────────────────────────────┼─────────────────┼─────────────────┼─────────────────┼─────────────────┼──────────────────────────┼──────────────────────────┼──────────────────────────┼──────────────────────────┼──────────────────┼──────────────────┼────────────────┼────────────────┼────────────────┼──────────────────┼──────────────────┼──────────────────┤
│  5353120015445259195275419.0022875.620.040.07           │ N            │ O            │ 1997-02-201997-02-031997-03-11    │ TAKE BACK RETURN  │ MAIL       │ multipliers affix. always ruthless acco     │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
│  53531200118451608201663538.0059229.840.070.06           │ N            │ O            │ 1996-12-161997-02-111997-01-11    │ COLLECT COD       │ FOB        │ s poach furiously blithe                    │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
│  53531200116167667417684637.0064152.820.050.06           │ N            │ O            │ 1997-02-181997-02-071997-02-23    │ DELIVER IN PERSON │ AIR        │ e regular dolphins. furiously f             │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
│  5353121011649727549727617.008900.150.010.07           │ N            │ O            │ 1997-06-251997-06-061997-07-23    │ NONE              │ RAIL       │ ully ironic deposits. slyly regular ide     │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
│  535312101969730519732421.001301.820.090.04           │ N            │ O            │ 1997-04-291997-05-291997-05-09    │ COLLECT COD       │ MAIL       │ ilent pinto beans? ironically bli           │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
│  5353121012602103852106327.0027134.190.080.02           │ N            │ O            │ 1997-05-081997-05-021997-05-22    │ TAKE BACK RETURN  │ REG AIR    │ . special ideas unwind care                 │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
│  5353121011353682336850448.0089239.200.070.02           │ N            │ O            │ 1997-04-201997-05-301997-04-29    │ NONE              │ SHIP       │ the furiously silent Tiresias.              │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
│  5353121011337786337786457.0013581.400.050.06           │ N            │ O            │ 1997-03-251997-05-161997-04-15    │ DELIVER IN PERSON │ FOB        │ he quickly pending platelets are furiously  │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
│  535312101172791272791369.0017458.470.080.00           │ N            │ O            │ 1997-04-121997-05-281997-05-02    │ TAKE BACK RETURN  │ SHIP       │ y. blithely ironi                           │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
│  5353121011001618501621720.0030391.200.000.00           │ N            │ O            │ 1997-04-021997-06-021997-04-06    │ NONE              │ FOB        │ le slyly after the ironic packages.         │            NULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULLNULL             │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
10 rows read in 1851.514 sec. Processed 1.2 billion rows, 215.47 GiB (648.16 thousand rows/s, 119.17 MiB/s)

@xudong963 xudong963 added this pull request to the merge queue Mar 5, 2024
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Mar 5, 2024
@BohuTANG BohuTANG merged commit f22ad96 into databendlabs:main Mar 5, 2024
@xudong963 xudong963 deleted the left_join_spill branch March 5, 2024 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants