Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SortMergeJoin: The query stuck when join filter is set and more matched rows than batch size #10491

Closed
Tracked by #9846
comphead opened this issue May 14, 2024 · 0 comments · Fixed by #10495
Closed
Tracked by #9846
Assignees

Comments

@comphead
Copy link
Contributor

          I'm not able to run such test just because of other SMJ issue not related to this PR:

If the join filter is set and for the same streaming index there are matching rows more or equal to a batch size then the query just stuck. Likely the problem is in polling state and it can be easily reproduced on main branch.

  #[tokio::test]
    async fn test_11() -> Result<()> {
        let ctx: SessionContext = SessionContext::new();

        let sql = "set datafusion.optimizer.prefer_hash_join = false;";
        let _ = ctx.sql(sql).await?.collect().await?;

        let sql = "set datafusion.execution.batch_size = 1";
        let _ = ctx.sql(sql).await?.collect().await?;

        let sql = "
        select * from (
        with
        t1 as (
            select 12 a, 12 b
            ),
        t2 as (
            select 12 a, 12 b
            )
            select t1.* from t1 join t2 on t1.a = t2.b where t1.a > t2.b
        ) order by 1, 2;
        ";

        let actual = ctx.sql(sql).await?.collect().await?;


        Ok(())
    }

I'll file a separate issue for this one, but perhaps we can go with this PR because potential problem you talking about cannot ever be hit because of the issue above

Originally posted by @comphead in #10304 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants