Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support join with enumerable on client #17031

Closed
dotlogix opened this issue Aug 8, 2019 · 7 comments
Closed

Support join with enumerable on client #17031

dotlogix opened this issue Aug 8, 2019 · 7 comments

Comments

@dotlogix
Copy link

dotlogix commented Aug 8, 2019

Describe what is not working as expected.
Why EF Core cant translate this query, its is a simple join on multiple columns and looks the same as in many tutorials for EF.
Is this not implemented yet and the tutorials are related to ef6 or what's wrong with this implementation?

Is there any way to solve this kind of Problem in EF Core?

protected override Task<List<TEntity>> LoadEntitiesAsync(DbContext context, IEnumerable<(string fsymId, string feItem, DateTime feFpEnd)> keys) {
return context.Set<TEntity>()
        .Join(keys,
          e => new { p1= e.FsymId, p2=e.FeItem, p3 = e.FeFpEnd },
          i => new { p1= i.fsymId, p2=i.feItem, p3 = i.feFpEnd },
          (e, i) => e)
        .ToListAsync();
		}

Further technical details

EF Core version: 2.2.6.0
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
Operating system:
IDE: (e.g. Visual Studio 2019)

@smitpatel
Copy link
Contributor

keys is client side enumerable. The join is purely client side. You can just get all TEntity-ies from server and do join on client. If not then how are you expecting this to be translated to server? Can you write a SQL translation for it?

@dotlogix
Copy link
Author

dotlogix commented Aug 8, 2019

Sure I would expect sth like this:

SELECT e.* FROM [table] e
JOIN (SELECT * FROM VALUES ((p1, p2, p3), ...) AS i (p1, p2, p3)) ON e.p1=i.p2 AND e.p2=i.p2 AND e.p3=i.p3

It should be used to select multiple entities by a 3 column composite key

@smitpatel
Copy link
Contributor

That is invalid SQL in SqlServer.

@dotlogix
Copy link
Author

dotlogix commented Aug 8, 2019

I executed this command on our local SQL instance and it worked as expected. I will try it again be be it is just a Syntax error. I wrote the Statement by Hand so maybe thats why it is invalid Syntax

SELECT *
FROM (VALUES (1,2)
, (3,4)
) t1 (c1, c2)

This is where I got it from it is valid SQL Server Statement

https://modern-sql.com/de/anwendung/select-ohne-from

@PawelGerr
Copy link

PawelGerr commented Aug 8, 2019

Interessting idea for JOIN with smaller collections.
The current implementation of EF requires the type (to join with) to be registered in OnModelCreating, i.e. something like

class MyClass
{ 
    string FsymId
    string FeItem
    DateTime FeFpEnd
}

void OnModelCreating(ModelBuilder builder)
{
    modelBuilder.Query<MyClass>();
}

After that we need a custom method/expression to generate the SQL. The easiest way would probably be the use of FromSql.

IQueryable<MyClass> keysQuery = ctx.CreateMagicallyQueryableFrom(keys);

----

var keysQuery = ctx.Query<MyClass>().FromSql("
SELECT * FROM ( VALUES ... )
");

Now, we could join with the keysQuery.

Don't know if it helps in your use case. But I'm usually working with big collection having hundreds or thousands or records so I'm using SqlBulkCopy to insert some data into temp tables then I use this temp table for joins. It looks like

// config
void OnModelCreating(ModelBuilder modelBuilder)
{
   modelBuilder.ConfigureTempTableEntity<MyClass>();
}

// usage
IEnumerable<MyClass> keys = ...;

using(ITempTableQuery<MyClass> tempTableQuery = await ctx.BulkInsertIntoTempTableAsync(keys))
{
    IQueryable<MyClass> keysQuery = tempTableQuery.Query;
   // join with keysQuery the usual way
}

@ajcvickers ajcvickers added this to the Backlog milestone Aug 9, 2019
@ajcvickers ajcvickers changed the title Composite Key Join not working as expected Join with enumerable on client not working as expected Aug 9, 2019
@ajcvickers ajcvickers changed the title Join with enumerable on client not working as expected Support join with enumerable on client Aug 9, 2019
@roji roji mentioned this issue Apr 19, 2023
34 tasks
@roji
Copy link
Member

roji commented Jun 13, 2023

This is now possible as part of #30426, by packing the local collection to JSON (or an array on PG). For example, see this test in our test suite:

LINQ:

public virtual Task Column_collection_Join_parameter_collection(bool async)
{
    var ints = new[] { 11, 111 };
    return AssertQuery(
        async,
        ss => ss.Set<PrimitiveCollectionsEntity>()
            .Where(c => c.Ints.Join(ints, i => i, j => j, (i, j) => new { I = i, J = j }).Count() == 2),
        entryCount: 1);
}

SQL:

@__ints_0='[11,111]' (Size = 4000)

SELECT [p].[Id], [p].[Bool], [p].[Bools], [p].[DateTime], [p].[DateTimes], [p].[Enum], [p].[Enums], [p].[Int], [p].[Ints], [p].[NullableInt], [p].[NullableInts], [p].[String], [p].[Strings]
FROM [PrimitiveCollectionsEntity] AS [p]
WHERE (
    SELECT COUNT(*)
    FROM OPENJSON([p].[Ints]) WITH ([value] int '$') AS [i]
    INNER JOIN OPENJSON(@__ints_0) WITH ([value] int '$') AS [i0] ON [i].[value] = [i0].[value]) = 2

@roji
Copy link
Member

roji commented Jun 13, 2023

Duplicate of #30426

@roji roji marked this as a duplicate of #30426 Jun 13, 2023
@roji roji closed this as not planned Won't fix, can't repro, duplicate, stale Jun 13, 2023
@roji roji removed this from the Backlog milestone Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants