Create lazy-loading and change-tracking proxy types lazily

[henrikdahl8240]

I have made this simple function:

private void buttonRoot_Click(object sender, EventArgs e)
{
    DateTime start = DateTime.Now;
    using (BikeShop_DbContext dbContext = new BikeShop_DbContext(new DbContextOptionsBuilder<BikeShop_DbContext>()
        //.UseLazyLoadingProxies()
        .UseSqlServer(@"data source=.;initial catalog=BikeShop;integrated security=True;MultipleActiveResultSets=True", opts => opts.CommandTimeout((int)TimeSpan.FromMinutes(10).TotalSeconds)).Options))
    {
        var root = dbContext.Roots.First();
        DateTime finish = DateTime.Now;
        MessageBox.Show(this, $"root.ID: {root.ID}.\n Start: {start}\nFinish: {finish}\nDuration: {finish - start}", "Root ID");
    }
}

On my machine, it takes 4:18 to execute, i.e. four minutes and 18 seconds.

It is important to notice, that I have commented out the invocation of .UseLazyLoadingProxies(). I obviously need these proxies, so it's just to compare.

So now I uncomment .UseLazyLoadingProxies() so it will also be run, i.e.:

private void buttonRoot_Click(object sender, EventArgs e)
{
    DateTime start = DateTime.Now;
    using (BikeShop_DbContext dbContext = new BikeShop_DbContext(new DbContextOptionsBuilder<BikeShop_DbContext>()
        .UseLazyLoadingProxies()
        .UseSqlServer(@"data source=.;initial catalog=BikeShop;integrated security=True;MultipleActiveResultSets=True", opts => opts.CommandTimeout((int)TimeSpan.FromMinutes(10).TotalSeconds)).Options))
    {
        var root = dbContext.Roots.First();
        DateTime finish = DateTime.Now;
        MessageBox.Show(this, $"root.ID: {root.ID}.\n Start: {start}\nFinish: {finish}\nDuration: {finish - start}", "Root ID");
    }
}

Now it takes 34:42 to execute, i.e. 34 minutes and 42 seconds. By simple subtraction, the invocation of .UseLazyLoadingProxies() adds approximately half an hour to the duration or makes initialization to take approximately 10 times longer time.

There was a lot of criticism on the initialization time in scope of EF 6. It seems like this gives exactly the same problem.

Can't you for instance lazy initialize or we could call it initialize on demand instead? Perhaps you have a better idea?

Do you agree, that it's a major problem, that use of your proxy feature forces initialization to take really long time?

I would like to mention, that I have in the magnitude of 4.000 entities in my model. Only one of these is being looked up by my small program example.

EF Core version: 3.1.2
Database provider: Microsoft.EntityFrameworkCore.SqlServer
Target framework: NET Core 3.1
Operating system: Windows 10 Professional
IDE: Visual Studio 2019 16.4.5.

[ajcvickers]

@henrikdahl8240 Can you attach your EF model (entity types and DbContext) so that we can investigate. This seems slower than expected even for 4000 entity types.

[henrikdahl8240]

Can't you give a suggestion for how to do that without challenging the immaterial rights. Do you for example have a reference for a Roslyn project which anonymizes all names in all projects in a solution?

I would like to share the model, but I would like to do it without disclosing anything for real.

[ErikEJ]

@henrikdahl8240 You can run an obfuscator?

[henrikdahl8240]

Yes, but I have never done that before. Can you suggest how to do that? Can I see the result myself? Is there one built-in in Visual Studio?

[ErikEJ]

https://docs.microsoft.com/en-us/visualstudio/ide/dotfuscator/?view=vs-2019
https://www.preemptive.com/dotfuscator/ce/docs/help/getting_started_gui.html

Then you can decompile the resulting dll and extract the relevant classes

[henrikdahl8240]

Thank you very much, Erik. Do you also have a suggestion concerning the decompilation step?

[ajcvickers]

@henrikdahl8240 If you don't want to share it publicly, then you can send it to me directly and we will keep it private. avickers at microsoft.com

[henrikdahl8240]

@ajcvickers It sounds excellent. I will put some efforts into that.

[ErikEJ]

I usually use dotPeek

[henrikdahl8240]

OK. I will take a look. Thank you for the hints, @ErikEJ

[henrikdahl8240]

@ajcvickers As you may see, I mailed you a soution on March 6. 2020 13:46. Perhaps you may run it to reproduce the problems yourself?

The mail details the few steps you should go through.

[ajcvickers]

@henrikdahl8240 Thanks--we got the code.

Note for team: I am able to reproduce this; culprit is model building but I haven't profiled to find out where. /cc @AndriySvyryd

Running the code below on my fast machine without lazy-loading proxies:

Creating model...
Done in 81904
Creating database...
Done in 113199
Running query...
Done in 124

With proxies:

Creating model...
Done in 701398
Creating database...
Done in 112623
Running query...
Done in 132

Model has 5,860 entity types with a total of 13,876 navigations.

EntityTypeConfigurations are already being used.

using (HenrikDahl_DbContext dbContext = new HenrikDahl_DbContext(new DbContextOptionsBuilder<HenrikDahl_DbCon
    .UseLazyLoadingProxies()
    .UseSqlServer(Your.SqlServerConnectionString, opts => opts.CommandTimeout((int)TimeSpan.FromMinutes(10).T
{
    Console.WriteLine("Creating model...");

    var timer = Stopwatch.StartNew();
    
    var model = dbContext.Model;
    timer.Stop();
    Console.WriteLine($"Done in {timer.ElapsedMilliseconds}");
    
    Console.WriteLine("Creating database...");

    timer = Stopwatch.StartNew();
    dbContext.Database.EnsureDeleted();
    dbContext.Database.EnsureCreated();
    timer.Stop();
    
    Console.WriteLine($"Done in {timer.ElapsedMilliseconds}");

    Console.WriteLine("Running query...");

    timer = Stopwatch.StartNew();
    var samoyed = dbContext.Samoyeds.FirstOrDefault();
    timer.Stop();
    
    Console.WriteLine($"Done in {timer.ElapsedMilliseconds}");
}

[AndriySvyryd]

It might be just the time it takes to create the proxy types

[henrikdahl8240]

Shouldn't proxy types generally be created as they are needed and not up-frot, i.e. created lazy-wise? Perhaps it could be an option for UseLazyLoadingProxies whether the proxy classes should be created eager-wise or lazy-wise. Otherwise I guess it will always be a pain in situations with relatively many entity types. I assume that the requirement of using "virtual" for all collections and navigation properties enable this.

I would imagine, that EF Core should rather shine with many entity types than with few entity types. Probably many can figure out to make such ORM stuff if it only needs to perform well in situations with few entity types. I assume that all non-trivial enterprise solutions will have many, many entity types and it's here EF Core should shine, at least from my point of view.