project with type provider builds in VS but not with "dotnet build"

[smoothdeveloper]

I'm facing issue with project referencing FSharp.Management not building from dotnet build

fsmgt.zip

When opening the same project in visual studio, it is possible to build.

Expected behavior

Able to build with dotnet build

Actual behavior

FSC : warning FS3005: Referenced assembly 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.PowerShell.dll' has assembly level attribute 'Microsoft.FSharp.Core.CompilerServices.TypeProviderAssemblyAttribute' but no public type provider classes were found [C:\tmp\fsmgt\fsmgt.fsproj]
FSC : warning FS3005: Referenced assembly 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.WMI.dll' has assembly level attribute 'Microsoft.FSharp.Core.CompilerServices.TypeProviderAssemblyAttribute' but no public type provider classes were found [C:\tmp\fsmgt\fsmgt.fsproj]
FSC : error FS3049: The type provider 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.PowerShell.dll' reported an error: The type provider designer assembly 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.PowerShell.dll' could not be loaded from folder 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40'. The exception reported was: System.IO.FileNotFoundException - Could not load file or assembly 'System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'. The system cannot find the file specified. [C:\tmp\fsmgt\fsmgt.fsproj]
FSC : error FS3049: The type provider 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40\FSharp.Management.WMI.dll' reported an error: The type provider designer assembly 'FSharp.Management.WMI.DesignTime' could not be loaded from folder 'C:\Users\gauthier.segay.nuget\packages\fsharp.management\0.4.5\lib\net40'. The exception reported was: System.IO.FileNotFoundException - Could not load file or assembly 'System.Management, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified. [C:\tmp\fsmgt\fsmgt.fsproj]
2 Warning(s)
2 Error(s)

Related information

• windows 10
• .NET sdk 5.0.100-rc.1.20452.10
• vs 2019

[smoothdeveloper]

Similar stuff happening with FSharp.Data.SqlClient, building in VS but not from command line using dotnet sdk compiler.

This boils down to full framework fsc.exe used in VS and dotnet sdk not working with type providers AFAIU:

error FS3033: The type provider 'FSharp.Data.SqlProgrammabilityProvider' reported an error: Could not load type 'System.Data.SqlClient.SqlConnection' from assembly 'System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'

This kind of a big deal and not sure how to work around the issue.

[cartermp]

As far as I can tell this type provider is still using old-style projects and targeting .NET Framework. I don't think dotnet build would be expected to work.

[smoothdeveloper]

@cartermp this is not the case with FSharp.Data.SqlClient which use new style SDK.

Also, are you saying that dotnet SDK version of compiler cannot compile projects using type providers when the target is .NET framework?

[cartermp]

No, but it's complicated. Old-style projects I would not expect to be buildable by the .NET SDK. New-style is .. it depends. How much messing around with loading different toolsets is being defined in the type provider? A common problem I see is an enormously complex build for type providers that attempt to programmatically load compilers and targets based on certain conditions. That stuff just rarely ever works.

[smoothdeveloper]

@cartermp thanks for giving hint about the underlying issue.

A common problem I see is an enormously complex build for type providers that attempt to programmatically load compilers and targets based on certain conditions. That stuff just rarely ever works.

In case of FSharp.Data.SqlClient, it has two "DesignTime" assemblies:

the netstandard one references the types from:

// Detected Target-Framework-Id: .NETStandard,Version=v2.0

// Referenced assemblies (in metadata order):
// FSharp.Core, Version=4.7.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
	// Assembly reference loading information:
	// There were some problems during assembly reference load, see below for more information!
	// Error: Could not find reference: FSharp.Core, Version=4.7.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a

// Microsoft.SqlServer.TransactSql.ScriptDom, Version=15.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91
	// Assembly reference loading information:
	// There were some problems during assembly reference load, see below for more information!
	// Error: Could not find reference: Microsoft.SqlServer.TransactSql.ScriptDom, Version=15.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91

// mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
	// Assembly reference loading information:
	// Info: Success - Loading from: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.1.4\mscorlib.dll

// netstandard, Version=2.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
	// Assembly reference loading information:
	// Info: Success - Loading from: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\2.1.4\netstandard.dll

// System.Configuration.ConfigurationManager, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
	// Assembly reference loading information:
	// Info: Success - Loading from: C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App\3.0.0-preview6-27804-01\System.Configuration.ConfigurationManager.dll

// System.Data.SqlClient, Version=4.4.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
	// Assembly reference loading information:
	// Info: Success - Loading from: C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App\2.1.4\System.Data.SqlClient.dll

// System.Runtime.Caching, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
	// Assembly reference loading information:
	// Info: Success - Loading from: C:\WINDOWS\Microsoft.NET\assembly\GAC_MSIL\System.Runtime.Caching\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Runtime.Caching.dll

// Detected Target-Framework-Id: .NETFramework,Version=v4.6.1

// Referenced assemblies (in metadata order):
// FSharp.Core, Version=4.4.3.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (unresolved)
// Microsoft.SqlServer.TransactSql.ScriptDom, Version=15.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91 (unresolved)
// Microsoft.SqlServer.Types, Version=14.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91 (unresolved)
// mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
	// Assembly reference loading information:
	// Info: Success - Found in Assembly List

// System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 (unresolved)
// System.Configuration, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a (unresolved)
// System.Core, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 (unresolved)
// System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 (unresolved)
// System.Numerics, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 (unresolved)
// System.Runtime.Caching, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
	// Assembly reference loading information:
	// Info: Success - Found in Assembly List

Is there anything which should be done in the implementation of that particular provider which would enable support from dotnet core version of fsc to compile net framework? (In which case we'd close this issue and improve the TP / TP SDK about those problems).

As a work around, is there a way to specify another fsc.exe to be used in those projects causing troubles so building from command line through dotnet build would call into the .NET framework version of the compiler?

edit: https://github.com/MarneeDear/FSharp.Data.SqlClient-dotnet/blob/master/fsc.props seems like a viable workaround.

I'd still like to better understand the whole lifecycle that lead to the issue happening, and get some pointers to the code paths in compiler dealing with this.

I assume the dotnet SDK compiler is loading the netstandard "DesignTime" assembly, but not sure what follows.

[isaacabraham]

@smoothdeveloper I think the best thing to do would be create an empty (new) TP using the latest TP template and see how that one works to diff between it and the SQLClient one. That was going to be my approach for the Azure one, although I never got around to it.

[cartermp]

I think your best bet is to stop trying to support net461. Just move only to .NET Standard 2.0 and simplify everything. The template here shows how to do that: https://github.com/fsprojects/FSharp.TypeProviders.SDK

[smoothdeveloper]

After a quick test removing .net framework target, a .net framework command line project fails to build in VS:

C:\dev\src\github.com\fsprojects\FSharp.Data.SqlClient\src\SqlClient.TestProjects\SqlClient.Tests.NET40\Program.fs(8,21): error FS3033: The type provider 'FSharp.Data.SqlCommandProvider' reported an error: Could not load file or assembly 'System.Configuration.ConfigurationManager, Version=4.0.1.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflection-only loader context. (Exception from HRESULT: 0x80131058)

This is despite adding that nuget package to the project, I'm not sure what is missing (in this case, that project is compiling with net48 target despite the name).

Also, the type provider targets net40 runtime, droping the .net framework binaries of the package would drop support for that target.

I guess my main concern is that dotnet core compiler resolves assemblies differently than .net framework even when targetting .net framework, eventually the .net framework compiler won't be updated anymore.

In this context, it may be worth the effort to make the dotnet core compiler behave the same when targetting .net framework.

[cartermp]

In this context, it may be worth the effort to make the dotnet core compiler behave the same when targetting .net framework.

Nooooo, that will absolutely never happen. @KevinRansom might get a heart attack if that were to happen.

I do not recommend supporting unsupported .NET FX runtimes like net40. It will truly be in the best long-term interest of this type provider to just target netstandard2.0 and build only with the .NET SDK compiler. Running into fundamental design issues like this are just what you're going to be signing yourself up for if you don't do that.

[KevinRansom]

@smoothdeveloper , @cartermp --- I will read this later ... the notion that I may have a heart attack is not attractive to me. Coffee and breakfast first, then i will have the fortitude to risk a medical event.

[KevinRansom]

@smoothdeveloper

• the coreclr version of fsc will require type providers to be compiled with the netstandard2.0 references if they are also expected to build with the desktop fcs compiler shipped in Visual Studio. So ... netstandard 2.0 TP's will run in VS and dotnet sdk.

• If that is not possible, then building the TP for desktop and also netcoreapp* and packaging in a nuget package should also work correctly.

• The current TP SDK demonstrates how to do this.

• Legacy .fsproj files are not expected to work correctly in dotnet sdk. Visual Studio, will of course maintain support for them, both for build and editing. We stopped deploying templates for legacy F# projects earlier this year If I remember correctly.

• Yeah ... we are focusing on 4.7.2+ and netcoreapp3.1+ for F# moving forwards. We expect to keep existing projects working in the scenarios they used to work in, but old projects don't get access to new language features or tooling.

• Before long we are going to deprecate the desktop FSC compiler and FSI in VS replacing them with coreclr versions. We will likely freeze the desktop fsc compiler and fsi and provide a mechanism to enable them, for developers who really need the old compiler, for a transition period.

I hope this clears things up, and at least explains some of the things you are seeing?

Kevin

[cartermp]

The current TP SDK demonstrates how to do this.

Actually, not anymore, since it was so fragile/fraught with complexity. It is now entirely .NET Standard 2.0-focused.

[smoothdeveloper]

Thanks for the feedback and fuller overview guys, it makes it clear that it is on the TP providers (!) side to deal with this.

I guess the challenge for now is to get net framework projects consume a net standard only release of the providers I'm dealing with and can be tracked in the specific projects instead.

I'm closing this, maybe should be tagged by-design or such.

[goswinr]

@smoothdeveloper @KevinRansom I have similar problem with the Excel provider.
I am using the latest Type provider SDK and target net48 in my own fork, but dotnet build still fails when using the Type provider with
error FS3033: The type provider 'FSharp.Interop.Excel.ExcelProvider.ProviderImplementation+ExcelProvider' reported an error: Could not create ExcelDataReader. NotSupportedException - No data is available for encoding 1252. For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method.
However, building in Visual Studio works just fine.
cc @quintusm
Is there a workaround that would allow me to use dotnet build ?

[kerams]

@goswinr, that does not seem like the original issue. I've seen the error, albeit on .NET Core and at runtime, and solved it with Encoding.RegisterProvider CodePagesEncodingProvider.Instance. Try calling that during the initialization of the design part of your TP, it could help.

[KevinRansom]

@goswinr --- could you provide a simplified repro or point to a github repo with instructions, and we can take a look ... thanks

[goswinr]

thanks, @kerams CodePagesEncodingProvider is .NET core only. 🤷‍♂️
@KevinRansom @vzarytovskii the problem is in how I build my own fork. With the Excelprovider Nuget it works fine. I noticed that the Runtime.dll from my own build is 1234 kB while the one from Nuget is just 45 kB in the Nuget.
But building the original TP seems broken too.
Anyway, I think you can close this issue for now, since it only affects my fork.

[smoothdeveloper]

@vzarytovskii / @KevinRansom can we remove "needs-repro", and reopen this issue?

I'm attaching another sample, which involves a type provider that supports netstandard, switching the target to net6.0 in the sample doesn't change the outcome.

sqlclienttp.zip

Maybe there is a way to improve the compiler report more details about the precise issue hit during the compilation and how to adjust for it?

Actual

Builds with:

• VS
• msbuild from developer command prompt

Fails to build with:

• dotnet build
• dotnet msbuild

error FS3033: The type provider 'FSharp.Data.SqlCommandProvider' r
eported an error: Could not load file or assembly 'System.Data.SqlClient, Version=4.4.0.0, Culture=neutral, PublicKeyTo
ken=b03f5f7f11d50a3a'. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflect
ion-only loader context. (0x80131058)
error FS3033: The type provider 'FSharp.Data.SqlCommandProvider' r
eported an error: Could not load file or assembly 'System.Data.SqlClient, Version=4.4.0.0, Culture=neutral, PublicKeyTo
ken=b03f5f7f11d50a3a'. Reference assemblies should not be loaded for execution. They can only be loaded in the Reflect
ion-only loader context. (0x80131058)

Expected

Builds in all those scenarios.

[jimfoye]

I have a project that targets net5.0-windows and uses SqlCommandProvider and I get this error with dotnet publish. But if I right click on the project in VS and select "Publish..." from the context menu, it works. I even call dotnet publish with the parameter -p:PublishProfile=FolderProfile.pubxml so I would think this is actually the exact equivalent of asking VS to do it. I'm not really sure what to do - I don't think the 5.0 target is the problem, right? (That's the highest version VS 2019 supports, so I'm stuck there). I'm trying to onboard someone with "see how great F# and type providers are" and I could use a good explanation on why I'm having this problem and so cannot automate the build entirely with a build script.

[Edit] I realize this is not a showstopper, dotnet publish is just building and copying files, I can do a build script that doesn't rely on it.