Please add support for AD authentication type

[NorDroN]

No description provided.

[MattB-msft]

This is an area we are exploring but have not committed too as of yet. In general we are strongly encouraging the use of ADFS for On Premises systems.

[impr3ssi0n]

Also interested in the possibility of the AD authentication type. Please add this.

[ttkoma]

Please any news by this?

[stepanhakobyan]

Please. We want to (and partially do) upgrade our services to .NET Core and further. But without .NET Core client libraries to connect to On Premise Dynamics 365 it is a no go.
Microsoft wants us to upgrade everything to .NET (core), but the Dynamics team does not, or provides a very difficult path. It should be easy and painless.

[MattB-msft]

Insofar as formal support for AD, AD Federated, and Remote AD domain federation, there is an increasing potential we will be able to support it in the future based on .net 5 baseline. Still TBD as we are seeking a way to support this that would not require a patch to onPremise.

The direct issue there is the realm of things that come with AD support that were baked into the previous client. Most of which are dropped with the WS-TRUST deprecation for Online.

You can use oAuth onPremise assuming you have an ADFS deployment to work with, we do document setting that up in the onpremise deployment documents for CRM 2016+. and this is what we would recommend when working with OnPrem for now.

[FixRM]

@MattB-msft, all server-to-server auth schemas are officially NOT supported for on-premise. It's documentation issue, I have support ticket about that: 2107050040001964. OAuth silent authentication seems partly working, but with IntegratedSecurity option but that means running in context of ActiveDirectory authenticated user process.

[impr3ssi0n]

Any news on this? We have migrated to v9 and want to migrate everything to .Net core to take full advantage of it.

[ONYXDave]

@MattB-msft you mentioned that there is a document on how to setup ADFS oAuth for CRM 2016+. Could you provide a link. All I can find is how to setup ADFS but without oAuth. Why was WS-Trust dropped just because it was depreciated for Online?

[FixRM]

@ONYXDave there is no such document, only this video: https://www.youtube.com/watch?v=26OnuDSOZf4

[MattB-msft]

This is documented folks.
It starts with configuring IFD: https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/deploy/configure-internet-facing-deployment?view=op-9-1
then once configured.. you use this command ( Doc) to enable OAuth in CRM OnPrem : https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/deploy/post-installation-configuration-guidelines-dynamics-365?view=op-9-1#configure-the-oauth-provider

Also, S2S was not stated to work for OnPrem @FixRM ,
We realized that the doc's were misleading there and fixed that recently to clearly state that S2S only works online.

for onPrem you need to use named user accounts

[FixRM]

I guess folks are asking how to setup S2S (non-user interactive) auth process @MattB-msft. This video basically tells how to do it with OAuth password flow which is suitable for the task. It requires lot of additional steps on top of documented configuration.

[MattB-msft]

@FixRM
In on-prem .. Application Users ( S2S / Confidential Client ) are not support at all. you would use a named user ( using oAuth is fine ) for that configuration. I will look thought the video and pass it on to our Auth folks to have a look at updating our documentation with any missing steps. Thanks for pointing it out.

Our documentation was not clear on the subject of application users up till recently when the question came up and we clarified the documentation.

There is an undocumented path of using Azure AAD for OnPrem that can support an Application User / S2S path. Its undocumented as there are unsupported modifications required to support Dynamics's Outlook Web Integration which we hope to sort out in the future.

[FixRM]

@MattB-msft that's right, we use named users. The problem here is to somehow get OAuth token without user interaction

[abelykh0]

Can you reuse some code from https://github.com/Data8/DataverseClient, since it is MIT?

[MattB-msft]

As long as you comply with the license rules associated to this repro. You are free to use it. Before you do though, which part are you interested in? There are several areas still under development. MattB

…

________________________________ From: Andrey Belykh ***@***.***> Sent: Wednesday, February 9, 2022 1:38:09 PM To: microsoft/PowerPlatform-DataverseServiceClient ***@***.***> Cc: MattB ***@***.***>; Mention ***@***.***> Subject: Re: [microsoft/PowerPlatform-DataverseServiceClient] Please add support for AD authentication type (#110) Can you reuse some code from https://github.com/Data8/DataverseClient, since it is MIT? — Reply to this email directly, view it on GitHub<#110 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACQUENGYSV62FGRXL4SI4NLU2LNEDANCNFSM4Z3PBRRQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[abelykh0]

As long as you comply with the license rules associated to this repro. You are free to use it. Before you do though, which part are you interested in? There are several areas still under development. MattB

If Microsoft could incorporate a code from https://github.com/Data8/DataverseClient into this line, would be great. Even greater, if you could implement missing AD authentication

[MattB-msft]

Ah, I'm sorry I misunderstood your question. I was looking at this thread on a phone.

You were asking if the DV ServiceClient could take code from another, non-Microsoft project. In short, 'it depends', there are a lot of rules we have to follow concerning accepting code from developers outside of Microsoft.

In the case of AD authentication support, that was intentionally removed from the Dataverse ServiceClient as part of the .net port. That code, along with the WS-TRUST supporting code, had a lot of dead code from the various evolutions over the years that CrmServiceClient has been in operation. There were (past-tense) technical issues with AD support in the .net core libs and the API targets that blocked the use of AD.

We are looking at adding support for OnPrem AD back in, however we may or may not add support for WS-TRUST for onPrem on .net core.

[abelykh0]

Ah, I'm sorry I misunderstood your question. I was looking at this thread on a phone.

You were asking if the DV ServiceClient could take code from another, non-Microsoft project. In short, 'it depends', there are a lot of rules we have to follow concerning accepting code from developers outside of Microsoft.

In the case of AD authentication support, that was intentionally removed from the Dataverse ServiceClient as part of the .net port. That code, along with the WS-TRUST supporting code, had a lot of dead code from the various evolutions over the years that CrmServiceClient has been in operation. There were (past-tense) technical issues with AD support in the .net core libs and the API targets that blocked the use of AD.

We are looking at adding support for OnPrem AD back in, however we may or may not add support for WS-TRUST for onPrem on .net core.

Thanks a lot for clarification.

[MattB-msft]

Good news / bad news sort of thing here folks. - Bit of an update...
The next drop of the client will have the groundwork for OnPrem in it using AD auth. Its full framework only though. so .net core will not work with AD Auth on Prem.

The initial bits will be limited in what they can do, all the Async operations should work, however .Clone operations do not work as of this check-in.

There is a lot more work required to support AD in .net core based on our current clients, we will continue to look at it as we move forward with this client.

[impr3ssi0n]

Good news / bad news sort of thing here folks. - Bit of an update... The next drop of the client will have the groundwork for OnPrem in it using AD auth. Its full framework only though. so .net core will not work with AD Auth on Prem.

The initial bits will be limited in what they can do, all the Async operations should work, however .Clone operations do not work as of this check-in.

There is a lot more work required to support AD in .net core based on our current clients, we will continue to look at it as we move forward with this client.

Good news. Hopefully .net core support in the future.

[abelykh0]

By the way, it is OK (for us) if the AD support is only for Windows on .Net 6, Linux can come later or never.

[MattB-msft]

Our goal is to allow AD authentication to work for .net core, currently though we have it working only in full framework. Depending on how far we get, we may ship the next drop with only .net framework support.

For transparency, the issue is more with how our API manages the configuration of various types of authentications for on-premises vs anything specific about AD itself, at least we have not discovered anything specific to .net core yet :)

[MattB-msft]

Ok folks, We have been at this for a few days now across a few teams.

Short story is this:
AD Auth over .net core is not going to work based on our current on-premise Server infrastructure.
The issue comes down to .net core + removing support both methods of TransportSecurty Bindings we use to secure AD credentials on the wire. Those protocols are tied to Windows's and so far the .net core team has decided to not support them.

We are discussing adding support for AD auth in a different way to the onprem servers, however that is not going to help with existing deployments as any change we do there would be patched into in a service release and apply only going forward.

At this point we are going to ship AD support for Full Framework in the next drop and turn our attention to making claims and oAuth work properly for On-Premises in full framework and .net core.

Sorry for the bad news there.

[dotfede]

@MattB-msft yeah, so:

1 - The entire purpose of this library is to move away from LEGACY Windows-only .NET 4.x. It is very important that you stop calling it "full" framework, because it's not "full" in any other way than "fully legacy", and no one cares about it, except people maintaining legacy software, and the entirety of Microsoft has clearly stated that REAL .NET (meaning .NET 6 and above, not the legacy one) is the way forward, only Power Platform people seem to not have noticed this.

2 - If the purpose of the library cannot be fulfilled due to whatever legacy reasons, then this library has no reason to exist at all, because Microsoft.Xrm.Sdk has been working just fine (for legacy .NET) for decades, and this library CANNOT replace it because it doesn't support the use cases supported by the former.

All in all, every single day I'm more and more thankful for having moved away from legacy, XML bloated, limitation-ridden CDS.

[impr3ssi0n]

I agree with fberasategui. Hope was on .net core. To migrate all services to the new platform. Otherwise, this library makes no sense.

[MattB-msft]

@fberasategui
I am unclear as to what your concern is, we are attempting to add in AD support at the request of the user population who is still working against the On-Premises configuration.
Regarding your objection to the term ".net Full Framework, or .net Framework" this is what the .net team calls that system. We didn't coin that term.

The issue, as I outlined, is that the .net core ( 3.x - 7 as if this writing) does not include support for the security / encryption system for the transport layer required to support AD authentication to the Dataverse server. It actually does not have anything to do with the Xrm Library.

@impr3ssi0n This lib is 100% .net core.
We are, however, we are limited by what .net core and its supporting modules can actually support, in this case AD to the Dataverse Servers on-premise. Thus, when running it under .net core there will be a set of features that are not enabled.

[dotfede]

@MattB-msft

I am unclear as to what your concern is

My concern is that unfortunately I am tied to maintaining legacy projects on your legacy platform, that's not even capable (not even with all the money and manpower in the world) to keep up with the advancements in technology your own company produces.

Luckily enough, the fruitless discussion about how your platform has no proper abstractions (from almost 2 years ago now, of which of course NOTHING has changed in 2 years) was an eye opener for me to realize that I should move off of it, and all our greenfield projects since then were based on a much better platform: mine.

we are attempting to add in AD support at the request of the user population who is still working against the On-Premises configuration

No you aren't. As other people have stated here, again: the main purpose of this library is to move away from legacy .NET. If it can't provide that, then it's worthless. Why would I refactor a huge CRM codebase with hundreds of plugins, external APIs, services, and a lot of stuff that works fine today using Microsoft.Xrm.Sdk, if by doing that I will still be stuck in legacy .NET instead of the real one?

we are limited by what .net core and its supporting modules can actually support

Ah, yes. This is 100% Microsoft mentality. If something isn't built-into the framework that means it can't exist. Ever. It's not like your team (which I'm sure has budgets in the order of several hundred million dollars annually) could spend resources on actually implementing whatever stuff is needed to support your legacy platform on real .NET.

[abelykh0]

Do you have a good article that describes how to connect to on premise using this library? Most of the documents I can find are about the old (Microsoft.Xrm.Sdk) library. Is it only possible if IFD is configured?

[MattB-msft]

@abelykh0 - its only possible with IFD + OAuth configured.
if you want to use the CrmServiceClient, that does support AD and IFD scenarios without oAuth

The Dataverse ServiceClient itself will support ADFS 2019+ for onPrem oAuth,
That said, at the moment, we are suggesting folks use the external auth hooks we provide to do it, as there is a bug in the way we are arriving at the authority information for OnPrem.

[MarkMpn]

I've now got a proof of concept extension to the Data8.PowerPlatform.Dataverse.Client package to support AD authentication as well as IFD. I've still got a lot of testing & tidying up to do before it's in a state I can release it, but I can now authenticate and execute requests against a couple of test servers without claims authentication configured using .NET Core 3.1.

[MattB-msft]

@MarkMpn Looking forward to seeing it.
I suspect you had to modify the Web.config file of the Server to allow this to work?

[MarkMpn]

@MattB-msft No server changes necessary, but the client has to do all the SSPI and XML signature work manually rather than using WCF.

Although it's .NET Core it will only run on Windows as it depends on the SSPI functions from the Windows SDK, not sure if there's an equivalent on other platforms that it could use but Windows is enough for me at the moment.

Thankfully it still seems to be able to reuse the DataContractSerializer to handle the requests and responses - please just keep KnownTypesResolver public!

[FixRM]

A little update for the folks who decided to go on with the OAuth instead of AD auth. As it turns out, it may not be necessary to configure ADFS Application Groups to use password OAuth flow. May be it's enough to Grant-AdfsApplicationPermission as said in Post installation steps. I remember that it was not working when I started experimenting and auth token was issued without required claims such as UPN if you request it without passing client secret. Now I it works... Unfortunately, I'm not ADFS expert and I didn't find any sane documentation on that, but this fact may simplify setup for you. For clean installation you may try:

1. Setup IFD + enable OAuth
2. AddAdfsClient
3. AdfsApplicationPermission
4. Use custom Auth Hook to get token manually

[MattB-msft]

@MattB-msft No server changes necessary, but the client has to do all the SSPI and XML signature work manually rather than using WCF.

Although it's .NET Core it will only run on Windows as it depends on the SSPI functions from the Windows SDK, not sure if there's an equivalent on other platforms that it could use but Windows is enough for me at the moment.

Ok, so you went that way. You're dropping into the Win32 Security API itself to access the bins necessary to create or emulate the behavior of the SymmetricSecurityBindingElement in full framework WCF, which is what we require for AD Auth?

We shied away from that due to a number of reasons, the binding to Windows Only stack, lack of support from the WCF team in .net core, other reasons that have to do with future investments in our communications infrastructure.
I am curious how you ended up doing the interface. If it's something we think we can clear though our security review process, we may ask for you to contribute to the main client, if your open to that.

Regarding the known types resolver, its public :) and will stay that way.

[MarkMpn]

@MattB-msft I've just pushed this code now, see https://github.com/Data8/DataverseClient/

I've also updated the package at https://www.nuget.org/packages/Data8.PowerPlatform.Dataverse.Client for anyone that wants to get started using this now.

[vmcbaptista]

@MarkMpn Thanks a lot for your efforts, this is something I'm planning to give a try in the next couple of weeks.
Is it something you're using or planning to use for production scenarios?

@MattB-msft Did your team by any chance had a look already in @MarkMpn work?
If yes, is it something you see that might be introduced to the official client or something that you would recommend to anyone needing to connect to On-Premises using AD authentication from .NET Core?

[MattB-msft]

@vmcbaptista
Yes, we have looked at it, and we are talking to our peers in identity and .net core about this. Given that what @MarkMpn is doing is using code that is working with the win32 SSPI API directly its our position that this is a feature that should be made available inside the .net core stack targeted at the windows hosts.

This is an ongoing discussion.

[Misiu]

@MattB-msft any public issues, we can track or vote on?
Official support would be awesome!

[MattB-msft]

Right now, the AD support conversation is centered around support for the underlying SSPI framework on windows.
If you want to help advocate, you can speak up on the WCF Git hub site..
one of several issues on this is here : dotnet/runtime#25320
.net is working on a series of updates to WCF now and may include the stuff we need in the upcoming work.

[Misiu]

@MattB-msft any updates? version 1.0.1 has been released. I'm currently using Data8 package for querying data from Dynamics, but I'd like to switch back to the MS package before I do a full rewrite of an old project.

[FixRM]

@MattB-msft what about adopting newly introduced https://github.com/CoreWCF/CoreWCF. I guess it' may even help migrate "Dataverse" to "Core" itself?

[Misiu]

@MattB-msft the package that @MarkMpn works great. I've just updated my project to .NET 7.0 RC2, dockerized it, and was able to use the AD auth type on a Linux host!

[peterwidmer]

Thank you @MarkMpn - I'm using your solution successfully with OnPremise AD!
@MattB-msft - I'd be really glad if you could support this officially!

[janis-veinbergs]

Good news, I've documented what is needed to get Dataverse ServiceClient work with on-premises CRM using only connection string.

• FD + enable OAuth
• AddAdfsClient
• AdfsApplicationPermission
• Use custom Auth Hook to get token manually

@FixRM spot on! Sans the last step :) The last step would be required if you have ADFS2016, but for ADFS2019 connection string is all you need.

But there is more to CRM configuration that is more of a workaround for existing bug in Dataverse ServiceClient. Read the whole thing at Using Dataverse Service Client to connect to OnPrem Dynamics 365 CRM (From .NET 6+) - i tried to document all the things needed and provide samples on how to validate your configuration and connection to CRM.

[FixRM]

Thanks, @janis-veinbergs! Great work! I thought it was always possible to workaround with CrmServiceClient.AuthOverrideHook?

[janis-veinbergs]

Thanks, @janis-veinbergs! Great work! I thought it was always possible to workaround with CrmServiceClient.AuthOverrideHook?

You are referring to .NET Core incompatible CrmServiceClient.

For DVSC and Adfs2016 you also have to resort to tokenProviderFunction. But Adfs2019 is supported natively. That means if some tools depend on DVSC, one can pass connection string and target OnPrem instance.

[FixRM]

I see, thank you @janis-veinbergs. I hope smaller brother will finally see some love MS. May be when 9.2 will arrive on-premise)

[janis-veinbergs]

I see, thank you @janis-veinbergs. I hope smaller brother will finally see some love MS. May be when 9.2 will arrive on-premise)

They drifted into azure so much that I'm not holding my breath:
Dynamics 365 application modernization on Microsoft Azure.

[FixRM]

I see, thank you @janis-veinbergs. I hope smaller brother will finally see some love MS. May be when 9.2 will arrive on-premise)

They drifted into azure so much that I'm not holding my breath: Dynamics 365 application modernization on Microsoft Azure.

It looks like their key investments are still in dead-born AI stuff. Meanwhile they are still married with .NET Framework and Windows ecosystem.