add `NoDiff` backend

[oscardssmith]

IMO it would be nice to have a NoDiff backed for use when we aren't expecting to do any differentiation (e.g. within OrdinaryDiffEq when the user is providing a jac and tgrad etc manually)

[topolarity]

This should even lead to some code-size improvements for --trim, since we can delete the unnecessary autodiff code.

[ChrisRackauckas]

I assume the behavior we would want would be to error if any call to DI is done, rather than no-op?

[gdalle]

It will error by default due to the lack of an appropriate method. My only concern is that the error might be a rather obscure MethodError, possibly not even in a very obvious function like DI.gradient but in a more subtle preliminary step like DI.pick_batchsize

[ChrisRackauckas]

I would think a purposeful error that is informative (to an LLM) would be very helpful.

[gdalle]

The issue being that we would need to define every single method of DI on NoDiff, like we do for all other backends. Which seems a bit over the top for something that is supposed to do nothing

[ChrisRackauckas]

Possibly yeah.. but I do think the better error messages would be worth it.

[oschulz]

I'd love to have such a NoAutoDiff <: AbstractADType too. For examples as a default argument for algorithms or computing contexts - I currently use Nothing or so, but a subtype of AbstractADType would be cleaner and avoid Union{Nothing,AbstractADType} in multiple places (and enable some things via dispatch that get tricky/awkward otherwise).

The issue being that we would need to define every single method of DI

I'd prefer that, because we could give better error messages, but even MethodError would do in my use cases. Algorihms that need a gradient would check if the user has passed NoAutoDiff() and give a specific error message, for example.

[oschulz]

I'd be happy to do a PR, also for DI if the consensus is that we want the better errors.

[ChrisRackauckas]

please do

[gdalle]

In DI maybe an error hint would be better than changing the method type? Fundamentally, a MethodError is the correct one semantically, but we can augment it with more help, as I do when the backend is not loaded for example: https://github.com/JuliaDiff/DifferentiationInterface.jl/blob/main/DifferentiationInterface/src/init.jl

[ChrisRackauckas]

yes an error hint that says "you called DI but NoAD mean that you shouldn't AD... are you sure you did this right?".

[oschulz]

Yep. And specific algorithms could catch those and then rethrow something even more specific, like "gradient-based whatever needs autodiff but you specified NoAutoDiff", etc.

[gdalle]

Well you can't catch an error hint, they would have to catch every MethodError and then sort it out

[oschulz]

Sorry, I mixed that up - if it's MethodError then algorithms will have to check if ad isa NoAutoDiff explicitly for algorithm-specific error. But if we throw a custom AutoDiffNotAvailable exception in DI, then it could be done more generically, resp. at higher/outer levels.

[oschulz]

But I'd be happy with a MethodError and error hint too.