CachingOptimizer is calling Utilities.get_fallback when ConstraintPrimal is not supported.
It would be useful if it was doing the same for ObjectiveValue and DualObjectiveValue.
Moreover, solvers that do support ObjectiveValue and DualObjectiveValue but that just throw infinite values at infeasibility certificate would be able to throw the appropriate error so that if there is a CachingOptimizer layer, it catches it and return the right value using get_fallback.
Solvers needing this: CSDP, SDPA, SDPT3, SeDuMi, OSQP (infinite ObjectiveValue for unbounded ray and does not support DualObjectiveValue).