Skip to content

Value-Function Differentiation

ValueFuncLayer is used when a scalar training loss depends on the optimal objective value

\[ \alpha(\hat y) := f_0(\hat z^\star,\hat y), \qquad \hat z^\star=\mathcal O(\hat y). \]

Unlike solution-map differentiation, the value-function derivative can be formed directly from the primal-dual optimum returned by the forward solve.

Canonical value function

For the canonical APQP, define

\[ \alpha(\hat y) = \min_{\tilde z} \left\{ \frac{1}{2}\tilde z^T P\tilde z +q(\hat y)^T\tilde z +c(\hat y) : A\tilde z=b(\hat y),\ G\tilde z\le h(\hat y) \right\}. \]

The parameter maps are

\[ \begin{aligned} q(\hat y) &=\tilde q+\sum_i Q_i\hat y_i, & c(\hat y) &=\tilde c+\sum_i d_i^T\hat y_i,\\ b(\hat y) &=\tilde b+\sum_i B_i\hat y_i, & h(\hat y) &=\tilde h+\sum_i H_i\hat y_i. \end{aligned} \]

See APQP and canonicalization for the full problem definition.

Direct derivative

The canonical Lagrangian is

\[ \mathcal L = \frac{1}{2}\tilde z^T P\tilde z +q(\hat y)^T\tilde z +c(\hat y) +\tilde\nu^T(A\tilde z-b(\hat y)) +\tilde\lambda^T(G\tilde z-h(\hat y)). \]

At an optimal primal-dual point, the envelope theorem removes the derivative of the optimizer itself. Differentiating the Lagrangian only through the explicit dependence on parameter block \(\hat y_i\) gives

\[ \boxed{ \mathrm D_{\hat y_i}\alpha(\hat y) = \tilde z^{\star,T}Q_i +d_i^T -\tilde\nu^{\star,T}B_i -\tilde\lambda^{\star,T}H_i }. \]

Each term has a direct interpretation:

Term Source of sensitivity
\(\tilde z^{\star,T}Q_i\) Linear objective coefficient
\(d_i^T\) Objective offset
\(-\tilde\nu^{\star,T}B_i\) Equality right-hand side
\(-\tilde\lambda^{\star,T}H_i\) Inequality right-hand side

The negative signs on the constraint terms follow from \(A\tilde z-b(\hat y)\) and \(G\tilde z-h(\hat y)\) in the Lagrangian.

Backpropagation to the forecaster

For \(\hat y_i=f_i(x,\theta)\) and a scalar loss \(\ell(\alpha(\hat y))\), the chain rule gives

\[ \mathrm D_\theta\ell = \mathrm D_\alpha\ell \sum_{i=1}^{N_p} \left( \mathrm D_{\hat y_i}\alpha(\hat y)\, \mathrm D_\theta f_i(x,\theta) \right). \]

PyTorch supplies \(\mathrm D_\alpha\ell\) and differentiates the forecaster. DiffAPQP supplies \(\mathrm D_{\hat y_i}\alpha(\hat y)\) from the forward primal-dual solution.

Why no adjoint KKT solve is needed

The forward pass already provides:

  • \(\tilde z^\star\);
  • \(\tilde\nu^\star\) and \(\tilde\lambda^\star\);
  • \(Q_i\), \(d_i\), \(B_i\), and \(H_i\) from canonicalization.

The backward pass therefore requires matrix-vector products but no full- or reduced-KKT linear solve. This is the sense in which value-function differentiation can be considered "differentiation for free": the expensive implicit adjoint solve is absent, although the forward optimization and backward arithmetic still remain.

The forward solve can use the same warm-start, update, and batch mechanisms described in the efficient forward pass.

Comparison with the solution map

Property Value-function layer Solution-map layer
Differentiable output \(\alpha(\hat y)\) \(\hat z^\star\)
Needs KKT adjoint solve No Yes
Needs active-set detection No Only for reduced KKT
Primal solution returned Yes, inspection only Yes, autograd connected
Appropriate loss Depends on optimal objective Depends on optimal decisions

Avoiding the adjoint system removes the numerical failure mode associated with solving a singular full or reduced KKT matrix. It does not guarantee that \(\alpha\) is differentiable everywhere. At parameter values where the value function is nonsmooth, the primal-dual solution may define a valid directional derivative or subgradient rather than a unique classical gradient.

Counterfactual learning

For an oracle value \(\alpha(y)\) computed from realized parameters, a common counterfactual loss is

\[ \ell_{\mathrm{value}} = \frac{1}{2} \left( \alpha(\hat y)-\alpha(y) \right)^2. \]

Its derivative with respect to the predicted parameters is

\[ \mathrm D_{\hat y}\ell_{\mathrm{value}} = \left( \alpha(\hat y)-\alpha(y) \right) \mathrm D_{\hat y}\alpha(\hat y). \]

This setting maps directly to value_batch from ValueFuncLayer. The returned primal_dict, equality duals, and inequality duals are available for inspection, but only value_batch is connected to autograd.

For losses built from the decisions themselves, use the solution-map theory and SolMapLayer.