Skip to content

Efficient Forward Pass

The forward pass evaluates the optimization map for every sample in a batch. DiffAPQP separates this operation into canonicalization, solution, and retrieval:

\[ \mathcal O = \mathcal R\circ\mathcal S\circ\mathcal C. \]

This decomposition exposes the repeated structure of decision-focused training and allows the forward solver to be chosen independently of the backward linear-system method.

Canonicalization, solution, retrieval

Given optimization parameters \(\hat y\), canonicalization produces

\[ \mathcal C(\hat y) = \big(P,q(\hat y),c(\hat y),A,b(\hat y),G,h(\hat y)\big). \]

The solver map returns a canonical primal-dual optimum:

\[ \big( \tilde z^\star, \tilde\nu^\star, \tilde\lambda^\star \big) = \mathcal S\big( P,q(\hat y),A,b(\hat y),G,h(\hat y) \big). \]

The scalar offset \(c(\hat y)\) changes the optimal value but not the optimizer. Finally, retrieval returns the named decisions in their original CVXPY shapes:

\[ \hat z^\star = \mathcal R(\tilde z^\star). \]

Layer construction extracts the APQP template and creates reusable standard-form problems. A forward call supplies each sample's \(\hat y\), stuffs the corresponding affine data, solves the instances, and retrieves the public outputs. See APQP and canonicalization for the data maps and the implementation pages for the recorded time_batch stages.

Disentangled forward and backward methods

The forward problem is an ordinary convex QP or LP. The backward pass needs only the resulting primal-dual optimum and canonical APQP data; it does not need to differentiate the solver iterations themselves. DiffAPQP can therefore use a QP/LP-compatible CVXPY solver in the forward pass and independently choose an adjoint linear solver in the backward pass.

This separation has two consequences:

  1. dedicated QP solvers can operate directly on the quadratic objective;
  2. the forward solver is not constrained to be the same numerical method used for implicit differentiation.

Available backends and DiffAPQP's configured defaults are listed on the solver page. Full- and reduced-KKT backward choices are described in solution-map differentiation.

Why solver-data update can help

Many iterative QP algorithms repeatedly solve a linear system. For an ADMM-style method, a representative system is

\[ \begin{pmatrix} P+\rho G^T G & A^T\\ A & 0 \end{pmatrix} \begin{pmatrix} \tilde z^{k+1}\\ w^{k+1} \end{pmatrix} = \begin{pmatrix} \rho G^T(\zeta^k-u^k)-q(\hat y)\\ b(\hat y) \end{pmatrix}. \]

For fixed \(\rho\), the coefficient matrix depends on \(P\), \(A\), and \(G\), which are constant throughout an APQP family. Parameter changes affect the right-hand side and bound data. A solver can therefore reuse its workspace, sparsity analysis, and, when its native algorithm permits, matrix factorizations.

In the DiffAPQP interface, update=True requests this in-place solver-data update path. The first solve creates the workspace; update becomes effective on later calls for the same batch slot. Exact reuse behavior remains solver-specific. See Warm-start and update lifecycle for the supported custom paths.

Why warm-starting can help

For a fixed dataset sample, predictions typically change gradually between training epochs:

\[ \hat y^{(1)},\hat y^{(2)},\ldots,\hat y^{(N_{\mathrm{epoch}})}. \]

The previous primal-dual iterate can therefore be a useful initialization for the next solve. DiffAPQP's explicit warm-start cache is indexed by stable sample identifiers:

\[ \text{sample id} \longmapsto \text{latest solver-native state}. \]

With warm_start=True, the first successful solve populates the cache. A later call with the same idx_list can reuse the state associated with each sample, even if the batch order changes.

Update and warm-start are distinct

Mechanism Reused object Indexed by First call
Solver-data update Solver workspace and supported data/factorization state Batch slot Builds workspace
Warm-start Solver-native primal-dual iterate and related state Stable sample id Populates sample cache

They can be enabled independently:

  • update-only reuses a batch slot's workspace and does not require idx_list;
  • warm-start-only reuses sample-specific iterates;
  • enabling both combines workspace reuse with sample-specific initialization.

These are forward-pass accelerations. They do not change whether SolMapLayer uses full or reduced KKT differentiation, and ValueFuncLayer still uses its direct value derivative.

Batched execution

Every batch row defines an independent value of \(\hat y\) with the same APQP template. DiffAPQP can solve those instances concurrently up to max_n_cores. This is outer sample parallelism; native solver or BLAS threads are a separate source of parallelism.

For SolMapLayer, max_n_cores also controls adjoint workers without changing the selected full- or reduced-KKT formulation. Linux uses the existing fork process pool, while Windows and macOS use shared-memory threads. See the SolMapLayer constructor.

The first forward call establishes the reusable per-slot problem list and the largest supported batch size for that layer instance. Input shapes, CPU conversion, worker behavior, and platform limitations are documented in:

Forward outputs and the backward hand-off

The forward solve provides all quantities needed by either backward strategy:

  • \(\tilde z^\star\), \(\tilde\nu^\star\), and \(\tilde\lambda^\star\) from the canonical APQP;
  • \(\hat z^\star=\mathcal R(\tilde z^\star)\) for a solution-dependent loss;
  • \(\alpha(\hat y)\) for a value-dependent loss;
  • canonical data derivatives \(Q_i\), \(d_i\), \(B_i\), and \(H_i\).

The next page depends on the selected differentiation target: