[ENH] set_cover Python: add SetCoverLagrangian bindings - #5122
Conversation
196eaa5 to
fad511d
Compare
Exposes the existing C++ SetCoverLagrangian class to Python via pybind11, resolving the TODO(user) at the end of set_cover.cc. Binds 12 of the class's public methods: multiplier initialization, reduced costs, subgradient, Lagrangian value, multiplier updates, gap computation, lower bound computation, and the parallel variants. Two methods are deliberately left unbound: - Optimize(): the header documents it as a dummy implementation and says the class is meant to be used through ComputeLowerBound(). - ThreePhase(): declared in set_cover_lagrangian.h:135, defined nowhere in the tree. Binding it links but produces an undefined symbol at module import, which fails the whole extension. Left unbound with a comment recording why. ComputeLowerBound() also turned out to null-dereference on a freshly constructed SetCoverLagrangian: it unconditionally calls the Parallel* methods, and those require the thread pool that UseNumThreads() constructs, which is null until that method is called. The test calls use_num_threads() first and documents the precondition, since nothing in the header states it. The list-valued parameters are typed const std::vector<double>& rather than absl::Span<const double>. No absl::Span type caster is registered in this module, so a Span-typed parameter cannot be satisfied from Python at all: pybind11 renders it as the raw C++ type in the signature and rejects any list. The existing Span-typed bindings in this file are unusable from Python for the same reason, reported separately. StrongVector types are converted to and from list[float], consistent with the existing subset_costs property. UpdateMultipliers() mutates through a pointer in C++; the Python binding returns the updated multipliers instead. Adds the //ortools/set_cover:set_cover_lagrangian bazel dep and 6 tests. CMake needs no change: the pybind target already links ::ortools. Built and tested on Linux (Bazel 8.7.0) against main at d9c0910: bazel test //ortools/set_cover/python:set_cover_test passes.
fad511d to
4dad6ea
Compare
|
@Mizux, rebased onto current What changed since the version you would have seen in April:
Two things the build turned up that were not visible from reading the header alone:
I built and ran a small benchmark on this exact build to check whether the new bound adds anything Two questions where I would rather follow your preference than guess:
|
|
Filed the Span/focus-API defect as its own issue: #5278. |
Disclosure: This PR was drafted with AI assistance (Claude). I reviewed every line and built and ran it myself; see the verification section for exactly what that covered.
What
Adds Python bindings for the existing
SetCoverLagrangianC++ class, resolving theTODO(user)at the end ofset_cover.cc. No algorithmic changes; this only exposes what already exists in C++.Why
The Python
set_covermodule exposes the greedy, steepest, GLS, tabu and dual-ascent optimizers, but not the Lagrangian relaxation code.SetCoverLagrangian::ComputeLowerBound()has no Python entry point at all, so Python users who want a Lagrangian dual bound, or multiplier-guided solutions built on top of one, have to reimplement the subgradient loop in Python and give up the compiled implementation.DualAscentOptimizeralready exposes a dual bound from Python, so this binding is not the only route to one. It is a genuinely different relaxation, and the question worth answering before adding 140 lines for it is whether that difference matters in practice. I benchmarked both, freshly, on this build.Benchmark: does the new bound add anything DualAscentOptimizer does not?
Four OR-Library SCP instances (Beasley's OR-Library,
scp41,scp42,scpa1,scpb1; a small sample given the time available, disclosed rather than silently limited), read via the already-boundread_orlib_scp. For each: a primal solution fromGreedySolutionOptimizer+SteepestSearch, then two lower bounds computed independently against that same primal cost.Dual ascent ranges are two independent runs with
set_num_random_passes(50), 10x the pass count needed to plateau (5 passes already gave the same bound within noise; 50 was to rule out an under-configured baseline before trusting the comparison). The Lagrangian bound was identical across both runs;ComputeLowerBoundhas no randomization.Across these four instances, the Lagrangian bound cuts the gap to the primal solution by roughly 2.5 to 3 times versus dual ascent, consistently. It also costs roughly 15 to 60 times more wall clock time, because
ComputeLowerBoundruns a fixed 1000-iteration subgradient loop (set_cover_lagrangian.cc), while dual ascent's passes are individually far cheaper. That is a real trade-off, not a free improvement: this binding is for offline or batch bound computation, not an interactive path.ComputeLowerBoundalso returns the reduced costs and multipliers alongside the bound, whichDualAscentOptimizerdoes not expose; those are what a caller would need to build a reduced-cost-guided heuristic on top of the bound.Two things the build surfaced that are worth flagging on their own
Building this confirmed two defects in
SetCoverLagrangianthat a Python (or C++) caller would hit regardless of this PR:ThreePhase()is declared but never defined.set_cover_lagrangian.h:135declaresvoid ThreePhase(Cost upper_bound);; there is no definition anywhere inset_cover_lagrangian.ccor elsewhere in the tree. Binding it producesundefined symbol: _ZN19operations_research18SetCoverLagrangian10ThreePhaseEdat module import, which fails the whole extension, not just that one call. I have left it unbound, with a comment explaining why.ComputeLowerBound()null-dereferences unlessUseNumThreads()is called first.thread_pool_isnullptruntilUseNumThreads()runs (set_cover_lagrangian.h:60,64), butComputeLowerBound()unconditionally calls theParallel*methods, which dothread_pool_->Schedule(...). On a freshly constructedSetCoverLagrangian,compute_lower_bound()segfaults. My test callsuse_num_threads()first and documents why; nothing in the header states this precondition.Neither is something this PR can fix without changing
set_cover_lagrangian.cc/.h, which is out of scope for a Python-bindings PR. Flagging both here since I found them building this.Methods exposed
initialize_lagrange_multipliers()InitializeLagrangeMultiplierscompute_reduced_costs(costs, multipliers)ComputeReducedCostsparallel_compute_reduced_costs(costs, multipliers)ParallelComputeReducedCostscompute_subgradient(reduced_costs)ComputeSubgradientparallel_compute_subgradient(reduced_costs)ParallelComputeSubgradientcompute_lagrangian_value(reduced_costs, multipliers)ComputeLagrangianValueparallel_compute_lagrangian_value(reduced_costs, multipliers)ParallelComputeLagrangianValueupdate_multipliers(step_size, lagrangian_value, upper_bound, reduced_costs, multipliers)UpdateMultipliersparallel_update_multipliers(...)ParallelUpdateMultiplierscompute_gap(reduced_costs, solution, multipliers)ComputeGapcompute_lower_bound(costs, upper_bound)ComputeLowerBounduse_num_threads(n)UseNumThreadsOptimize()andThreePhase()are not exposed; see above for both.These are the 12 remaining public methods
SetCoverLagrangiandeclares itself. The inheritedSetCoverOptimizersurface (time limits,run_time(),ResetLimits()) is not exposed here, and is not exposed for any other class in this file either.StrongVectortypes (SubsetCostVector,ElementCostVector) are converted to and fromlist[float], consistent with the existingsubset_costsproperty.UpdateMultipliersreturns the updated multipliers, since the C++ out-pointer is not idiomatic in Python.List parameters are typed
const std::vector<double>&, notabsl::Span<const double>. This module registers noabsl::Spantype caster, so a Span-typed parameter cannot be satisfied from Python at all: pybind11 renders it as the raw C++ type in the signature and rejects any list. The existing Span-typed bindings in this file have the same problem; reported separately.Changes
ortools/set_cover/python/set_cover.cc: include,usingdeclarations forCost/ElementCostVector/SetCoverLagrangian, aVectorDoubleToElementCostVectorhelper mirroring the existingVectorDoubleToSubsetCostVector, and thepy::class_<SetCoverLagrangian>block.ortools/set_cover/python/set_cover_test.py: 6 tests: multiplier initialization, reduced costs, subgradient and Lagrangian value, multiplier update, lower bound, and serial/parallel agreement.ortools/set_cover/python/BUILD.bazel: added the//ortools/set_cover:set_cover_lagrangiandep.ortools/set_cover/python/CMakeLists.txt: no change needed, the pybind target already links::ortools.Verification
Built and tested on Linux (Bazel 8.7.0, x86_64) against
mainatd9c0910:clang-formatclean against the repo.clang-format; the Python file isblackclean at the file's existing line width. The benchmark above ran against this same build.Notes
main; no longer depends on [BUG] set_cover Python: fix UB in the all_subsets property #5121.compute_lower_boundalone if that fits the intended scope better; the benchmark above is what I'd point to for why the rest earns its place.Checklist
main