Skip to content

[BUG] set_cover Python: fix UB in the all_subsets property - #5121

Open
jg-codes wants to merge 1 commit into
google:mainfrom
jg-codes:fix-set-cover-python-bindings
Open

[BUG] set_cover Python: fix UB in the all_subsets property#5121
jg-codes wants to merge 1 commit into
google:mainfrom
jg-codes:fix-set-cover-python-bindings

Conversation

@jg-codes

@jg-codes jg-codes commented Apr 3, 2026

Copy link
Copy Markdown

What

Fixes undefined behaviour in the set_cover Python wrapper (ortools/set_cover/python/set_cover.cc) and adds a regression test.

Disclosure: drafted with AI assistance (Claude). See the verification section for exactly what was built and run.

The bug

SetCoverModel.all_subsets terminates the interpreter when read from Python. The property is set_cover.cc:201-213, and it has two independent defects in one statement:

  1. begin() and end() come from different objects. SetCoverModel::all_subsets() returns by value (set_cover_model.h:211):

    std::vector<SubsetIndex> all_subsets() const { return all_subsets_; }

    The property calls it twice, so model.all_subsets().begin() and model.all_subsets().end() are iterators into two distinct temporaries. That pair does not delimit a valid range, and the transform runs off the end of the first temporary's buffer. This is the dominant defect, and it is why the property has never worked.

  2. The output iterator is also invalid. subsets.begin() on a default-constructed, zero-capacity vector: a write through an iterator that was never valid.

The neighbouring columns and rows properties are fine: columns() and rows() return const references (:189, :192), so calling them twice is harmless. Their pre-size-and-write-through-begin() pattern is correct for a reference-returning accessor. all_subsets() is the only by-value accessor in the file, and it is the only one that crashes.

Reproduction

from ortools.set_cover.python import set_cover

model = set_cover.SetCoverModel()
model.add_empty_subset(1.0)
model.add_element_to_last_subset(0)
model.add_empty_subset(1.0)
model.add_element_to_last_subset(1)

print(model.all_subsets)  # process dies here

Segmentation fault, exit 139, on the released ortools 9.15.6755 wheel (macOS arm64, CPython 3.13.11). The std::transform statement at the v9.15 tag is identical to the one on main.

The fix

Bind the returned vector to a local, then append through back_inserter. Binding the local also removes a third full copy that reserve(model.all_subsets().size()) would otherwise make.

VectorIntToVectorSubsetIndex has the second defect too, and is fixed here for consistency, though it is not currently reachable from Python: all of its callers take absl::Span, and this module registers no absl::Span type caster. That is a separate, unrelated defect, reported independently rather than folded into this PR.

Verification

Built and tested on Linux (Bazel 8.7.0, x86_64) against main at d9c0910:

new test applied WITHOUT the fix:  FAIL (Segmentation fault)
new test applied WITH the fix:     PASSED

clang-format clean against the repo .clang-format.

Checklist

  • Read CONTRIBUTING.md and the PR template, targeting main
  • CLA signed
  • Minimal diff: the all_subsets fix and its test
  • No unrelated formatting or refactoring
  • AI-assisted: disclosed above

@google-cla

google-cla Bot commented Apr 3, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@jg-codes
jg-codes force-pushed the fix-set-cover-python-bindings branch from d6afbef to 5a9195b Compare April 3, 2026 18:43
@jg-codes jg-codes changed the title [BUG] Fix undefined behavior and swapped getter/setter in set_cover Python bindings [BUG] set_cover Python: fix UB in vector transforms and swapped GuidedTabuSearch bindings Apr 3, 2026
@jg-codes
jg-codes changed the base branch from stable to main April 3, 2026 18:51
@jg-codes
jg-codes force-pushed the fix-set-cover-python-bindings branch from 5a9195b to 25129a2 Compare April 3, 2026 20:18
jg-codes added a commit to jg-codes/or-tools that referenced this pull request Apr 3, 2026
Exposes the existing C++ SetCoverLagrangian class to Python via pybind11.
Resolves the TODO(user) at the end of set_cover.cc.

Bindings cover all public methods: multiplier initialization, reduced
costs, subgradient, Lagrangian value, multiplier updates, gap
computation, three-phase algorithm, lower bound computation, and
their parallel variants.

Also includes the bug fixes from google#5121 (UB in vector transforms,
swapped GuidedTabuSearch getter/setter) since this patch builds on
the corrected file.

Added BUILD dep on :set_cover_lagrangian and 6 regression tests.
jg-codes added a commit to jg-codes/or-tools that referenced this pull request Apr 3, 2026
Exposes the existing C++ SetCoverLagrangian class to Python via pybind11.
Resolves the TODO(user) at the end of set_cover.cc.

Bindings cover all public methods: multiplier initialization, reduced
costs, subgradient, Lagrangian value, multiplier updates, gap
computation, three-phase algorithm, lower bound computation, and
their parallel variants.

Also includes the bug fixes from google#5121 (UB in vector transforms,
swapped GuidedTabuSearch getter/setter) since this patch builds on
the corrected file.

Added BUILD dep on :set_cover_lagrangian and 6 regression tests.
@jg-codes

Copy link
Copy Markdown
Author

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

I've signed the CLA. Anything else needed from your side?

@Mizux Mizux added the Solver: Set Cover Solver in set_cover/ label Jun 26, 2026
Reading model.all_subsets from Python terminates the interpreter. Two
separate defects in the same statement:

1. SetCoverModel::all_subsets() returns by value (set_cover_model.h:211),
   and the property called it twice, so begin() and end() came from two
   distinct temporaries. That iterator pair does not delimit a range, so
   the transform walks off the end of the first temporary's buffer. This
   is the dominant defect and it is the reason the property has never
   worked.

2. The output iterator was subsets.begin() on a default-constructed,
   zero-capacity vector, which is a write through an invalid iterator.

Both are fixed by binding the returned vector to a local and appending
through back_inserter(). Binding the local also removes a third full
copy of the vector that the reserve() call would otherwise make.

Note the neighbouring columns and rows properties are not affected:
columns() and rows() return const references (set_cover_model.h:189,
:192), so calling them twice is harmless. They pre-size and write
through begin(), which is correct for a reference-returning accessor.

VectorIntToVectorSubsetIndex had defect 2 as well and is fixed here for
consistency, but it is not reachable from Python: all of its callers take
absl::Span, and this module registers no absl::Span type caster. That is
reported separately.

Adds a regression test for the property. The focus-based paths cannot be
regression-tested from Python until the Span casters are addressed.

The swapped GuidedTabuSearch lagrangian getter/setter reported in the
first version of this PR was fixed upstream in a1e13b3, so that hunk is
dropped.
@jg-codes
jg-codes force-pushed the fix-set-cover-python-bindings branch from 25129a2 to dab4410 Compare August 1, 2026 22:36
@jg-codes

jg-codes commented Aug 1, 2026

Copy link
Copy Markdown
Author

@Mizux, rebased onto current main, just force-pushed. The diagnosis changed since I opened this, so the whole PR may be worth a fresh look.

a1e13b37 ("set_cover: export from google3") fixed the swapped GuidedTabuSearch::{Set,Get}LagrangianFactor bindings, so that hunk and its test are gone. Thanks for fixing it.

model.all_subsets kills the interpreter, and the empty-vector write is only half of why

The property is set_cover.cc:201-213. Two defects in one statement:

1. begin() and end() come from different objects. SetCoverModel::all_subsets() returns by value (set_cover_model.h:211):

std::vector<SubsetIndex> all_subsets() const { return all_subsets_; }

The property calls it twice, so model.all_subsets().begin() and model.all_subsets().end() are iterators into two distinct temporaries. That pair does not delimit a range, and the transform runs off the end of the first temporary's buffer.

2. The output iterator is invalid too. subsets.begin() on a default-constructed, zero-capacity vector.

I originally reported only the second one, but both need fixing.

The neighbouring columns and rows properties are fine: columns() and rows() return const references (:189, :192), so calling them twice is harmless. Their pre-size-and-write-through-begin() form is correct for a reference-returning accessor. all_subsets() is the only by-value one, and it is the only property that crashes.

Reproduction

48 elements on a 6×8 grid, 24 subsets, each covering the cells within L1 distance 2, all costs 1.0:

from ortools.set_cover.python import set_cover

ROWS, COLS = 6, 8
model = set_cover.SetCoverModel()
for r in range(ROWS):
    for c in range(COLS):
        if (r + c) % 2:
            continue
        model.add_empty_subset(1.0)
        for dr in range(-2, 3):
            for dc in range(-2, 3):
                nr, nc = r + dr, c + dc
                if abs(dr) + abs(dc) <= 2 and 0 <= nr < ROWS and 0 <= nc < COLS:
                    model.add_element_to_last_subset(nr * COLS + nc)

print("subsets:", model.num_subsets)              # 24
print("coverable:", model.compute_feasibility())  # True
print("ids:", model.all_subsets)                  # process dies here
subsets: 24
coverable: True
[1]    segmentation fault

Exit 139 on 6 of 6 runs. Solving the same model is fine: next_solution() returns True at cost 10.0, exit 0 on 5 of 5. So it is the property, not the model. ortools 9.15.6755, CPython 3.13.11, macOS 26.5.2 arm64; the statement at the v9.15 tag is identical to the one on main.

There is no exception to catch and nothing in the log, which is what makes it awkward in practice.

The fix

Bind the returned vector to a local, then append through back_inserter. Binding the local also removes a third full copy that reserve(model.all_subsets().size()) would otherwise make.

One more correction to my original description

I claimed the UB reached every Python call passing a focus list. That is wrong. Those overloads take absl::Span, and this module registers no absl::Span caster: pybind11 falls back to printing the raw C++ type and rejects any list, so VectorIntToVectorSubsetIndex is not reachable from Python at all. It has the same output-iterator defect and I fixed it here for consistency, but all_subsets is the reachable instance. The Span problem is filed separately.

That also means the focus paths cannot carry a regression test from Python.

Diff and verification

17 added, 12 removed in set_cover.cc, plus 6 lines of test. Branch jg-codes:fix-set-cover-python-bindings, compare. clang-format clean against the repo .clang-format.

Built and ran bazel test //ortools/set_cover/python:set_cover_test on Linux (Bazel 8.7.0, x86_64) against main at d9c0910:

new test applied WITHOUT the fix:  FAIL (Segmentation fault)
new test applied WITH the fix:     PASSED

Disclosure: drafted with AI assistance (Claude). Every claim above, the segfault, the exit codes, and the bazel result, is from a run I executed myself.

Happy to close this if you would rather take it internally.

@jg-codes

jg-codes commented Aug 1, 2026

Copy link
Copy Markdown
Author

Filed the Span/focus-API defect as its own issue: #5278.

@jg-codes jg-codes changed the title [BUG] set_cover Python: fix UB in vector transforms and swapped GuidedTabuSearch bindings [BUG] set_cover Python: fix UB in the all_subsets property Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Solver: Set Cover Solver in set_cover/

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants