Skip to content

L-shaped: fix subproblem indexing when root_scenarios is used - #817

Open
DLWoodruff wants to merge 1 commit into
Pyomo:mainfrom
DLWoodruff:lshaped-root-scenarios-551
Open

L-shaped: fix subproblem indexing when root_scenarios is used#817
DLWoodruff wants to merge 1 commit into
Pyomo:mainfrom
DLWoodruff:lshaped-root-scenarios-551

Conversation

@DLWoodruff

Copy link
Copy Markdown
Collaborator

Fixes #551 (also explains the crash in discussion #542).

The bug

farmer_lshapedhub.py (like any L-shaped run using the root_scenarios option) builds one scenario directly into the root problem. Such scenarios have no eta variable and no Benders subproblem — root.eta is indexed by the non-root scenarios only. But set_ls in mpisppy/utils/lshaped_cuts.py sized global_subproblem_count and the local-to-global subproblem index map over all scenario names.

With a single hub rank the misindexing is coincidentally harmless (the positional keys and the dropped root scenario line up, and the extra array slot stays zero) — which is why the 2-process run in run_all.py never crashed. With two or more hub ranks, a rank owning a scenario that comes after a root scenario writes its cut data to a global slot past the end of all_root_etas, producing the reported

File ".../pyomo/contrib/benders/benders_cuts.py", line 379, in generate_cut
    root_eta = self.all_root_etas[global_subproblem_ndx]
IndexError: list index out of range

Note this is narrower than the diagnosis in the issue thread: mpi-sppy's LShapedCutGeneratorData subclass already bypasses Pyomo's call-everywhere/argmin rank-assignment protocol by imposing mpi-sppy's own scenario-to-rank map — the map was just wrong when root_scenarios was nonempty.

The fix

set_ls now indexes over the non-root scenarios only, matching the eta ordering built by lshaped.py (getattr(ls, "has_root_scens", False) so the cross_scen_spoke.py caller, whose opt object has no such attribute and uses all scenarios, keeps its exact previous behavior).

Tests

  • mpisppy/tests/test_lshaped_cuts.py (new, serial, no solver): unit tests of the set_ls bookkeeping, including per-rank views of the failing 3-rank configuration. The root-scenario cases fail on main and pass with the fix. Wired into run_coverage.bash and the CI unit-test job.
  • straight_tests.py: new end-to-end run of farmer_lshapedhub.py with a 3-rank hub (one scenario per rank, middle scenario in the root problem) — the exact shape of the L-shaped/benders cuts Index error #551 repro. Reproduces the IndexError on main; with the fix it converges to the farmer optimum.

Verified locally with gurobi_persistent: the original -np 6 ... --xhatlshaped repro from #551, plus -np 4 and -np 2, all converge to −108390 with gap 0; netdes with --cross-scenario-cuts still runs clean; test_maximization.py (5 L-shaped tests) passes.

🤖 Generated with Claude Code

…yomo#551)

Scenarios built into the root problem (root_scenarios) have no eta
variable and no Benders subproblem, but set_ls in lshaped_cuts.py sized
global_subproblem_count and the local-to-global subproblem index map
over all scenario names. With a single hub rank the misindexing was
coincidentally harmless, but with two or more hub ranks a rank owning a
scenario that follows a root scenario wrote to a global slot past the
end of all_root_etas, giving the IndexError in generate_cut reported in
issue Pyomo#551 (and discussion Pyomo#542).

set_ls now indexes over the non-root scenarios only, matching the eta
ordering built by lshaped.py. The cross_scen_spoke.py caller has no
has_root_scens attribute and keeps its previous behavior via getattr.

Adds serial unit tests for the set_ls bookkeeping and a multi-rank hub
L-shaped smoke run (farmer with a root scenario, 3 hub ranks) to
straight_tests.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.44%. Comparing base (3749661) to head (32bb9f9).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #817      +/-   ##
==========================================
+ Coverage   76.33%   76.44%   +0.10%     
==========================================
  Files         170      170              
  Lines       22628    22631       +3     
==========================================
+ Hits        17274    17301      +27     
+ Misses       5354     5330      -24     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@DLWoodruff
DLWoodruff marked this pull request as ready for review July 24, 2026 19:24
@DLWoodruff
DLWoodruff requested a review from bknueven July 24, 2026 19:25

@bknueven bknueven left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code obviously isn't used much by us 🤦‍♂️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

L-shaped/benders cuts Index error

2 participants