Convert timed mipgap to use termination callback mechanics

.github/workflows/test_pr_and_main.yml

@@ -80,7 +80,7 @@ jobs:
         run: |
           conda install mpi4py pandas setuptools
           pip install pyomo sphinx sphinx_rtd_theme sphinx-copybutton dill gridx-egret cplex pybind11
-          pip install xpress coverage
+          pip install gurobipy xpress coverage
 
       - name: Build pyomo extensions
         run: |
@@ -165,6 +165,14 @@ jobs:
         run: |
           coverage run $COV_ARGS -m pytest mpisppy/tests/test_prox_approx_e2e.py -v
 
+      - name: Test timed mipgap
+        run: |
+          coverage run $COV_ARGS -m pytest mpisppy/tests/test_timed_mipgap.py -v
+
+      - name: Test termination callback
+        run: |
+          coverage run $COV_ARGS -m pytest mpisppy/utils/callbacks/termination/tests/test_termination.py -v
+
       - name: Test docs
         run: |
           cd ./doc/

doc/src/extensions.rst

@@ -28,7 +28,11 @@ command-line flags:
   (see :ref:`w_oscillation`)
 - ``--interrupt-W-oscillations <file>`` -- activates W-oscillation
   interruption (slamming; implies detection; see :ref:`w_oscillation`)
-- ``--mipgaps-json <file>`` -- activates the mipgapper extension
+- ``--mipgaps-json <file>`` -- activates the legacy mipgapper schedule mode
+- ``--starting-mipgap <float>`` (required; ``--mipgap-ratio`` defaults to
+  ``0.1``) -- activates the mipgapper's auto-gap mode for cylinders
+- ``--timed-mipgap <curve>`` -- activates the timed MIP gap extension and sets
+  the timed MIP gap curve as ``gap:time`` pairs (e.g., ``"0.02:100 0.05:200"``)
 - ``--user-defined-extensions <module>`` -- loads a custom extension
 - ``--grad-rho`` -- activates gradient-based rho (see :ref:`rho_setting`)
 - ``--use-norm-rho-updater`` -- activates the norm rho updater
@@ -82,22 +86,51 @@ now describe a few of the extensions in the release.
 mipgapper.py
 ^^^^^^^^^^^^
 
-This is a good extension to look at as a first example. It takes a
-dictionary with iteration numbers and mipgaps as input and changes the
-mipgap at the corresponding iterations. The dictionary is provided in
-the options dictionary in ``["gapperoptions"]["mipgapdict"]``.  There
-is an example of its use in ``examples.sizes.sizes_demo.py``.
+This is a good extension to look at as a first example. It can either take a
+dictionary with iteration numbers and mipgaps as input, or it can run in
+auto-gap mode when used from ``generic_cylinders``.
 
-Instead of an options dictionary, when run with cylinders the options
-``["gapperoptions"]["starting_mipgap"]`` and ``["gapperoptions"]["mipgap_ratio"]``
-can be set. The ``starting_mipgap`` will be the initial value used,
-and as the cylinders close the relative optimality gap the extension will set the subproblem
-mipgaps as the ``min(starting_mipgap, mipgap_ratio * problem_ratio)``, where
-the ``problem_ratio`` is the relative optimality gap on the overall problem
-as computed by the cylinders.
+The dictionary form is provided in the options dictionary in
+``["gapperoptions"]["mipgapdict"]``. There is an example of its use in
+``examples.sizes.sizes_demo.py``.
+
+When run with cylinders, the options ``["gapperoptions"]["starting_mipgap"]``
+and ``["gapperoptions"]["mipgap_ratio"]`` can be set instead. The
+``starting_mipgap`` is the initial value used, and as the cylinders close the
+relative optimality gap the extension sets the subproblem mipgaps as
+``min(starting_mipgap, mipgap_ratio * problem_ratio)``, where
+``problem_ratio`` is the relative optimality gap on the overall problem as
+computed by the cylinders.
 
 This extension can also be used with the Lagrangian and subgradient spokes.
 
+timed_mipgap.py
+^^^^^^^^^^^^^^^^
+
+This extension installs a solver termination callback that stops a persistent
+MIP solve once a user-specified run time has been reached *and* the current
+relative gap is already below a target threshold. The option is given as a
+string of ordered ``gap:time`` pairs in
+``options["timed_mipgap"]["timecurve"]``. For example,
+``"0.02:100 0.05:200"`` means: after 100 seconds, stop if the relative gap is
+below 2%; after 200 seconds, stop if it is below 5%.
+
+This is a soft, time-dependent stopping rule: it does not force termination at
+the specified times unless the incumbent and bound are already close enough.
+It is useful when early PH iterations do not need tight subproblem solves, but
+later iterations may still benefit from stronger solves when the solver is
+making progress.
+
+When using ``generic_cylinders.py``, enable it with:
+
+- ``--timed-mipgap "0.02:100 0.05:200"``
+
+The extension currently requires a persistent solver with supported termination
+callbacks; at present this includes CPLEX, Gurobi, and Xpress persistent
+interfaces. The ``gap:time`` pairs are validated in the order provided and
+must be strictly increasing in both gap and time; duplicate gap entries are
+rejected.
+
 fixer.py
 ^^^^^^^^
 

doc/src/generic_cylinders.rst

@@ -230,6 +230,10 @@ Some extensions can be activated directly from the command line:
 
 - ``--fixer`` -- Fix variables that have converged
 - ``--mipgaps-json <file>`` -- MIP gap schedule from a JSON file
+- ``--starting-mipgap <float>`` (required; ``--mipgap-ratio`` defaults to
+  ``0.1``) -- auto MIP gap mode for cylinders
+- ``--timed-mipgap <curve>`` -- Time-dependent MIP gap termination
+  curve as ``gap:time`` pairs
 - ``--user-defined-extensions <module>`` -- Load a custom extension module
 - ``--wtracker`` -- Track W (Lagrange-multiplier) values per iteration
   and write a convergence report at the end of the run
@@ -390,7 +394,9 @@ any ``mipgap`` set elsewhere.
 
 For iteration-aware mipgap, use ``--iter0-mipgap`` and
 ``--iterk-mipgap`` (plus their per-spoke variants), or
-``--mipgaps-json <path>`` for a mipgap-only schedule.
+``--mipgaps-json <path>`` for a mipgap-only schedule. For
+auto-tuning mipgap during decomposition, use ``--starting-mipgap``;
+``--mipgap-ratio`` defaults to ``0.1``.
 ``--max-solver-threads`` sets a system-level thread cap that wins
 over any inline ``threads`` value; use it on shared HPC nodes.
 

mpisppy/cylinders/hub.py

@@ -8,6 +8,7 @@
 ###############################################################################
 
 import abc
+import math
 import logging
 import mpisppy.log
 
@@ -86,6 +87,22 @@ def get_update_string(self):
             return '  ' + self.latest_ib_char
         return self.latest_ob_char+' '+self.latest_ib_char
 
+    @staticmethod
+    def _format_trace_value(value, width, decimals):
+        """Format a trace value in fixed notation unless it under/overflows it."""
+        fixed = f"{value:{width}.{decimals}f}"
+        if not math.isfinite(value):
+            return fixed
+
+        if len(fixed) > width:
+            return f"{value:{width}.{decimals}e}"
+
+        fixed_core = fixed.strip().lstrip("-")
+        if value != 0 and fixed_core == f"0.{('0' * decimals)}":
+            return f"{value:{width}.{decimals}e}"
+
+        return fixed
+
     def screen_trace(self):
         current_iteration = self.current_iteration()
         abs_gap, rel_gap = self.compute_gaps()
@@ -96,7 +113,11 @@ def screen_trace(self):
             row = f'{"Iter.":>5s}  {"   "}  {"Best Bound":>14s}  {"Best Incumbent":>14s}  {"Rel. Gap":>12s}  {"Abs. Gap":>14s}'
             global_toc(row, True)
             self.print_init = False
-        row = f"{current_iteration:5d}  {update_source}  {best_bound:14.4f}  {best_solution:14.4f}  {rel_gap*100:12.3f}%  {abs_gap:14.4f}"
+        best_bound_s = self._format_trace_value(best_bound, 14, 4)
+        best_solution_s = self._format_trace_value(best_solution, 14, 4)
+        rel_gap_s = self._format_trace_value(rel_gap * 100, 12, 3)
+        abs_gap_s = self._format_trace_value(abs_gap, 14, 4)
+        row = f"{current_iteration:5d}  {update_source}  {best_bound_s}  {best_solution_s}  {rel_gap_s}%  {abs_gap_s}"
         global_toc(row, True)
         self.clear_latest_chars()
 

mpisppy/extensions/timed_mipgap.py

@@ -7,96 +7,120 @@
 # full copyright and license information.
 ###############################################################################
 '''
-This class is implemented as an extension to be used in mpi-sppy to add a callback to a persistent
-solver to implement a time-dependent target MIP gap.
-For now, the only solver supported is GurobiPersistent.
+This class is implemented as an extension to be used in mpi-sppy to add a termination callback to a
+persistent solver and implement a time-dependent target MIP gap.
 '''
 
+import math
+
 import mpisppy.extensions.extension
 import mpisppy.utils.sputils as sputils
-from pyomo.solvers.plugins.solvers.gurobi_persistent import GurobiPersistent
-
-from gurobipy import GRB
+from mpisppy.utils.callbacks.termination import (
+    set_termination_callback,
+    supports_termination_callback,
+)
 
 class TimedMIPGapCB(mpisppy.extensions.extension.Extension):
     '''
-    This extension adds a solver callback function that implements a time-dependent target MIP gap.
-    The curve is defined by a sequence of (time,gap) pairs, monotonically increasing in both dimensions.
-    For each (t,g) pair, when the solver reaches time t, it relaxes the target MIP gap to g.
+    This extension adds a solver termination callback that implements a time-dependent target MIP gap.
+    The curve is defined by an ordered mapping of gap:time pairs, monotonically increasing in both
+    dimensions. For each (t,g) pair, when the solver reaches time t, it terminates if the relative gap
+    has already improved below g.
 
     This class requires the following options:
     'timed_mipgap':
         'timecurve': string of gap:time pairs separated by spaces. Example: "0.02:100 0.05:200"
 
     Attributes
     ----------
-    timecurve: dict of {gap:time} with sequence of (time,gap) pairs.
-
-    Reference
-    ---------
-    Based on post found at:
-    https://support.gurobi.com/hc/en-us/articles/360047717291-How-do-I-use-callbacks-to-terminate-the-solver-
-    (Oct 2023)
+    timecurve: ordered dict of {gap:time} pairs.
     '''
 
     def __init__(self, ph):
-
+        super().__init__(ph)
         self.ph = ph
-
         self._set_options()
 
     def _set_options(self):
         ph = self.ph
         if 'timed_mipgap' not in ph.options:
             raise RuntimeError('Did not find "timed_mipgap" options')
         timecurve_str = ph.options['timed_mipgap']['timecurve']
-        self.timecurve = {float(ent.split(':')[0]):float(ent.split(':')[1]) for ent in timecurve_str.split(' ')}
-
-    def iter0_post_solver_creation(self):
-        ph = self.ph
-        for sname, s in ph.local_scenarios.items():
-            if not hasattr(s, '_solver_plugin'):
-                raise RuntimeError('Solver must be created before calling callback extension')
-            if not (sputils.is_persistent(s._solver_plugin)):
-                raise RuntimeError('Solvers must be persistent for callback definition')
-            if not s._solver_plugin.has_instance():
-                raise RuntimeError('Solver must be instantiated before calling callback extension')
-            if not isinstance(s._solver_plugin,GurobiPersistent):
-                raise RuntimeError('Currently, only GurobiPersistent solver is supported.')
-
-            def cb_fun(cb_m,cb_opt,cb_wh):
-                '''
-                callback function
-                '''
-                return self._timecurve_cb(cb_m,cb_opt,cb_wh,self.timecurve)
-            s._solver_plugin.set_callback(cb_fun)
+        self.timecurve = dict()
+        prev_gap = None
+        prev_time = None
+        for entry in timecurve_str.split():
+            try:
+                gap_str, time_str = entry.split(':', 1)
+                gap = float(gap_str)
+                solve_time = float(time_str)
+            except ValueError as exc:
+                raise RuntimeError(
+                    'Timed MIP gap option "timecurve" entries must have format "gap:time"'
+                ) from exc
+
+            if not math.isfinite(gap) or not math.isfinite(solve_time):
+                raise RuntimeError(
+                    'Timed MIP gap option "timecurve" entries must use finite gap and time values'
+                )
+            if gap in self.timecurve:
+                raise RuntimeError(
+                    f'Timed MIP gap option "timecurve" has duplicate gap entry {gap}'
+                )
+            if prev_gap is not None and (gap <= prev_gap or solve_time <= prev_time):
+                raise RuntimeError(
+                    'Timed MIP gap option "timecurve" must be strictly increasing in both gap and time'
+                )
+
+            self.timecurve[gap] = solve_time
+            prev_gap = gap
+            prev_time = solve_time
+
+        if not self.timecurve:
+            raise RuntimeError('Timed MIP gap option "timecurve" must not be empty')
 
     @staticmethod
-    def _timecurve_cb(cb_model, cb_opt, cb_where,
-                     timecurve_dict):
-        '''
-        Inputs
-        ------
-        cb_model: Pyomo ConcreteModel
-        cb_opt: SolverFactory model of gurobi_persistent type
-        cb_where: argument that indicates where in the algorith the callback is being called from
-
-        timecurve_dict: dict of {gap:time} with:
-            time: time in sec
-            gap: target MIP gap (in p.u.)
-        Note that (time,gap) only define a meaningful timecurve if there is monotonicity, so monotonicity
-        is assumed. However, it is not verified.
-
-        '''
-
-        if cb_where == GRB.Callback.MIP:
-            grb_m = cb_opt._solver_model # gurobipy model
-            runtime = grb_m.cbGet(GRB.Callback.RUNTIME)
-            objbst = grb_m.cbGet(GRB.Callback.MIP_OBJBST)
-            objbnd = grb_m.cbGet(GRB.Callback.MIP_OBJBND)
-            gap = abs((objbst - objbnd) / objbst)
-
-            for tc_gap,tc_t in timecurve_dict.items():
-                if runtime > tc_t and gap < tc_gap:
-                    grb_m.terminate()
-
+    def _compute_relative_gap(best_obj, best_bound):
+        '''Return relative gap or None if a bound is unavailable.'''
+        if best_obj is None or best_bound is None:
+            return None
+        if not math.isfinite(best_obj) or not math.isfinite(best_bound):
+            return None
+
+        return abs(best_obj - best_bound) / max(
+            1e-10,
+            abs(best_obj),
+            abs(best_bound),
+        )
+
+    def _should_terminate(self, runtime, best_obj, best_bound):
+        '''Return True when the timecurve says to terminate.'''
+        if runtime is None or not math.isfinite(runtime):
+            return False
+
+        gap = self._compute_relative_gap(best_obj, best_bound)
+        if gap is None:
+            return False
+
+        for tc_gap, tc_t in self.timecurve.items():
+            if runtime > tc_t and gap < tc_gap:
+                return True
+
+        return False
+
+    def pre_solve(self, s):
+        if not hasattr(s, '_solver_plugin'):
+            raise RuntimeError('Solver must be created before calling callback extension')
+        if not (sputils.is_persistent(s._solver_plugin)):
+            raise RuntimeError('Solvers must be persistent for callback definition')
+        if not s._solver_plugin.has_instance():
+            raise RuntimeError('Solver must be instantiated before calling callback extension')
+        if not supports_termination_callback(s._solver_plugin):
+            raise RuntimeError(
+                'Timed MIP gap requires a persistent solver with supported termination callbacks'
+            )
+
+        def cb_fun(runtime, best_obj, best_bound):
+            return self._should_terminate(runtime, best_obj, best_bound)
+
+        set_termination_callback(s._solver_plugin, cb_fun)

mpisppy/generic/parsing.py

@@ -145,6 +145,7 @@ def add_decomp_args(cfg):
     cfg.coeff_rho_args()
     cfg.sensi_rho_args()
     cfg.reduced_costs_rho_args()
+    cfg.timed_mipgap_args()
 
     cfg.add_to_config("user_defined_extensions",
                       description="Space-delimited module names for user extensions",

mpisppy/tests/test_cylinder_hub_formatting.py

@@ -0,0 +1,29 @@
+###############################################################################
+# mpi-sppy: MPI-based Stochastic Programming in PYthon
+#
+# Copyright (c) 2024, Lawrence Livermore National Security, LLC, Alliance for
+# Sustainable Energy, LLC, The Regents of the University of California, et al.
+# All rights reserved. Please see the files COPYRIGHT.md and LICENSE.md for
+# full copyright and license information.
+###############################################################################
+"""Tests for adaptive trace formatting in cylinders.hub."""
+
+import unittest
+
+from mpisppy.cylinders.hub import Hub
+
+
+class TestTraceFormatting(unittest.TestCase):
+    def test_fixed_format_is_preserved(self):
+        self.assertEqual(Hub._format_trace_value(123.4567, 14, 4), "      123.4567")
+
+    def test_small_values_switch_to_scientific(self):
+        self.assertIn("e", Hub._format_trace_value(1.2e-8, 14, 4))
+        self.assertIn("e", Hub._format_trace_value(-4.9e-5, 14, 4))
+
+    def test_large_values_switch_to_scientific(self):
+        self.assertIn("e", Hub._format_trace_value(1234567890.0, 14, 4))
+
+
+if __name__ == "__main__":
+    unittest.main()