exorcist user-facing docs

docs/environment.yaml

@@ -33,6 +33,7 @@ dependencies:
   - git+https://github.com/OpenFreeEnergy/kartograf@main
   - git+https://github.com/OpenFreeEnergy/konnektor@main
   - git+https://github.com/OpenFreeEnergy/lomap@main
+  - git+https://github.com/OpenFreeEnergy/exorcist@main
 
 # These are added automatically by RTD, so we include them here
 # for a consistent environment.

docs/guide/execution/exorcist_execution.rst

@@ -0,0 +1,105 @@
+.. userguide_exorcist:
+
+Execution with Exorcist Workers
+===============================
+
+Using the API to execute an Alchemical Network
+----------------------------------------------
+
+You can execute the network of simulation units defined by an ``AlchemicalNetwork`` (see `create_alchemical_network`) using ``openfe.orchestration``:
+
+
+First, we build a graph of tasks to be executed from the ``AlchemicalNetwork``:
+
+.. code:: python
+
+    from openfe.orchestration import build_task_db_from_alchemical_network
+    from openfe.storage import warehouse
+
+    alchemical_network = openfe.AlchemicalNetwork.from_json("alchemicalNetwork.json")
+
+    # create a Warehouse to define where simulation data is stored
+    my_warehouse = FileSystemWarehouse()
+
+    # store the AlchemicalNetwork in the Warehouse
+    my_warehouse.store_setup_tokenizable(alchemical_network)
+
+    # build a database of tasks from the AlchemicalNetwork
+    db_path = Path(my_warehouse.root_dir) / "tasks.db"
+    task_db = build_task_db_from_alchemical_network(alchemical_network, my_warehouse, db_path)
+
+
+Next, we call ``worker.execute_unit()`` to execute the next available task in the warehouse:
+
+.. code:: python
+
+    # execution: build the worker
+    from openfe.orchestration import Worker
+
+    worker = Worker(warehouse=my_warehouse, task_db_path=db_path)
+
+    execution = worker.execute_unit(scratch=pathlib.Path("path/to/local/scratch"))
+
+
+Each time ``worker.execute_unit`` is called, the worker will pick up the next valid task in the AlchemicalNetwork's task graph.
+
+
+Using the CLI
+-------------
+
+.. Note: this is a proof-of-concept for use with RBFEs, tbd if we want to expose this right now.
+
+.. code:: bash
+
+    openfe plan-rbfe-network ... --warehouse
+
+
+.. code:: bash
+
+    openfe worker warehouse/
+
+To run a single task to completion.
+To take full advantage of the worker model, you can run multiple ``openfe worker`` commands concurrently.
+
+The following is an example script that runs up to 4 workers at a time, with each automatically picking up the next valid unit to be executed.
+
+.. code:: bash
+
+    #!/usr/bin/env bash
+    set -euo pipefail
+
+    # Command + args as an array (so spaces are handled correctly)
+    CMD=(openfe worker warehouse)
+
+    MAX_JOBS=4
+    pids=()
+
+    cleanup() {
+    echo "Stopping workers..."
+    for pid in "${pids[@]:-}"; do
+        kill "$pid" 2>/dev/null || true
+    done
+    wait 2>/dev/null || true
+    exit 0
+    }
+    trap cleanup INT TERM
+
+    run_job() {
+    while true; do
+        "${CMD[@]}"
+    done
+    }
+
+    # Start initial workers
+    for ((i=0; i<MAX_JOBS; i++)); do
+    run_job &
+    pids+=("$!")
+    done
+
+    # Keep 4 running; if one exits, start another
+    while true; do
+    wait -n
+    run_job &
+    pids+=("$!")
+    done
+

docs/guide/execution/index.rst

@@ -2,13 +2,17 @@
 
 Execution
 =========
+The planning and preparation of a campaign of alchemical simulations using ``openfe`` is intended to be achievable on a local workstation in a matter of minutes.
+
+The *execution* of these simulations however requires a large amount of computational power, and beyond running single calculations locally, is intended to be distributed across a HPC environment.
+
+The simplest way to run a Transformation is to use the :ref:`quickrun CLI tool <userguide_quickrun>`.
 
-With a :class:`.Transformation` defined, the next step is to execute this.
-The easiest way to run it is to use the :ref:`quickrun CLI tool <userguide_quickrun>`.
 More advanced options are available through first considering the
 :ref:`theory of the execution model<userguide_execution_theory>`
 then :ref:`reading on the available Python functions<reference_execution>`.
 
 .. toctree::
    quickrun_execution
+   exorcist_execution
    execution_theory

docs/guide/execution/quickrun_execution.rst

@@ -3,9 +3,10 @@
 Execution with Quickrun
 =======================
 
-The planning and preparation of a campaign of alchemical simulations using ``openfe`` is intended to be achievable on a local workstation in a matter of minutes.
-The *execution* of these simulations however requires a large amount of computational power, and beyond running single calculations locally, is intended to be distributed across a HPC environment.
-Doing this requires storing and sending the details of the simulation from the local workstation to a HPC environment, which can be done via the :func:`.Transformation.to_json` function which :ref:`creates a saved JSON version of the data<dumping_transformations>`.
+``openfe quickrun`` is the simplest way to execute simulations on a per-Transformation basis.
+
+The :func:`.Transformation.to_json` function :ref:`creates a saved JSON version of the data<dumping_transformations>`.
+
 These serialized JSON files are the currency of executing a campaign of simulations and contain all the information required to execute a single simulation.
 
 To read the ``Transformation`` information and execute the simulation, the command line interface provides the ``openfe quickrun`` command, the full details of which are given in :ref:`the CLI reference section<cli_quickrun>`.

docs/reference/api/defining_and_executing_simulations.rst

@@ -13,9 +13,11 @@ Executing Simulations
 
 .. autosummary::
     :nosignatures:
+    :recursive:
     :toctree: generated/
 
     execute_DAG
+    storage
 
 General classes
 ---------------

environment.yml

@@ -56,6 +56,7 @@ dependencies:
     - git+https://github.com/OpenFreeEnergy/kartograf@main  # needs gufe which pins to pydantic < 2.13. update this once gufe v1.13 is released on conda-forge
     - git+https://github.com/OpenFreeEnergy/konnektor@main
     - git+https://github.com/OpenFreeEnergy/gufe@main
+    - git+https://github.com/OpenFreeEnergy/exorcist@main
   - run_constrained:
     # drop this pin when handled upstream in espaloma-feedstock
     - smirnoff99frosst>=1.1.0.1  #https://github.com/openforcefield/smirnoff99Frosst/issues/109

news/warehouse.rst

@@ -0,0 +1,23 @@
+**Added:**
+
+* Added ``openfe.storage.warehouse``, which includes ``WarehouseBaseClass`` and the example implementation ``FileSystemWarehouse``.
+
+**Changed:**
+
+* <news item>
+
+**Deprecated:**
+
+* <news item>
+
+**Removed:**
+
+* Removed the unused methods ``metadatastore``, ``resultclient``, and ``resultserver`` from ``openfe.storage``.
+
+**Fixed:**
+
+* <news item>
+
+**Security:**
+
+* <news item>