`Optimize1qGatesDecomposition` deadlocks on Python-defined gates

[splch]

Environment

• Qiskit version: 2.5.0
• Python version: 3.10-3.14
• Operating system: macOS, Linux, Windows

What is happening?

Since #15567, run_optimize_1q_gates_decomposition enters rayon while holding the GIL (L457). Workers fetching a Python-defined gate's matrix (L404) block in PyGILState_Ensure while the main thread blocks on the rayon latch -> deadlock (0 % CPU, immune to Ctrl-C).

So transpile() hangs at optimization_level >= 1 for any target with Python-defined 1q gates - e.g. qiskit-ionq and qiskit-braket-provider native backends (see qiskit-community/qiskit-ionq#266).

How can we reproduce the issue?

import numpy as np
from qiskit import QuantumCircuit, generate_preset_pass_manager
from qiskit.circuit import Gate, Measure, Parameter
from qiskit.transpiler import Target

class PhasedX(Gate):
    def __init__(self, phi):
        super().__init__("phased_x", 1, [phi])
    def __array__(self, dtype=None, copy=None):
        phi = float(self.params[0])
        return np.array([[0, np.exp(-1j * phi)], [np.exp(1j * phi), 0]], dtype=complex)

target = Target(num_qubits=2)
target.add_instruction(PhasedX(Parameter("phi")))
target.add_instruction(Measure())

qc = QuantumCircuit(2)
qc.append(PhasedX(0.1), [0])
qc.append(PhasedX(0.2), [1])
qc.measure_all()

pm = generate_preset_pass_manager(optimization_level=2, target=target)
print(pm.run(qc))  # hangs

Completes on 2.4.2, or with QISKIT_IN_PARALLEL=TRUE (serial path).

What should happen?

Transpilation should complete. The pass may skip Python-defined gates, but must not deadlock.

Any suggestions?

Release the GIL around the par_iter, prefetch Python matrices before it, or fall back to serial when a run contains a Python-defined op.

[mtreinish]

Thanks for reporting this, it is clearly a bug on inspecting the code again with fresh eyes. I was also able to reproduce it locally.

It looks like we had a clear testing gap on Optimize1qGatesDecomposition with a custom gate in the target which let this slip in. The other multithreaded passes added in 2.5.0 are already releasing the GIL prior to entering a multithreaded part for this exact reason, but we missed it on optimize 1q decomp.

The only small complexity on your suggestion is just wiring through the py::detach() call as needed isn't straightforward because the pass function as written is also used from the C API where python isn't accessible. So we'll need to split out the python entrypoint from the rust native one and likely split the internals to facilitate releasing the gil for the parallel portion and then reacquiring it for the serial output construction. But it's a small change to make to the structure of the code.

I'll push up a fix PR shortly and we can tag it for inclusion in a 2.5.1 release.

[splch]

Thank you very much @mtreinish! I really appreciate it 😄