Skip to content

Fix the round-2 windows walls + port the MSYS2 sweep for the static shape - #17

Merged
ronaldtse merged 1 commit into
mainfrom
fix/msys-vpath-threads
Sep 13, 2026
Merged

ronaldtse merged 1 commit into
mainfrom
fix/msys-vpath-threads

Conversation

@ronaldtse

@ronaldtse ronaldtse commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

What this fixes

Two compile walls from tebako-runtime-python PR #16's windows leg (run 34725669811, /tmp log analysis in the commit body):

  1. Python/sysmodule.c:3924: 'VPATH' undeclared — the use is #ifdef MS_WINDOWS-gated; MSVC gets VPATH from PC/pyconfig.h, and the autotools sysmodule.o rule passes only -DABIFLAGS/$(MULTIARCH_CPPFLAGS), so the first msys compile of sysmodule.c dies.
  2. Python/thread_pthread.h:413: 'native_id' undeclared / :440: implicit declaration of 'pause' — the pthread ladder has no __MINGW32__ rung; upstream's pthread path is POSIX-only and the Windows thread support upstream actually tests is thread_nt.h behind the MSVC build. MSYS2's proven answer is not to grow pthread rungs but to select the NT thread model on mingw.

Then the full sweep: all 103 patches of the MSYS2 MINGW-packages mingw-w64-python series were triaged for our build shape (static-ish, --disable-shared, ucrt64, MS_WINDOWS defined by pyport_ms_windows_msys). Every compile/link-wall patch is ported; every skip is documented below. Two CI rounds after the first push surfaced three more promotions (0005, 0078, 0037) and the syslibs extension — each is called out with its run id.

Ported (MSYS2 number → repo patch → why)

New this round (18):

  • 0001configure_nt_threads_msys — the thread wall above: --with-nt-threads defaults yes on *-*-mingw*, NT_THREADS defined, pthread/semaphore probes skipped, _POSIX_THREADS undef'd where winpthreads leaks it via <windows.h>. thread.c compiles thread_nt.h; thread_pthread.h never enters the build.
  • 0005configure_mingw_defaults_msyspromoted from the "cross-only" skip pile on CI evidence (run 34730588522). Despite the MSYS2 title ("default cross configuration"), its configure.ac loader sources Misc/config_mingw for EVERY *-*-mingw* host, native included — and that file pins exactly the probes the first CI leg died on: ac_cv_func_alarm=ignore (the link probe finds alarm in the mingw import libs, but ucrt64's <unistd.h> declares it only under __USE_MINGW_ALARMsignalmodule.c:398 implicit declaration; MSVC has no HAVE_ALARM), ac_cv_file__dev_ptmx=ignore + ac_cv_file__dev_ptc=no (the MSYS2 shell emulates /dev/ptmx, so the native probe defines HAVE_DEV_PTMX and os_openpty_impl reaches the STREAMS ioctl(I_PUSH) calls mingw-w64 lacks → posixmodule.c:9000; os.openpty simply does not exist on Windows, matching MSVC), ftruncate/truncate pinned off (wrong mingw semantics; the _chsize path is the Windows-correct one), getpeername forced on. Misc/cross_mingw32 rides along verbatim (inert natively).
  • 0008posixmodule_nt_msys — the module must be named nt on mingw (PyInit_nt vs the posix registration; os.py imports nt when MACHDEP=win32). Boot/link wall.
  • 0016frozenmain_drop_msysfrozenmain.o references PC/frozen_dllmain.c symbols → exe link wall.
  • 0017socketmodule_msys — without it _socket never builds on mingw (header gates, SOCKET_LIBS, ws2tcpip probes).
  • 0036signal_ws2_32_msyssignalmodule.c:773 calls WSAGetLastError() under MS_WINDOWS; _signal is a bootstrap module so its .o rides MODULE_OBJS into the _bootstrap_python/_freeze_module/exe links, which carry $(LIBS) $(MODLIBS) $(SYSLIBS) but never the factory's TEBAKO_LIBS. The trailing -lws2_32 folds into BASEMODLIBSMODLIBS (makesetup), a var the factory's modlib_rewrites never touches. Applies after 0008 (its context line @MODULE_POSIXMODULE_NAME@ is 0008's output).
  • 0037io_winconsoleio_msyspromoted from the "feature parity" skip pile on CI evidence (run 34732802820 — a LINK wall this time). Upstream's autotools build never compiles Modules/_io/winconsoleio.c (the MSVC vcxproj carries it instead), so once 0002's HAVE_WINDOWS_CONSOLE_IO arm makes the console code paths live on mingw, the _freeze_module link dies on _get_console_type (Parser/myreadline.c:273/:276), _PyIO_get_console_type and winconsoleio_spec (Modules/_io/_iomodule.c:324/:584). The patch appends _io/winconsoleio.c to the _io bootstrap line (the file's whole body is #ifdef HAVE_WINDOWS_CONSOLE_IO — POSIX builds gain an empty object) and registers _testconsole on win32 (test-only, rides along verbatim).
  • 0041configure_inet_pton_msys — 0017's probe pair.
  • 0045timemodule_msys — ucrt _timezone spellings (strict-ANSI risk).
  • 0054sysmodule_vpath_msys — the VPATH wall above.
  • 0056bootstrap_python_link_flags_msys_bootstrap_python defines wmain; mingw's CRT only calls it under -municode → freeze-tool link wall on every build.
  • 0059ssl_module_msys — static openssl needs -lws2_32 -lcrypt32; plus the cert-store widening.
  • 0060configure_winsock_netdb_msyspromoted on cross-probe evidence. Without it every netdb probe fails on mingw (PY_CHECK_FUNC compiles void *x=<fn> against <netdb.h>, which on mingw-w64 declares none of them), leaving HAVE_GETHOSTNAME/HAVE_SHUTDOWN undefined while MSVC's PC/pyconfig.h defines both — so the 0017-enabled _socket would lose socket.gethostname() (#ifdef HAVE_GETHOSTNAME, socketmodule.c:5863) and shutdown versus the Windows reference. Verified by this PR's cross probe (before: all netdb HAVE_ undef; after: both defined). HAVE_GETADDRINFO/HAVE_GETNAMEINFO stay undefined exactly as upstream's PC build has them.
  • 0063configure_exeext_msys — deterministic .exe (upstream: empty EXEEXT + a case-probe).
  • 0075configure_disable_posix_modules_msys — the POSIX-only module set is n/a on mingw (subsumes 0058).
  • 0078configure_noerror_incompat_ptr_msyspromoted from the "warning-grade" skip pile: GCC 14 escalated -Wincompatible-pointer-types from warning to ERROR (CI ships gcc 16). The MSVC-oriented MS_WINDOWS branches of posixmodule.c pass wchar_t** where the CRT prototypes take const wchar_t* const* (PathCchSkipRoot :5269, _wexecv/_wexecve :7062/:7150, _wspawnv/_wspawnve :7823/:7945) — upstream code MSVC accepts. MSYS2's answer is the blanket demotion on mingw hosts rather than diverging from upstream at every call site (the qualifiers are ABI-honored either way).
  • 0097mmapmodule_no_seh_msys — the mmap gate passes on mingw (sys/mman.h exists), so the __try/__except SEH code hits GCC → compile error.
  • 0100configure_pthread_name_maxlen_msys_PYTHREAD_NAME_MAXLEN is used UNGUARDED in _threadmodule.c's Windows branch → compile wall.

Already in the tree from the earlier rounds (5): 0002 (pyport_ms_windows_msys), 0004 (configure_machdep_msys), 0007 (configure_pc_include_msys), 0052 (configure_c_locale_coercion_msys), 0096 (pycore_time_timeval_msys).

Plus one tebako deviation (the shape is ours; the lib set is MSYS2's):

  • configure_syslibs_msys — the factory's Mlibs.modlib_rewrites rewrites the generated Makefile's MODULE__SSL_LDFLAGS/MODULE__HASHLIB_LDFLAGS (and zlib/binascii) to absolute .a paths, dropping the -lws2_32 -lcrypt32 tail 0059 appends. Every link pulling _ssl.o/_hashlib.o then dies on undefined WSAStartup/CertOpenSystemStore/BCryptGenRandom. MSYS2 never needs the indirection (shared build: libpython3.dll resolves the tails through its own import list). Fix source-side: a host-conditional WINDOWS_SYSLIBS rides SYSLIBS, which sits after MODLIBS on every link line and is never rewritten downstream. Extended at run 34732802820 with the -lversion -lshlwapi -lpathcch tail of 0010's mingw LIBS hunk (0010 itself stays skipped as shared-build machinery): the same run's _freeze_module link died on GetFileVersionInfoSizeW/GetFileVersionInfoW/VerQueryValueW (Python/sysmodule.c:1666-1674-lversion) and __imp_PathCchSkipRoot/__imp_PathCchCombineEx (Python/fileutils.c:2160/:2407, Modules/posixmodule.c:5269-lpathcch; pathconfig.c's <shlwapi.h> arm → -lshlwapi). MSVC gets those libs from #pragma comment(lib, ...) in the sources — which GCC silently ignores. Full set: -lws2_32 -lcrypt32 -ladvapi32 -lbcrypt -lversion -lshlwapi -lpathcch on *-*-mingw*, empty elsewhere.

All generated-configure/pyconfig.h.in hunks are machine-exact regens with the upstream-pinned autoconf 2.72, not hand-syncs — proven by regenerating over the assembled 24-patch tree (configure/pyconfig.h.in come out byte-identical). The full series applied to pristine 3.14.7 in manifest order reproduces an independently staged sequential-MSYS2 reference tree byte-for-byte.

Skipped (every other patch in the 103, grouped)

  • Feature parity, not compile walls (deferred; flagged for a follow-up runtime-parity PR): 0018 (enables winreg/msvcrt/_winapi/winsound/overlapped — note _winapi is subprocess-critical on Windows, first candidate for the next round), 0038/0061/0073 (multiprocessing set), 0003 (sys flags), 0012 + 0066–0070 (dynload_win set — ucrt64 has dlfcn/dlopen; the VFS-path LoadLibrary question is factory/contract scope), 0062 (mmap enablement — the gate already passes), 0074 (selectmodule — examined this round: no compile wall (the POSIX branch compiles on mingw), but without it select() on sockets takes the _PyIsSelectable_fd range-check branch MSVC never runs → runtime divergence; joins the parity batch), 0042 (frozen importlib path-separator order — runtime behavior), 0043 (mingw stack reserve — link-flag parity with MSVC's bpo-40458 fix; no wall), 0050 (clock probes — MSVC-feature parity / winpthread dependency hygiene; in our shape the probes succeed AND link cleanly against default ucrt64 libs, cross-probe verified — if a winpthread undefined-ref ever appears at the factory, this is the queued fix), 0065 (pathcch forward-slash normalization — runtime correctness on / paths, not a wall; also a parity-batch candidate). (0037 was in this bucket in the first pass — promoted on link-wall evidence, see above.)
  • Shared-build / DLL naming / packaging (we are --disable-shared): 0009, 0010 (its mingw LIBS hunk is folded into the syslibs deviation — see above), 0011, 0013, 0014, 0015, 0028, 0029, 0034, 0039, 0047, 0048, 0049, 0071, 0086, 0088, 0090, 0093, 0098 (ABIFLAGS is empty in our shape), 0103.
  • Behavior / install-path, non-wall (portable fallbacks verified to exist): 0019–0022, 0026, 0027, 0030, 0035, 0040, 0057 (getpath — PYTHONHOME drives runtime), 0083, 0085, 0091, 0099.
  • Warning-grade only (no -Werror on these classes): 0031, 0033, 0081, 0082. (0078 was in this bucket in the first pass — promoted, see above.)
  • Cross-build only (we build natively in the ucrt64 shell): 0055, 0080. (0005 was mis-filed here in the first pass — its config_mingw loads natively; promoted, see above.)
  • 32-bit / debug / policy: 0006, 0072, 0092, 0094 (_WIN32_WINNT floor).
  • Tests/CI: 0044, 0051, 0053, 0079, 0084, 0087, 0095, 0102.
  • Inapplicable in our shape: 0023–0025 + 0076 (_ctypes disabled via the factory's Setup.local), 0077 (_uuid disabled), 0089 (_wmi not in the autoconf module set at all), 0032 (gettext absent in the CI env), 0046/0064 (dlopen probes harmless — mingw-w64 ships dlfcn), 0058 (subsumed by 0075), 0101 (autoconf version pin — we regenerate with the pinned 2.72 ourselves).

Tooling changes the sweep forced

  • Tfs::SourcePrep#audit now applies the series cumulatively in manifest order instead of git apply --check per patch on pristine. A quilt series cannot satisfy per-patch-pristine (0036's context line @MODULE_POSIXMODULE_NAME@ only exists after 0008). On a mid-series failure the remaining patches report a not-checked failure. Specs pin the dependent-context and not-checked-tail behaviors (103 examples, +2); the lint/README/onboarder comments follow.
  • tools/compile_smoke WALL_OBJECTS gains the round-2 walls and the patched module TUs per the regression-list contract: Python/sysmodule.o, Python/thread.o, Modules/posixmodule.o, Modules/signalmodule.o, Modules/timemodule.o, Modules/socketmodule.o, Modules/_threadmodule.o, Programs/_bootstrap_python.o — plus a platform-conditional WALL_OBJECTS_MSYS for Modules/_io/winconsoleio.o: the TU exists only in the patched tree (0037 adds it to the _io bootstrap line), so the pristine POSIX leg has no rule for it and must not fail on its absence, while on windows-msys a missing rule IS the regression. Patch-named .c auto-adds now probe the generated Makefile for a literal rule first: an n/a module (e.g. _ssl without an openssl dev package in the smoke env) emits NO rule, and make's implicit %.o:%.c fallback would otherwise compile it with the wrong flags and fail. Skips are reported in the OK line.

Verification (local; the native ucrt64 compile is CI's oracle)

  • tools/lint 3.14.7: 24-patch series applies clean to the pristine tarball (cumulative audit).
  • Series output tree byte-identical to the independently staged manifest-order reference; shipped configure/pyconfig.h.in equal the pinned autoconf 2.72 regen of the shipped configure.ac exactly.
  • rspec 103/0, tools/validate_manifests OK.
  • POSIX smoke leg: tools/compile_smoke 3.14.7 --platform linux-gnu — all 12 wall TUs compile on the pristine tree (regression proof for the POSIX asset).
  • Cross configure (x86_64-w64-mingw32-gcc 14, --host override, --disable-shared, NO manual ac_cv presets — config_mingw loads itself): NT_THREADS defined with _POSIX_THREADS undef'd; nt posixmodule.c and _signal signalmodule.c -lws2_32 in Setup.bootstrap; EXE/BUILDEXE=.exe; MODULE__SOCKET_STATE=yes with the ws2_32 tail; HAVE_GETHOSTNAME/HAVE_SHUTDOWN defined (MSVC parity); HAVE_ALARM/HAVE_DEV_PTMX/HAVE_TRUNCATE/HAVE_FTRUNCATE pinned off; HAVE_GETPEERNAME on; _posixshmem/grp/resource n/a; _PYTHREAD_NAME_MAXLEN=32766; the extended SYSLIBS tail; -Wno-error=incompatible-pointer-types in CFLAGS_NODIST; -DVPATH + -I./PC on the make -n Python/sysmodule.o line.
  • Cross compile+link sweep: all 13 wall TUs compile clean; the full core (libpython3.14.a, 191 objects) cross-compiles with zero errors; and Programs/_freeze_module — the run 34732802820 casualty — now cross-LINKS clean (winconsoleio.o in the _io objects; version/pathcch resolved by the extended SYSLIBS).

CI history: run 34730588522 failed on exactly the two probe walls 0005 pins (alarm, /dev/ptmx) plus the _bootstrap_python.o cascade from them; the earlier walls (sysmodule/thread) compiled. Run 34732802820 got through every compile and died at the _freeze_module LINK on the console symbols (0037) and the version/pathcch libs (the syslibs extension). The 0078 const-qualification errors were found by the local cross compile before the second push.

@ronaldtse
ronaldtse force-pushed the fix/msys-vpath-threads branch from 2feefd0 to 13572cc Compare September 13, 2026 02:20
… shape

Two observed walls from tebako-runtime-python PR #16 (run 34725669811):

- Python/sysmodule.c:3924 VPATH undeclared — the use is MS_WINDOWS-gated
  and MSVC gets VPATH from PC/pyconfig.h; the autotools sysmodule.o rule
  passes only -DABIFLAGS/$(MULTIARCH_CPPFLAGS). Ported MSYS2 0054
  (-DVPATH='"$(VPATH)"' on the sysmodule.o rule).
- Python/thread_pthread.h:413 native_id / :440 pause — the pthread
  ladder has no __MINGW32__ rung (upstream's pthread path is POSIX-only;
  the Windows thread support upstream tests is thread_nt.h behind the
  MSVC build). Ported MSYS2's proven answer, 0001: --with-nt-threads
  defaults yes on *-*-mingw*, AC_DEFINEs NT_THREADS, skips the pthread
  probe set; pythread.h/pycore_pythread.h/pycore_condvar.h undef
  _POSIX_THREADS (winpthreads leaks it via <windows.h>), so thread.c
  compiles thread_nt.h and thread_pthread.h never enters the build.

Then the full sweep of the MSYS2 MINGW-packages mingw-w64-python series
(103 patches) for our build shape (static-ish, --disable-shared, ucrt64,
MS_WINDOWS via pyport_ms_windows_msys). Ported, in MSYS2 numbering:
0001 (above), 0005 (mingw configure defaults — Misc/config_mingw loads
for EVERY mingw host, native included; despite the MSYS2 title it is
not cross-only: it pins ac_cv_func_alarm/ftruncate/truncate and
ac_cv_file__dev_ptmx/ptc off — run 34730588522's native ucrt64 walls:
signalmodule.c:398 implicit alarm() (the link probe finds alarm in the
import libs; ucrt64's unistd.h declares it only under
__USE_MINGW_ALARM) and posixmodule.c:9000 ioctl/I_PUSH (the msys shell
emulates /dev/ptmx, so HAVE_DEV_PTMX came out set and os_openpty_impl
reached the STREAMS ioctls mingw-w64 lacks) — and forces getpeername
on), 0008 (nt module naming — PyInit_nt vs posix registration,
os.py needs nt at MACHDEP=win32), 0016 (frozenmain.o references
PC/frozen_dllmain.c symbols — exe link wall), 0017 + 0041 (socketmodule
mingw support + the inet_pton probes — without them _socket never
builds), 0036 (_signal signalmodule.c -lws2_32 — signalmodule.c:773's
MS_WINDOWS WSAGetLastError walls the bootstrap links, which never see
the factory's TEBAKO_LIBS), 0037 (winconsoleio + _testconsole —
run 34732802820's _freeze_module LINK wall: upstream's autotools build
never compiles Modules/_io/winconsoleio.c (the MSVC vcxproj carries
it), so once 0002's HAVE_WINDOWS_CONSOLE_IO arm makes the console code
paths live on mingw the link dies on _get_console_type
(Parser/myreadline.c:273/:276), _PyIO_get_console_type and
winconsoleio_spec (Modules/_io/_iomodule.c); the TU is
HAVE_WINDOWS_CONSOLE_IO-guarded, an empty object on POSIX), 0045 (ucrt
_timezone spellings), 0054 (above), 0056 (_bootstrap_python defines
wmain — needs -municode), 0059 (static openssl needs -lws2_32 -lcrypt32
+ the cert-store widening), 0060 (winsock.h for the netdb probes —
without it every netdb probe fails on mingw,
HAVE_GETHOSTNAME/HAVE_SHUTDOWN stay undefined while PC/pyconfig.h
defines them, and socket.gethostname / shutdown vanish from the
0017-enabled _socket; proven by this PR's cross probe), 0063
(deterministic .exe), 0075 (the POSIX-only module set is n/a on mingw;
subsumes 0058), 0078 (GCC 14 escalated -Wincompatible-pointer-types to
an error; the MSVC-oriented MS_WINDOWS branches (PathCchSkipRoot,
_wexecv/_wspawnv families) pass wchar_t** where the CRT wants const
wchar_t* const* — MSYS2 demotes to warning on mingw rather than
diverging from upstream at every call site; gate-critical on any
current toolchain — CI ships gcc 16), 0097 (mmap's __try/__except is a
GCC error once the gate passes on mingw), 0100 (_PYTHREAD_NAME_MAXLEN
is used UNGUARDED in _threadmodule.c's Windows branch). With 0002,
0004, 0007, 0052, 0096 already in the tree from the earlier rounds, the
series is 24 patches. One tebako deviation: configure_syslibs_msys —
the factory's modlib_rewrites rewrite MODULE__SSL_LDFLAGS/
MODULE__HASHLIB_LDFLAGS to absolute .a paths, dropping 0059's -lws2_32
-lcrypt32 tail, so a host-conditional WINDOWS_SYSLIBS rides SYSLIBS,
which sits after MODLIBS on every link line and is never rewritten
downstream. The lib set is MSYS2's: the original four plus the
-lversion -lshlwapi -lpathcch tail of 0010's mingw LIBS hunk (0010
itself stays skipped as shared-build machinery; MSVC gets the same
libs via #pragma comment(lib, ...) in the sources, which GCC ignores —
run 34732802820's undefined GetFileVersionInfoSizeW/GetFileVersionInfoW/
VerQueryValueW (-> -lversion) and __imp_PathCchSkipRoot/
__imp_PathCchCombineEx (-> -lpathcch) are exactly those pragmas going
silent). All generated-configure hunks are machine-exact regens with
the upstream-pinned autoconf 2.72 (byte-identical regen against
pristine proven), not hand-syncs. The deliberate skips (shared-build/
DLL naming, feature parity like 0018's _winapi, cross-only, tests/CI,
warning-grade) are grouped in the PR body.

Two tooling changes the sweep forced:

- Tfs::SourcePrep#audit now applies the series CUMULATIVELY in manifest
  order instead of git apply --check per patch on pristine: a quilt
  series cannot satisfy per-patch-pristine (0036's context line is
  @MODULE_POSIXMODULE_NAME@, which only exists after 0008). On a
  mid-series failure the remaining patches report a not-checked
  failure. Specs pin the dependent-context and not-checked-tail
  behavior; the lint/README/onboarder comments follow.
- tools/compile_smoke WALL_OBJECTS gains the round-2 walls and the
  patched module TUs (sysmodule, thread, posixmodule, signalmodule,
  timemodule, socketmodule, _threadmodule, _bootstrap_python — the
  regression-list contract), plus a platform-conditional
  WALL_OBJECTS_MSYS for Modules/_io/winconsoleio.o: the TU exists only
  in the patched tree (0037 adds it to the _io bootstrap line), so the
  pristine POSIX leg has no rule for it and must not fail on its
  absence, while on windows-msys a missing rule IS the regression.
  Patch-named .c auto-adds now probe the generated Makefile for a
  literal rule first: an n/a module (e.g. _ssl without an openssl dev
  package in the smoke env) emits NO rule, and make's implicit %.o:%.c
  fallback would otherwise compile it with the wrong flags and fail.
  Skips are reported in the OK line.

Verified locally (macOS + homebrew mingw-w64 gcc 14 — the ucrt64
compile itself is CI's oracle): the 24-patch series applies clean to
pristine 3.14.7 (tools/lint) and reproduces the independently staged
manifest-order MSYS2 reference tree byte-for-byte; the shipped
configure and pyconfig.h.in equal the pinned autoconf 2.72 regen of the
shipped configure.ac exactly (regen-fidelity checked on the assembled
tree); rspec 103/0; validate_manifests OK; the POSIX smoke leg compiles
all 12 wall TUs on the pristine tree; a cross configure
(--host=x86_64-w64-mingw32, --disable-shared, no manual ac_cv presets —
config_mingw loads itself) confirms NT_THREADS defined with
_POSIX_THREADS undef'd, nt posixmodule.c and _signal signalmodule.c
-lws2_32 in Setup.bootstrap, EXE/BUILDEXE=.exe, MODULE__SOCKET_STATE=
yes with the ws2_32 tail, HAVE_GETHOSTNAME/HAVE_SHUTDOWN defined (MSVC
parity), HAVE_ALARM/HAVE_DEV_PTMX/HAVE_TRUNCATE pinned off,
HAVE_GETPEERNAME forced on, _posixshmem/grp/resource n/a,
_PTHREAD_NAME_MAXLEN=32766, the extended SYSLIBS tail
(-lws2_32 -lcrypt32 -ladvapi32 -lbcrypt -lversion -lshlwapi -lpathcch),
-Wno-error=incompatible-pointer-types in CFLAGS_NODIST, and -DVPATH on
the make -n Python/sysmodule.o compile line with -I./PC present. And
the compile+link sweep: all 13 wall TUs cross-compile clean, the FULL
core (libpython3.14.a — 191 objects) cross-compiles with zero errors,
and Programs/_freeze_module — the run 34732802820 casualty — now
cross-LINKS clean (winconsoleio.o in the _io objects, the extended
SYSLIBS resolving version/pathcch).
@ronaldtse
ronaldtse force-pushed the fix/msys-vpath-threads branch from 13572cc to 71883d3 Compare September 13, 2026 03:02
@ronaldtse
ronaldtse merged commit 0fb8cd8 into main Sep 13, 2026
10 checks passed
ronaldtse pushed a commit to tamatebako/tebako-runtime-python that referenced this pull request Sep 13, 2026
v0.3.2 is the v0.3.1 catalog plus the MSYS2-MINGW-packages sweep for
the static ucrt64 shape (tamatebako/python#17): 18 more patches ported
— the observed round-2 walls (0054 -DVPATH on the sysmodule.o rule,
0001 --with-nt-threads default + the _POSIX_THREADS undefs) plus the
series audit's compile/link/module-reach set (0005 mingw configure
defaults — the alarm/dev-ptmx probe pins that walled the native ucrt64
smoke leg, 0008 nt module naming, 0016 frozenmain exe-link, 0017+0041
socketmodule + inet_pton probes, 0036 _signal -lws2_32, 0037
winconsoleio in the _io bootstrap line, 0045 ucrt _timezone spellings,
0056 -municode for _bootstrap_python, 0059 ssl/hashlib -l tails, 0060
winsock netdb probes, 0063 deterministic EXEEXT, 0075 POSIX-only
modules n/a, 0078 -Wno-error=incompatible-pointer-types for gcc>=14,
0097 mmapmodule SEH, 0100 _PTHREAD_NAME_MAXLEN) and the syslibs
deviation extended with 0010's -lversion -lshlwapi -lpathcch tail.
The version catalog is unchanged.
ronaldtse added a commit to tamatebako/tebako-runtime-python that referenced this pull request Sep 13, 2026
* Return the windows-2022 leg on the source factory's scenario axis (TODO.python/05 item 2)

The windows row comes back to the matrix (63e8336's descope), now
buildable: tamatebako/python's platform-scenario axis ships the
msys2/ucrt64 patch series as tfs-python-<base>-src-windows-msys.tar.gz
assets (tamatebako/python#14).

- .github/matrix.json: re-add the windows-2022 row, exactly as
  descoped. The windows-gated legs in _build-platform.yml (msys2
  provisioning, build, symbol provenance, boot smoke) and publish.yml's
  windows job need no edits — they light up on the row's legs.
- SourceFetcher: scenario-aware assets (the ruby factory's
  scenario_asset_names convention, minus ruby's msys two-pass split —
  CPython needs one tree per scenario). fetch/tarball_sha256 take an
  optional platform; a mingw host names the -windows-msys asset, POSIX
  keeps the unsuffixed back-compat asset. Builder threads @platform.
- compute_matrix.rb: jit lines skip windows (upstream's JIT target
  whitelist admits MSVC windows only — the musl skip's sibling), and a
  windows leg exists only when the pinned source_release's SHA256SUMS
  carries the line's windows-msys asset — missing asset = loud skip,
  not a red leg. The SHA256SUMS read is lazy and scoped to a windows
  env row surviving the filters (POSIX triggers stay fetch-free), and
  an unreadable pin surfaces as a named exit-64 error, never a stack
  trace.
- contract.yml: source_release v0.2.0 -> v0.3.0 (the release carrying
  the scenario axis). EXPECTED RED until tamatebako/python#14 merges
  and v0.3.0 is cut: the pinned SHA256SUMS 404s, and lint.yml's
  unfiltered compute_matrix fails named. Once v0.3.0 exists this PR
  goes green with no further edits (the #2 expected-red precedent).

Verified locally: rspec 14/0 (new source_fetcher_spec covers the
scenario grammar over a file:// mirror); check_contract green; POSIX
matrices byte-identical to main; PLATFORM_FILTER=windows against a
staged SHA256SUMS yields exactly the 3.14.7 leg with loud skips for
the other five catalog lines, and the all-missing case yields an empty
matrix with six loud skips; ruby -c sweep clean.

* contract: consume source release v0.3.1 (the errmap.h msys include fix)

v0.3.1 is the v0.3.0 catalog plus the MSYS2-MINGW-packages 0007 port
(tamatebako/python#16): PC/errmap.h is checked-in source and upstream's
Makefile grants -I$(srcdir)/PC to MSVC only; the windows-msys scenario
tarball now compiles Objects/exceptions.o through. The version catalog
is unchanged.

* contract: consume source release v0.3.2 (the msys compile-wall sweep)

v0.3.2 is the v0.3.1 catalog plus the MSYS2-MINGW-packages sweep for
the static ucrt64 shape (tamatebako/python#17): 18 more patches ported
— the observed round-2 walls (0054 -DVPATH on the sysmodule.o rule,
0001 --with-nt-threads default + the _POSIX_THREADS undefs) plus the
series audit's compile/link/module-reach set (0005 mingw configure
defaults — the alarm/dev-ptmx probe pins that walled the native ucrt64
smoke leg, 0008 nt module naming, 0016 frozenmain exe-link, 0017+0041
socketmodule + inet_pton probes, 0036 _signal -lws2_32, 0037
winconsoleio in the _io bootstrap line, 0045 ucrt _timezone spellings,
0056 -municode for _bootstrap_python, 0059 ssl/hashlib -l tails, 0060
winsock netdb probes, 0063 deterministic EXEEXT, 0075 POSIX-only
modules n/a, 0078 -Wno-error=incompatible-pointer-types for gcc>=14,
0097 mmapmodule SEH, 0100 _PTHREAD_NAME_MAXLEN) and the syslibs
deviation extended with 0010's -lversion -lshlwapi -lpathcch tail.
The version catalog is unchanged.

* contract: consume source release v0.3.3 (the mingw shared-build machinery)

v0.3.3 is the v0.3.2 catalog plus the MSYS2-MINGW-packages
shared-build port (tamatebako/python#18): 0009/0010 (declspec exports
+ the enable_shared mingw configure arms — LDLIBRARY=
libpython3.14.dll.a / DLLLIBRARY=libpython3.14.dll), the dynload_win
coherence cluster (0011 PYD_PLATFORM_TAG, 0012 dynload_win + PC/dl_nt.o,
0013 SHLIB_SUFFIX=.pyd, 0014 mingw SOABI/EXT_SUFFIX, 0015 MS_DLL_ID),
the two interpreter boot walls the PR's own smoke leg forced (0018:
winreg/msvcrt/_winapi as bootstrap-static modules — _bootstrap_external
imports winreg at top level on win32; 0034: sysmodule's winver/dllhandle
guard MS_COREDLL -> defined(MS_WINDOWS) && defined(Py_ENABLE_SHARED)),
0066 (the _Py_CheckPython3 declaration conflict), and 0039 (-municode
exe link + the win resources + pythonw.exe). Observable rename on
windows-msys: extension modules are .cp314-mingw_x86_64_ucrt_gnu.pyd.
The version catalog is unchanged.

* windows-msys: build --enable-shared + ship the runtime DLL (issue 40's answer)

The v0.3.3 source release carries the MSYS2-series machinery (0009/0010
shared build, 0011-0015, 0018, 0034, 0039, 0066); the first trp leg
against it (run 34745803168) failed exactly as predicted on the static
shape: the stdlib extensions link as loadable PE modules
(array.cp314-mingw_x86_64_ucrt_gnu.pyd, Makefile:3624) and PE modules
cannot carry undefined symbols — the link died on _Py_Dealloc & co
because --disable-shared gives them no libpython to resolve against.
On PE the answer is structural, not a patch: build --enable-shared and
ship the DLL.

- configure: --enable-shared on windows-msys (POSIX keeps
  --disable-shared, the shipped single-exe shape).
- substitute_makefiles: the exe link recipe substitution is now
  platform-shaped. The 0039 patch (win_resources_pythonw_msys) rewrites
  the rule to `...$(LINKFORSHARED) -municode -o $@ Programs/python.o ...
  $(SYSLIBS) python_exe.o` — the old POSIX regex never matched that
  line, so the prereq side rewrote python.o -> tebako_python.o while the
  recipe kept linking Programs/python.o: the driver link was SILENTLY
  lost (the fs TU compiled but never linked). The msys substitution
  strips -municode (the fs TU defines plain main; the mingw CRT demands
  wmain otherwise) and keeps python_exe.o (the version/icon resource).
  verify_substitutions now also requires the rewritten recipe line
  (-o $@ Programs/tebako_python.o) — the silent-loss class fails closed.
- abi: EXT_SUFFIX regex accepts the mingw spelling
  (cp314-mingw_x86_64_ucrt_gnu.pyd) alongside the POSIX cpython-* one.
- make: on msys pre-build the DLL target SERIALLY before the -j make.
  Upstream's `$(DLLLIBRARY) libpython$(LDVERSION).dll.a` two-target rule
  (-Wl,--out-implib=$@) is parallel-unsafe: observed locally, the
  interleaved import-library write replaced the 39 MB libpython3.14.dll
  with the 1.2 MB import stub. The serial pre-build materializes both
  files with the correct expansion; the parallel make then finds the
  rule up-to-date and never re-runs it.
- finalize stages libpython<X.Y>.dll beside the runtime package as
  <package>.dll (unique per leg — two same-ABI legs share the PE name
  and would collide in the merged release workspace); a missing DLL is
  a named error (exit 130), never a silent static regression. The image
  layout card declares runtime_dll (additive, schema_minor 3,
  spec 22 §2.1); PythonVersion#msys_dll_name stays the name's single
  owner. tools/boot_smoke materializes the PE-named copy next to the
  in-leg exe before any scenario boots it (the OS loader resolves it
  before any driver code runs).
- upload_release: expected_facets requires <package>.dll on windows —
  a leg that failed to stage it is an incomplete release, not an
  optional facet. (The dll manifest facet, install_as included, was
  already modeled; the comments claiming "nothing ships it today" are
  corrected.)
- The mingw support set never reaches the user: LIBS="-static-libgcc
  -l:libwinpthread.a" static-links libgcc/libwinpthread INTO the
  runtime's PE modules (ruby factory's proven Mlibs::MSYS_DLL_LIBRARIES
  recipe; LIBS lands in the link tail, after the objects, so the
  archives resolve). Verified against a local cross build of the v0.3.3
  tree: libpython3.14.dll (39,320,791 bytes) imports system DLLs only
  (KERNEL32/ADVAPI32/VERSION/WS2_32/bcrypt/api-ms-win-*) —
  libgcc_s_seh-1.dll and libwinpthread-1.dll are gone; the exe imports
  system + libpython3.14.dll only. A bare windows machine installs
  nothing (audience rule 1).
- README's windows-boundary section describes the shared shape.

Gates: rspec 14/0; check_contract.rb green; ruby -c sweep green. The
recipe substitutions were cross-checked by driving the real
exe_recipe_substitution against the real v0.3.3 Makefile.pre shapes
(msys probe tree + POSIX smoke trees): each pair matches its own shape
exactly once; the msys result line verified char-exact. trp#16 stays
draft — no merge without the owner's go-ahead.

* contract: consume source release v0.3.4 (the sysconfig posix-mode pair)

v0.3.4 (tamatebako/python#19) ports MSYS2 0003 (sys._is_mingw flags) +
0020 (mingw builds take the posix sysconfig path) — the answer to the
first --enable-shared leg's pybuilddir.txt wall (run 34748169561:
KeyError 'LIBPL'; upstream's nt path never merges the Makefile-parsed
build_time_vars). 0034's hunk 3 returns to MSYS2-verbatim context.
Windows-msys asset verified on the release (sha256
d5628491ee94b21dce0c92ba6913342cd68c3f99e7a569428ab16a4278803052).

* contract: consume source release v0.3.5 (the module-libs cluster)

v0.3.5 (tamatebako/python#20) ports MSYS2 0061 + 0073
(_multiprocessing: the win32 enable condition, then -lws2_32), 0074
(select: _MSC_VER → MS_WIN32 guards + -lws2_32), and 0076 (_ctypes:
-lole32 -loleaut32 -luuid) — the answer to the shared-extension link
wall the v0.3.4 leg hit (run 34750005177: undefined __WSAFDIsSet /
__imp_select / __imp_WSAGetLastError at the select.pyd and
_multiprocessing.pyd links; PE resolves every symbol at link time and
upstream's PY_STDLIB_MOD stanzas carry no per-module libs — MSVC gets
them via PCbuild). Windows-msys asset verified on the release (sha256
18f20a10922e55da3ce2213769cccf2db4a732b8898e2c128173051ed7b1f8b3;
the POSIX assets are byte-identical with v0.3.4's).

* contract: consume source release v0.3.6 (the mmap + legacy-DLL trio)

v0.3.6 (tamatebako/python#21) ports MSYS2 0062 (mmap builds on win32 —
its absence killed ensurepip in `make install` on the v0.3.5 leg, run
34753278108: pip's vendored cachecontrol imports mmap), 0067
(PYTHONLEGACYWINDOWSDLLLOADING — the bpo-36085 DEFAULT_DIRS search
never consults PATH, so the build-tree check removed
_decimal/_zstd/_testinternalcapi whose dep DLLs live in
<ucrt64>/bin), and 0064 (have_dynamic_loading for the dlopen-gated
test modules); 0076 returns to MSYS2-verbatim context. Windows-msys
asset verified on the release (sha256
20ec675d03bb3612e4c69026fa723d97cdd323a2faf38a22bee32105236f1c23;
the POSIX assets are byte-identical with v0.3.5's).

* windows-msys: install with prefix=/A/t + stage pip explicitly (round-6)

CPython's install rules spell every destination $(DESTDIR)$(dir) with no
separator; with the baked drive-qualified prefix (A:/t) that concatenates
into one junk component (stage + "A:/t/..." — the failing recipes are
mostly `-`-prefixed, so make exited green over the wreckage and
staged_prefix_tree's "no A/t tree" was the first truth, exit 105 on run
34756006093). The msys install now overrides prefix to the POSIX spelling
/A/t — every install dir var derives unexpanded from $(prefix) (autoconf
convention), so the override flows to every rule — and skips ensurepip:
pip's --root rebase drive-strips the built python's BUILD-TREE prefix,
never the staged one, so its pip landed under <stage>/<build-tree path>.

place_pip stages pip explicitly instead: the bundled wheel's own
installer run into the staged lib/pythonX.Y/site-packages (the series'
0026 site.py branch puts the runtime's site-packages there — verified
against the applied tree), mirroring ensurepip's -c invocation but with
--target deciding placement. The paths ride argv so the msys runtime
translates them for the native exe (a path embedded in the -c payload
crosses verbatim), and --no-compile mirrors ensurepip: .pyc records
written here would spell the stage paths — dead links in the mounted
image.

POSIX legs keep the old recipe exactly (ensurepip still rides the
install). Recorded follow-up: whether ensurepip's --root placement ever
landed pip in the POSIX staged trees either (the build-tree prefix
rebase exists there too) — checked separately, not in this commit.

* ci: spell the provenance exe with .exe on the windows leg (round-7)

msys stat() transparency lets foo.exe satisfy `[ -s foo ]`, so the
step's extensionless-or-.exe dance always kept the extensionless
spelling; the native nm/objdump receive the argument verbatim, get
ENOENT, and the script's discarded stderr read back as a stripped exe
(five failed provenance checks on run 34758424181 — the exe itself was
fine: the same build's place_pip had just run it). The name carries
.exe on this leg by construction (Platform#exe_suffix) — spell it out,
the ruby factory's ruby.exe spelling. The script now also prints nm's
own diagnostic when the symbol-table capture comes back empty, so a
silent nm failure never reads as "stripped" again.

* ci: capture-then-test the disassembly greps (the SIGPIPE false negative)

grep -q exits on its first match; under pipefail the still-writing
printf then dies of SIGPIPE and the pipeline reads 141 — a false
"not found". POSIX legs never noticed: their objdump honors
--disassemble=main tightly (a screenful, under the pipe buffer). A
binutils PE objdump's span runs from the symbol to the next data
island — 4,700 lines on the windows leg — so grep matched early,
printf was killed, and run 34759975017 failed the forwarding check
while its own evidence printout showed main calling
tebako_driver_boot at 140da1789. Same discipline as the nm side
already documented in this script: capture first, then test.

* windows-msys: bare boot pins driver + getpath warnings, not exit status (round-9)

Run 34760958198's windows leg failed the bare scenario on its POSIX
assertions: the child printed 'unreached' and exited 0, with only the
driver warning and getpath's "Could not find platform independent
libraries <prefix>" on stderr.

Mechanism, established against the 3.14.7 sources: the encodings import
at startup is unconditional (pylifecycle.c _PyUnicode_InitEncodings ->
unicodeobject.c _PyCodec_InitRegistry, fatal on failure), so a child
that ran user code necessarily FOUND a stdlib. The channel is upstream:
Modules/getpath.py's winreg section (os_name == 'nt' and
use_environment and winreg) appends the HKCU/HKLM
SOFTWARE\Python\PythonCore\<ver>\PythonPath registry values to sys.path
— and the GHA runner has a registered host python. Bare boot on windows
is therefore environment-dependent by upstream design: exit status and
the encodings fatal are not assertable there.

The msys bare scenario now pins the deterministic surface instead: the
driver's bare-boot warning (ours, every platform) plus getpath's
platstdlib warning (the prefix fell back to the baked-in mount root,
which never exists on the runner). POSIX keeps the strict trio
(non-zero exit + driver warning + encodings fatal) — no registry
fallback exists there.

---------

Co-authored-by: tebako-ci <tebako@ribose.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants