Fix the round-2 windows walls + port the MSYS2 sweep for the static shape - #17
Merged
Merged
Conversation
ronaldtse
force-pushed
the
fix/msys-vpath-threads
branch
from
September 13, 2026 02:20
2feefd0 to
13572cc
Compare
… shape Two observed walls from tebako-runtime-python PR #16 (run 34725669811): - Python/sysmodule.c:3924 VPATH undeclared — the use is MS_WINDOWS-gated and MSVC gets VPATH from PC/pyconfig.h; the autotools sysmodule.o rule passes only -DABIFLAGS/$(MULTIARCH_CPPFLAGS). Ported MSYS2 0054 (-DVPATH='"$(VPATH)"' on the sysmodule.o rule). - Python/thread_pthread.h:413 native_id / :440 pause — the pthread ladder has no __MINGW32__ rung (upstream's pthread path is POSIX-only; the Windows thread support upstream tests is thread_nt.h behind the MSVC build). Ported MSYS2's proven answer, 0001: --with-nt-threads defaults yes on *-*-mingw*, AC_DEFINEs NT_THREADS, skips the pthread probe set; pythread.h/pycore_pythread.h/pycore_condvar.h undef _POSIX_THREADS (winpthreads leaks it via <windows.h>), so thread.c compiles thread_nt.h and thread_pthread.h never enters the build. Then the full sweep of the MSYS2 MINGW-packages mingw-w64-python series (103 patches) for our build shape (static-ish, --disable-shared, ucrt64, MS_WINDOWS via pyport_ms_windows_msys). Ported, in MSYS2 numbering: 0001 (above), 0005 (mingw configure defaults — Misc/config_mingw loads for EVERY mingw host, native included; despite the MSYS2 title it is not cross-only: it pins ac_cv_func_alarm/ftruncate/truncate and ac_cv_file__dev_ptmx/ptc off — run 34730588522's native ucrt64 walls: signalmodule.c:398 implicit alarm() (the link probe finds alarm in the import libs; ucrt64's unistd.h declares it only under __USE_MINGW_ALARM) and posixmodule.c:9000 ioctl/I_PUSH (the msys shell emulates /dev/ptmx, so HAVE_DEV_PTMX came out set and os_openpty_impl reached the STREAMS ioctls mingw-w64 lacks) — and forces getpeername on), 0008 (nt module naming — PyInit_nt vs posix registration, os.py needs nt at MACHDEP=win32), 0016 (frozenmain.o references PC/frozen_dllmain.c symbols — exe link wall), 0017 + 0041 (socketmodule mingw support + the inet_pton probes — without them _socket never builds), 0036 (_signal signalmodule.c -lws2_32 — signalmodule.c:773's MS_WINDOWS WSAGetLastError walls the bootstrap links, which never see the factory's TEBAKO_LIBS), 0037 (winconsoleio + _testconsole — run 34732802820's _freeze_module LINK wall: upstream's autotools build never compiles Modules/_io/winconsoleio.c (the MSVC vcxproj carries it), so once 0002's HAVE_WINDOWS_CONSOLE_IO arm makes the console code paths live on mingw the link dies on _get_console_type (Parser/myreadline.c:273/:276), _PyIO_get_console_type and winconsoleio_spec (Modules/_io/_iomodule.c); the TU is HAVE_WINDOWS_CONSOLE_IO-guarded, an empty object on POSIX), 0045 (ucrt _timezone spellings), 0054 (above), 0056 (_bootstrap_python defines wmain — needs -municode), 0059 (static openssl needs -lws2_32 -lcrypt32 + the cert-store widening), 0060 (winsock.h for the netdb probes — without it every netdb probe fails on mingw, HAVE_GETHOSTNAME/HAVE_SHUTDOWN stay undefined while PC/pyconfig.h defines them, and socket.gethostname / shutdown vanish from the 0017-enabled _socket; proven by this PR's cross probe), 0063 (deterministic .exe), 0075 (the POSIX-only module set is n/a on mingw; subsumes 0058), 0078 (GCC 14 escalated -Wincompatible-pointer-types to an error; the MSVC-oriented MS_WINDOWS branches (PathCchSkipRoot, _wexecv/_wspawnv families) pass wchar_t** where the CRT wants const wchar_t* const* — MSYS2 demotes to warning on mingw rather than diverging from upstream at every call site; gate-critical on any current toolchain — CI ships gcc 16), 0097 (mmap's __try/__except is a GCC error once the gate passes on mingw), 0100 (_PYTHREAD_NAME_MAXLEN is used UNGUARDED in _threadmodule.c's Windows branch). With 0002, 0004, 0007, 0052, 0096 already in the tree from the earlier rounds, the series is 24 patches. One tebako deviation: configure_syslibs_msys — the factory's modlib_rewrites rewrite MODULE__SSL_LDFLAGS/ MODULE__HASHLIB_LDFLAGS to absolute .a paths, dropping 0059's -lws2_32 -lcrypt32 tail, so a host-conditional WINDOWS_SYSLIBS rides SYSLIBS, which sits after MODLIBS on every link line and is never rewritten downstream. The lib set is MSYS2's: the original four plus the -lversion -lshlwapi -lpathcch tail of 0010's mingw LIBS hunk (0010 itself stays skipped as shared-build machinery; MSVC gets the same libs via #pragma comment(lib, ...) in the sources, which GCC ignores — run 34732802820's undefined GetFileVersionInfoSizeW/GetFileVersionInfoW/ VerQueryValueW (-> -lversion) and __imp_PathCchSkipRoot/ __imp_PathCchCombineEx (-> -lpathcch) are exactly those pragmas going silent). All generated-configure hunks are machine-exact regens with the upstream-pinned autoconf 2.72 (byte-identical regen against pristine proven), not hand-syncs. The deliberate skips (shared-build/ DLL naming, feature parity like 0018's _winapi, cross-only, tests/CI, warning-grade) are grouped in the PR body. Two tooling changes the sweep forced: - Tfs::SourcePrep#audit now applies the series CUMULATIVELY in manifest order instead of git apply --check per patch on pristine: a quilt series cannot satisfy per-patch-pristine (0036's context line is @MODULE_POSIXMODULE_NAME@, which only exists after 0008). On a mid-series failure the remaining patches report a not-checked failure. Specs pin the dependent-context and not-checked-tail behavior; the lint/README/onboarder comments follow. - tools/compile_smoke WALL_OBJECTS gains the round-2 walls and the patched module TUs (sysmodule, thread, posixmodule, signalmodule, timemodule, socketmodule, _threadmodule, _bootstrap_python — the regression-list contract), plus a platform-conditional WALL_OBJECTS_MSYS for Modules/_io/winconsoleio.o: the TU exists only in the patched tree (0037 adds it to the _io bootstrap line), so the pristine POSIX leg has no rule for it and must not fail on its absence, while on windows-msys a missing rule IS the regression. Patch-named .c auto-adds now probe the generated Makefile for a literal rule first: an n/a module (e.g. _ssl without an openssl dev package in the smoke env) emits NO rule, and make's implicit %.o:%.c fallback would otherwise compile it with the wrong flags and fail. Skips are reported in the OK line. Verified locally (macOS + homebrew mingw-w64 gcc 14 — the ucrt64 compile itself is CI's oracle): the 24-patch series applies clean to pristine 3.14.7 (tools/lint) and reproduces the independently staged manifest-order MSYS2 reference tree byte-for-byte; the shipped configure and pyconfig.h.in equal the pinned autoconf 2.72 regen of the shipped configure.ac exactly (regen-fidelity checked on the assembled tree); rspec 103/0; validate_manifests OK; the POSIX smoke leg compiles all 12 wall TUs on the pristine tree; a cross configure (--host=x86_64-w64-mingw32, --disable-shared, no manual ac_cv presets — config_mingw loads itself) confirms NT_THREADS defined with _POSIX_THREADS undef'd, nt posixmodule.c and _signal signalmodule.c -lws2_32 in Setup.bootstrap, EXE/BUILDEXE=.exe, MODULE__SOCKET_STATE= yes with the ws2_32 tail, HAVE_GETHOSTNAME/HAVE_SHUTDOWN defined (MSVC parity), HAVE_ALARM/HAVE_DEV_PTMX/HAVE_TRUNCATE pinned off, HAVE_GETPEERNAME forced on, _posixshmem/grp/resource n/a, _PTHREAD_NAME_MAXLEN=32766, the extended SYSLIBS tail (-lws2_32 -lcrypt32 -ladvapi32 -lbcrypt -lversion -lshlwapi -lpathcch), -Wno-error=incompatible-pointer-types in CFLAGS_NODIST, and -DVPATH on the make -n Python/sysmodule.o compile line with -I./PC present. And the compile+link sweep: all 13 wall TUs cross-compile clean, the FULL core (libpython3.14.a — 191 objects) cross-compiles with zero errors, and Programs/_freeze_module — the run 34732802820 casualty — now cross-LINKS clean (winconsoleio.o in the _io objects, the extended SYSLIBS resolving version/pathcch).
ronaldtse
force-pushed
the
fix/msys-vpath-threads
branch
from
September 13, 2026 03:02
13572cc to
71883d3
Compare
ronaldtse
pushed a commit
to tamatebako/tebako-runtime-python
that referenced
this pull request
Sep 13, 2026
v0.3.2 is the v0.3.1 catalog plus the MSYS2-MINGW-packages sweep for the static ucrt64 shape (tamatebako/python#17): 18 more patches ported — the observed round-2 walls (0054 -DVPATH on the sysmodule.o rule, 0001 --with-nt-threads default + the _POSIX_THREADS undefs) plus the series audit's compile/link/module-reach set (0005 mingw configure defaults — the alarm/dev-ptmx probe pins that walled the native ucrt64 smoke leg, 0008 nt module naming, 0016 frozenmain exe-link, 0017+0041 socketmodule + inet_pton probes, 0036 _signal -lws2_32, 0037 winconsoleio in the _io bootstrap line, 0045 ucrt _timezone spellings, 0056 -municode for _bootstrap_python, 0059 ssl/hashlib -l tails, 0060 winsock netdb probes, 0063 deterministic EXEEXT, 0075 POSIX-only modules n/a, 0078 -Wno-error=incompatible-pointer-types for gcc>=14, 0097 mmapmodule SEH, 0100 _PTHREAD_NAME_MAXLEN) and the syslibs deviation extended with 0010's -lversion -lshlwapi -lpathcch tail. The version catalog is unchanged.
ronaldtse
added a commit
to tamatebako/tebako-runtime-python
that referenced
this pull request
Sep 13, 2026
* Return the windows-2022 leg on the source factory's scenario axis (TODO.python/05 item 2) The windows row comes back to the matrix (63e8336's descope), now buildable: tamatebako/python's platform-scenario axis ships the msys2/ucrt64 patch series as tfs-python-<base>-src-windows-msys.tar.gz assets (tamatebako/python#14). - .github/matrix.json: re-add the windows-2022 row, exactly as descoped. The windows-gated legs in _build-platform.yml (msys2 provisioning, build, symbol provenance, boot smoke) and publish.yml's windows job need no edits — they light up on the row's legs. - SourceFetcher: scenario-aware assets (the ruby factory's scenario_asset_names convention, minus ruby's msys two-pass split — CPython needs one tree per scenario). fetch/tarball_sha256 take an optional platform; a mingw host names the -windows-msys asset, POSIX keeps the unsuffixed back-compat asset. Builder threads @platform. - compute_matrix.rb: jit lines skip windows (upstream's JIT target whitelist admits MSVC windows only — the musl skip's sibling), and a windows leg exists only when the pinned source_release's SHA256SUMS carries the line's windows-msys asset — missing asset = loud skip, not a red leg. The SHA256SUMS read is lazy and scoped to a windows env row surviving the filters (POSIX triggers stay fetch-free), and an unreadable pin surfaces as a named exit-64 error, never a stack trace. - contract.yml: source_release v0.2.0 -> v0.3.0 (the release carrying the scenario axis). EXPECTED RED until tamatebako/python#14 merges and v0.3.0 is cut: the pinned SHA256SUMS 404s, and lint.yml's unfiltered compute_matrix fails named. Once v0.3.0 exists this PR goes green with no further edits (the #2 expected-red precedent). Verified locally: rspec 14/0 (new source_fetcher_spec covers the scenario grammar over a file:// mirror); check_contract green; POSIX matrices byte-identical to main; PLATFORM_FILTER=windows against a staged SHA256SUMS yields exactly the 3.14.7 leg with loud skips for the other five catalog lines, and the all-missing case yields an empty matrix with six loud skips; ruby -c sweep clean. * contract: consume source release v0.3.1 (the errmap.h msys include fix) v0.3.1 is the v0.3.0 catalog plus the MSYS2-MINGW-packages 0007 port (tamatebako/python#16): PC/errmap.h is checked-in source and upstream's Makefile grants -I$(srcdir)/PC to MSVC only; the windows-msys scenario tarball now compiles Objects/exceptions.o through. The version catalog is unchanged. * contract: consume source release v0.3.2 (the msys compile-wall sweep) v0.3.2 is the v0.3.1 catalog plus the MSYS2-MINGW-packages sweep for the static ucrt64 shape (tamatebako/python#17): 18 more patches ported — the observed round-2 walls (0054 -DVPATH on the sysmodule.o rule, 0001 --with-nt-threads default + the _POSIX_THREADS undefs) plus the series audit's compile/link/module-reach set (0005 mingw configure defaults — the alarm/dev-ptmx probe pins that walled the native ucrt64 smoke leg, 0008 nt module naming, 0016 frozenmain exe-link, 0017+0041 socketmodule + inet_pton probes, 0036 _signal -lws2_32, 0037 winconsoleio in the _io bootstrap line, 0045 ucrt _timezone spellings, 0056 -municode for _bootstrap_python, 0059 ssl/hashlib -l tails, 0060 winsock netdb probes, 0063 deterministic EXEEXT, 0075 POSIX-only modules n/a, 0078 -Wno-error=incompatible-pointer-types for gcc>=14, 0097 mmapmodule SEH, 0100 _PTHREAD_NAME_MAXLEN) and the syslibs deviation extended with 0010's -lversion -lshlwapi -lpathcch tail. The version catalog is unchanged. * contract: consume source release v0.3.3 (the mingw shared-build machinery) v0.3.3 is the v0.3.2 catalog plus the MSYS2-MINGW-packages shared-build port (tamatebako/python#18): 0009/0010 (declspec exports + the enable_shared mingw configure arms — LDLIBRARY= libpython3.14.dll.a / DLLLIBRARY=libpython3.14.dll), the dynload_win coherence cluster (0011 PYD_PLATFORM_TAG, 0012 dynload_win + PC/dl_nt.o, 0013 SHLIB_SUFFIX=.pyd, 0014 mingw SOABI/EXT_SUFFIX, 0015 MS_DLL_ID), the two interpreter boot walls the PR's own smoke leg forced (0018: winreg/msvcrt/_winapi as bootstrap-static modules — _bootstrap_external imports winreg at top level on win32; 0034: sysmodule's winver/dllhandle guard MS_COREDLL -> defined(MS_WINDOWS) && defined(Py_ENABLE_SHARED)), 0066 (the _Py_CheckPython3 declaration conflict), and 0039 (-municode exe link + the win resources + pythonw.exe). Observable rename on windows-msys: extension modules are .cp314-mingw_x86_64_ucrt_gnu.pyd. The version catalog is unchanged. * windows-msys: build --enable-shared + ship the runtime DLL (issue 40's answer) The v0.3.3 source release carries the MSYS2-series machinery (0009/0010 shared build, 0011-0015, 0018, 0034, 0039, 0066); the first trp leg against it (run 34745803168) failed exactly as predicted on the static shape: the stdlib extensions link as loadable PE modules (array.cp314-mingw_x86_64_ucrt_gnu.pyd, Makefile:3624) and PE modules cannot carry undefined symbols — the link died on _Py_Dealloc & co because --disable-shared gives them no libpython to resolve against. On PE the answer is structural, not a patch: build --enable-shared and ship the DLL. - configure: --enable-shared on windows-msys (POSIX keeps --disable-shared, the shipped single-exe shape). - substitute_makefiles: the exe link recipe substitution is now platform-shaped. The 0039 patch (win_resources_pythonw_msys) rewrites the rule to `...$(LINKFORSHARED) -municode -o $@ Programs/python.o ... $(SYSLIBS) python_exe.o` — the old POSIX regex never matched that line, so the prereq side rewrote python.o -> tebako_python.o while the recipe kept linking Programs/python.o: the driver link was SILENTLY lost (the fs TU compiled but never linked). The msys substitution strips -municode (the fs TU defines plain main; the mingw CRT demands wmain otherwise) and keeps python_exe.o (the version/icon resource). verify_substitutions now also requires the rewritten recipe line (-o $@ Programs/tebako_python.o) — the silent-loss class fails closed. - abi: EXT_SUFFIX regex accepts the mingw spelling (cp314-mingw_x86_64_ucrt_gnu.pyd) alongside the POSIX cpython-* one. - make: on msys pre-build the DLL target SERIALLY before the -j make. Upstream's `$(DLLLIBRARY) libpython$(LDVERSION).dll.a` two-target rule (-Wl,--out-implib=$@) is parallel-unsafe: observed locally, the interleaved import-library write replaced the 39 MB libpython3.14.dll with the 1.2 MB import stub. The serial pre-build materializes both files with the correct expansion; the parallel make then finds the rule up-to-date and never re-runs it. - finalize stages libpython<X.Y>.dll beside the runtime package as <package>.dll (unique per leg — two same-ABI legs share the PE name and would collide in the merged release workspace); a missing DLL is a named error (exit 130), never a silent static regression. The image layout card declares runtime_dll (additive, schema_minor 3, spec 22 §2.1); PythonVersion#msys_dll_name stays the name's single owner. tools/boot_smoke materializes the PE-named copy next to the in-leg exe before any scenario boots it (the OS loader resolves it before any driver code runs). - upload_release: expected_facets requires <package>.dll on windows — a leg that failed to stage it is an incomplete release, not an optional facet. (The dll manifest facet, install_as included, was already modeled; the comments claiming "nothing ships it today" are corrected.) - The mingw support set never reaches the user: LIBS="-static-libgcc -l:libwinpthread.a" static-links libgcc/libwinpthread INTO the runtime's PE modules (ruby factory's proven Mlibs::MSYS_DLL_LIBRARIES recipe; LIBS lands in the link tail, after the objects, so the archives resolve). Verified against a local cross build of the v0.3.3 tree: libpython3.14.dll (39,320,791 bytes) imports system DLLs only (KERNEL32/ADVAPI32/VERSION/WS2_32/bcrypt/api-ms-win-*) — libgcc_s_seh-1.dll and libwinpthread-1.dll are gone; the exe imports system + libpython3.14.dll only. A bare windows machine installs nothing (audience rule 1). - README's windows-boundary section describes the shared shape. Gates: rspec 14/0; check_contract.rb green; ruby -c sweep green. The recipe substitutions were cross-checked by driving the real exe_recipe_substitution against the real v0.3.3 Makefile.pre shapes (msys probe tree + POSIX smoke trees): each pair matches its own shape exactly once; the msys result line verified char-exact. trp#16 stays draft — no merge without the owner's go-ahead. * contract: consume source release v0.3.4 (the sysconfig posix-mode pair) v0.3.4 (tamatebako/python#19) ports MSYS2 0003 (sys._is_mingw flags) + 0020 (mingw builds take the posix sysconfig path) — the answer to the first --enable-shared leg's pybuilddir.txt wall (run 34748169561: KeyError 'LIBPL'; upstream's nt path never merges the Makefile-parsed build_time_vars). 0034's hunk 3 returns to MSYS2-verbatim context. Windows-msys asset verified on the release (sha256 d5628491ee94b21dce0c92ba6913342cd68c3f99e7a569428ab16a4278803052). * contract: consume source release v0.3.5 (the module-libs cluster) v0.3.5 (tamatebako/python#20) ports MSYS2 0061 + 0073 (_multiprocessing: the win32 enable condition, then -lws2_32), 0074 (select: _MSC_VER → MS_WIN32 guards + -lws2_32), and 0076 (_ctypes: -lole32 -loleaut32 -luuid) — the answer to the shared-extension link wall the v0.3.4 leg hit (run 34750005177: undefined __WSAFDIsSet / __imp_select / __imp_WSAGetLastError at the select.pyd and _multiprocessing.pyd links; PE resolves every symbol at link time and upstream's PY_STDLIB_MOD stanzas carry no per-module libs — MSVC gets them via PCbuild). Windows-msys asset verified on the release (sha256 18f20a10922e55da3ce2213769cccf2db4a732b8898e2c128173051ed7b1f8b3; the POSIX assets are byte-identical with v0.3.4's). * contract: consume source release v0.3.6 (the mmap + legacy-DLL trio) v0.3.6 (tamatebako/python#21) ports MSYS2 0062 (mmap builds on win32 — its absence killed ensurepip in `make install` on the v0.3.5 leg, run 34753278108: pip's vendored cachecontrol imports mmap), 0067 (PYTHONLEGACYWINDOWSDLLLOADING — the bpo-36085 DEFAULT_DIRS search never consults PATH, so the build-tree check removed _decimal/_zstd/_testinternalcapi whose dep DLLs live in <ucrt64>/bin), and 0064 (have_dynamic_loading for the dlopen-gated test modules); 0076 returns to MSYS2-verbatim context. Windows-msys asset verified on the release (sha256 20ec675d03bb3612e4c69026fa723d97cdd323a2faf38a22bee32105236f1c23; the POSIX assets are byte-identical with v0.3.5's). * windows-msys: install with prefix=/A/t + stage pip explicitly (round-6) CPython's install rules spell every destination $(DESTDIR)$(dir) with no separator; with the baked drive-qualified prefix (A:/t) that concatenates into one junk component (stage + "A:/t/..." — the failing recipes are mostly `-`-prefixed, so make exited green over the wreckage and staged_prefix_tree's "no A/t tree" was the first truth, exit 105 on run 34756006093). The msys install now overrides prefix to the POSIX spelling /A/t — every install dir var derives unexpanded from $(prefix) (autoconf convention), so the override flows to every rule — and skips ensurepip: pip's --root rebase drive-strips the built python's BUILD-TREE prefix, never the staged one, so its pip landed under <stage>/<build-tree path>. place_pip stages pip explicitly instead: the bundled wheel's own installer run into the staged lib/pythonX.Y/site-packages (the series' 0026 site.py branch puts the runtime's site-packages there — verified against the applied tree), mirroring ensurepip's -c invocation but with --target deciding placement. The paths ride argv so the msys runtime translates them for the native exe (a path embedded in the -c payload crosses verbatim), and --no-compile mirrors ensurepip: .pyc records written here would spell the stage paths — dead links in the mounted image. POSIX legs keep the old recipe exactly (ensurepip still rides the install). Recorded follow-up: whether ensurepip's --root placement ever landed pip in the POSIX staged trees either (the build-tree prefix rebase exists there too) — checked separately, not in this commit. * ci: spell the provenance exe with .exe on the windows leg (round-7) msys stat() transparency lets foo.exe satisfy `[ -s foo ]`, so the step's extensionless-or-.exe dance always kept the extensionless spelling; the native nm/objdump receive the argument verbatim, get ENOENT, and the script's discarded stderr read back as a stripped exe (five failed provenance checks on run 34758424181 — the exe itself was fine: the same build's place_pip had just run it). The name carries .exe on this leg by construction (Platform#exe_suffix) — spell it out, the ruby factory's ruby.exe spelling. The script now also prints nm's own diagnostic when the symbol-table capture comes back empty, so a silent nm failure never reads as "stripped" again. * ci: capture-then-test the disassembly greps (the SIGPIPE false negative) grep -q exits on its first match; under pipefail the still-writing printf then dies of SIGPIPE and the pipeline reads 141 — a false "not found". POSIX legs never noticed: their objdump honors --disassemble=main tightly (a screenful, under the pipe buffer). A binutils PE objdump's span runs from the symbol to the next data island — 4,700 lines on the windows leg — so grep matched early, printf was killed, and run 34759975017 failed the forwarding check while its own evidence printout showed main calling tebako_driver_boot at 140da1789. Same discipline as the nm side already documented in this script: capture first, then test. * windows-msys: bare boot pins driver + getpath warnings, not exit status (round-9) Run 34760958198's windows leg failed the bare scenario on its POSIX assertions: the child printed 'unreached' and exited 0, with only the driver warning and getpath's "Could not find platform independent libraries <prefix>" on stderr. Mechanism, established against the 3.14.7 sources: the encodings import at startup is unconditional (pylifecycle.c _PyUnicode_InitEncodings -> unicodeobject.c _PyCodec_InitRegistry, fatal on failure), so a child that ran user code necessarily FOUND a stdlib. The channel is upstream: Modules/getpath.py's winreg section (os_name == 'nt' and use_environment and winreg) appends the HKCU/HKLM SOFTWARE\Python\PythonCore\<ver>\PythonPath registry values to sys.path — and the GHA runner has a registered host python. Bare boot on windows is therefore environment-dependent by upstream design: exit status and the encodings fatal are not assertable there. The msys bare scenario now pins the deterministic surface instead: the driver's bare-boot warning (ours, every platform) plus getpath's platstdlib warning (the prefix fell back to the baked-in mount root, which never exists on the runner). POSIX keeps the strict trio (non-zero exit + driver warning + encodings fatal) — no registry fallback exists there. --------- Co-authored-by: tebako-ci <tebako@ribose.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Two compile walls from tebako-runtime-python PR #16's windows leg (run 34725669811,
/tmplog analysis in the commit body):Python/sysmodule.c:3924: 'VPATH' undeclared— the use is#ifdef MS_WINDOWS-gated; MSVC getsVPATHfromPC/pyconfig.h, and the autotoolssysmodule.orule passes only-DABIFLAGS/$(MULTIARCH_CPPFLAGS), so the first msys compile of sysmodule.c dies.Python/thread_pthread.h:413: 'native_id' undeclared/:440: implicit declaration of 'pause'— the pthread ladder has no__MINGW32__rung; upstream's pthread path is POSIX-only and the Windows thread support upstream actually tests isthread_nt.hbehind the MSVC build. MSYS2's proven answer is not to grow pthread rungs but to select the NT thread model on mingw.Then the full sweep: all 103 patches of the MSYS2 MINGW-packages
mingw-w64-pythonseries were triaged for our build shape (static-ish,--disable-shared, ucrt64,MS_WINDOWSdefined bypyport_ms_windows_msys). Every compile/link-wall patch is ported; every skip is documented below. Two CI rounds after the first push surfaced three more promotions (0005, 0078, 0037) and the syslibs extension — each is called out with its run id.Ported (MSYS2 number → repo patch → why)
New this round (18):
configure_nt_threads_msys— the thread wall above:--with-nt-threadsdefaults yes on*-*-mingw*,NT_THREADSdefined, pthread/semaphore probes skipped,_POSIX_THREADSundef'd where winpthreads leaks it via<windows.h>.thread.ccompilesthread_nt.h;thread_pthread.hnever enters the build.configure_mingw_defaults_msys— promoted from the "cross-only" skip pile on CI evidence (run 34730588522). Despite the MSYS2 title ("default cross configuration"), its configure.ac loader sourcesMisc/config_mingwfor EVERY*-*-mingw*host, native included — and that file pins exactly the probes the first CI leg died on:ac_cv_func_alarm=ignore(the link probe findsalarmin the mingw import libs, but ucrt64's<unistd.h>declares it only under__USE_MINGW_ALARM→signalmodule.c:398implicit declaration; MSVC has noHAVE_ALARM),ac_cv_file__dev_ptmx=ignore+ac_cv_file__dev_ptc=no(the MSYS2 shell emulates/dev/ptmx, so the native probe definesHAVE_DEV_PTMXandos_openpty_implreaches the STREAMSioctl(I_PUSH)calls mingw-w64 lacks →posixmodule.c:9000;os.openptysimply does not exist on Windows, matching MSVC),ftruncate/truncatepinned off (wrong mingw semantics; the_chsizepath is the Windows-correct one),getpeernameforced on.Misc/cross_mingw32rides along verbatim (inert natively).posixmodule_nt_msys— the module must be namednton mingw (PyInit_ntvs the posix registration;os.pyimportsntwhenMACHDEP=win32). Boot/link wall.frozenmain_drop_msys—frozenmain.oreferencesPC/frozen_dllmain.csymbols → exe link wall.socketmodule_msys— without it_socketnever builds on mingw (header gates,SOCKET_LIBS, ws2tcpip probes).signal_ws2_32_msys—signalmodule.c:773callsWSAGetLastError()underMS_WINDOWS;_signalis a bootstrap module so its.oridesMODULE_OBJSinto the_bootstrap_python/_freeze_module/exe links, which carry$(LIBS) $(MODLIBS) $(SYSLIBS)but never the factory'sTEBAKO_LIBS. The trailing-lws2_32folds intoBASEMODLIBS→MODLIBS(makesetup), a var the factory'smodlib_rewritesnever touches. Applies after 0008 (its context line@MODULE_POSIXMODULE_NAME@is 0008's output).io_winconsoleio_msys— promoted from the "feature parity" skip pile on CI evidence (run 34732802820 — a LINK wall this time). Upstream's autotools build never compilesModules/_io/winconsoleio.c(the MSVC vcxproj carries it instead), so once 0002'sHAVE_WINDOWS_CONSOLE_IOarm makes the console code paths live on mingw, the_freeze_modulelink dies on_get_console_type(Parser/myreadline.c:273/:276),_PyIO_get_console_typeandwinconsoleio_spec(Modules/_io/_iomodule.c:324/:584). The patch appends_io/winconsoleio.cto the_iobootstrap line (the file's whole body is#ifdef HAVE_WINDOWS_CONSOLE_IO— POSIX builds gain an empty object) and registers_testconsoleon win32 (test-only, rides along verbatim).configure_inet_pton_msys— 0017's probe pair.timemodule_msys— ucrt_timezonespellings (strict-ANSI risk).sysmodule_vpath_msys— the VPATH wall above.bootstrap_python_link_flags_msys—_bootstrap_pythondefineswmain; mingw's CRT only calls it under-municode→ freeze-tool link wall on every build.ssl_module_msys— static openssl needs-lws2_32 -lcrypt32; plus the cert-store widening.configure_winsock_netdb_msys— promoted on cross-probe evidence. Without it every netdb probe fails on mingw (PY_CHECK_FUNCcompilesvoid *x=<fn>against<netdb.h>, which on mingw-w64 declares none of them), leavingHAVE_GETHOSTNAME/HAVE_SHUTDOWNundefined while MSVC'sPC/pyconfig.hdefines both — so the 0017-enabled_socketwould losesocket.gethostname()(#ifdef HAVE_GETHOSTNAME, socketmodule.c:5863) andshutdownversus the Windows reference. Verified by this PR's cross probe (before: all netdb HAVE_ undef; after: both defined).HAVE_GETADDRINFO/HAVE_GETNAMEINFOstay undefined exactly as upstream's PC build has them.configure_exeext_msys— deterministic.exe(upstream: empty EXEEXT + a case-probe).configure_disable_posix_modules_msys— the POSIX-only module set is n/a on mingw (subsumes 0058).configure_noerror_incompat_ptr_msys— promoted from the "warning-grade" skip pile: GCC 14 escalated-Wincompatible-pointer-typesfrom warning to ERROR (CI ships gcc 16). The MSVC-orientedMS_WINDOWSbranches ofposixmodule.cpasswchar_t**where the CRT prototypes takeconst wchar_t* const*(PathCchSkipRoot:5269,_wexecv/_wexecve:7062/:7150,_wspawnv/_wspawnve:7823/:7945) — upstream code MSVC accepts. MSYS2's answer is the blanket demotion on mingw hosts rather than diverging from upstream at every call site (the qualifiers are ABI-honored either way).mmapmodule_no_seh_msys— the mmap gate passes on mingw (sys/mman.hexists), so the__try/__exceptSEH code hits GCC → compile error.configure_pthread_name_maxlen_msys—_PYTHREAD_NAME_MAXLENis used UNGUARDED in_threadmodule.c's Windows branch → compile wall.Already in the tree from the earlier rounds (5): 0002 (
pyport_ms_windows_msys), 0004 (configure_machdep_msys), 0007 (configure_pc_include_msys), 0052 (configure_c_locale_coercion_msys), 0096 (pycore_time_timeval_msys).Plus one tebako deviation (the shape is ours; the lib set is MSYS2's):
configure_syslibs_msys— the factory'sMlibs.modlib_rewritesrewrites the generated Makefile'sMODULE__SSL_LDFLAGS/MODULE__HASHLIB_LDFLAGS(and zlib/binascii) to absolute.apaths, dropping the-lws2_32 -lcrypt32tail 0059 appends. Every link pulling_ssl.o/_hashlib.othen dies on undefinedWSAStartup/CertOpenSystemStore/BCryptGenRandom. MSYS2 never needs the indirection (shared build: libpython3.dll resolves the tails through its own import list). Fix source-side: a host-conditionalWINDOWS_SYSLIBSridesSYSLIBS, which sits afterMODLIBSon every link line and is never rewritten downstream. Extended at run 34732802820 with the-lversion -lshlwapi -lpathcchtail of 0010's mingwLIBShunk (0010 itself stays skipped as shared-build machinery): the same run's_freeze_modulelink died onGetFileVersionInfoSizeW/GetFileVersionInfoW/VerQueryValueW(Python/sysmodule.c:1666-1674→-lversion) and__imp_PathCchSkipRoot/__imp_PathCchCombineEx(Python/fileutils.c:2160/:2407,Modules/posixmodule.c:5269→-lpathcch;pathconfig.c's<shlwapi.h>arm →-lshlwapi). MSVC gets those libs from#pragma comment(lib, ...)in the sources — which GCC silently ignores. Full set:-lws2_32 -lcrypt32 -ladvapi32 -lbcrypt -lversion -lshlwapi -lpathcchon*-*-mingw*, empty elsewhere.All generated-
configure/pyconfig.h.inhunks are machine-exact regens with the upstream-pinned autoconf 2.72, not hand-syncs — proven by regenerating over the assembled 24-patch tree (configure/pyconfig.h.income out byte-identical). The full series applied to pristine 3.14.7 in manifest order reproduces an independently staged sequential-MSYS2 reference tree byte-for-byte.Skipped (every other patch in the 103, grouped)
_winapiis subprocess-critical on Windows, first candidate for the next round), 0038/0061/0073 (multiprocessing set), 0003 (sys flags), 0012 + 0066–0070 (dynload_win set — ucrt64 has dlfcn/dlopen; the VFS-path LoadLibrary question is factory/contract scope), 0062 (mmap enablement — the gate already passes), 0074 (selectmodule — examined this round: no compile wall (the POSIX branch compiles on mingw), but without itselect()on sockets takes the_PyIsSelectable_fdrange-check branch MSVC never runs → runtime divergence; joins the parity batch), 0042 (frozen importlib path-separator order — runtime behavior), 0043 (mingw stack reserve — link-flag parity with MSVC's bpo-40458 fix; no wall), 0050 (clock probes — MSVC-feature parity / winpthread dependency hygiene; in our shape the probes succeed AND link cleanly against default ucrt64 libs, cross-probe verified — if a winpthread undefined-ref ever appears at the factory, this is the queued fix), 0065 (pathcch forward-slash normalization — runtime correctness on/paths, not a wall; also a parity-batch candidate). (0037 was in this bucket in the first pass — promoted on link-wall evidence, see above.)--disable-shared): 0009, 0010 (its mingwLIBShunk is folded into the syslibs deviation — see above), 0011, 0013, 0014, 0015, 0028, 0029, 0034, 0039, 0047, 0048, 0049, 0071, 0086, 0088, 0090, 0093, 0098 (ABIFLAGS is empty in our shape), 0103.-Werroron these classes): 0031, 0033, 0081, 0082. (0078 was in this bucket in the first pass — promoted, see above.)config_mingwloads natively; promoted, see above.)_WIN32_WINNTfloor)._ctypesdisabled via the factory's Setup.local), 0077 (_uuiddisabled), 0089 (_wminot in the autoconf module set at all), 0032 (gettext absent in the CI env), 0046/0064 (dlopen probes harmless — mingw-w64 ships dlfcn), 0058 (subsumed by 0075), 0101 (autoconf version pin — we regenerate with the pinned 2.72 ourselves).Tooling changes the sweep forced
Tfs::SourcePrep#auditnow applies the series cumulatively in manifest order instead ofgit apply --checkper patch on pristine. A quilt series cannot satisfy per-patch-pristine (0036's context line@MODULE_POSIXMODULE_NAME@only exists after 0008). On a mid-series failure the remaining patches report a not-checked failure. Specs pin the dependent-context and not-checked-tail behaviors (103 examples, +2); the lint/README/onboarder comments follow.tools/compile_smokeWALL_OBJECTS gains the round-2 walls and the patched module TUs per the regression-list contract:Python/sysmodule.o,Python/thread.o,Modules/posixmodule.o,Modules/signalmodule.o,Modules/timemodule.o,Modules/socketmodule.o,Modules/_threadmodule.o,Programs/_bootstrap_python.o— plus a platform-conditionalWALL_OBJECTS_MSYSforModules/_io/winconsoleio.o: the TU exists only in the patched tree (0037 adds it to the_iobootstrap line), so the pristine POSIX leg has no rule for it and must not fail on its absence, while on windows-msys a missing rule IS the regression. Patch-named.cauto-adds now probe the generated Makefile for a literal rule first: an n/a module (e.g._sslwithout an openssl dev package in the smoke env) emits NO rule, and make's implicit%.o:%.cfallback would otherwise compile it with the wrong flags and fail. Skips are reported in the OK line.Verification (local; the native ucrt64 compile is CI's oracle)
tools/lint 3.14.7: 24-patch series applies clean to the pristine tarball (cumulative audit).configure/pyconfig.h.inequal the pinned autoconf 2.72 regen of the shippedconfigure.acexactly.rspec103/0,tools/validate_manifestsOK.tools/compile_smoke 3.14.7 --platform linux-gnu— all 12 wall TUs compile on the pristine tree (regression proof for the POSIX asset).x86_64-w64-mingw32-gcc14,--hostoverride,--disable-shared, NO manual ac_cv presets —config_mingwloads itself):NT_THREADSdefined with_POSIX_THREADSundef'd;nt posixmodule.cand_signal signalmodule.c -lws2_32inSetup.bootstrap;EXE/BUILDEXE=.exe;MODULE__SOCKET_STATE=yeswith the ws2_32 tail;HAVE_GETHOSTNAME/HAVE_SHUTDOWNdefined (MSVC parity);HAVE_ALARM/HAVE_DEV_PTMX/HAVE_TRUNCATE/HAVE_FTRUNCATEpinned off;HAVE_GETPEERNAMEon;_posixshmem/grp/resourcen/a;_PYTHREAD_NAME_MAXLEN=32766; the extendedSYSLIBStail;-Wno-error=incompatible-pointer-typesinCFLAGS_NODIST;-DVPATH+-I./PCon themake -n Python/sysmodule.oline.libpython3.14.a, 191 objects) cross-compiles with zero errors; andPrograms/_freeze_module— the run 34732802820 casualty — now cross-LINKS clean (winconsoleio.oin the_ioobjects;version/pathcchresolved by the extendedSYSLIBS).CI history: run 34730588522 failed on exactly the two probe walls 0005 pins (
alarm,/dev/ptmx) plus the_bootstrap_python.ocascade from them; the earlier walls (sysmodule/thread) compiled. Run 34732802820 got through every compile and died at the_freeze_moduleLINK on the console symbols (0037) and theversion/pathcchlibs (the syslibs extension). The 0078 const-qualification errors were found by the local cross compile before the second push.