Commit 575619f
committed
Port the mingw shared-build machinery: MSYS2 0009/0010 + the dynload_win cluster (0011-0015) + 0018 + 0034 + 0066 + 0039; the smoke leg goes --enable-shared
The wall (tebako-runtime-python PR #16, run 34735362057, leg
windows/3.14.7): the full factory build reaches the extension modules
and dies linking Modules/array.cpython-314.so — undefined references
to PyExc_EOFError/_Py_Dealloc/PyUnicode_* and the rest of the core
API. On PE/Windows a loadable module cannot carry undefined symbols at
link time, and statically embedding the core per-extension breaks
interpreter-state identity — the --disable-shared + .so extensions
shape is impossible by construction on Windows. The intended shape is
--enable-shared (the factory's PythonVersion#msys_dll_name and
upload_release dll-facet path were written for it; the ruby factory
ships that way).
Eight MSYS2 MINGW-packages ports close it, inserted in series order:
- 0009 -> exports_declspec_msys (verbatim): HAVE_DECLSPEC_DLL on
__MINGW32__ + the cygwin arms (auto-import functions, dllimport
data, exported PyMODINIT_FUNC). Without it the core exports nothing
in the MSVC style and the import library binds no API surface.
Inert for the static build: the special-linkage block still gates on
defined(Py_ENABLE_SHARED) || defined(__CYGWIN__).
- 0010 -> configure_shared_mingw_msys, with ONE hunk elided (the
closing LIBS hunk — one carrier for -lversion/-lshlwapi/-lpathcch/
-lbcrypt, the syslibs deviation's post-MODLIBS SYSLIBS position; a
pre-MODLIBS LIBS copy of -lbcrypt would not resolve libcrypto's
static refs). The ported hunks: the enable_shared MINGW arms
(LDLIBRARY=libpython$(LDVERSION).dll.a, DLLLIBRARY=
libpython$(LDVERSION).dll — the factory's msys_dll_name —,
BLDLIBRARY=-L. -lpython$(LDVERSION)), the static LDLIBRARY=.a arm,
LDSHARED/LDCXXSHARED=$(CC) -shared -Wl,--enable-auto-image-base, and
LIBPYTHON/MODULE_DEPS_SHARED for win32 — what puts $(BLDLIBRARY) on
every stdlib extension link and $(LDLIBRARY) on its dependency list.
The 0009+0010 cross proof then exposed the wall cascade; each further
port is forced by named evidence (local cross build,
x86_64-w64-mingw32-gcc 14 on macOS, configure --host=
x86_64-w64-mingw32 --enable-shared):
1. The libpython3.14.dll link died: Modules/getpath.o references
PyWin_DLLhModule (Modules/getpath.c:811-816, compiled only under
Py_ENABLE_SHARED && MS_WINDOWS — the static tree never misses it),
and upstream defines the symbol only in PC/dl_nt.c, which the
autoconf build never compiles. 0012 (dynload_win_msys, verbatim)
cures it: DYNLOADFILE=dynload_win.o plus PC/dl_nt.o in
extra_machdep_objs. The loader switch is itself forced — base mingw
has no dlopen (configure falls to dynload_stub.o = a shared build
that cannot load anything; the v0.3.2 skip note's "ucrt64 has
dlfcn" was wrong).
2. 0012 drags its coherence cluster, previously deferred as interop
naming: the dynload_win.o compile rule passes
-DPYD_PLATFORM_TAG='"$(PYD_PLATFORM_TAG)"' (0011), and
dynload_win's import filetab is .pyd-based
(Include/internal/pycore_importdl.h, upstream), so the build-time
suffix must agree — 0013 (SHLIB_SUFFIX=.pyd + the import-lib
install rule) and 0014 (mingw SOABI=cp<ver><thread>-
<PYD_PLATFORM_TAG>, EXT_SUFFIX=.<SOABI>.pyd). Observable rename:
windows-msys extension modules change spelling from
.cpython-314.so to .cp314-mingw_x86_64_ucrt_gnu.pyd. And dl_nt.c's
PyWin_DLLVersionString = MS_DLL_ID needs 0015's definition
(MS_DLL_ID via CFLAGS_NODIST; upstream defines it only in the MSVC
pyconfig).
3. dynload_win.c then failed to COMPILE: its static _Py_CheckPython3
collides with pylifecycle.h's external declaration under
MS_WINDOWS. 0066 (verbatim) drops the declaration; no other caller
exists.
4. python.exe then failed to link: Programs/python.c provides wmain
under MS_WINDOWS, and linking wmain needs -municode — without it
the mingw CRT pulls the WinMain startup (crtexewin) and dies
(crtexewin.c:62: undefined reference to `WinMain'). 0039 (one
anchor elided — its closing configure.ac hunk rides on 0010's
elided LIBS case; the FIELD3/RCFLAGS additions apply verbatim in a
fresh mingw case at the same spot) rewrites the interpreter link
with -municode + the python_exe.o version/icon resource, adds
pythonw.exe (-municode -mwindows, the GUI variant every windows
install ships), the DLL's python_nt.o version resource, the
windres/RCFLAGS configure machinery, and the altbininstall
python3w.exe lines.
This PR's own CI then exposed the next wall (run 34741183330, leg
smoke 3.14.7 (windows-msys)): with the link shape fixed, every make
target died at frozen-module regen — ModuleNotFoundError: No module
named 'winreg' / Fatal Python error: _PyImport_InitExternal: external
importer setup failed (Makefile:1957/1960, frozen_modules/abc.h,
codecs.h). Root cause: Lib/importlib/_bootstrap_external.py:36 imports
winreg at module top level whenever sys.platform == 'win32', and the
autoconf build never builds winreg (upstream compiles it MSVC-only via
PCbuild) — so EVERY mingw interpreter boot dies in import setup,
_bootstrap_python included. 0018 (winreg_modules_msys, verbatim +
regen) cures it: winreg/msvcrt/_winapi become bootstrap-static modules
(Setup.bootstrap.in + PY_STDLIB_MOD), winsound gains -lwinmm and
_overlapped -lws2_32 (Setup.stdlib.in), crtdbg.h is debug-gated in
_winapi.c/msvcrtmodule.c, and dynamic_annotations.c's _MSC_VER guard
becomes MS_WINDOWS.
The second CI leg then moved one step deeper (run 34742370411, job
103684133612): with winreg in place the interpreter boots past
_PyImport_InitExternal and dies at Lib/site.py:330 — _get_path's win32
branch (sys.platform == 'win32' via configure_machdep_msys) reads
sys.winver -> AttributeError: module 'sys' has no attribute 'winver'
(Makefile:1964/1967, the same frozen-headers regen). Upstream
sysmodule.c installs winver/dllhandle only under MS_COREDLL, an
MSVC-project-only macro (PC/pyconfig.h) the autotools build never
defines — while the definitions it guards DO exist in this link (0012
compiles PC/dl_nt.o, 0015 defines MS_DLL_ID). 0034
(sysmodule_coredll_msys — the transformation verbatim, the hunks
re-authored against 3.14.7: the upstream hunk-3 context rides MSYS2's
own 0003 sys._is_mingw_ucrt patch, which this series does not carry)
makes the guard defined(MS_WINDOWS) && defined(Py_ENABLE_SHARED); in
the static shape the guard stays false (static behavior unchanged),
POSIX untouched, no configure.ac change so no regen.
Deliberately skipped from the cluster's neighborhood (recorded in the
PR body): 0067 (PYTHONLEGACYWINDOWSDLLLOADING — uninstalled-tree test
convenience; its mingw_smoketests.py hunk targets an MSYS2-CI-local
file), 0068 (GetPythonImport libpython-name port — without it the
wrong-python-version guard is inert, not broken: the .pyd imports
libpython3.14.dll, which never matches the MSVC "python" prefix),
0069 (backslash normalization — its precondition is MSYS2-fork
forward-slash path handling this series does not carry), 0070
(suppress the python3.dll preload under mingw — with PY3_DLLNAME=
libpython3.14.dll the preload resolves to the already-loaded core DLL,
a harmless no-op; its hunk context also rides on the skipped 0067).
tools/compile_smoke follows the factory's new linkage shape (the
coverage contract — the smoke IS the factory's shape): the windows-msys
leg configures --enable-shared and gains the one LINK regression
target — Modules/array$(EXT_SUFFIX), the exact run-34735362057
casualty, never probed, EXT_SUFFIX read off the generated Makefile (it
now resolves to the .pyd spelling). The POSIX leg is untouched
(pristine tree, upstream's static default, 12 wall TUs).
Verified locally (macOS + homebrew mingw-w64 gcc 14; the native ucrt64
build is CI's oracle): the 35-patch series applies clean to pristine
3.14.7 (tools/lint) — the 34-patch prefix reproduces the reference
tree byte-for-byte, and 0034's only target (Python/sysmodule.c) is
touched by no other entry; shipped configure/pyconfig.h.in equal the
pinned autoconf 2.72 regen exactly; rspec 103/0; validate_manifests
OK; the POSIX smoke leg compiles all 12 wall TUs on the pristine tree.
Cross proof: configure yields the MSYS2 spellings (LDLIBRARY=
libpython3.14.dll.a, DLLLIBRARY=libpython3.14.dll, DYNLOADFILE=
dynload_win.o, MACHDEP_OBJS=PC/dl_nt.o, SHLIB_SUFFIX=.pyd,
EXT_SUFFIX=.cp314-mingw_x86_64_ucrt_gnu.pyd, PY_ENABLE_SHARED=1); the
build links libpython3.14.dll (38 MB), python.exe, pythonw.exe, and
Modules/array.cp314-mingw_x86_64_ucrt_gnu.pyd — the
run-34735362057 casualty under its new name; the DLL export table
carries PyExc_EOFError/_Py_Dealloc/PyUnicode_FromString (the wall's
undefined symbols) among 3678 exports; the .pyd imports
libpython3.14.dll; the DLL's own imports add libgcc_s_seh-1.dll and
libwinpthread-1.dll — the two mingw runtime DLLs the factory's windows
layout must stage beside the exe. The 0018 shape: winreg/msvcrt/
_winapi compile into the core archive (PC/winreg.o,
PC/msvcrtmodule.o, Modules/_winapi.o), the generated Modules/config.c
registers all three in _PyImport_Inittab, and PyInit_winreg exports
from the DLL. The 0034 shape: Python/sysmodule.o compiles for the
target and the python.exe link still closes. Both boot-time fixes are
provable only at run time — the CI smoke leg is the oracle.1 parent 0fb8cd8 commit 575619f
30 files changed
Lines changed: 1901 additions & 185 deletions
File tree
- patches/3.14
- tools
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
| 27 | + | |
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
| |||
33 | 33 | | |
34 | 34 | | |
35 | 35 | | |
36 | | - | |
| 36 | + | |
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
50 | 50 | | |
51 | 51 | | |
52 | | - | |
| 52 | + | |
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
| 21 | + | |
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
| 46 | + | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| |||
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
44 | | - | |
| 44 | + | |
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | | - | |
| 16 | + | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
| 34 | + | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | | - | |
| 64 | + | |
65 | 65 | | |
66 | 66 | | |
67 | 67 | | |
| |||
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
79 | | - | |
| 79 | + | |
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
| |||
91 | 91 | | |
92 | 92 | | |
93 | 93 | | |
94 | | - | |
| 94 | + | |
95 | 95 | | |
96 | 96 | | |
97 | 97 | | |
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | | - | |
| 109 | + | |
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
| |||
121 | 121 | | |
122 | 122 | | |
123 | 123 | | |
124 | | - | |
| 124 | + | |
125 | 125 | | |
126 | 126 | | |
127 | 127 | | |
| |||
136 | 136 | | |
137 | 137 | | |
138 | 138 | | |
139 | | - | |
| 139 | + | |
140 | 140 | | |
141 | 141 | | |
142 | 142 | | |
| |||
151 | 151 | | |
152 | 152 | | |
153 | 153 | | |
154 | | - | |
| 154 | + | |
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| |||
166 | 166 | | |
167 | 167 | | |
168 | 168 | | |
169 | | - | |
| 169 | + | |
170 | 170 | | |
171 | 171 | | |
172 | 172 | | |
| |||
181 | 181 | | |
182 | 182 | | |
183 | 183 | | |
184 | | - | |
| 184 | + | |
185 | 185 | | |
186 | 186 | | |
187 | 187 | | |
| |||
196 | 196 | | |
197 | 197 | | |
198 | 198 | | |
199 | | - | |
| 199 | + | |
200 | 200 | | |
201 | 201 | | |
202 | 202 | | |
| |||
212 | 212 | | |
213 | 213 | | |
214 | 214 | | |
215 | | - | |
| 215 | + | |
216 | 216 | | |
217 | 217 | | |
218 | | - | |
| 218 | + | |
219 | 219 | | |
220 | 220 | | |
221 | 221 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
40 | 36 | | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | | - | |
| 15 | + | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
26 | 33 | | |
27 | | - | |
| 34 | + | |
28 | 35 | | |
29 | 36 | | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
0 commit comments