Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/matrix.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
{ "host": "ubuntu-22.04", "container": "ghcr.io/tamatebako/tpkg-builder-x86_64-linux-musl", "os": "linux-musl", "arch": "x86_64" },
{ "host": "ubuntu-22.04-arm", "container": "ghcr.io/tamatebako/tpkg-builder-aarch64-linux-musl","os": "linux-musl", "arch": "arm64" },
{ "host": "macos-15-intel", "container": null, "os": "macos", "arch": "x86_64" },
{ "host": "macos-14", "container": null, "os": "macos", "arch": "arm64" }
{ "host": "macos-14", "container": null, "os": "macos", "arch": "arm64" },
{ "host": "windows-2022", "container": null, "os": "windows", "arch": "x86_64" }
]
}
11 changes: 8 additions & 3 deletions .github/workflows/_build-platform.yml
Original file line number Diff line number Diff line change
Expand Up @@ -274,14 +274,19 @@ jobs:
ci/check_symbol_provenance.sh --exe \
"runtime-packages/tebako-runtime-${{ needs.compute.outputs.tebako_version }}-${{ matrix.python }}-${{ matrix.host_id }}"

# The exe name carries .exe on this leg (Platform#exe_suffix) and it
# must be spelled out: msys stat() transparency lets foo.exe satisfy
# `[ -s foo ]`, but the native nm/objdump open the argument verbatim
# — an extensionless spelling is ENOENT to them and reads back as a
# stripped exe (run 34758424181's false wall). The ruby factory
# spells ruby.exe the same way.
- name: Check symbol provenance (windows)
if: matrix.os == 'windows'
shell: msys2 {0}
run: |
set -euo pipefail
exe="runtime-packages/tebako-runtime-${{ needs.compute.outputs.tebako_version }}-${{ matrix.python }}-${{ matrix.host_id }}"
[ -s "$exe" ] || exe="$exe.exe"
ci/check_symbol_provenance.sh --exe "$exe"
ci/check_symbol_provenance.sh --exe \
"runtime-packages/tebako-runtime-${{ needs.compute.outputs.tebako_version }}-${{ matrix.python }}-${{ matrix.host_id }}.exe"

# The boot smoke self-selects the scenario set per platform (the
# POSIX set vs the windows bare + named-error trio) and resolves the
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Build artifacts
.build/
.build-inspect/
runtime-packages/

# Ruby tooling (factory convention: the lockfile floats)
Expand Down
12 changes: 8 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,14 @@ BOOT_LIVE gate + raw early-boot passthrough
the gnu legs), and the `fcntl64` export (tamatebako/tebako#529).
The POSIX boot-smoke CI legs were red-by-design on the old v2.1.5
pin and are green on the v2.1.10+ line. The windows row
is descoped out of the 02 matrix (TODO.python/05 — CPython upstream
has zero mingw support; the port is a tamatebako/python msys2/ucrt64
patch
series), and `tools/boot_smoke` hard-kills a wedged child against a
(63e8336's descope) returns with TODO.python/05 item 2: CPython
upstream has zero mingw support, so the windows leg builds from the
line's msys2/ucrt64 scenario tree
(`tfs-python-<base>-src-windows-msys.tar.gz`, tamatebako/python's
patches/<line>/ series) — `scripts/compute_matrix.rb` skips a windows
leg loudly when the pinned `source_release` ships no such asset for
the line (and skips jit lines on windows outright: upstream's JIT
whitelist admits MSVC targets only), and `tools/boot_smoke` hard-kills a wedged child against a
CLOCK_MONOTONIC deadline instead of wedging with it. The
linked-driver decision and the preload-shim re-exec rationale are
recorded in README.md — do not reopen them here; the wrapper pattern
Expand Down
29 changes: 18 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,11 @@ boundary below).

The env image ships the stdlib plus a **declarative site-packages
whitelist** (`build/site-packages.yml`; v1 keeps `pip` only, with its
dist-info — pip's `importlib.metadata` self-check reads it). Everything
else ensurepip installed is pruned at image assembly.
dist-info — pip's `importlib.metadata` self-check reads it). pip reaches
the staged site-packages via `make install`'s ensurepip on POSIX and via
an explicit bundled-wheel placement on msys (the install's `--root`
rebase drive-strips the build-tree prefix — `PythonBuild#place_pip`).
Anything else the install staged is pruned at image assembly.

**The pip form is `python3 -m pip`.** The image's `bin/` directory is
pruned wholesale: the ensurepip console scripts' shebangs spell the
Expand All @@ -125,15 +128,19 @@ interpreter by convention.

## The windows boundary

The v1 windows leg (ucrt64, `--disable-shared`) is the **driver-contract
surface only**: the exe boots the driver, answers `--tebako-image`/
`TEBAKO_RUNTIME_IMAGE` with the same named errors, and runs the
interpreter only when nothing was mounted (bare/dev mode). There is no
preload tier on windows — with any mount the fs TU exits **69** with a
named error (roadmap 30 phase 2). The `--disable-shared` choice means no
libpython DLL facet ships; `scripts/upload_release.rb` already models
the dll facet opportunistically if a future `--enable-shared` leg
appears (the ruby factory's issue-40 analog).
The windows leg (ucrt64, `--enable-shared` — issue 40's answer: on PE a
loadable module cannot carry undefined symbols, so the extensions link
`libpython<X.Y>.dll`; the ruby factory ships the same shape) is the
**driver-contract surface only**: the exe boots the driver, answers
`--tebako-image`/`TEBAKO_RUNTIME_IMAGE` with the same named errors, and
runs the interpreter only when nothing was mounted (bare/dev mode).
There is no preload tier on windows — with any mount the fs TU exits
**69** with a named error (roadmap 30 phase 2). The shared build ships
the DLL as a `<package>.dll` release facet (the release manifest's
`dll.install_as` names the PE spelling the store materializes beside
the exe), and the mingw support set (libgcc/libwinpthread) is
statically linked into the runtime's own PE modules — a bare machine
installs nothing.

## The artifacts (per version × triplet)

Expand Down
38 changes: 33 additions & 5 deletions build/lib/tebako_python_builder/builder.rb
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,13 @@ def initialize(repo_root:, python_version:, tebako_version:, prefix:, output:, #
end

def run # rubocop:disable Metrics/MethodLength
# A flavored line (x.y.z-jit) consumes the SAME pristine source
# tarball as its base — the flavor is a configure-time ability of
# the line, never a second source artifact (the source factory's
# SHA256SUMS names base versions only).
(tarball, sha256) = fetcher.fetch(@python.base_version)
# A flavored line (x.y.z-jit) consumes the SAME source tarball as its
# base — the flavor is a configure-time ability of the line, never a
# second source artifact (the source factory's SHA256SUMS names base
# versions only). The platform selects the scenario: a mingw/ucrt
# host builds from the line's patched windows-msys tree, every POSIX
# host from the unsuffixed pristine one.
(tarball, sha256) = fetcher.fetch(@python.base_version, platform: @platform)
puts "-- Building tebako runtime for python #{@python_version} " \
"(tebako #{@tebako_version}, #{@platform.host_id}, #{File.basename(tarball)})"
link_unit_dir = link_unit.stage(File.join(@prefix, "link-unit"))
Expand Down Expand Up @@ -129,6 +131,32 @@ def finalize(built_exe)
FileUtils.mkdir_p(File.dirname(output))
FileUtils.cp(built_exe, output)
FileUtils.chmod(0o755, output)
stage_python_dll(built_exe) if @platform.msys?
end

# msys only (the --enable-shared shape): stage the just-linked
# libpython<X.Y>.dll next to the runtime executable under the
# PACKAGE's name (<runtime>.dll — unique per leg: two same-ABI legs
# share the PE name and would collide in the merged release
# workspace; the manifest's dll.install_as flows the PE name to the
# store entry, and tools/boot_smoke materializes it in-leg). The DLL
# links next to the exe in the build tree (the Makefile's
# $(DLLLIBRARY) rule); its absence means the build regressed to a
# static shape — a named error, never a silent skip. The name's
# single owner is PythonVersion#msys_dll_name (invariant 10).
def stage_python_dll(built_exe)
dll = File.join(File.dirname(built_exe), @python.msys_dll_name)
unless File.file?(dll)
raise TebakoPythonBuilder::Error.new(
"expected the shared build's #{@python.msys_dll_name} next to #{built_exe} " \
"(the windows-msys leg must configure --enable-shared — issue 40)", 130
)
end

dest = "#{output.sub(/\.exe\z/, "")}.dll"
FileUtils.cp(dll, dest)
FileUtils.chmod(0o755, dest)
puts "-- Runtime DLL: #{dest} (installs as #{@python.msys_dll_name})"
end

def assemble_and_pack_image(build, sha256, link_unit, link_unit_dir, tfs)
Expand Down
10 changes: 9 additions & 1 deletion build/lib/tebako_python_builder/image_builder.rb
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,13 @@ class ImageBuilder
# effective root, so the interpreter follows TEBAKO_MOUNT_ROOT (the
# factory owns both sides of the grant; the boot smoke asserts the
# chain end-to-end). The additive preload_shim grant (schema_minor 2)
# names exactly the staged shim's in-image path.
# names exactly the staged shim's in-image path. The additive
# runtime_dll basename (schema_minor 3) is emitted for MSYS builds
# only — flowed from PythonVersion#msys_dll_name, the name's single
# owner (invariant 10); the driver exports it as TEBAKO_RUNTIME_DLL
# into the handoff env, and the tfs PE closure walk excludes a bare
# import name matching it (spec 22 §2.1 — the OS's basename-reuse
# rule binds the already-loaded copy). POSIX builds omit the key.
LAYOUT_DECLARATION = {
"schema" => "layout",
"schema_version" => 1,
Expand Down Expand Up @@ -192,6 +198,8 @@ def deploy_layout(tree)
)
unless @platform.msys?
declaration["preload_shim"] = File.join("lib", "tebako", @link_unit.preload_shim_name)
else
declaration["runtime_dll"] = @python.msys_dll_name
end
File.write(path, YAML.dump(declaration))
puts " ... env image layout declaration: #{path}"
Expand Down
Loading
Loading