Skip to content

selftests/bpf: restructure the Makefile as a layered build - #13415

Closed
kernel-patches-daemon-bpf[bot] wants to merge 9 commits into
bpf-next_basefrom
series/1150503=>bpf-next
Closed

selftests/bpf: restructure the Makefile as a layered build#13415
kernel-patches-daemon-bpf[bot] wants to merge 9 commits into
bpf-next_basefrom
series/1150503=>bpf-next

Conversation

@kernel-patches-daemon-bpf

Copy link
Copy Markdown

Pull request for series with
subject: selftests/bpf: restructure the Makefile as a layered build
version: 7
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

Upstream branch: 5e289c5
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503
version: 7

@kernel-patches-review-bot

Copy link
Copy Markdown
commit 96dd7c3765aa55b423ff7444415b95a550824e7c
Author: Mykola Lysenko <nickolay.lysenko@gmail.com>

selftests/bpf: drop stale lines, restore two header dependencies

This patch removes dead build rules and dependency entries that no longer
serve any purpose, and restores two header dependencies in working form.

> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 118a7c356e26c..c7c7f27dd8605 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -4,9 +4,7 @@ include ../../../scripts/Makefile.arch
>  include ../../../scripts/Makefile.include
>
>  CXX ?= $(CROSS_COMPILE)g++
> -OBJCOPY ?= $(CROSS_COMPILE)objcopy
>
> -CURDIR := $(abspath .)
>  TOOLSDIR := $(abspath ../../..)

This isn't a bug, but would it be worth splitting the OBJCOPY and CURDIR
variable removals into their own small cleanup patch? The subject and the
bulk of the changelog focus on stale build rules and the two restored
header dependencies, while these variable removals are unused/redundant
definitions that get their own paragraph at the end.

[ ... ]



AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: selftests/bpf: drop stale lines, restore two header dependencies
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32662817569

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

Upstream branch: 5e289c5
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503
version: 7

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

@kernel-patches-review-bot

Copy link
Copy Markdown
commit 79d6c61a930a7bcff9e424734672a09c86fbdcf2
Author: Mykola Lysenko <nickolay.lysenko@gmail.com>

selftests/bpf: move shared build definitions into Makefile.buildvars

This commit moves shared build definitions (tree layout, tool locations,
flag assembly, feature probes, signing key paths, and permissive-mode
helpers) from the main Makefile into a new Makefile.buildvars file so
they can be included by multiple makefiles.

> commit 79d6c61a930a7bcff9e424734672a09c86fbdcf2
> Author: Mykola Lysenko <nickolay.lysenko@gmail.com>
> Date:   Sun Aug 23 12:42:40 2026 -0700
>
>     selftests/bpf: move shared build definitions into Makefile.buildvars
>
>     Move the build definitions a test runner needs - tree layout and
>     tool locations, flag assembly, feature probes, signing key paths, the
>     permissive-mode helpers - into Makefile.buildvars, in their current
>     order, as preparation for building each test runner instance in its
>     own sub-make: the definitions become includable by more than one
>     makefile. No rules or recipes are changed.
>
>     The include sits directly after ../lib.mk, ahead of the
>     ../../../build/Makefile.feature evaluation, so the definitions are in
>     place for everything the top Makefile parses afterwards - in particular
>     the LLVM feature probe captures srctree and PKG_CONFIG from their new
>     location, and the top Makefile's remaining uses of both are all
>     recipe-time expansions. Two evaluation-time changes follow from
>     the new position and a third is made on the way, each commented in
>     place: CFLAGS and LDFLAGS are
>     reassembled around a snapshot taken before lib.mk, keeping the
>     pre-split order;
>     CLANG_SYS_INCLUDES becomes an immediate assignment (its inputs are
>     final by this point, and expanding it forks shells); and the
>     CLANG_HAS_ARENA_ASAN probe now sees CLANG as resolved from
>     LLVM=<suffix-or-path> rather than the bare 'clang' default it saw
>     above the include, while the CLANG_CPUV4 probe, needed before lib.mk,
>     keeps the old behavior.
>
>     BPF_GCC, TEST_KMODS and the knobs the runner never reads (SKIP_*,
>     submake_extras, TEST_KMOD_TARGETS, the VMLINUX_BTF block) stay in the
>     Makefile - the first two are needed before lib.mk is included, where
>     Makefile.buildvars cannot yet be, and reach the runners from there once
>     a later patch adds them: TEST_KMODS exported in the environment, BPF_GCC
>     as the bpf_gcc flavor's compiler parameter. Definitions that reach no
>     runner, directly or as a sub-make parameter, also stay - except
>     HOST_INCLUDE_DIR, which moves with the conditional block it shares
>     with the host build directories - and the four that derive from
>     Makefile.buildvars values sit in a block right after the include.
>
>     Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
>     Assisted-by: Claude:claude-fable-5 shellcheck
>     Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>

This isn't a bug, but the second paragraph has some lines that look
unfinished mid-paragraph ("place: CFLAGS and LDFLAGS are" at 29
columns, then "pre-split order;" at 16 columns), as if the text was
edited but not rewrapped. Would it help to reflow those lines?

The third paragraph walks through which variables moved and which
stayed ("BPF_GCC, TEST_KMODS and the knobs the runner never reads...
also stay - except HOST_INCLUDE_DIR..."), restating what the diff shows.
The key detail - that BPF_GCC and TEST_KMODS must be defined before
../lib.mk - is there, but it's somewhat buried in the inventory. Would
trimming the list and keeping just the reason make this clearer?

> diff --git a/tools/testing/selftests/bpf/Makefile.buildvars b/tools/testing/selftests/bpf/Makefile.buildvars
> new file mode 100644
> index 0000000000000..dc0be9ee72bc0
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/Makefile.buildvars
> @@ -0,0 +1,184 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Shared toolchain, path and flag definitions for the BPF selftests
> +# build, included by both Makefile and Makefile.runner. Definitions
> +# follow the order they had in the Makefile before the split.
> +
> +TOOLSDIR := $(abspath ../../..)

[ ... ]

> +# Before the split these definitions sat above the ../lib.mk include:
> +# they appended to the flags inherited from the environment, and
> +# lib.mk's additions (USERCFLAGS/USERLDFLAGS among them) came last.
> +# This file is parsed after lib.mk at the top level and in its place
> +# in the runner sub-makes, so the same order is rebuilt here. The
> +# includer snapshots the inherited flags in INHERITED_CFLAGS and

This isn't a bug, but the header and several comments are written from
the patch's point of view rather than as documentation of the current
state. The header says definitions "follow the order they had in the
Makefile before the split", and this block starts with "Before the
split these definitions sat above the ../lib.mk include". A reader
arriving later has no "before" to compare against.

The header also says this file is "included by both Makefile and
Makefile.runner", but Makefile.runner doesn't exist until the next
commit. If someone lands here during a bisect, that sentence is
inaccurate.

Would these comments read better stated as the current invariant ("lib.mk
appends to CFLAGS/LDFLAGS; this file's additions must come first, so
the includer snapshots them in INHERITED_*"), and should the header
mention Makefile.runner only once it exists?



AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: selftests/bpf: move shared build definitions into Makefile.buildvars
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32662817569

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

@kernel-patches-review-bot

Copy link
Copy Markdown
commit 2c5b93f3e52b47182259f3354c543be38f34cd63
Author: Mykola Lysenko <nickolay.lysenko@gmail.com>

selftests/bpf: build each test runner instance in its own sub-make

Replace the DEFINE_TEST_RUNNER/DEFINE_TEST_RUNNER_RULES double-expansion
machinery - whose rules never appear in the source as written, are
invisible to make's own debugging facilities, and are easy to break
for one flavor while testing another - with Makefile.runner: one
sub-make invocation per test runner
instance (test_progs, test_progs-no_alu32, test_progs-cpuv4,
test_progs-bpf_gcc, test_maps), each in its own single-flavor namespace
written in plain make - no define/eval layer, no per-flavor guards,
no accumulating vpath directives.

> Replace the DEFINE_TEST_RUNNER/DEFINE_TEST_RUNNER_RULES double-expansion
> machinery - whose rules never appear in the source as written, are
> invisible to make's own debugging facilities, and are easy to break
> for one flavor while testing another - with Makefile.runner: one
> sub-make invocation per test runner
> instance (test_progs, test_progs-no_alu32, test_progs-cpuv4,
> test_progs-bpf_gcc, test_maps), each in its own single-flavor namespace
> written in plain make - no define/eval layer, no per-flavor guards,
> no accumulating vpath directives.
> The main Makefile keeps everything that exists once - tool sub-builds,
> vmlinux.h, signing keys, tests.h generation, standalone binaries and
> the kselftest lib.mk contract - and delegates to the runner through
> explicit per-instance rules.
> The rules for one flavor's BPF objects and skeletons live in
> Makefile.skel, instantiated by the main Makefile for the default
> flavor and by each runner sub-make that builds BPF objects. lib.mk's
> install rule copies the default flavor's BPF objects - the files the
> flat copy of every flavor's objects resolved to before. No file is
> built twice in one build: the main Makefile produces the default
> flavor's artifacts before the unflavored test_progs sub-make runs,
> and each flavored runner writes its own subdirectory plus its binary
> in $(OUTPUT) (the bpftool
> link in $(OUTPUT) is refreshed by both unflavored sub-makes, as
> before).
> The runner instances link userspace objects the main Makefile
> builds once; the
> flavored instances link the shared objects instead of compiling
> their own identical copies.

This isn't a bug, but the wrapping looks unusual here. Several lines
break after only a few words in the middle of a sentence (e.g., "with
Makefile.runner: one / sub-make invocation per test runner / instance"
and "and each flavored runner writes its own subdirectory plus its
binary / in $(OUTPUT) (the bpftool / link in $(OUTPUT) is refreshed").

Could the changelog be reflowed to a consistent width? Kernel changelogs
are conventionally wrapped at around 72-75 columns with blank lines
between paragraphs.

> Smaller behaviour changes that come with the move, for the record:
> the skeleton recipes' build-log lines take the msg helper's format
> and stream, the shared userspace objects log as CC rather than
> EXT-OBJ/LIB-OBJ, and TEST-HDR lines lose their runner tag; the signed
> light-skeleton rule lists the key and certificate it uses as
> prerequisites; the extras copy skips an empty file list and, in
> permissive mode, does not wait for files whose build was skipped;
> $(OUTPUT)/test_maps waits for the same shared prerequisites as the
> other runners; the bare-name convenience targets for the
> linked-skeleton constituents (make linked_funcs1.bpf.o) are gone; the
> .d include no longer special-cases the clean, docs-clean and
> emit_tests goals,
> which a runner sub-make never runs; and the test objects' first-build
> ordering lists only the generated headers the runner itself consumes,
> the rest being built by the main Makefile before any runner starts.

This isn't a bug, but the behaviour changes paragraph is written as one
sentence with roughly ten semicolon-separated clauses spanning about 20
lines. At least two of these are user-visible interface changes (the
bare-name convenience targets like "make linked_funcs1.bpf.o" are gone,
and the .d include no longer special-cases clean/docs-clean/emit_tests)
rather than cosmetic log changes.

Could this be turned into a bulleted list with the interface-affecting
changes called out separately?

> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile

[ ... ]

> @@ -672,76 +416,109 @@
>  	+$(MAKE) -C libarena libarena_asan.skel.h $(LIBARENA_MAKE_ARGS)
>  endif
>
> -# Define test_progs test runner.
> -TRUNNER_TESTS_DIR := prog_tests
> -TRUNNER_BPF_PROGS_DIR := progs
> -TRUNNER_EXTRA_SOURCES := test_progs.c		\
> -			 cgroup_helpers.c	\
> -			 trace_helpers.c	\
> -			 network_helpers.c	\
> -			 testing_helpers.c	\
> -			 btf_helpers.c		\
> -			 cap_helpers.c		\
> -			 unpriv_helpers.c 	\
> -			 sysctl_helpers.c	\
> -			 netlink_helpers.c	\
> -			 jit_disasm_helpers.c	\
> -			 io_helpers.c		\
> -			 test_loader.c		\
> -			 xsk.c			\
> -			 disasm.c		\
> -			 disasm_helpers.c	\
> -			 json_writer.c 		\
> -			 $(VERIFY_SIG_HDR)		\
> -			 flow_dissector_load.h	\
> -			 ip_check_defrag_frags.h	\
> -			 bpftool_helpers.c	\
> -			 usdt_1.c usdt_2.c	\
> -			 $(LIBARENA_SKEL)	\
> -			 $(LIBARENA_ASAN_SKEL)
> -TRUNNER_LIB_SOURCES := find_bit.c
> -TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read				\
> -		       $(OUTPUT)/liburandom_read.so			\
> -		       $(OUTPUT)/xdp_synproxy				\
> -		       $(OUTPUT)/sign-file				\
> -		       $(OUTPUT)/uprobe_multi				\
> -		       $(TEST_KMOD_TARGETS)				\
> -		       ima_setup.sh 					\
> -		       $(VERIFY_SIG_SETUP)				\
> -		       $(wildcard progs/btf_dump_test_case_*.c)		\
> -		       $(wildcard progs/*.bpf.o)
> -TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE
> -TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS) -DENABLE_ATOMICS_TESTS
> -$(eval $(call DEFINE_TEST_RUNNER,test_progs))
> +# Generated test list headers
> +
> +define gen_tests_hdr
> +	$(call msg,TEST-HDR,,$@)
> +	$(Q)(echo '/* Generated header, do not edit */';		\
> +		  sed -n -E 's/^void (serial_)?test_([a-zA-Z0-9_]+)\((void)?\).*/DEFINE_TEST(\2)/p'	\
> +		$(@D)/*.c | sort) > $@
> +endef
> +
> +prog_tests/tests.h: $(wildcard prog_tests/*.c)
> +	$(gen_tests_hdr)
> +
> +map_tests/tests.h: $(wildcard map_tests/*.c)
> +	$(gen_tests_hdr)
> +
> +# Test runner instances, one sub-make each (see Makefile.runner).
>
> -# Define test_progs-no_alu32 test runner.
> -TRUNNER_BPF_BUILD_RULE := CLANG_NOALU32_BPF_BUILD_RULE
> -TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS)
> -$(eval $(call DEFINE_TEST_RUNNER,test_progs,no_alu32))
> +# The LLVM feature-probe results and TEST_KMODS are exported
> +# to the runner sub-makes. CC is passed explicitly instead: exporting it
> +# would also leak lib.mk's CC into the libbpf sub-build, which computes
> +# its own. ('export NAME' on an undefined
> +# variable creates an empty one, so these stay below the definitions.)
> +export LLVM_LDLIBS LLVM_LDFLAGS TEST_KMODS
> +export INHERITED_CFLAGS INHERITED_LDFLAGS
>
> -# Define test_progs-cpuv4 test runner.
> +RUNNER_MAKE := $(MAKE) -f Makefile.runner OUTPUT=$(OUTPUT) CC='$(CC)'	\
> +	       CLANG='$(CLANG)'
> +
> +# Everything a runner instance references but does not know how to build.
> +RUNNER_PREREQS := $(INCLUDE_DIR)/vmlinux.h $(BPFOBJ) $(BPFTOOL)		\
> +		  $(TRUNNER_BPFTOOL) $(RESOLVE_BTFIDS)			\
> +		  $(OUTPUT)/veristat					\
> +		  $(VERIFY_SIG_HDR) $(PRIVATE_KEY) $(VERIFICATION_CERT)	\
> +		  $(LIBARENA_SKEL) $(LIBARENA_ASAN_SKEL)		\
> +		  prog_tests/tests.h map_tests/tests.h			\
> +		  $(RUNNER_OBJS)					\
> +		  $(OUTPUT)/urandom_read $(OUTPUT)/liburandom_read.so	\
> +		  $(OUTPUT)/xdp_synproxy $(OUTPUT)/sign-file		\
> +		  $(OUTPUT)/uprobe_multi $(TEST_KMOD_TARGETS)
> +
> +# The main Makefile cannot tell whether $(OUTPUT)/test_progs is stale -
> +# only the runner sub-make knows its full dependency graph. FORCE makes
> +# the delegating rules below always run; their dependents still rebuild
> +# on mtime only.
> +FORCE:
> +
> +# The default flavor's BPF objects and skeletons are consumed here as
> +# well as inside the runner sub-makes: bench, xskxceiver, xdp_* and
> +# test_cpp depend on individual skeletons, and lib.mk's
> +# install rule copies the BPF objects. Instantiate the shared rules
> +# (Makefile.skel) for the default flavor, so those consumers depend on
> +# exactly the files they use; the unflavored runners' delegation rules
> +# below list the whole set as prerequisites, so those sub-makes find the
> +# files this Makefile owns up to date.
> +RDIR := $(OUTPUT)
> +FLAVOR :=
> +BINARY := test_progs
> +BPF_CC := $(CLANG)
> +BPF_CC_MSG := CLNG-BPF
> +BPF_SYS_INCLUDES := $(CLANG_SYS_INCLUDES)
> +BPF_CC_FLAGS := -O2 $(BPF_TARGET_ENDIAN) -mcpu=v3
> +BPF_DEFINES := -DENABLE_ATOMICS_TESTS
> +include Makefile.skel
> +
> +DEFAULT_RUNNER_ARGS := RUNNER=test_progs FLAVOR= TESTS_DIR=prog_tests	\
> +	BPF_CC='$(BPF_CC)' BPF_CC_MSG=$(BPF_CC_MSG)			\
> +	BPF_SYS_INCLUDES='$(BPF_SYS_INCLUDES)'				\
> +	BPF_CC_FLAGS='$(BPF_CC_FLAGS)' BPF_DEFINES=$(BPF_DEFINES)
> +
> +$(OUTPUT)/test_progs: $(RUNNER_PREREQS) $(BPF_OBJS) $(ALL_SKELS) FORCE
> +	+$(Q)$(RUNNER_MAKE) $(DEFAULT_RUNNER_ARGS)
> +
> +$(OUTPUT)/test_progs-no_alu32: $(RUNNER_PREREQS) FORCE
> +	+$(Q)$(RUNNER_MAKE) RUNNER=test_progs FLAVOR=no_alu32 TESTS_DIR=prog_tests \
> +		BPF_CC='$(CLANG)' BPF_CC_MSG=CLNG-BPF BPF_SYS_INCLUDES='$(CLANG_SYS_INCLUDES)'	\
> +		BPF_CC_FLAGS='-O2 $(BPF_TARGET_ENDIAN) -mcpu=v2'
> +
>  ifneq ($(CLANG_CPUV4),)
> -TRUNNER_BPF_BUILD_RULE := CLANG_CPUV4_BPF_BUILD_RULE
> -TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS) -DENABLE_ATOMICS_TESTS
> -$(eval $(call DEFINE_TEST_RUNNER,test_progs,cpuv4))
> +$(OUTPUT)/test_progs-cpuv4: $(RUNNER_PREREQS) FORCE
> +	+$(Q)$(RUNNER_MAKE) RUNNER=test_progs FLAVOR=cpuv4 TESTS_DIR=prog_tests \
> +		BPF_CC='$(CLANG)' BPF_CC_MSG=CLNG-BPF BPF_SYS_INCLUDES='$(CLANG_SYS_INCLUDES)'	\
> +		BPF_CC_FLAGS='-O2 $(BPF_TARGET_ENDIAN) -mcpu=v4'		\
> +		BPF_DEFINES=-DENABLE_ATOMICS_TESTS
>  endif

This isn't a bug, but since DEFAULT_RUNNER_ARGS already factors the
clang argument list, would a shared CLANG_RUNNER_ARGS (with just -mcpu=
and BPF_DEFINES varying per flavor) read better than repeating
BPF_CC/BPF_CC_MSG/BPF_SYS_INCLUDES in each of the three clang flavor
recipes?

> diff --git a/tools/testing/selftests/bpf/Makefile.runner b/tools/testing/selftests/bpf/Makefile.runner
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/Makefile.runner
> @@ -0,0 +1,199 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Build one BPF test-runner instance: test_progs, one of its flavors
> +# (no_alu32, cpuv4, bpf_gcc), or test_maps.

[ ... ]

> +# Keep in sync with the CFLAGS/LDFLAGS additions in ../lib.mk, in its
> +# order (the selftests include path is spelled from srctree here; lib.mk
> +# spells it from its own directory).
> +ifneq ($(LLVM),)
> +CFLAGS += -Wno-address-of-packed-member
> +CFLAGS += -Wno-gnu-variable-sized-type-not-at-end
> +endif
> +CFLAGS += -D_GNU_SOURCE=
> +CFLAGS += -I$(srctree)/tools/testing/selftests
> +CFLAGS += $(USERCFLAGS)
> +LDFLAGS += $(USERLDFLAGS)

The runner rebuilds the CFLAGS that ../lib.mk used to add, but spells
the selftests include path from $(srctree) instead of lib.mk's
$(top_srcdir). These are not equivalent.

lib.mk derives its path from a realpath, so it is always absolute:

  tools/testing/selftests/lib.mk:61  top_srcdir = $(selfdir)/../../..
  tools/testing/selftests/lib.mk:203 CFLAGS += -I${top_srcdir}/tools/testing/selftests

$(srctree) is exported by the kernel top-level Makefile and is relative
for common cases: '.' for an in-tree build and '..' for a build in a
subdirectory of the source tree. Makefile.buildvars' fallback only fires
when srctree is empty, so it does not normalize a relative value.

The test-object recipe compiles with the working directory changed to
the output directory:

  $(Q)(cd $(@D) && $(CC) -I. $(CFLAGS) -MMD -MT $@ -c $(CURDIR)/$< $(LDLIBS) -o $(@F))

so a relative -I is resolved against $(RDIR), not against the source
tree, and the flag silently names a non-existent directory when the bpf
selftests are built through the kernel top level.

The only include in prog_tests/ or map_tests/ that needs that directory
is prog_tests/test_xsk.h's #include "../kselftest.h", and it is
satisfied earlier by COMMON_CFLAGS' -I$(CURDIR), so this does not break
today. Using an absolute spelling here would keep the runner's
compilation environment identical to the top level's in every
configuration. Was the relative-srctree case considered?



AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: selftests/bpf: build each test runner instance in its own sub-make
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32662817569

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

Upstream branch: 5e289c5
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503
version: 7

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

Upstream branch: 5e289c5
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503
version: 7

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

Upstream branch: d83fba2
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503
version: 7

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

Upstream branch: ce36e38
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503
version: 7

@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

Upstream branch: 05ea1b6
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503
version: 7

prozak added 5 commits August 25, 2026 12:46
The generic '$(OUTPUT)/%:%.c' rule links with '$(LINK.c) $^', so
every prerequisite of such a binary lands on the compiler driver's
command line. That only works while none of them is a header: gcc
tolerates a stray .h argument on a link line, but clang treats it as
a request to precompile the header and fails with "cannot specify -o
when generating multiple output files".

Filter headers out of the recipe, with a comment: header
prerequisites on these targets are useful for dependency tracking -
the next patch declares one - and the constraint is easy to
rediscover the hard way.

Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Four target-specific lines name objects nothing builds. Three name
the BPF objects by their pre-rename names: commit afef88e
("selftests/bpf: Store BPF object files with .bpf.o extension") left
them without a target; the flow_dissector_load.o dependency names an
intermediate the generic one-step compile+link rule does not produce.

Three linked-skeleton dependency map entries (xsk_xdp_progs,
xdp_hw_metadata, xdp_features) were dead on arrival: commit
f0a249d ("selftests/xsk: get rid of built-in XDP program"),
commit 297a3f1 ("selftests/bpf: Simple program to dump XDP RX
metadata") and commit 4dba3e7 ("selftests/bpf: introduce XDP
compliance test tool") each added one, mimicking the neighboring
entries without adding the skeleton to LINKED_SKELS, the only place
the -deps map is read from; these three skeletons are generated by
the regular pattern rule, which never consults it.

The '-fno-inline' pair (test_l4lb_noinline, test_xdp_noinline) had
stopped taking effect even earlier, with commit 74b5a59
("selftests/bpf: Replace test_progs and test_maps w/ general rule"):
since then the BPF compile recipe expands TRUNNER_BPF_CFLAGS, a
simply-expanded copy of BPF_CFLAGS taken when the runner rules are
instantiated, which a target-specific 'BPF_CFLAGS +=' cannot reach.
Both programs have compiled without the flag since, and nothing was
lost: every function they define
carries a noinline annotation, except the SEC() entry points, which
nothing in the file calls, and the single __always_inline helper in
each, a deliberate exception the flag never overrode; and
compiling with -fno-inline restored yields byte-identical objects.

The two header dependencies with remaining value are restored in
working form. flow_dissector_load.h moves to the binary itself, which
is linked straight from its .c by the generic '$(OUTPUT)/%:%.c' rule
- editing the header now rebuilds it - on the line that already lists
the binary's helper object. cgroup_getset_retval_hooks.h -
added by commit e7215f5 ("selftests/bpf: Make sure
bpf_{g,s}et_retval is exposed everywhere") days before the rename
orphaned it - lived in the top directory, outside the progs/*.h
blanket prerequisite of the BPF object rules, so editing it never
rebuilt the BPF object. Move it under progs/, next to its only BPF
consumer, where the blanket prerequisite covers it - the arrangement
several other headers shared between progs/ and prog_tests/ already
use. Its userspace consumer is tracked exactly by the
compiler-generated dependency files.

'CURDIR := $(abspath .)' redefines make's builtin to the value it
already has, and 'OBJCOPY ?= $(CROSS_COMPILE)objcopy' defines a
variable nothing in the selftests build or the included kselftest
infrastructure ever reads; drop both.

Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
With BPF_STRICT_BUILD=0, eleven recipes append the same "|| { remove
the target, print a SKIP marker, report success }" tail, each spelled
out inline. Factor the tail into skip_on_fail; every call site keeps its
exact message and behavior.

The permissive fragments of other shapes are not suffixes and keep
their current form: the rsync --ignore-missing-args flags, the
missing-input guards in the skeleton recipes, the link rule's
wildcard handling and the test_kmods copy wrapper.

Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
$(VERIFICATION_CERT) and $(PRIVATE_KEY) come from a single genkey
invocation, but a rule with two normal targets runs its recipe once
per stale target. Today only $(VERIFICATION_CERT) is ever a
prerequisite (of $(VERIFY_SIG_HDR)), so the recipe runs once; the
runner patch later in this series lists both files as prerequisites
of the same target, at which point both runs would execute, under -j
concurrently, and the openssl invocations would race on the same
output files.

Rewrite it as a pattern rule, the same workaround the test_kmods
rule already uses, with the stem narrowed to the fixed name: genkey
hardcodes signing_key.pem / signing_key.der, and a wider pattern
would claim every .pem/.der under $(BUILD_DIR) and "succeed" without
producing the requested file.

Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
The verifier/tests.h recipe is a $(shell ...) expansion: the command
runs while make expands the recipe line - including under make -n -
its exit status is discarded, and the resulting (empty) expansion is
what make actually executes.

Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

Upstream branch: 1555de3
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503
version: 7

prozak added 4 commits August 25, 2026 12:46
The bench binary links every benchs/bench_*.c object, but the link
rule names them one by one in a hand-maintained list, which has to be
extended by hand for every new benchmark although the pattern rule
already builds any bench_*.c placed in benchs/. Derive the list with
a wildcard instead: a new benchmark is compiled and linked in by
dropping its source there (its skeleton dependency line, when it has
one, is still declared next to the others).

The derived list is sorted, which changes the link order of the bench
objects (previously roughly chronological) and with it the symbol
layout of the binary; no benchmark behaves differently. The trailing
'#' terminator goes away together with the block it closed: the two
remaining entries after the variable are fixed, so the append
friendliness it provided no longer buys anything.

Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
The four skeleton generation recipes (regular, light, signed light and
linked) are near-identical pipelines - link via "bpftool gen
object" three times, compare the second and third results as a
regression test for bpftool's determinism, generate the skeleton (and
possibly subskeleton), remove intermediates - duplicated with small
variations inside DEFINE_TEST_RUNNER_RULES, where every line pays the
double-expansion escaping tax.

Move the pipeline into gen_bpf_skel.sh; the differences between the
four variants become the --lskel, --sign and --subskel flags. Signing
takes the key and certificate from $PRIVATE_KEY and $VERIFICATION_CERT
in the environment, like the bpftool binary comes from $BPFTOOL; the
script checks for both up front, before any intermediate exists. The
distinct linked/llinked intermediate infixes are kept but derived
inside the script (its header says why they matter). The intermediates
are now named after the output header rather than the input object
(foo.linked1.o instead of foo.bpf.linked1.o for the single-object
variants; the linked-skeleton variant already used that form), which
keeps the stems distinct just the same.

The permissive-mode missing-input checks at the top of the recipes -
when the object's compile already failed and was skipped, skip the
skeleton quietly instead of running bpftool against a missing file -
are kept, factored into a skip_if_missing helper.
The build-log messages stay folded behind the guard as in the original
recipes, so a skipped skeleton prints SKIP-SKEL and nothing else. What
does change is the failure path: the script removes the intermediates
it created (the recipes left them behind), the determinism check
reports through cmp plus one message instead of a diff listing, and
the linked-skeleton recipe prints its GEN-SKEL line before the link
rather than after it, like the other three always did.

Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Assisted-by: Claude:claude-fable-5 shellcheck
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Move the build definitions a test runner needs - tree layout and
tool locations, flag assembly, feature probes, signing key paths, the
permissive-mode helpers - into Makefile.buildvars, in their current
order, as preparation for building each test runner instance in its
own sub-make: the definitions become includable by more than one
makefile. No rules or recipes are changed.

The include sits directly after ../lib.mk, ahead of the
../../../build/Makefile.feature evaluation, so the definitions are in
place for everything the top Makefile parses afterwards - in particular
the LLVM feature probe captures srctree and PKG_CONFIG from their new
location, and the top Makefile's remaining uses of both are all
recipe-time expansions. Two evaluation-time changes follow from
the new position and a third is made on the way, each commented in
place: CFLAGS and LDFLAGS are
reassembled around a snapshot taken before lib.mk, keeping the
pre-split order;
CLANG_SYS_INCLUDES becomes an immediate assignment (its inputs are
final by this point, and expanding it forks shells); and the
CLANG_HAS_ARENA_ASAN probe now sees CLANG as resolved from
LLVM=<suffix-or-path> rather than the bare 'clang' default it saw
above the include, while the CLANG_CPUV4 probe, needed before lib.mk,
keeps the old behavior.

BPF_GCC, TEST_KMODS and the knobs the runner never reads (SKIP_*,
submake_extras, TEST_KMOD_TARGETS, the VMLINUX_BTF block) stay in the
Makefile - the first two are needed before lib.mk is included, where
Makefile.buildvars cannot yet be, and reach the runners from there once
a later patch adds them: TEST_KMODS exported in the environment, BPF_GCC
as the bpf_gcc flavor's compiler parameter. Definitions that reach no
runner, directly or as a sub-make parameter, also stay - except
HOST_INCLUDE_DIR, which moves with the conditional block it shares
with the host build directories - and the four that derive from
Makefile.buildvars values sit in a block right after the include.

Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Assisted-by: Claude:claude-fable-5 shellcheck
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Replace the DEFINE_TEST_RUNNER/DEFINE_TEST_RUNNER_RULES double-expansion
machinery - whose rules never appear in the source as written, are
invisible to make's own debugging facilities, and are easy to break
for one flavor while testing another - with Makefile.runner: one
sub-make invocation per test runner
instance (test_progs, test_progs-no_alu32, test_progs-cpuv4,
test_progs-bpf_gcc, test_maps), each in its own single-flavor namespace
written in plain make - no define/eval layer, no per-flavor guards,
no accumulating vpath directives.

The main Makefile keeps everything that exists once - tool sub-builds,
vmlinux.h, signing keys, tests.h generation, standalone binaries and
the kselftest lib.mk contract - and delegates to the runner through
explicit per-instance rules.

The rules for one flavor's BPF objects and skeletons live in
Makefile.skel, instantiated by the main Makefile for the default
flavor and by each runner sub-make that builds BPF objects. lib.mk's
install rule copies the default flavor's BPF objects - the files the
flat copy of every flavor's objects resolved to before. No file is
built twice in one build: the main Makefile produces the default
flavor's artifacts before the unflavored test_progs sub-make runs,
and each flavored runner writes its own subdirectory plus its binary
in $(OUTPUT) (the bpftool
link in $(OUTPUT) is refreshed by both unflavored sub-makes, as
before).

The runner instances link userspace objects the main Makefile
builds once; the
flavored instances link the shared objects instead of compiling
their own identical copies.

This also fixes a latent parallel-build race: the runner objects
including libbpf's internal headers now order against the bpftool
sub-build that installs them into $(INCLUDE_DIR).

Smaller behaviour changes that come with the move, for the record:
the skeleton recipes' build-log lines take the msg helper's format
and stream, the shared userspace objects log as CC rather than
EXT-OBJ/LIB-OBJ, and TEST-HDR lines lose their runner tag; the signed
light-skeleton rule lists the key and certificate it uses as
prerequisites; the extras copy skips an empty file list and, in
permissive mode, does not wait for files whose build was skipped;
$(OUTPUT)/test_maps waits for the same shared prerequisites as the
other runners; the bare-name convenience targets for the
linked-skeleton constituents (make linked_funcs1.bpf.o) are gone; the
.d include no longer special-cases the clean, docs-clean and
emit_tests goals,
which a runner sub-make never runs; and the test objects' first-build
ordering lists only the generated headers the runner itself consumes,
the rest being built by the main Makefile before any runner starts.

The runner sub-makes receive CC and CLANG as resolved by lib.mk on
their command line: Makefile.buildvars probes clang's capabilities
with $(CLANG), and a runner has to reach the same verdict as the top
level when LLVM=<suffix or path> selects a non-default toolchain.
The unflavored in-tree instance skips the extras copy as before; the
runner compares the realpath of $(OUTPUT) with $(CURDIR), make's
physical working directory, so a tree reached through a symlink
still counts as in-tree. The userspace objects depend on the
generated tests.h of the runner that includes them only
(test_progs.o on prog_tests/tests.h, test_maps.o on
map_tests/tests.h); before, every extra object of a runner depended
on that runner's tests.h. The other headers those objects depended
on - flow_dissector_load.h, ip_check_defrag_frags.h, the libarena
skeletons - are included by none of them and are dropped.

Co-developed-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Assisted-by: Claude:claude-fable-5 shellcheck
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
@kernel-patches-daemon-bpf

Copy link
Copy Markdown
Author

At least one diff in series https://patchwork.kernel.org/project/netdevbpf/list/?series=1150503 expired. Closing PR.

@kernel-patches-daemon-bpf
kernel-patches-daemon-bpf Bot deleted the series/1150503=>bpf-next branch August 27, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant