bpf: Cancel special fields in resizable hashtab on recycle - #13437
bpf: Cancel special fields in resizable hashtab on recycle#13437kernel-patches-daemon-bpf[bot] wants to merge 4 commits into
Conversation
|
Upstream branch: 5e289c5 |
AI reviewed your patch. Please fix the bug or email reply why it's not a bug. In-Reply-To-Subject: |
AI reviewed your patch. Please fix the bug or email reply why it's not a bug. In-Reply-To-Subject: |
AI reviewed your patch. Please fix the bug or email reply why it's not a bug. In-Reply-To-Subject: |
AI reviewed your patch. Please fix the bug or email reply why it's not a bug. In-Reply-To-Subject: |
|
Forwarding comment 5397478581 via email |
|
Forwarding comment 5397558898 via email |
|
Forwarding comment 5397565336 via email |
|
Forwarding comment 5397588881 via email |
71e031f to
909ca3a
Compare
|
Upstream branch: d83fba2 |
71e35bf to
1f152be
Compare
909ca3a to
b010507
Compare
|
Upstream branch: ce36e38 |
1f152be to
e3a5ee0
Compare
b010507 to
975b11a
Compare
|
Upstream branch: 05ea1b6 |
e3a5ee0 to
ee3e356
Compare
975b11a to
f8c8078
Compare
|
Upstream branch: 1555de3 |
rhtab_delete_elem() and rhtab_map_update_existing() eagerly call bpf_obj_free_fields() when an element is deleted or its value is replaced. This runs kptr destructors in the caller's execution context, which is unsafe for BPF programs running in NMI context (e.g. perf_event programs attached to hardware PMU overflows): referenced kptr destructors may take locks or otherwise cannot run in NMI. Commit a3a81d2 ("bpf: Cancel special fields on map value recycle") switched the hash map and array recycle paths to bpf_obj_cancel_fields(), which only cancels NMI-safe fields (timer, workqueue, task_work), but it missed the resizable hashtab. rhtab_map_update_existing() even documents the intended "cancel" semantics while still calling bpf_obj_free_fields(). Fix the resizable hashtab the same way: * rhtab_delete_elem() and rhtab_map_update_existing() now cancel only NMI-safe fields. Referenced kptrs stay attached to the recycled element and are destroyed by rhtab_mem_dtor() once the element is eventually freed, keeping the reference accounting balanced. * rhtab_map_update_elem() initializes the special fields of a freshly allocated element. The bpf memory allocator may return a recycled element that still owns a referenced kptr, and check_and_init_map_value() would zero that slot, dropping the reference without releasing it. rhtab_init_map_value() initializes the remaining fields (spin lock, timer, workqueue, task_work, refcount) but leaves kptr slots untouched, matching the hash map semantics. Verified with a selftest: a perf_event (NMI) program overwrites a rhtab element that holds a referenced task kptr, and a second phase deletes and re-inserts the element to exercise the recycle path. Before the patch the NMI update eagerly released the kptr and the recycle path zeroed the inherited slot; after the patch the kptr is inherited on both paths and the probe observes it non-NULL. Fixes: a3a81d2 ("bpf: Cancel special fields on map value recycle") Signed-off-by: Yuan Chen <chenyuan@kylinos.cn>
bpf_ma_set_dtor() duplicates the map's btf_record for the bpf_mem_alloc destructor. For kptr fields backed by the program BTF (MEM_ALLOC kptrs, e.g. objects allocated with bpf_obj_new()/bpf_percpu_obj_new()), btf_record_dup() only borrows the reference, matching what btf_parse_fields() did for the map's own record. The duplicated record, however, is released later from the deferred bpf_mem_alloc destructor workqueue (free_mem_alloc_deferred), by which time the program BTF may already have been freed: bpf_map_free() drops the map's own reference, and the RCU callback can run before the workqueue. Reading field->kptr.btf in btf_record_free() (via btf_is_kernel()) is then a use-after-free, detected by KASAN as "slab-use-after-free in btf_is_kernel" when a map with a MEM_ALLOC kptr field is destroyed. Hold a reference on program BTF for the lifetime of the duplicated record and drop it right before the record is freed. The last btf_put() only schedules the object for RCU destruction, so btf_record_free() can still safely read the field descriptors. The rhtab kptr selftests exercise this path on every map teardown and triggered the bug under KASAN; with this fix they pass cleanly. Fixes: 1df97a7 ("bpf: Register dtor for freeing special fields") Signed-off-by: Yuan Chen <chenyuan@kylinos.cn>
A perf_event program running in NMI context overwrites a rhtab element whose value holds a referenced task kptr. The old kptr must stay attached to the element (cancel semantics, matching hash maps); before the rhtab recycle fix the NMI update eagerly released it and the probe observed NULL. The test asserts the NMI program actually ran, so the probe result is meaningful. A second phase deletes and re-inserts the element 2000 times. The re-insertion may recycle the freed element, which still owns the kptr; before the fix the alloc path zeroed the inherited slot via check_and_init_map_value(), leaking the reference, and the probe never observed a non-NULL pointer. The test requires at least one recycle to inherit the kptr, and also verifies that plain (non-special) value bytes still round-trip through the recycled element on every iteration. The NMI phase is skipped when no hardware PMU is available. Signed-off-by: Yuan Chen <chenyuan@kylinos.cn>
BPF_MAP_TYPE_RHASH allows spin locks, timers, workqueues, task_work,
kptrs (referenced, untrusted, per-cpu) and refcounts in map values.
The recycle fix only changes kptr slot handling, so verify each field
combination end to end:
* lock_kptr: bpf_spin_lock + referenced kptr + plain data in one
value. BPF_F_LOCK syscall updates/lookups must work before and
after many delete/re-insert recycle cycles, the referenced kptr
must be inherited on recycled elements (zeroing it would leak the
reference), and the plain bytes must round-trip every iteration.
* timer: arm a bpf_timer and verify it fires, delete the element and
verify the timer is cancelled, then re-insert (possibly recycling
the freed element) and arm a fresh timer again.
* kptr_untrusted: the untrusted kptr must survive the recycle like a
referenced one.
* kptr_percpu: the per-cpu kptr reference must survive the recycle
(zeroing it would leak the reference).
On the unfixed kernel the three kptr subtests fail at the recycle
assertions while the lock and timer paths still pass, isolating the
behavior change to kptr slots only.
Signed-off-by: Yuan Chen <chenyuan@kylinos.cn>
ee3e356 to
f8e2f1a
Compare
Pull request for series with
subject: bpf: Cancel special fields in resizable hashtab on recycle
version: 1
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=1150975