| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs fixes from Christian Brauner:
"This contains fixes for the current development cycle.
All of them came out of a review of the mount code that started with a
bug report. The review modeled the corner cases of mount propagation,
unmounting and mount reference counting and turned up a lot of bugs.
Most of them years old. Most fixes come with a selftest.
- Rework connected mounts.
A mount that is unmounted together with its parent can stay
attached to the parent to keep its mountpoint covered. That happens
when the mountpoint is removed with rmdir(), unlink() or rename(),
when a detached tree is dissolved, and for locked mounts in any
umount that isn't synchronous, including the teardown of their
mount namespace. The parent then owns the child and drops it on its
own final mntput(). So any reference from the child's superblock
back to one of its ancestors becomes a cycle that is never freed.
A loop device backed by an image on a tmpfs and mounted on that
same tmpfs is enough. Remove the directory the tmpfs is mounted on
from the host, let the container's mount namespace exit, and the
loop device, the tmpfs and the filesystem on the loop device are
leaked for good. The same works with autofs, zram, ecryptfs,
binfmt_misc, fuse passthrough, zloop, a mass storage gadget and md,
and the selftests have reproducers for them. This has been possible
since v4.1. It is also why "put_mnt_ns(): leave mounts connected"
was reverted in -rc5. Keeping every mount of a dying mount
namespace connected made these cycles trivial to create.
Every unmounted mount is now detached from its parent. Where the
mountpoint has to stay covered the mount leaves a cover on the
parent instead, allocated together with the mount. A lookup on the
unmounted parent that hits a cover finds an empty immutable
directory or file on the private nullfs instance. Nothing leads
from a cover to another mount, so no unmounted mount owns another
one and no cycle can form.
This is visible to userspace. A formerly connected mount can no
longer be reached through its unmounted parent and ".." inside it
leads nowhere, as for every other lazily unmounted mount.
With that the private nullfs instance becomes reachable from
userspace, so it now refuses mounts on top, is mounted read-only,
and refuses fsnotify marks and file locks. Its inodes are shared by
every holder and a watch or a lock would otherwise reach across
users. may_decode_fh() now also decides its subtree check under a
single mount_lock hold, as a racing umount could otherwise let it
decode into what a locked child covered.
- umount:
* Don't silently unmount busy mounts.
Since v4.13 propagate_umount() takes down propagated copies of
the victim with children as long as each child is an overmount or
another copy of the victim, but propagate_mount_busy() only ever
checked copies without children or with just an overmount. A
container that moved a tree beneath its copy of a host mount lost
that tree from under its open file descriptor to a plain umount()
on the host. propagate_mount_busy() now applies the same rules,
walking each chain of copies once.
* Don't let a migrating task hide its reference from umount().
mnt_get_count() sums the per-cpu counters under mount_lock but
the mntget() and mntput() fast paths don't take it. A task that
takes a reference on a cpu the sum has already passed and drops
it after migrating to one the sum hasn't reached yet hides the
reference it held to begin with, and umount() succeeds with the
file still open. Gets and puts now live in separate per-cpu
counters and all puts are summed before all gets with a full
barrier in between, the way srcu_readers_active_idx_check() does
it. mntget() is unchanged and mntput() gains an smp_wmb().
* Check each submount for references right before unmounting it.
shrink_submounts() and mark_mounts_for_expiry() checked all their
victims up front. Unmounting the first could move a busy
overmount to where the next victim's propagated copy is looked up
and it was then unmounted without a check.
* Never expire a locked mount.
A shrinkable mount moved beneath a locked mount with
MOVE_MOUNT_BENEATH takes over the lock, and umount() of an
unlocked ancestor expired it and revealed what it covered. That
umount() now fails with EBUSY as it does for any other locked
child. A lazy umount still takes the whole tree.
- Overmounts and locked mounts:
* Unhash a dentry before detaching the mounts on it.
unlink(), rmdir() and rename() detach the mounts on the victim
but only d_delete() it once its inode is unlocked, a window that
includes an expedited RCU grace period. In between, a lookup from
a mount namespace in which the dentry is a mountpoint found it
hashed, positive and uncovered. Drop the dentry first, as
d_invalidate() already does.
* Don't reveal overmounted entries in refwalk.
A refwalk that had grabbed the dentry before the unlink never
rechecked it the way rcuwalk does with d_seq and mount_lock.
Without any artificial widening three walkers read the covered
file 27 times in a minute. step_into() now fails an unhashed
dentry marked DCACHE_CANT_MOUNT with -ESTALE and the walk is
retried.
* Keep covered mounts covered in OPEN_TREE_NAMESPACE.
Creating such a mount namespace only takes a user namespace and
the copy followed bind mount rules: no children without
AT_RECURSIVE and no unbindable mounts with it. An unprivileged
user could see what mounts covered in the source, such as the
parts of /proc and /sys that container runtimes mask. If the
caller doesn't own the source mount namespace a non-recursive
copy of a mount with something mounted below the requested
directory is now refused and a recursive copy includes unbindable
mounts, the way unshare() copies.
* Keep the lock on a mount that a propagated copy is moved beneath.
MNT_LOCKED moved to any mount that ended up beneath a locked
mount, propagated copies included. A host mount and umount on a
directory covered by a locked mount in a less privileged mount
namespace left that cover unlocked for the namespace's owner to
remove. Only mounts the caller places beneath take over the lock
now.
* Handle mount locking for automounts correctly. Which copies to
lock was decided by the mount namespace of the task that
triggered the automount. A task in a user namespace that
triggered one on a host mount through a file descriptor got the
host's own automount locked while its own copy stayed unlocked
and could have nosuid, nodev and noexec cleared. Use the owner of
the mount namespace the mount lands in.
- Use-after-free and crashes:
* Refuse an automount below a mount that is in no namespace.
The private clones overlayfs uses for its layers have the
MNT_NS_INTERNAL error pointer as their namespace, which
finish_automount() let through and count_mounts() dereferenced. A
fanotify filesystem mark on an overlayfs lower layer hands out
file descriptors on such a clone. With debugfs as the lower layer
opening "tracing" oopses with namespace_sem held for writing and
every mount operation on the system blocks from then on.
* Reset the old parent's ->overmount in mnt_change_mountpoint().
When propagate_umount() moved an overmount off a mount that a
file descriptor kept alive, MOVE_MOUNT_BENEATH through that
descriptor later followed the stale pointer into the freed
overmount.
* statmount() with STATMOUNT_BY_FD and pivot_root() read the parent
of a mount that may be unmounted and only held by a file
descriptor, while the parent's final mntput() can free it.
statmount() now reads it under mount_lock and pivot_root() first
checks that both mounts are in the caller's mount namespace.
* Queue a mount only once for mount notifications. A mount
reparented by one umount_tree() and taken down by the next under
the same namespace_sem hold, as in shrink_submounts(), was queued
twice. That cut the mounts queued in between out of notify_list
while it still pointed at them, and once they were freed every
later mount operation walked freed memory.
* Don't let a pseudo dentry become the root of a mount. A bind
mount of a bpf token file did that with a DCACHE_NORCU dentry,
which is freed without an RCU grace period while lockless path
walks may still look at it. Refuse to clone such a mount.
* Don't inherit MNT_UMOUNT in clone_mnt(). A bind mount of a lazily
unmounted nsfs or pidfs mount through its file descriptor started
out flagged as unmounted. Among other things __detach_mounts()
then dropped the namespace's reference on it, the mount outlived
its namespace and mount_setattr() through the descriptor read the
freed namespace. A recursive bind mount of such a mount also
copied the unmounted stack still attached to it. That now fails
with EINVAL, copying the mount itself still works.
* Remove the fsnotify marks of a mount namespace in free_mnt_ns()
instead of the RCU callback that frees the namespace, where
taking the group mutexes meant sleeping in softirq context.
- Propagation and copies:
* Keep a copied mount unbindable. Since v6.17 clone_mnt() didn't
copy the unbindable flag, so every mount namespace created with
CLONE_NEWNS had bindable copies of all unbindable mounts. This
had been fixed once before.
* Refuse MOVE_MOUNT_SET_GROUP on an unbindable mount. It made the
mount an unbindable slave, a state nothing else can produce, or
silently dropped the unbindable flag. CRIU applies MS_UNBINDABLE
after restoring sharing and isn't affected.
* Check a recursive bind mount for mount namespace loops.
Recursively bind mounting a tree from another mount namespace
could put a mount of a namespace's file inside that same
namespace, which then pins itself and all its mounts. Repeating
it leaks without limit, the reproducer took Shmem from 380 kB to
65916 kB. The copy is now checked with check_for_nsfs_mounts()
before it is grafted, as move_mount() does.
* Look at the topmost mount for a mount namespace file.
attach_recursive_mnt() never looked at the topmost mount of the
source's chain of overmounts. If that was the chain's only mount
namespace file an existing mount at a propagated destination got
buried below the root of the nsfs file where no path walk reaches
it.
* Don't put a mountpoint on a dentry that's being removed.
attach_recursive_mnt() makes a mountpoint of the source's root
without its inode lock, so a racing rmdir() of that directory
could leave a mount on it that nothing ever detaches.
d_set_mounted() now checks cant_mount() as well.
- nullfs:
* Take no inode lock for readdir of an immutable directory.
The root of every empty mount namespace is the same nullfs
directory and iterate_dir() held its i_rwsem across
->iterate_shared(). A reader whose buffer faults on a FUSE mount
of its own holds it for as long as its server wants, and with an
exclusive locker queued behind it every lookup that misses the
dcache, every create and every mount in that directory waits. One
user of an empty mount namespace stalls all others. Directories
with the new FOP_IMMUTABLE flag skip the lock.
* Refuse to reconfigure internal superblocks through fspick(),
MS_REMOUNT or the read-only remount that a synchronous umount()
of the root does. For nullfs only root in the initial user
namespace could do it, but the superblock is shared by every
mount namespace and the flags showed up in statfs() for all of
them.
* Don't update the access time on nullfs and refuse F_SET_RW_HINT
on an immutable inode.
- unshare: Free an nsproxy that was never installed with
nsproxy_free() when set_cred_ucounts() fails. put_nsproxy() dropped
active references that were never taken, which triggered a warning
and hid the caller's own namespaces from listns().
- Smaller changes: mount_setattr() checks the target before it walks
the tree to allocate peer group ids, unshare() puts the old
fs_struct before the old namespaces, dissolve_on_fput() drops the
file's reference to the tree itself, disconnect_mount() is
simplified and the documentation of the propagated unmount rule is
brought up to date"
* tag 'vfs-7.3-rc7.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (58 commits)
namespace: simplify disconnect_mount()
selftests/filesystems: test covered mounts
namespace: rework connected mounts
nullfs: add an empty immutable regular file
selftests/filesystems: check that reading the root of an empty mount namespace stalls nobody
selftests/filesystems: add a helper that holds a readdir in a page fault
readdir: take no inode lock on an immutable directory
nullfs: refuse file locks
fsnotify: let a filesystem refuse marks on its objects
namespace: nothing is mounted on or written through knullfs
namespace: keep the private nullfs instance in knullfs
fhandle: decide the subtree check under mount_lock
selftests/filesystems: check that an automount below an overlay layer is refused
selftests/filesystems: check the atime of the empty mount namespace root
selftests/filesystems: check that a lock lands on the right mount and stays
namespace: keep the lock on a mount that a propagated copy is moved beneath
namespace: never expire a locked mount
nullfs: don't update the access time
namespace: handle mount locking for automounts correctly
namespace: refuse an automount below a mount that is in no namespace
...
|
|
Active references on the namespaces of an nsproxy are taken when the
nsproxy is installed into a task in switch_task_namespaces() and
copy_namespaces() and dropped again by put_nsproxy() through
deactivate_nsproxy().
But ksys_unshare() calls put_nsproxy() on an nsproxy that was never
installed when set_cred_ucounts() fails. The new namespaces of that
nsproxy go from zero to minus one and the namespaces shared with the
caller lose a reference that belongs to the nsproxy the caller keeps
using:
WARNING: kernel/nscommon.c:171 at __ns_ref_active_put+0x1cd/0x230
nsproxy_ns_active_put
deactivate_nsproxy
ksys_unshare
Afterwards the namespaces the caller lives in aren't listed by listns()
anymore. set_cred_ucounts() only fails when alloc_ucounts() can't
allocate and it only allocates when the real uid of the caller differs
from its effective uid.
Commit cefd55bd2159 ("nsproxy: fix free_nsproxy() and simplify
create_new_namespaces()") separated the two cases on purpose.
nsproxy_free() frees an nsproxy that was prepared but never installed
and that's what a failed setns() uses in put_nsset(). Export it and use
it for a failed unshare() as well.
Fixes: a98621a0f187 ("unshare: fix nsproxy leak in ksys_unshare() on set_cred_ucounts() failure")
Cc: stable@vger.kernel.org # v7.1+
Link: https://patch.msgid.link/20260930-work-mount-fixes-3-v1-16-be34c83956ae@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
unshare(CLONE_NEWNS) always copies the fs_struct and copy_mnt_ns()
points root and pwd of that copy into the new mount namespace. The old
fs_struct pins the old root and pwd until ksys_unshare() frees it at the
end and switch_task_namespaces() will already put the old mount
namespace before.
This causes pointless work in user namespaces because the copied mounts
are locked and stay connected. Switch and free the fs_struct before
switching the namespaces. The old root and pwd are wasted while their
mount namespace is still alive which also keeps both puts on the
mntput() fast path.
setns() already does this right as it updates root and pwd before the
namespaces are switched.
Link: https://patch.msgid.link/20260926-work-mount-fixes-2-v1-7-f357abf3d17b@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Add a sched_cache_grp pointer to task_struct so that scheduler code
can access the cache group directly via the task, without going
through mm->sched_cache_grp. This decouples the scheduler's hot-path
accesses from the mm_struct.
Each task holds its own refcount on the sched_cache_group, separate
from the reference held by its mm_struct. The reference is acquired
in copy_mm() (fork) and exec_mmap() (exec), and released in exit_mm().
This fixes the use-after-free when account_mm_sched() reaches the group
through a task whose mm is being switched, as reported by Hyunwoo:
https://lore.kernel.org/lkml/apPb-Dr4nPYuHQOK@v4bel/
Convert all scheduler code in fair.c and exit.c to use
p->sched_cache_grp instead of p->mm->sched_cache_grp.
Keep the fork/exec/exit reference management out of the generic mm
paths: add sched_cache_fork(), sched_cache_fork_cleanup(),
sched_cache_exec_mmap() and sched_cache_exit_mm() in
kernel/sched/cache_sched.c (with empty stubs for !CONFIG_SCHED_CACHE),
so fs/exec.c, kernel/fork.c and kernel/exit.c each call one helper
instead of open-coding the refcounting under #ifdef. Also add
sched_cache_group_get() and task_cache_group_get().
Fixes: df0d98475954 ("sched/cache: Introduce infrastructure for cache-aware load balancing")
Closes: https://lore.kernel.org/lkml/apPb-Dr4nPYuHQOK@v4bel/
Closes: https://lore.kernel.org/all/343a7e07-7fad-4979-9c9b-82ec038c293c@linux.dev/
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Reported-by: Zenghui Yu (Huawei) <zenghui.yu@linux.dev>
Co-developed-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: <stable@kernel.org> #7.2.x
Link: https://patch.msgid.link/ae7081dc54736bf115215f9867abb2711a7403fb.1790035273.git.tim.c.chen@linux.intel.com
|
|
As Jann demonstrated, it is entirely feasible to access the mm through vfork().
Therefore we need to allocate a private hash on vfork() as well as any other
CLONE_VM user.
Specifically, it must be avoided to have (private) futex waiters before
allocating the private hash.
Fixes: ee9dce44362b ("futex: Drop CLONE_THREAD requirement for private default hash alloc")
Reported-by: Jann Horn <jannh@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260911090447.GT788244@noisy.programming.kicks-ass.net
|
|
dup_task_struct() copies the parent's task_struct, including ret_stack.
ftrace_graph_init_task() clears the copied function graph state, but it
currently runs after copy_exec_state().
For non-CLONE_VM forks, copy_exec_state() allocates a new task_exec_state.
If that allocation fails, copy_process() reaches bad_fork_free and
free_task() calls ftrace_graph_exit_task(). Since the child still carries
the parent's ret_stack pointer, the unwind frees the parent's active
function graph return stack. The parent subsequently accesses freed memory
from function_graph_enter_regs().
KASAN reports:
[ 22.190920] ==================================================================
[ 22.195899] BUG: KASAN: slab-use-after-free in function_graph_enter_regs+0xa76/0xb90
[ 22.200747] Write of size 8 at addr ff110000054dc0a8 by task repro/1
[ 22.205134]
[ 22.210770] CPU: 0 UID: 0 PID: 1 Comm: repro Not tainted 7.2.0-07732-g9328b3b03bdc-dirty #3 PREEMPT(lazy)
[ 22.212576] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
[ 22.213750] Call Trace:
[ 22.215271] <TASK>
[ 22.216242] ? ftrace_stub_direct_tramp+0x10/0x10
[ 22.217774] dump_stack_lvl+0x4e/0x70
[ 22.220531] print_report+0x157/0x4b4
[ 22.223202] ? fixup_red_left+0x9/0x30
[ 22.224407] ? complete_report_info+0x83/0x110
[ 22.226679] ? function_graph_enter_regs+0xa76/0xb90
[ 22.228084] kasan_report+0xce/0x100
[ 22.230109] ? function_graph_enter_regs+0xa76/0xb90
[ 22.232860] ? stack_trace_save+0x4/0xd0
[ 22.234156] function_graph_enter_regs+0xa76/0xb90
[ 22.236090] ? kasan_save_stack+0x30/0x50
[ 22.237752] ? __pfx_function_graph_enter_regs+0x10/0x10
[ 22.238694] ? ring_buffer_lock_reserve+0x345/0xf80
[ 22.239628] ? stack_trace_save+0x4/0xd0
[ 22.242121] ? stack_trace_save+0x4/0xd0
[ 22.243588] ftrace_graph_func+0xda/0x160
[ 22.245362] ? ftrace_stub_direct_tramp+0x10/0x10
[ 22.246520] 0xffffffffa0000095
[ 22.250528] ? stack_trace_save+0x9/0xd0
[ 22.251757] ? ring_buffer_unlock_commit+0x11d/0x5c0
[ 22.253152] stack_trace_save+0x9/0xd0
[ 22.254264] kasan_save_stack+0x30/0x50
[ 22.273631] kasan_save_track+0x14/0x30
[ 22.276763] kasan_save_free_info+0x3b/0x70
[ 22.278296] __kasan_slab_free+0x43/0x70
[ 22.280157] kmem_cache_free+0xbf/0x3b0
[ 22.282963] ? ftrace_stub_direct_tramp+0x10/0x10
[ 22.284001] free_task+0xa2/0x160
[ 22.285699] ? ftrace_stub_direct_tramp+0x10/0x10
[ 22.286752] copy_process+0x2aae/0x7bc0
Initialize the child function graph state immediately after
dup_task_struct(), before the first fallible operation.
Cc: stable@vger.kernel.org
Fixes: 6b1c66c9cca9 ("exec_state: relocate dumpable information")
Reviewed-by: Bradley Morgan <include@grrlz.net>
Link: https://patch.msgid.link/20260822195321.962383-2-Jeremy.Jean@oss.cyber.gouv.fr
Assisted-by: Codex:gpt-5
Signed-off-by: Jérémy Jean <Jeremy.Jean@oss.cyber.gouv.fr>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull non-MM updates from Andrew Morton:
- "ocfs2/dlm: bound peer-controlled lengths in the o2dlm" (Bryam
Vargas)
Validate and bound all input lengths and count fields in the o2dlm
migration and recovery receive handlers to prevent memory corruption
and kernel panics from malformed cluster messages
- "ocfs2: validate xattr entry bounds" (Cen Zhang)
Validate OCFS2 extended attribute entry name and value bounds during
metadata reads to prevent out-of-range memory accesses during
retrieval or listing operations.
- "taskstats: fix cgroupstats invalid fd handling and add selftests"
(Yiyang Chen)
Return -EBADF when cgroupstats receives an invalid file descriptor to
prevent caller hangs and misleading success ACKs. Add a kselftest to
validate valid cgroup v1 queries and verify proper error handling
across different Netlink flag combinations.
- "misc lib/raid/ improvements v2" (Christoph Hellwig)
Improve benchmark-based algorithm selection for the XOR and RAID6
libraries, add KUnit benchmark tests, and cleanup minor
implementation details.
- "ocfs2: cluster: o2hb_region_pin() fixes" (Joseph Qi)
Fix sleeping-in-atomic, lock order inversion and error-path cleanup
bugs in o2hb_region_pin() by releasing o2hb_live_lock across sleeping
configfs_depend_item() calls and using unlocked variants from
callback context. Ensure failed pin attempts properly decrement user
counts and unpin partially initialized heartbeat regions to prevent
memory leaks and unprotected states.
- "lib/ucs2_string.c: fix out-of-bounds read in ucs2_strnlen()"
(Vincent Mailhol)
Fix an off-by-one which could cause an out-of-bounds read.
- "ocfs2: harden heartbeat teardown races" (Cen Zhang)
Fix two OCFS2 heartbeat/o2net teardown races found by KASAN.
- "taskstats: tidy up the cpumask command path" *Bradley Morgan)
make two small cleanups in kernel/taskstats.c.
- "ocfs2: validate active orphan slots during inode read" (ZhengYuan
Huang)
Validate active ordinary and append-DIO orphan slots read from OCFS2
dinodes at the metadata boundary to prevent corrupted slot indices
from causing out-of-bounds array accesses.
- "ocfs2: bound-check both readdir re-validation scans" (Zhan Xusheng)
Enforce strict boundary checks on directory entry record lengths and
offset calculations during OCFS2 directory re-scans to prevent
out-of-bounds memory reads and directory position corruption.
* tag 'mm-nonmm-stable-2026-08-22-16-57' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (95 commits)
mailmap: fix bouncing address for Taniya Das
ocfs2: bound-check dir entries in the inline-data re-validation scan
ocfs2: bound-check dir entries in the readdir re-validation scan
squashfs: avoid thundering-herd cache wakeups
prctl: fix PR_SET_MM_AUXV losing the forced AT_NULL terminator
mailmap: update email address for Linfeng Sun
lib/interval_tree: fix allocation warning messages
checkpatch: add NOKPROBE_SYMBOL to the whitelist of lines that can occur immediately after functions
Squashfs: check block offset is not negative
signal: factor out the kernel reserved si_code check
ocfs2: fix readdir position truncation on 32-bit kernels
ocfs2: fix cached cluster count after suballocator reclaim
ocfs2: fix circular locking dependency in ocfs2_init_acl()
ocfs2: validate DIO orphan slot during inode read
ocfs2: validate orphan slot during inode read
selftests/prctl: fix non-anonymous VMA mapping in set-anon-vma-name test
MAINTAINERS: add IRC and patchwork for LTP
include/linux/list.h: mark list_add and __list_add as __always_inline
tools/mm: prevent page_owner_sort from truncating input
hung_task: update DETECT_HUNG_TASK_BLOCKER Kconfig help
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull futex fixes from Ingo Molnar:
- Enforce that the private futex owner shares the mm when attaching
(Kyle Zeng, Thomas Gleixner)
- Fix race on the initial mm->futex.phash.ref allocation (Hyunwoo Kim)
- Fix might_sleep() warning in futex_pivot_pending() (Peter Zijlstra)
* tag 'locking-urgent-2026-08-22' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
futex: Fix might_sleep() warning in futex_pivot_pending()
futex: Fix race on the initial mm->futex.phash.ref allocation
futex: Clean up the redundant exit/exec functions
futex/pi: Plug private futex exec() race
futex: Sanitize and document task_struct::futex::state transitions
futex/pi: Reject cross-mm private futex owners
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull SMP core updates from Thomas Gleixner:
- Reduce the preemption disabled sections in smp_call_function*().
The various smp call functions keep preemption disabled accross the
full operation which includes the wait for completion. Especially the
latter can take some time when one of the target CPUs is not
immediately responding to the IPI, which can result in large latency
spikes.
To improve this provide a per task CPU mask to track the CPUs to wait
for. That makes the information required for the wait task local and
therefore allows to reenable preemption before the wait.
While this comes with moderate extra memory cost this reduces SMP
function call induced latency measured in a fleet for high priority
tasks from ~17ms to ~1.5ms (~90%).
- Reduce the overhead of the CSD debug code by replacing the heavy
memory barriers with smp_store_release()/acquire()
- Remove obsolute unused hotplug states
* tag 'smp-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
scftorture: Remove preempt_disable() in scftorture_invoke_one()
smp: Remove preempt_disable() from on_each_cpu_cond_mask()
smp: Remove preempt_disable() from smp_call_function()
smp: Enable preemption early in smp_call_function_many_cond()
smp: Alloc percpu csd data in smpcfd_prepare_cpu() only once
smp: Use task-local IPI cpumask in smp_call_function_many_cond()
smp: Refactor remote CPU selection in smp_call_function_any()
smp: Enable preemption early in smp_call_function_single()
smp: Disable preemption explicitly in __csd_lock_wait()
cpu/hotplug: Remove CPUHP_AP_ARM_CORESIGHT_CTI_STARTING
smp: Use release stores for csd_lock_record() state
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull performance events updates from Ingo Molnar:
"uprobes updates:
- Fix a category of bugs with optimized uprobes that can clobber the
redzone area with call instruction storing return address on stack
where user code may keep temporary data without adjusting RSP.
Fix this by moving the optimized uprobes on top of 10-bytes NOP
instruction, so we can squeeze another instruction to escape the
redzone area before doing the call (Jiri Olsa, Andrii Nakryiko)
- Switch uretprobes_srcu to SRCU-fast-updown, to improve performance
(Puranjay Mohan)
Intel CPU PMU driver updates:
- Optimize ACR handling in match_prev_assignment() (Dapeng Mi)
- Fix various PMU driver bugs and data leaks (Dapeng Mi)
- Fix Intel PT stop/start with no update (Adrian Hunter)
Intel uncore PMU driver updates:
- Fix various uncore PMU setup robustness bugs (Zide Chen)
AMD uncore PMU driver updates:
- Add group validation (Sandipan Das)
.. and misc fixes and updates by Dapeng Mi, Randy Dunlap and Zide Chen"
* tag 'perf-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (42 commits)
perf/x86: Optimize ACR handling in match_prev_assignment()
perf/x86/intel: Fix intel_cap handling on hybrid PMUs
perf/x86: Remove stale fixed counter helper and fix hybrid PMU access
perf/x86/intel: Unwind cpuc state if PEBS buffer setup fails
perf/x86: Guard intel_pmu_cpu_dead() against invalid hybrid PMU casts
perf/x86: Free hybrid state on PMU init failure
perf/x86: Unregister PMI handler on PMU init failure
perf/x86/intel/pt: Fix stop/start with no update
perf/x86/intel/pt: Use bitwise access for PERF_HES_STOPPED
perf/x86/intel/pt: Factor out pt_config_enable()
uprobes: Switch uretprobes_srcu to SRCU-fast-updown
srcu: Add lock guard for srcu_fast_updown flavor
perf/x86/intel/pt: Drop kernel-doc for deleted struct members
perf/x86/amd/uncore: Add group validation
selftests/bpf: Add tests for forked/cloned optimized uprobes
selftests/bpf: Add tests for uprobe nop10 red zone clobbering
selftests/bpf: Add reattach tests for uprobe syscall
selftests/bpf: Change uprobe/usdt trigger bench code to use nop10
selftests/bpf: Change uprobe syscall tests to use nop10
selftests/bpf: Emit nop,nop10 instructions combo for x86_64 arch
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull kthread vfs updates from Christian Brauner:
"This stops kernel threads from sharing filesystem state with
userspace. This work is about 3 cycles old and has been in -next
for about that time.
When the kernel boots init_task creates PID 1 and then kthreadd. From
that point every kthread and PID 1 share the same fs_struct. That is
why pivot_root() has to rewrite the fs_struct of all kthreads. The
rewriting exists so that kthreads can use init's filesystem state when
they want to. It also means userspace can move the ground out from
under the kernel.
PID 1 now gets a completely separate fs_struct. All kthreads are
anchored in a private SB_KERNMOUNT instance of nullfs that cannot be
mounted on and cannot be used to follow other mounts. Userspace init
can no longer affect kthread filesystem state and kthreads can no
longer affect userspace fs state without explicit opting in to that.
Path lookup from a kthread now fails by default. It makes it
deliberately hard to offload security sensitive operations into init's
filesystem state from a kthread.
Places that legitimately need to look something up there opt in
through the new scoped_with_init_fs() which temporarily overrides the
caller's fs_struct with init's. usermodehelpers remain the only kernel
tasks that genuinely share init's filesystem state, since they execute
random binaries in the root filesystem (excellent...).
The visible result is that /proc/2/root is a nullfs with an empty
mountinfo while /proc/1/root is the real root"
* tag 'vfs-7.3-rc1.kthread' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (26 commits)
initramfs_test: use test init/exit hooks to override init fs
fs: stop rewriting paths for PF_EXITING | PF_DUMPCORE
fs: stop rewriting kthread fs structs
fs: start all kthreads in nullfs
nullfs: make nullfs multi-instance
devtmpfs: create private mount namespace
fs: add umh argument to struct kernel_clone_args
fs: stop sharing fs_struct between init_task and pid 1
af_unix: use scoped_with_init_fs() for coredump socket lookup
initramfs: use scoped_with_init_fs() for rootfs unpacking
pnfs/blocklayout: use scoped_with_init_fs() for SCSI device lookup
ksmbd: use scoped_with_init_fs() for VFS path operations
ksmbd: use scoped_with_init_fs() for filesystem info path lookup
ksmbd: use scoped_with_init_fs() for share path resolution
fs: use scoped_with_init_fs() for kernel_read_file_from_path_initns()
coredump: use scoped_with_init_fs() for coredump path resolution
btrfs: use scoped_with_init_fs() for update_dev_time()
scsi: target: use scoped_with_init_fs() for APTPL metadata
scsi: target: use scoped_with_init_fs() for ALUA metadata
crypto: ccp: use scoped_with_init_fs() for SEV file access
...
|
|
futex_exit_release() and futex_exec_release() are identical now. That means
also exit_mm_release() and exec_mm_release() are identical.
Consolidate the whole lot and remove the redundant copies.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Kyle Zeng <kylebot@openai.com>
Acked-by: Peter Zijlstra <peterz@infradead.org>
|
|
max_threads is initialized once by fork_init()->set_max_threads(), and
thereafter is mostly read in hot path, such as copy_process().
The fact that it is mostly read and not written to makes it candidates for
__read_mostly declarations.
I have already tested on my machine(arm64,256core,kernel-7.1.3) with
'hackbench -P -l 500 -g 100/500/1000/2000' and the results show that there
is an average of 13% improvement in performance.
before patch:
hackbench -g 100 500 1000 2000
1.094 4.651 13.305 36.322
after patch:
hackbench -g 100 500 1000 2000
0.823 4.487 11.107 32.987
Above data are the average values obtained from multiple tests, and there
was indeed some fluctuation in the data during the tests.
Link: https://lore.kernel.org/20260707092354.465847-1-jansef.jian@hj-micro.com
Signed-off-by: JianChunfu <jansef.jian@hj-micro.com>
Reviewed-by: Lorenzo Stoakes <ljs@kernel.org>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Kees Cook <kees@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
|
smp_call_function_many_cond() uses the per-CPU cfd->cpumask as the list
of remote CPUs to wait for. That is safe while the caller remains pinned
to the current CPU for the whole operation, because another task cannot
run on the same CPU and reuse the per-CPU mask.
The synchronous wait is the long-latency part of the operation. To make
that wait preemptible, the mask iterated by csd_lock_wait() must remain
stable even if the task is preempted or migrates. If the wait used the
per-CPU cfd->cpumask after dropping CPU pinning, another task scheduled
on the original CPU could enter smp_call_function_many_cond() and
overwrite the mask while the first task is still iterating it.
Give each task private IPI cpumask storage and use it as the wait mask in
smp_call_function_many_cond(). Other cpumask storage choices do not fit
this use case:
- Per-CPU storage is the state that becomes unsafe once the wait is
made preemptible. After the caller drops CPU pinning, another task
scheduled on the original CPU can enter smp_call_function_many_cond()
and reuse the same per-CPU mask.
- Stack storage is not suitable for large NR_CPUS or
CONFIG_CPUMASK_OFFSTACK=y configurations. The wait mask needs to
scale with cpumask_size(), and putting that storage on the stack is
not acceptable on large systems.
- Allocating the mask inside smp_call_function_many_cond() would put an
allocation and a failure path in the generic IPI path. A sleeping
allocation is not suitable because callers have historically only
provided a preempt-disabled context, not a sleepable one. GFP_ATOMIC
would avoid sleeping, but a failure fallback would make the latency
improvement opportunistic instead of guaranteed.
The users are not limited to a small, pre-identifiable class of tasks. On
x86, ordinary tasks can reach this path through TLB flushes during exit,
unmap and reclaim, so allocating the mask only for a known subset of
tasks is not straightforward.
The memory cost is explicit: one word is added to task_struct. When
cpumask_size() fits in that word, the mask is stored inline and no
separate allocation is needed. Larger systems allocate cpumask_size() per
task; on x86-64 NR_CPUS=8192 this is 1 KiB per task. For context,
x86 already carries several KiB of per-task architecture and FPU state,
depending on the enabled features and configuration. That does not make
the extra cpumask free, but it puts the large-NR_CPUS case in
perspective.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-5-zhouchuyi@bytedance.com
|
|
Removing struct uprobe_trampoline object and it's tracking code,
because it's not needed. We can do same thing directly on top of
struct vm_area_struct objects.
This makes the code simpler and allows easy propagation of the
trampoline vma object into child process in following change.
Note the original code called destroy_uprobe_trampoline if the
optimiation failed, but it only freed the struct uprobe_trampoline
object, not the vma. The new vma leak is fixed in following change.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://patch.msgid.link/20260703114917.238144-3-jolsa@kernel.org
|
|
Currently, copy_process() can bail out to free_task() before p->bpf_storage
has been initialized, with this call graph (shown here for the
!CONFIG_MEMCG case):
copy_process
dup_task_struct
arch_dup_task_struct
[copies the entire task_struct, including ->bpf_storage member]
[RLIMIT_NPROC check fails]
delayed_free_task
free_task
bpf_task_storage_free
rcu_dereference(task->bpf_storage)
bpf_local_storage_destroy
In this case, the nascent task's ->bpf_storage member that
bpf_local_storage_destroy() operates on is a plain copy of the parent's
->bpf_storage pointer, not a real initialized pointer.
This leads to badness (kernel hangs, UAF).
This is reachable as long as the process calling fork() has been inserted
into a task storage map.
Cc: stable@kernel.org
Fixes: a10787e6d58c ("bpf: Enable task local storage for tracing programs")
Signed-off-by: Jann Horn <jannh@google.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
|
|
Add a umh field to struct kernel_clone_args. When set, copy_fs() copies
from pid 1's fs_struct instead of the kthread's fs_struct. This ensures
usermodehelper threads always get init's filesystem state regardless of
their parent's (kthreadd's) fs.
Usermodehelper threads are not allowed to create mount namespaces
(CLONE_NEWNS), share filesystem state (CLONE_FS), or be started from
a non-initial mount namespace. No usermodehelper currently does this so
we don't need to worry about this restriction.
Set .umh = 1 in user_mode_thread(). At this stage pid 1's fs points to
rootfs which is the same as kthreadd's fs, so this is functionally
equivalent.
Link: https://patch.msgid.link/20260601-work-kthread-nullfs-v4-20-77ee053060e0@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Add a real_fs field to task_struct that always mirrors the fs field.
This lays the groundwork for distinguishing between a task's permanent
fs_struct and one that is temporarily overridden via scoped_with_init_fs().
When a kthread temporarily overrides current->fs for path lookup, we
need to know the original fs_struct for operations like exit_fs() and
unshare_fs_struct() that must operate on the real, permanent fs.
For now real_fs is always equal to fs. It is maintained alongside fs in
all the relevant paths: exit_fs(), unshare_fs_struct(),
switch_fs_struct(), and copy_fs().
Link: https://patch.msgid.link/20260601-work-kthread-nullfs-v4-4-77ee053060e0@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Don't open-code the guts of replacing current's fs struct.
Link: https://patch.msgid.link/20260601-work-kthread-nullfs-v4-1-77ee053060e0@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block fixes from Jens Axboe:
- blk-cgroup locking rework and fixes:
- fix a use-after-free in __blkcg_rstat_flush()
- defer freeing policy data until after an RCU grace period
- defer the blkcg css_put until the blkg is unlinked from
the queue
- unwind the queue_lock nesting under RCU / blkcg->lock
across the lookup, create, associate and destroy paths
- NVMe fixes via Keith:
- Fix a crash and memory leak during invalid cdev teardown,
and related cdev cleanups (Maurizio, John)
- nvmet fixes: handle TCP_CLOSING in the tcp state_change
handler, reject short AUTH_RECEIVE buffers, handle inline
data with a nonzero offset in rdma, fix an sq refcount leak,
and allocate ana_state with the port (Maurizio, Michael,
Bryam, Wentao, Rosen)
- nvme-fc fix to not cancel requests on an IO target before it
is initialized (Mohamed)
- nvme-apple fix to prevent shared tags across queues on Apple
A11 (Nick)
- Various smaller fixes and cleanups (John)
- MD fixes via Yu Kuai:
- raid1/raid10 fixes for writes_pending and barrier reference
leaks on write and discard failures, plus REQ_NOWAIT handling
fixes (Abd-Alrhman)
- raid5 discard accounting and validation, and a batch of fixes
for stripe batch races (Yu Kuai, Chen)
- Protect raid1 head_position during read balancing (Chen)
- block bio-integrity fixes: correct an error injection static key
decrement, fix GFP flag confusion in bio_integrity_alloc_buf(), and
handle REQ_OP_ZONE_APPEND in __bio_integrity_action() (Christoph)
- Fixes for bio_iov_iter_bounce_write(): revert the iov_iter after a
short copy, and respect the iov_iter nofault flag (Qu)
- Invalidate the cached plug timestamp after a task switch, and clear
PF_BLOCK_TS in copy_process() (Usama)
- Fix the IORING_URING_CMD_REISSUE flags check in blkdev_uring_cmd()
(Yitang)
- Remove a redundant plug in __submit_bio() (Wen)
- Don't warn when reclassifying a busy socket lock in nbd (Deepanshu)
* tag 'block-7.2-20260625' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (45 commits)
block: handle REQ_OP_ZONE_APPEND in __bio_integrity_action
block: fix GFP_ flags confusion in bio_integrity_alloc_buf
block, bfq: don't grab queue_lock to initialize bfq
mm/page_io: don't nest queue_lock under rcu in bio_associate_blkg_from_page()
blk-cgroup: don't nest queue_lock under blkcg->lock in blkcg_destroy_blkgs()
blk-cgroup: don't nest queue_lock under rcu in bio_associate_blkg()
blk-cgroup: don't nest queue_lock under rcu in blkg_lookup_create()
blk-cgroup: don't nest queue_lock under rcu in blkcg_print_blkgs()
blk-cgroup: delay freeing policy data after rcu grace period
blk-cgroup: protect iterating blkgs with blkcg->lock in blkcg_print_stat()
md/raid5: avoid R5_Overlap races while breaking stripe batches
md/raid5: use stripe state snapshot in break_stripe_batch_list()
blk-cgroup: defer blkcg css_put until blkg is unlinked from queue
blk-cgroup: fix UAF in __blkcg_rstat_flush()
block, bfq: protect async queue reset with blkcg locks
nbd: don't warn when reclassifying a busy socket lock
block: fix incorrect error injection static key decrement
md/raid5: let stripe batch bm_seq comparison wrap-safe
md/raid1: protect head_position for read balance
md/raid1: free r1_bio when REQ_NOWAIT is set and read would block on retry
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull MM updates from Andrew Morton:
- "selftests/mm: clean up build output and verbosity" (Li Wang)
Remove some noise from the MM selftests build
- "mm: Free contiguous order-0 pages efficiently" (Ryan Roberts)
Speed up the freeing of a batch of 0-order pages by first scanning
them for coalescing opportunities. This is applicable to vfree() and
to the releasing of frozen pages
- "mm/damon: introduce DAMOS failed region quota charge ratio"
(SeongJae Park)
Address a DAMOS usability issue: The DAMOS quota often exhausts
prematurely because it charges for all memory attempted, causing slow
and inconsistent performance when actions fail on unreclaimable
memory.
To fix this, a new feature lets users set a smaller, flexible quota
charge ratio (via a numerator and denominator) for failed regions.
Since failed actions cause less overhead, reducing their quota cost
ensures more predictable and efficient DAMOS processing
- "selftests/cgroup: improve zswap tests robustness and support large
page sizes" (Li Wang)
Fix various spurious failures and improves the overall robustness of
the cgroup zswap selftests
- "fix MAP_DROPPABLE not supported errno" (Anthony Yznaga)
Fix an issue in the mlock selftests on arm32
- "mm: huge_memory: clean up defrag sysfs with shared" (Breno Leitao)
Some maintenance work in the huge_memory code
- "treewide: fixup gfp_t printks" (Brendan Jackman)
Use the special vprintf() gfp_t conversion in various places
- "mm: Fix vmemmap optimization accounting and initialization" (Muchun
Song)
Fix several bugs in the vmemmap optimization, mainly around incorrect
page accounting and memmap initialization in the DAX and memory
hotplug paths. It also fixes pageblock migratetype initialization and
struct page initialization for ZONE_DEVICE compound pages
- "mm/damon: repost non-hotfix reviewed patches in damon/next tree"
A sprinkle of unrelated minor bugfixes for DAMON
- "mm: remove page_mapped()" (David Hildenbrand)
Remove this function from the tree, replacing it with folio_mapped()
- "mm/damon: let DAMON be paused and resumed" (SeongJae Park)
Allow DAMON to be paused and resumed without losing its current state
- "kasan: hw_tags: Disable tagging for stack and page-tables" (Muhammad
Usama Anjum)
Simplify and speed up kasan by removing its ineffective tagging of
stacks and page tables
- "mm/damon/reclaim,lru_sort: monitor all system rams by default"
(SeongJae Park)
Simplify deployment on diverse hardware like NUMA systems by updating
DAMON_RECLAIM and DAMON_LRU_SORT to automatically monitor the
physical address range covering all System RAM areas by default,
replacing the overly restrictive behavior that only targeted the
single largest memory block to save on negligible overhead
- "mm/damon/sysfs: document filters/ directory as deprecated" (SeongJae
Park)
Update some DAMON docs
- "mm: use spinlock guards for zone lock" (Dmitry Ilvokhin)
Switch zone->lock handling over to using the guard() mechanisms
- "mm/filemap: tighten mmap_miss hit accounting" (fujunjie)
Fix a flaw where the mmap_miss counter over-credited page cache hits
during fault-arounds and page-fault retries. This results in
significant reduction of redundant synchronous mmap readahead I/O,
drastically cutting down execution time and gigabytes read for sparse
random or strided memory access workloads
- "selftests/cgroup: Fix false positive failures in test_percpu_basic"
(Li Wang)
Fix a couple of false-positives in the cgroup kmem selftests
- "mm/damon/reclaim: support monitoring intervals auto-tuning"
(SeongJae Park)
Add a new parameter to DAMON permitting DAMON_RECLAIM to
automatically tune DAMON's sampling and aggregation intervals
- "mm/damon/stat: add kdamond_pid parameter" (SeongJae Park)
Change DAMON_STAT to provide the pid of its kdamond
- "mm/kmemleak: dedupe verbose scan output" (Breno Leitao)
Remove large amounts of duplicated backtraces from the verbose-mode
kmemleak output
- "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part 1)" (David
Hildenbrand)
Reduce our use of CONFIG_HAVE_BOOTMEM_INFO_NODE, with a view to
removing it entirely in a later series
- "mm/damon: validate min_region_size to be power of 2" (Liew Rui Yan)
Prevent users from passing a non-power-of-2 value of `addr_unit', as
this later results in undesirable behavior
- "mm: document read_pages and simplify usage" (Frederick Mayle)
- "tools/mm/page-types: Fix misc bugs" (Ye Liu)
Fix three issues in tools/mm/page-types.c
- "mm: misc cleanups from __GFP_UNMAPPED series" (Brendan Jackman)
Implement several cleanups in the page allocator and related code
- "mm, swap: swap table phase IV: unify allocation" (Kairui Song)
Unify the allocation and charging of anon and shmem swap in folios,
provides better synchronization, consolidates the metadata
management, hence dropping the stat |
