| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/modules/linux
Pull module updates from Petr Pavlu:
- Ignore local labels and mapping symbols during module load
These symbols were already ignored during kallsyms address-to-symbol
lookups. Filter them out earlier during module loading to reduce
lookup overhead and prevent them from appearing in /proc/kallsyms.
- Fix potentially incorrect access of SetOnce<T> when setting Rust
module parameters.
- Fix a potential read of an uninitialized count when printing taint
information for unloaded modules.
- Allocate the kmod argument array using kmalloc_array() in preparation
for conversion to type-aware allocators.
- Clean up includes in linux/kmod.h
Add explicit includes of linux/umh.h to files that use usermode
helpers. Then clean up the #include list in linux/kmod.h, notably
removing the compatibility linux/umh.h include.
- Remove the error-injection.h include from linux/module.h
Add explicit includes of linux/error-injection.h to files that use
the ALLOW_ERROR_INJECTION() macro and currently rely on it being
provided indirectly via linux/module.h. Then remove the unnecessary
linux/error-injection.h include from linux/module.h.
- Replace __MODULE_STRING() with __stringify() in various drivers in
preparation for removing the former macro.
* tag 'modules-7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/modules/linux:
drivers: Replace __MODULE_STRING() with __stringify()
module: Fix uninitialized count read of unloaded taint tracking record
rust: module_param: Fix potentially incorrect access of `SetOnce<T>`
module: Remove the error-injection.h include from linux/module.h
drm/xe: Include error-injection.h for ALLOW_ERROR_INJECTION()
drm/i915: Include error-injection.h for ALLOW_ERROR_INJECTION()
syscalls: Include error-injection.h for ALLOW_ERROR_INJECTION()
net: Include error-injection.h for ALLOW_ERROR_INJECTION()
block: Include error-injection.h for ALLOW_ERROR_INJECTION()
module: Bring includes in linux/kmod.h up to date
time/jiffies: Include linux/sysctl.h for proc_int_u2k_conv_uop(), ...
umh, treewide: Explicitly include linux/umh.h where needed
modules/kmod: Allocate argv with kmalloc_array()
module/kallsyms: Ignore local labels and mapping symbols during module load
module/kallsyms: Refactor current symbol filtering into is_core_symbol()
module/kallsyms: Rename is_mapping_symbol() to is_ignored_kernel_symbol()
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull VDSO updates from Thomas Gleixner:
"Updates for the VDSO subsystem:
- Ensure that the CLOCK_AUX base calculations are properly scaled
against the base timekeeper. The update function failed to take the
fractional nanosecond part into account, which results in a
rounding error.
- Cleanup and consolidate the vsyscall headers to get rid of
pointless copies all over the architectures.
- Replace a runtime conditional in the time getter function for
namespaced clocks by simple pointer arithmetic which speeds up the
read function"
* tag 'timers-vdso-2026-10-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
vdso/vsyscall: Keep the CLOCK_AUX base scaled
vdso/math64: Add and use __iter_div64_u64_rem()
vdso/math64: Use OPTIMIZER_HIDE_VAR() in __iter_div_u64_rem()
vdso/gettimeofday: Avoid runtime conditional in do_hres_timens()
vdso: Remove __ASSEMBLER__ guards from vsyscall.h
vdso: Add asm-generic/vdso/vsyscall.h
vdso: Remove unnecessary vdso/vsyscall.h includes
riscv/hwprobe: Include vdso/datapage.h
arm64/vdso: Delete asm/vdso/vsyscall.h inclusions
x86/vdso: Delete asm/vdso/vsyscall.h inclusions
sparc/vdso: Delete asm/vdso/vsyscall.h inclusion
s390/vdso: Delete asm/vdso/vsyscall.h inclusions
sparc/vdso: Move __VDSO_PAGES to asm/vdso.h
x86/vdso: Move vDSO page definitions to asm/vdso.h
vdso: Delete asm-generic/vdso/vsyscall.h
vdso/vsyscall: Define fallback time synchronization functions in vsyscall.c
vdso/getrandom: Define fallback __arch_get_vdso_u_rng_data() in getrandom.c
vdso/gettimeofday: Define fallback __arch_get_vdso_u_time_data() in gettimeofday.c
vdso: Move the kernel-side time update helpers to a new header
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull NOHZ updates from Thomas Gleixner:
"Updates for the tick/NOHZ machinery:
- Add the BLOCK_SOFTIRQ to the hotplug safe local softirq pending
mask to prevent false positive error messages. A BLOCK_SOFTIRQ can
be pending when a CPU goes offline as the subsequent cleanup will
take care of the outstanding completions.
- Remove a few redundant invocations of local_irq_save/restore() and
timekeeping_max_deferment() to speed up the NOHZ idle transitions"
* tag 'timers-nohz-2026-10-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
tick/nohz: Avoid unused timekeeping_max_deferment() calls
tick/nohz: Remove redundant local_irq_save()/restore()
tick/nohz: Add BLOCK_SOFTIRQ to the hotplug safe mask
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer core updates from Thomas Gleixner:
"Updates for the core timekeeping and timers subsystem:
- Futher cleanups for the POSIX timer exit/exec related code to
simplify the code flow and to make it more robust against future
changes.
- Prevent tearing of timekeeper::ktime_sec which can be accessed
without the sequence count protection.
The timekeeper is updated by memcpy() with an uneven sequence count
to force readers into the retry path. On 64-bit machine the
ktime_sec member can be accessed atomically without the sequence
count protection. That is only correct when the update side
memcpy() guarantees atomicity at the 64-bit granularity. That's not
guaranteed due to debug options, compilers and instruction
availability. If the memcpy() does not provide the atomicity then
the unprotected readers can observe store tearing.
To prevent this write the ktime_sec member before the memcpy() so
that a non-atomic memcpy() will write the same data. A similar fix
has been done for the xtime_sec member, but the ktime_sec
equivalent was missed.
- Prevent time64_to_tm() day truncation on 32-bit kernels
The function receives a 64-bit seconds value but stores the
quotient of the computation in a long, which truncates the result
for large input values. Cure this by using div_s64_rem() which
stores the quotient in s64 and only leaves the seconds-within-day
reminder in a long, which is guaranteed to fit. This fix comes with
a corresponding kunit test to prevent further mishaps.
- Mark data-racy accesses to the hrtimer_sleeper::task member.
The recent updates to KCSAN flagged these benign races. Annotate
them with new helper functions, convert all existing usage sites
and mark the task member private after that so that future users
which fail to use the helpers can be detected automatically.
- Address similar benign KCSAN flagged issues in the hrtimer, timer
and timer migration code.
- Fixes for a few selftest issues"
* tag 'timers-core-2026-10-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (26 commits)
selftests/timers: clocksource-switch: Fix unchecked open()/read()
selftests: timers: Measure the CPU timers on the clock they count
selftests: timers: Count what tick is worth in the drift estimate
time/kunit: Add time64_to_tm() case beyond 32-bit day count
time: Prevent time64_to_tm() day truncation on 32-bit
timekeeping: Use READ_ONCE/WRITE_ONCE() for ktime_sec to prevent tearing
timers/migration: Mark racy updates to tmigr_event::ignore field
timers: Mark racy updates to hlist_node::pprev field
hrtimer: Apply READ_ONCE() to lockless base->running loads
hrtimer: Mark the hrtimer_sleeper structure's ->task field __private
hrtimer: Update hrtimer_resolution only if value changes
futex: Use accessor for hrtimer_sleeper ->task field in requeue
rtmutex: Use accessor for hrtimer_sleeper ->task field
net: pktgen: Use accessor for hrtimer_sleeper ->task field
timers: Use accessor for hrtimer_sleeper ->task field in sleep_timeout.c
futex: Use accessor for hrtimer_sleeper ->task field in waitwake.c
io-uring/rw: Use accessor for hrtimer_sleeper ->task field
wait: Use accessor for hrtimer_sleeper ->task field
aio: Use accessor for hrtimer_sleeper ->task field
hrtimer: Mark data-racy accesses to hrtimer_sleeper ->task field
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler updates from Ingo Molnar:
"Kconfig updates:
- Simplify PREEMPT_DYNAMIC (Mark Rutland)
Fair scheduler updates:
- Remove redundant RUN_TO_PARITY check (Elif Topuz)
- Declare augmented callbacks per field, fix
rb_add_augmented_cached() descent (Yiwei Lin, Peter Zijlstra)
- Restart fair hrtick after same-task repicks (Shubhang Kaushik)
- Improve the scheduling latency of short slice tasks (Vincent
Guittot)
Deadline/RT scheduler updates:
- Check start_dl_timer expiry with ktime_before() (Liang Hao)
- irq_work: CPU-hotplug improvements on PREEMPT_RT (Sebastian Andrzej
Siewior)
Energy-aware scheduling updates:
- Implement the steal_governor, introduce preferred CPUs and
steal-driven vCPU backoff (Shrikanth Hegde)
Multi-processor load-balancing updates:
- Enable preferred SMT siblings on NVIDIA Olympus (Andrea Righi)
- Randomize equally shallow idle CPU picks (Christian Loehle)
- Remove redundant core_sched_seq (Hui Su)
Cache-aware scheduling updates:
- Avoid publishing partial LLC utilization (Jemmy Wong)
NUMA-balancing updates:
- Reset NUMA fault locality after scan period update (Eric Kim)
Proxy execution updates:
- Ongoing work to implement sleeping owner handling for proxy
execution (John Stultz, Vasily Gorbik)
- Preparatory steps to make sched_ext compatible with proxy execution
(Andrea Righi)
Scheduler infrastructure updates & fixes:
- Remove the RCU wait from PSI trigger destruction (Shakeel Butt)
- Avoid modifying wait-entry on failed priority registration
(Hui Su)
- Use the new static key API for sched_feat (Hongyan Xia)
- Convert paravirt_steal to new static key APIs (Hongyan Xia)
- cputime: Don't account idle time twice after dyntick-idle
(Stian Halseth)
Scheduler debugging updates:
- Set TIF_NEED_RESCHED before calling __trace_set_need_resched()
(Sechang Lim)
- Fix context analysis errors in non-preferred CPU push (Shrikanth
Hegde)
Misc cleanups and fixes by Christian Loehle, Shubhang Kaushik,
Thomas Huth and Vladimir Zapolskiy"
* tag 'sched-core-2026-10-18' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (64 commits)
sched: Replace nr_iowait_cpu() with a bool accessor
Revert "sched/fair: Remove dead code on enqueue_task_fair()"
sched/cache: Avoid publishing partial LLC utilization
sched/cputime: Don't account idle time twice after dyntick-idle
sched/psi: Remove the RCU wait from trigger destruction
sched/eevdf: Remove redundant RUN_TO_PARITY check
sched/wait: Avoid modifying entry on failed priority registration
rbtree_test: Use rb_add_augmented_cached() for the cached augmented test
rbtree: Update augmented data on the way down in rb_add_augmented_cached()
rbtree: Declare augmented callbacks per field with RB_AUG()
rbtree_test: Use rb_add() and rb_add_cached() for the basic tests
smpboot: Don't park the thread if work is pending
irq_work: Flush lazy work CPU down on PREEMPT_RT
irq_work: Update a comment regarding CPU hotplug invocation
sched/topology: Add asymmetric SMT packing override
sched/fair: Honor asymmetric SMT priority in idle selection
sched: Set TIF_NEED_RESCHED before calling __trace_set_need_resched()
sched/core: Fix context analysis errors in non-preferred CPU push
virt/steal_governor: Enable the driver
virt/steal_governor: Implement steal_governor policy loop
...
|
|
On idle exit the dyntick-idle accounting accounts the time up to now,
then the tick is restarted on its old period. The first tick accounts a
whole TICK_NSEC to whatever runs, although the part of that period
before the idle exit has just been accounted as idle time, or with
IRQ_TIME_ACCOUNTING as IRQ time. That is up to a full tick per idle
exit, and /proc/stat reports more idle time than wall time.
Record how much of the current tick period dyntick-idle has accounted,
and leave it out of the tick that ends the period.
Fixes: cf6444c3e1bb7 ("tick/sched: Unify idle cputime accounting")
Signed-off-by: Stian Halseth <stian@itx.no>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://lore.kernel.org/all/20261004142724.3896396-1-stian@itx.no/
Link: https://patch.msgid.link/20261007142950.547604-1-stian@itx.no
|
|
The vDSO basetime of a clock is stored in the scaled nanoseconds of
tkr_mono, so that the reader can floor the base and the cycle delta
together in vdso_calc_ns().
vdso_time_update_aux() instead shifts the base down to nanoseconds, adds
the offset, and shifts it back up, which zeroes the fractional nanoseconds
of xtime_nsec. The reader then floors the base and the delta separately:
ktime_get_aux(): base + ((delta * mult + xtime_nsec) >> shift)
vdso: base + (xtime_nsec >> shift)
+ ((delta * mult) >> shift)
Since floor(a) + floor(b) <= floor(a + b), the vDSO reports 0 or 1 ns
below the syscall for the same clock. It is not a monotonicity problem:
across an update the step is floor(a + d) - floor(a) - floor(d), which is
0 or 1, never negative.
Add the offset in scaled nanoseconds as the other high resolution clocks
do, and normalise with __iter_div64_u64_rem() so that the stored base
stays below one second and the userspace fast-path does not iterate more
in __iter_div_u64_rem().
Only the sub-second field changes. (a + (b << shift)) >> shift is exactly
(a >> shift) + b, so the seconds carried out of the normalisation are the
same as before; what the old form dropped was the low shift bits of the
remainder.
monotonic_to_aux.tv_nsec is a normalised timespec64 fraction, so it stays
below NSEC_PER_SEC even for a negative offset, and the sum stays below
2 * (NSEC_PER_SEC << shift). The largest shift clocks_calc_mult_shift()
can pick is 32, which makes that 8.6e18 against a u64 limit of 1.8e19.
Fixes: 380b84e168e5 ("vdso/vsyscall: Update auxiliary clock data in the datapage")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20261002153104.1347978-4-zhanxusheng@xiaomi.com
|
|
The vDSO basetimes for CLOCK_MONOTONIC and CLOCK_BOOTTIME are stored in
the scaled nanoseconds of tkr_mono, so normalising them requires a
division by NSEC_PER_SEC << shift. That divisor does not fit the u32
parameter of __iter_div_u64_rem(), so update_vdso_time_data() open-codes
the same iterative division twice.
Repeated subtraction is the appropriate form at these two sites because
the quotient never exceeds one. accumulate_nsecs_to_secs() keeps
xtime_nsec below one scaled second, and the offset added to it is a
normalised timespec64 fraction, so the dividend stays below twice the
divisor. That bound holds on every architecture, which matters more here
than the cost of a division on any particular one.
Add __iter_div64_u64_rem(), the u64-divisor counterpart of
__iter_div_u64_rem(), and use it at both sites. Return the quotient as a
u32 for consistency with the u32-divisor variant; the bound above leaves
no use for a wider type.
Store the remainder directly into the basetime, as the coarse clocks
already do. The CLOCK_BOOTTIME copy of the CLOCK_MONOTONIC values then
takes both fields from the same place.
Folding the two open-coded loops into one inlined helper removes 16 bytes
of vsyscall.o text on x86-64 with gcc 13.
No functional change.
Suggested-by: David Laight <david.laight.linux@gmail.com>
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20261002153104.1347978-3-zhanxusheng@xiaomi.com
|
|
__arch_update_vdso_clock() and __arch_sync_vdso_time_data() are only
used in vsyscall.c. vsyscall.h has become a dumping ground for different
kinds of fallbacks.
Move the fallback function definitions into vsyscall.c.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260909-vdso-asm-generic-vsyscall-h-v1-4-63ae856cc659@linutronix.de
|
|
The vdso/ header namespace is meant for symbols which are usable from
vDSO userspace code. However vdso_update_begin() and vdso_update_end()
are only usable from the kernel.
Move them into a new header in the linux/ namespace. Also rename them to
make clear that they are only for time-related data.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260909-vdso-asm-generic-vsyscall-h-v1-1-63ae856cc659@linutronix.de
|
|
The existing date range test covers +/-80,000 years around the epoch,
far below the day count at which a 32-bit long wraps (2^31 days,
roughly 5.88 million years).
Add a case at exactly 2^31 days after the epoch. Without the preceding
fix a 32-bit kernel returns tm_year=-5879541 and wrong month, day,
day-of-year and weekday; with it the result matches 64-bit kernels.
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: LLM
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20260925165702.51677-3-kmehltretter@gmail.com
|
|
time64_to_tm() accepts a 64-bit seconds value, but stores the quotient in
long. On a 32-bit kernel the day count therefore wraps when it reaches
2^31, even though the corresponding year remains representable in
struct tm.
This is reachable when formatting externally supplied timestamps. For
example, NILFS recovery copies the little-endian 64-bit ss_create field
from an on-disk segment summary into time64_t. Its sysfs attributes then
print that value with %ptTs, whose formatter calls time64_to_tm(). A
corrupted image can consequently produce an architecture-dependent printed
date.
Keep the day quotient in s64 while leaving the seconds-within-day remainder
as long. Use div_s64_rem() for the weekday calculation so 32-bit builds do
not require compiler runtime division helpers.
Fixes: e6c2682a1da3 ("time: Add time64_to_tm()")
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: LLM
Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Link: https://patch.msgid.link/20260925165702.51677-2-kmehltretter@gmail.com
|
|
The timekeeper update path uses a bulk memcpy() to synchronize the
timekeeper structure, which is not guaranteed to be atomic. This allows for
torn reads in ktime_get_seconds() which bypasses the sequence counter
protection for performance.
To prevent reading a torn ktime_sec value, enforce atomic-like
access by using WRITE_ONCE() for the critical field before the bulk
memcpy() in timekeeping_update_from_shadow(). Correspondingly, use
READ_ONCE() in ktime_get_seconds() to ensure a fresh, consistent load
from memory.
The same was done for xtime_sec and ktime_get_real_seconds() in commit
d7fc133bf91f ("timekeeping: Use READ_ONCE/WRITE_ONCE() for xtime_sec to
prevent tearing").
Signed-off-by: Thomas Weißschuh (Schneider Electric) <thomas.weissschuh@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260903-timekeeping-ktime_sec-v1-1-b33017536aa9@linutronix.de
|
|
The tmigr_event structure's ignore field is sometimes accessed locklessly,
but the __tmigr_cpu_activate(), tmigr_cpu_new_timer(), and
__tmigr_cpu_deactivate() functions do not mark accesses to this field.
Therefore, use READ_ONCE() for the tmigr_cpu_new_timer() function's
lockless load and WRITE_ONCE() for the __tmigr_cpu_activate() and
__tmigr_cpu_deactivate() functions' stores.
KCSAN located this issue.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260919002523.3133928-2-paulmck@kernel.org
|
|
hlist_unhashed_lockless() locklessly samples the hlist_node structure's
pprev field, but detach_timer(), hlist_move_list(), and hlist_splice_init()
all use plain C-language stores to update this field, despite the "The
READ_ONCE() is paired with the various WRITE_ONCE() in hlist helpers that
are defined below" in the hlist_unhashed_lockless() header comment.
Therefore use WRITE_ONCE() for these hlist_node::pprev updates.
KCSAN located this issue.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260919002523.3133928-1-paulmck@kernel.org
|
|
Updates to base->running are protected by the hrtimer base lock, but some
loads are lockless. Therefore, prevent compiler mischief by applying
READ_ONCE() to the lockless loads.
KCSAN located this issue.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260919001428.3133388-12-paulmck@kernel.org
|
|
Currently, the hrtimer_switch_to_hres() function unconditionally updates
hrtimer_resolution to HIGH_RES_NSEC. This causes KCSAN false positives
in kernels built with CONFIG_KCSAN_STRICT=y. It also causes unnecessary
cache-line traffic on the many code paths that load hrtimer_resolution,
which is especially regrettable because the value of hrtimer_resolution
normally changes only once during the lifetime of a given kernel.
Therefore, update the value of hrtimer_resolution to HIGH_RES_NSEC only
if it does not already have that value.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Link: https://patch.msgid.link/20260919001428.3133388-10-paulmck@kernel.org
|
|
The hrtimer_sleeper structure's ->task field is used as a flag to indicate
that the associated hrtimer has expired. This means that the hrtimer
handler can be storing to this field while other code is loading from it
to check for expiry. Note that additional races appear for hrtimers that
can be restarted, which could be argued to be a user error. However, that
is no reason to let the compiler introduce additional confusion, and to
this end, the hrtimer_sleeper_task_get() was introduced, use of which also
has the benefit of avoiding open-code access to hrtimer_sleeper innards.
Therefore, apply this accessor to schedule_hrtimeout_range_clock().
KCSAN located this issue.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Dmitry Ilvokhin <d@ilvokhin.com>
Link: https://patch.msgid.link/20260919001428.3133388-6-paulmck@kernel.org
|
|
The hrtimer_sleeper structure's ->task field is used as a flag to indicate
that the associated hrtimer has expired. This means that the hrtimer
handler can be storing to this field while other code is loading from it
to check for expiry. Note that additional races appear for hrtimers that
can be restarted, which could be argued to be a user error. However,
that is no reason to let the compiler introduce additional confusion.
Therefore, mark data-racy accesses to the hrtimer_sleeper ->task field
using READ_ONCE() (using a new hrtimer_sleeper_task_get() access function)
and WRITE_ONCE() (using a new hrtimer_sleeper_task_set() access function).
KCSAN located this issue.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Dmitry Ilvokhin <d@ilvokhin.com>
Link: https://patch.msgid.link/20260919001428.3133388-1-paulmck@kernel.org
|
|
tick_nohz_next_event() limits a CPU's sleep interval to the maximum
deferment supported by the current clocksource when that CPU owns the
do_timer() duty. If the duty is unassigned, the limit also applies when
the CPU's TS_FLAG_DO_TIMER_LAST flag is set.
After the early timer checks, the function currently reads the maximum
deferment unconditionally. It then replaces the result with KTIME_MAX
unless one of the two conditions above applies.
timekeeping_max_deferment() performs a seqcount-protected read of the
shared timekeeper and follows its clocksource pointer. Check the do_timer
state first and avoid this work when the result would be discarded. This
leaves the resulting expiry unchanged and reduces accesses to timekeeper
data that is modified regularly.
On x86-64 this removes 18-20 dynamically executed instructions, including
the call, from the common non-owner path when the seqcount does not retry.
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Rik van Riel <riel@surriel.com>
Link: https://patch.msgid.link/20260918133408.2834751-1-usama.arif@linux.dev
|
|
tick_nohz_update_jiffies() is only reached through tick_irq_enter() from
hard interrupt entry, where local interrupts are already disabled.
tick_do_update_jiffies64() requires interrupts to be disabled. The
local_irq_save()/local_irq_restore() pair around it therefore provides no
additional interrupt exclusion.
Remove it and assert the caller invariant instead.
On x86-64, this avoids the redundant RFLAGS save, CLI, saved-IF test and
conditional branch on the tick-stopped path.
On a 252-CPU host running a Thrift RPC service, tick_irq_enter() ran
165,116 times/s out of 212-244k interrupts/s. Only calls with
TS_FLAG_STOPPED reach the optimized path, so this is an upper bound on
its execution rate.
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://patch.msgid.link/20260916141530.3423310-1-usama.arif@linux.dev
|
|
Return ULONG_MAX for values that don't fit an unsigned long.
proc_int_u2k_conv_uop() now correctly rejects a result above INT_MAX.
This is the erroneous behaviour that is being fixed. The input has to
exceed ULONG_MAX / HZ for the product to wrap, so the value below is
specific to CONFIG_HZ=1000:
# echo 18446744073709552 > /proc/sys/net/ipv4/tcp_keepalive_time
# cat /proc/sys/net/ipv4/tcp_keepalive_time
0
That value is now rejected with an error.
The original bound ("*u_ptr > INT_MAX / HZ") was removed in commit
2dc164a48e6f ("sysctl: Create converter functions with two new macros").
Fixes: 2dc164a48e6f ("sysctl: Create converter functions with two new macros")
Cc: stable@vger.kernel.org
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
jiffies.c uses proc_int_u2k_conv_uop(), proc_int_k2u_conv_kop(),
proc_int_conv(), proc_dointvec_conv() and proc_doulongvec_minmax_conv(),
which are declared in linux/sysctl.h. It currently gets that header
indirectly through linux/module.h -> linux/kmod.h, but the latter no longer
needs to include linux/sysctl.h.
Include linux/sysctl.h directly in jiffies.c ahead of removing the
unnecessary include from linux/kmod.h.
Acked-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer race fixes from Ingo Molnar:
- Fix timer signal <-> exec() race, to prevent UAF (Thomas Gleixner)
- Clean up POSIX CPU timers right after de_thread(), to prevent UAF
(Hyunwoo Kim)
- Fix POSIX CPU timers race between expiry and timer_settime(),
to prevent UAF (Thomas Gleixner)
* tag 'timers-urgent-2026-09-20' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
posix-cpu-timers: Prevent freeing a timer which is queued on the expiry list
exec: Cleanup POSIX timers right after de_thread()
signal: Prevent exec() race
|
|
Kijo analyzed another race in the POSIX CPU timer code:
Commit bf635681c906 converted cpu_timer::firing from a tristate value to a
boolean. This lost the distinction between "not owned by the firing list"
and "still owned, but delivery was canceled". The resulting race is:
expiry handler timer_settime() timer_delete()
-------------- --------------- --------------
collect timer onto
private firing list
firing = true
observes firing = true
firing = false
return TIMER_RETRY
wait for handler
observes firing = false
finish deletion
unhash and free timer
resume list traversal
read freed elist.next
-> UAF
The firing bit is clearly the wrong indicator since that commit.
Check whether the timer is queued on the expiry list or not instead. If it
is queued clear the firing bit to prevent signal delivery as before and
return TIMER_RETRY so the caller unlocks the timer which allows the expiry
code to make progress and remove it from the list.
Fixes: bf635681c906 ("posix-cpu-timers: Cleanup the firing logic")
Reported-by: Kijo Park <red993688@gmail.com>
Debugged-by: Kijo Park <red993688@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Cc: stable@vger.kernel.org
|
|
Now that POSIX CPU timers cannot be enqueued on a task after PF_EXITING is
set and process wide timers cannot be enqueued when PF_EXITING is set and
the last thread in the group is exiting, it is possible to mop up POSIX
timers in do_exit() completely.
This requires to cancel an eventually pending POSIX CPU timer task work
right there because do_exit() invokes exit_task_work() later, which would
be acting on torn down data.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://patch.msgid.link/20260911090541.947206186@kernel.org
|
|
To prepare for cleaning up POSIX CPU timers in do_exit(), prevent
enqueueing POSIX CPU timers on a task which has PF_EXITING set. Queueing a
timer on such a task is pointless because the task won't expire the timer
anymore.
The same applies to process wide timers when tsk::signal::flags has
SIGNAL_GROUP_EXIT set.
Pretending that the timer is armed allows to keep the POSIX timer mechanism
"working" so that the timer stays accessible up to the point where a task
is unhashed.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://patch.msgid.link/20260911090541.892445443@kernel.org
|
|
Right now POSIX CPU timers use task::exit_state to check whether a task is
exiting. That works correctly, but exit_state is set later in do_exit() and
too late for allowing to cleanup POSIX timers earlier.
It does not matter in case of exit whether the cutoff is a bit earlier.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://patch.msgid.link/20260911090541.837831386@kernel.org
|
|
They are only used in the POSIX CPU timer code. No point in exposing them
globally and parsing them for nothing.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260911090541.785749315@kernel.org
|
|
Move the POSIX timer group exit handling into the posix timer code and
provide a proper stub when POSIX timers are disabled in Kconfig.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260911090541.733966325@kernel.org
|
|
Move it to the POSIX timer code and provide a proper stub when POSIX timers
are disabled in Kconfig.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260911090541.680668094@kernel.org
|
|
Return MAX_JIFFY_OFFSET for all the values truncated when val (u64) is
passed to msecs_to_jiffies (u32). This aligns with how very large
millisecond values get translated into MAX_JIFFY_OFFSET.
Fixes: b96b5c6708ea ("sysctl: Replace do_proc_do{int,ulong,uint}vec with do_proc_vec")
Suggested-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Add the range check back to do_proc_ulong_conv_ms_jiffies that commit
b96b5c6708ea ("sysctl: Replace do_proc_do{int,ulong,uint}vec with
do_proc_vec") incorrectly removed. Append "_minmax" to the end of
do_proc_ulong_conv_ms_jiffies so it is clear that there should be a
range check.
Fixes: b96b5c6708ea ("sysctl: Replace do_proc_do{int,ulong,uint}vec with do_proc_vec")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Add the range check to do_proc_int_conv_ms_jiffies_minmax that commit
d174174c6776 ("sysctl: replace SYSCTL_INT_CONV_CUSTOM macro with
functions") incorrectly removed.
Fixes: d174174c6776 ("sysctl: replace SYSCTL_INT_CONV_CUSTOM macro with functions")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Rearming a queued timer with nonzero slack can leave the timerqueue out
of order. remove_and_enqueue_same_base() checks the new soft expiry
against its neighbours' hard expiries, then stores the new hard expiry
in the node without requeueing it.
For example, with A at 10 and B at 20, rearming A at 11 with slack 30
passes the neighbour check but leaves A's hard expiry of 41 before B's
20. The same function also caches the soft expiry in base->expires_next
when updating or inserting the first timer, giving next-event selection
an earlier deadline than the queue head's hard expiry.
Set the timer expiry before handling the queue. Use its stored hard
expiry for the in-place ordering check and both updates to
base->expires_next.
The early update is safe because remove_and_enqueue_same_base() runs
with base->cpu_base->lock held. The lock keeps the queue stable while
hrtimer_can_update_in_place() checks the new expiry against both
neighbours. If the check fails, timerqueue_linked_del() removes the node
without comparing expiry values before it is reinserted.
Fixes: eddffab8282e3 ("hrtimer: Keep track of first expiring timer per clock base")
Fixes: 343f2f4dc5425 ("hrtimer: Try to modify timers in place")
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Assisted-by: LLM
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260910143442.2018-1-parri.andrea@gmail.com
|
|
朱恺乾 reported and decoded the following race condition when a broadcast
device is replaced:
CPUA CPUB
__tick_broadcast_oneshot_control()
bc = tick_broadcast_device.evtdev;
tick_install_broadcast_device(dev)
clockevents_exchange_device(cur, dev)
shutdown(cur);
detach(cur);
cur->handler = noop;
tick_broadcast_device.evtdev = dev;
tick_broadcast_set_event(bc, next_event); <- FAIL: arms a detached device.
If the original broadcast device has a restricted interrupt affinity mask
and the last CPU in that mask goes offline then the BUG() in
tick_cleanup_dead_cpu() triggers because the clockevent device is not in
detached state.
The reason for this is that tick_install_broadcast_device() is not
serialized vs. tick broadcast operations.
The obvious cure is to serialize tick_install_broadcast_device() with
tick_broadcast_lock against a concurrent tick broadcast operation.
That requires to split clockevents_exchange_device() into two parts, one
which does the exchange, shutdown and detach operation and the other which
drops the module reference count. This is required because the module
reference cannot be dropped while holding tick_broadcast_lock.
Let clockevents_exchange_device() do both operations as before, but let the
broadcast device code take the two step approach and do the device
exchange under tick_broadcast_lock and drop the module reference count
after releasing it.
Fixes: f8381cba04ba ("[PATCH] tick-management: broadcast functionality")
Reported-by: 朱恺乾 <zhukaiqian@xiaomi.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Tested-by: 刘术高 <liushugao@xiaomi.com>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/87cymdsu0r.ffs@tglx
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
Pull s390 updates from Vasily Gorbik:
- Add a cpuidle driver with polling and enabled wait states using the
existing CPU idle infrastructure and idle governor to improve latency
for frequent sleep/wakeup cycles. Remove the obsolete tick delay
heuristic and generic arch_needs_cpu() hook. Add the corresponding
driver entry to MAINTAINERS
- Add kCFI support using the generic support provided by Clang
- Enable Clang CONTEXT_ANALYSIS for various architecture code and for
char, PCI, CIO and virtio drivers. Add required lock annotations,
exclude unsupported mm helpers and remove conditional PCI locking
- Fix secure storage access exception handling and reintroduce
DCACHE_WORD_ACCESS previously removed as a workaround
- Fix cpum_cf perf crashes when CPUs are brought online while per-task
events are active. Allocate and remove per-CPU counter data from CPU
hotplug callbacks
- Fix a deadlock when an s390dbf debug area is unregistered while one
of its debugfs files is being written to
- Fix MVIY_PERCPU() with binutils older than 2.39, where an assembler
macro silently omitted an instruction needed to repair interrupted
operations after CPU migration
- Remove/replace cond_resched() calls which are no-ops with the
supported s390 preemption models
- Fix AP queue depth and maximum message length decoding according to
the architecture. Current hardware is not affected, but future
hardware could report values which were handled incorrectly
- Reflect the configured CPU state in cpu_enabled_mask so deconfigured
CPUs are not presented as available for onlining
- Restore the vDSO GNU_EH_FRAME program header which was lost when the
build switched to direct linker invocation, and mark it read-only
- Add SCLP action qualifiers used by Spyre for card initialization,
recoverable error and telemetry reporting
- Move KMSAN interrupt flag helpers out of line to fix
-Wstatic-in-inline build warnings
- Use level-specific page table entry accessors for hugetlb entries and
ptep_get() when accessing crashed kernel memory in kdump
- Make forced AP bus rescans killable so that a user process blocked
behind an ongoing scan can still be terminated with SIGKILL
- Rework pkey ioctl error paths to remove duplicated cleanup code and
avoid freeing error pointers
- Allow the protected guest SWIOTLB buffer to be allocated outside the
first 2GB. Also enable dynamic SWIOTLB growth and the coherent atomic
pool fallback to improve I/O behavior when the initial pool is
exhausted
- Add program check statistics and spinlock contention tracepoints.
Increase the lockdep chain capacity to keep lockdep enabled for
complex code paths such as btrfs
- Simplify IPL, trap and syscall code and remove the obsolete
unistd_32.h generation entry
* tag 's390-7.3-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux: (59 commits)
s390/percpu: Fix MVIY_PERCPU() with older binutils
s390/debug: Fix deadlock during unregister
s390/cpum_cf: Handle CPU hotplug via prepare/dead callbacks
s390: Enable CONTEXT_ANALYSIS for various directories
s390/mm: Add __context_unsafe() attribute to gmap helper functions
s390/mm: Add __context_unsafe() attribute to do_secure_storage_access()
s390/sysinfo: Add context analysis attributes
s390/irqflags: Add out-of-line definitions of arch_local_irq_*() for KMSAN
s390/virtio: Enable CONTEXT_ANALYSIS
s390/cio: Enable CONTEXT_ANALYSIS
s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
s390/pci: Enable CONTEXT_ANALYSIS
s390/pci: Rework __zpci_event_availability() to remove conditional locking
s390/pci: Rework __zpci_event_error() to remove conditional locking
s390/char: Enable CONTEXT_ANALYSIS
s390/con3215: Add __must_hold() attribute to raw3215_make_room()
s390/ap: Fix MAPML computation
s390/cio: Remove cond_resched() calls
s390: Remove cond_resched() calls
KVM: s390: Remove cond_resched() calls
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fixes from Ingo Molnar:
- Fix timer debugobjects state corruption on CPU offlining (Thomas
Gleixner)
- Fix ARM get_cycles() regression causing boot hangs on certain ARM
configs (Nathan Chancellor)
* tag 'core-urgent-2026-08-22' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
timer: Keep debugobjects state consistent in migrate_timer_list()
* tag 'timers-urgent-2026-08-22' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
ARM: Fix get_cycles() after delay_read_timer() conversion
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl
Pull sysctl updates from Joel Granados:
- Fix kernel-doc warnings by adjusting in file documentation
- Consolidate do_proc_* function into do_proc_vec
Consolidate three slightly different implementations of applying a
converter on all elements of a vector. Fixes to this function now
propagate to the three types.
- Replace CONFIG_PROC_SYSCTL with CONFIG_SYSCTL (they were the same)
and restrict cad_pid modi |
