diff options
| author | Peter Zijlstra <peterz@infradead.org> | 2026-05-12 02:56:17 +0000 |
|---|---|---|
| committer | Peter Zijlstra <peterz@infradead.org> | 2026-06-02 12:26:07 +0200 |
| commit | 1628b25248d0742b2ce9c7cfa59cd183e35f37e1 (patch) | |
| tree | 6f204daf3a4dcc54e962e2fba54fa8bc18211068 /kernel/sched | |
| parent | 4c2a20413d7fb3fc3dd7adf233a4f82bb203fb58 (diff) | |
sched: Add blocked_donor link to task for smarter mutex handoffs
Add link to the task this task is proxying for, and use it so
the mutex owner can do an intelligent hand-off of the mutex to
the task that the owner is running on behalf.
[jstultz: This patch was split out from larger proxy patch]
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Juri Lelli <juri.lelli@redhat.com>
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Connor O'Brien <connoro@google.com>
Signed-off-by: John Stultz <jstultz@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260512025635.2840817-8-jstultz@google.com
Diffstat (limited to 'kernel/sched')
| -rw-r--r-- | kernel/sched/core.c | 14 |
1 files changed, 13 insertions, 1 deletions
diff --git a/kernel/sched/core.c b/kernel/sched/core.c index c7552869d5c4..4c6ceff3855e 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6827,7 +6827,17 @@ static void proxy_migrate_task(struct rq *rq, struct rq_flags *rf, * Find runnable lock owner to proxy for mutex blocked donor * * Follow the blocked-on relation: - * task->blocked_on -> mutex->owner -> task... + * + * ,-> task + * | | blocked-on + * | v + * blocked_donor | mutex + * | | owner + * | v + * `-- task + * + * and set the blocked_donor relation, this latter is used by the mutex + * code to find which (blocked) task to hand-off to. * * Lock order: * @@ -6969,6 +6979,7 @@ find_proxy_task(struct rq *rq, struct task_struct *donor, struct rq_flags *rf) * rq, therefore holding @rq->lock is sufficient to * guarantee its existence, as per ttwu_remote(). */ + owner->blocked_donor = p; } WARN_ON_ONCE(owner && !owner->on_rq); return owner; @@ -7125,6 +7136,7 @@ pick_again: clear_task_blocked_on(prev, NULL); rq_set_donor(rq, next); + next->blocked_donor = NULL; if (unlikely(next->is_blocked && next->blocked_on)) { next = find_proxy_task(rq, next, &rf); if (!next) { |