From 8a570b19b4b16a8a3b5ffa2b332bd5613110b2d8 Mon Sep 17 00:00:00 2001 From: D Scott Phillips Date: Tue, 14 Jul 2026 16:11:58 -0700 Subject: KVM: arm64: vgic: Avoid double-deactivate of IRQs in the nested context In the nested state, the physical interrupt has already been deactivated through the HW bit in the LR. The extra deactivation would be harmless but can hit an errata case on AmpereOne, so avoid it here. On AmpereOne, deactivating a physical interrupt through ICC_DIR_EL1 or ICC_EOIR1_EL1 (depending on EOImode) which is not active, but is the highest priority pending interrupt causes the cpu to lose the interrupt pending state and also prevents the delivery of future interrupts. Fixes: 6dd333c8942b2 ("KVM: arm64: GICv3: nv: Plug L1 LR sync into deactivation primitive") Signed-off-by: D Scott Phillips Cc: stable@vger.kernel.org Link: https://lore.kernel.org/linux-arm-kernel/20260710222128.416581-1-scott@os.amperecomputing.com/ Link: https://patch.msgid.link/20260714231158.496808-1-scott@os.amperecomputing.com Signed-off-by: Marc Zyngier --- Documentation/arch/arm64/silicon-errata.rst | 4 ++++ arch/arm64/kvm/vgic/vgic-v3.c | 8 +++++++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/Documentation/arch/arm64/silicon-errata.rst b/Documentation/arch/arm64/silicon-errata.rst index 014aa1c215a1..88b4aa45a206 100644 --- a/Documentation/arch/arm64/silicon-errata.rst +++ b/Documentation/arch/arm64/silicon-errata.rst @@ -55,10 +55,14 @@ stable kernels. +----------------+-----------------+-----------------+-----------------------------+ | Ampere | AmpereOne | AC03_CPU_38 | AMPERE_ERRATUM_AC03_CPU_38 | +----------------+-----------------+-----------------+-----------------------------+ +| Ampere | AmpereOne | AC03_CPU_57 | N/A | ++----------------+-----------------+-----------------+-----------------------------+ | Ampere | AmpereOne AC04 | AC04_CPU_10 | AMPERE_ERRATUM_AC03_CPU_38 | +----------------+-----------------+-----------------+-----------------------------+ | Ampere | AmpereOne AC04 | AC04_CPU_23 | AMPERE_ERRATUM_AC04_CPU_23 | +----------------+-----------------+-----------------+-----------------------------+ +| Ampere | AmpereOne AC04 | AC04_CPU_29 | N/A | ++----------------+-----------------+-----------------+-----------------------------+ +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A510 | #2457168 | ARM64_ERRATUM_2457168 | +----------------+-----------------+-----------------+-----------------------------+ diff --git a/arch/arm64/kvm/vgic/vgic-v3.c b/arch/arm64/kvm/vgic/vgic-v3.c index 9e841e7afd4a..7aa417440f6a 100644 --- a/arch/arm64/kvm/vgic/vgic-v3.c +++ b/arch/arm64/kvm/vgic/vgic-v3.c @@ -275,7 +275,13 @@ void vgic_v3_deactivate(struct kvm_vcpu *vcpu, u64 val) lr = vgic_v3_compute_lr(vcpu, irq) & ~ICH_LR_ACTIVE_BIT; } - if (lr & ICH_LR_HW) + /* + * In the nested state, the irq has already been deactivated via the HW + * bit in the LR. Deactivating again would be harmless except AmpereOne + * errata AC03_CPU_57, AC04_CPU_29 could cause irq delivery to break if + * the deactivation hits the highest priority pending irq. + */ + if ((lr & ICH_LR_HW) && !vgic_state_is_nested(vcpu)) vgic_v3_deactivate_phys(FIELD_GET(ICH_LR_PHYS_ID_MASK, lr)); vgic_v3_fold_lr(vcpu, lr); -- cgit v1.2.3 From 5aab4d46278df6e9e94bd0a92df9a135a5fe16ad Mon Sep 17 00:00:00 2001 From: Fuad Tabba Date: Sun, 19 Jul 2026 17:32:57 +0100 Subject: KVM: arm64: Update Fuad Tabba's email address Switch my KVM/arm64 reviewer entry to fuad.tabba@linux.dev, my canonical address for kernel work, and add a .mailmap entry so the existing tabba@google.com commits map onto it. Signed-off-by: Fuad Tabba Link: https://patch.msgid.link/20260719163257.3156529-1-fuad.tabba@linux.dev Signed-off-by: Marc Zyngier --- .mailmap | 1 + MAINTAINERS | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/.mailmap b/.mailmap index 735470e21075..91003fb86136 100644 --- a/.mailmap +++ b/.mailmap @@ -291,6 +291,7 @@ Frank Rowand Frank Rowand Frank Rowand Frank Zago +Fuad Tabba Gao Xiang Gao Xiang Gao Xiang diff --git a/MAINTAINERS b/MAINTAINERS index a674e36529f7..5ab6fc150426 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14192,7 +14192,7 @@ F: virt/kvm/* KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64) M: Marc Zyngier M: Oliver Upton -R: Fuad Tabba +R: Fuad Tabba R: Joey Gouly R: Steffen Eiden R: Suzuki K Poulose -- cgit v1.2.3 From cbfe2b24a1ea9de35032dbdd100fdc700f5be92d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Carlos=20L=C3=B3pez?= Date: Wed, 15 Jul 2026 12:51:37 +0200 Subject: KVM: arm64: vgic: Fix race between LPI release and re-registration MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fix a potential race between decrementing an LPI's reference count and evicting that structure from the LPI xarray. LPI structures are maintained in the VGIC LPI xarray (dist->lpi_xa). When the reference count of an LPI structure drops to zero, vgic_release_lpi_locked() removes the structure from the xarray and frees it under the xarray lock. However, the release of an LPI can race with a concurrent LPI re-registration with the same INTID via vgic_add_lpi() on another CPU, since the reference count drop and the xarray eviction are not performed in a single atomic step. This can happen e.g. if the guest issues a DISCARD while the LPI is still referenced from a vCPU's active-pending list (ap_list), and the same INTID is re-mapped via MAPTI. Particularly, vgic_release_lpi_locked() is called from two distinct paths: direct release via vgic_put_irq(), and deferred release via vgic_release_deleted_lpis(). During direct release, the issue can result in deleting a newly registered LPI from the xarray: CPU0 (Releasing LPI) CPU1 (Adding new LPI) ==================== ===================== vgic_put_irq() __vgic_put_irq() refcount_dec_and_test() vgic_add_lpi() xa_lock_irqsave() old_irq = xa_load(.., intid) vgic_try_get_irq_ref(old_irq) == false new IRQ inserted --> __xa_store(.., intid, ..) xa_unlock_irqrestore() xa_lock_irqsave(); vgic_release_lpi_locked() __xa_erase(.., irq->intid) <-- BUG: new IRQ is erased kfree_rcu(old_irq) During the deferred release path, the old IRQ can be leaked: CPU0 (Releasing LPI) CPU1 (Adding new LPI) ==================== ===================== vgic_put_irq_norelease() __vgic_put_irq() refcount_dec_and_test() irq->pending_release = true vgic_add_lpi() xa_lock_irqsave() old_irq = xa_load(.., intid) vgic_try_get_irq_ref(oldirq) == false BUG: old IRQ overwritten --> __xa_store(.., intid, ..) xa_unlock_irqrestore() vgic_release_deleted_lpis() xa_lock_irqsave() xa_for_each() { .. } <-- old IRQ with pending_release = true is gone, so it cannot be released To fix the direct release path, move the reference count drop inside the xarray lock, making sure that vgic_add_lpi() never encounters the to-be-released LPI. In the deferred release path, the refcount drop must happen under a raw spinlock, so the xarray lock cannot be grabbed, and the same solution does not work. Instead, update vgic_add_lpi(), so that if it evicts an LPI from the xarray, it takes on the responsibility of freeing it. Consequently, an LPI may now be freed concurrently after a deferred release drops the refcount, so accessing the pending_release field is no longer safe from use-after-free. Delete all uses of the flag, and update vgic_release_deleted_lpis() to identify orphaned LPIs purely based on their refcount. Reported-by: Claude:claude-opus-4-6 Fixes: 3a08a6ca7c37 ("KVM: arm64: vgic-v3: Use bare refcount for VGIC LPIs") Fixes: d54594accf73 ("KVM: arm64: vgic-v3: Erase LPIs from xarray outside of raw spinlocks") Signed-off-by: Carlos López Link: https://patch.msgid.link/20260715105137.3973823-4-clopez@suse.de Signed-off-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic-its.c | 24 ++++++++++++++++-------- arch/arm64/kvm/vgic/vgic.c | 18 ++++++++---------- include/kvm/arm_vgic.h | 3 --- 3 files changed, 24 insertions(+), 21 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 740b39875728..5c5d1772147d 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -116,18 +116,26 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, kfree(irq); irq = oldirq; } else { - ret = xa_err(__xa_store(&dist->lpi_xa, intid, irq, 0)); - } - - xa_unlock_irqrestore(&dist->lpi_xa, flags); + /* + * The entry is either empty or contains a dead LPI (refcount=0) + * from the deferred release path, pending cleanup by + * vgic_release_deleted_lpis(). Evict and free it if present. + */ + oldirq = __xa_store(&dist->lpi_xa, intid, irq, 0); + ret = xa_err(oldirq); + if (ret) { + xa_unlock_irqrestore(&dist->lpi_xa, flags); + kfree(irq); - if (ret) { - xa_release(&dist->lpi_xa, intid); - kfree(irq); + return ERR_PTR(ret); + } - return ERR_PTR(ret); + if (oldirq && !WARN_ON_ONCE(refcount_read(&oldirq->refcount))) + kfree_rcu(oldirq, rcu); } + xa_unlock_irqrestore(&dist->lpi_xa, flags); + /* * We "cache" the configuration table entries in our struct vgic_irq's. * However we only have those structs for mapped IRQs, so we read in diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index ccb7e3a90cd0..74bace10a22e 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -146,11 +146,7 @@ static __must_check bool __vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) static __must_check bool vgic_put_irq_norelease(struct kvm *kvm, struct vgic_irq *irq) { - if (!__vgic_put_irq(kvm, irq)) - return false; - - irq->pending_release = true; - return true; + return __vgic_put_irq(kvm, irq); } void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) @@ -167,12 +163,14 @@ void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq) guard(spinlock_irqsave)(&dist->lpi_xa.xa_lock); } - if (!__vgic_put_irq(kvm, irq)) + if (!irq_is_lpi(kvm, irq->intid)) return; - xa_lock_irqsave(&dist->lpi_xa, flags); - vgic_release_lpi_locked(dist, irq); - xa_unlock_irqrestore(&dist->lpi_xa, flags); + if (refcount_dec_and_lock_irqsave(&irq->refcount, + &dist->lpi_xa.xa_lock, &flags)) { + vgic_release_lpi_locked(dist, irq); + xa_unlock_irqrestore(&dist->lpi_xa, flags); + } } static void vgic_release_deleted_lpis(struct kvm *kvm) @@ -184,7 +182,7 @@ static void vgic_release_deleted_lpis(struct kvm *kvm) xa_lock_irqsave(&dist->lpi_xa, flags); xa_for_each(&dist->lpi_xa, intid, irq) { - if (irq->pending_release) + if (!refcount_read(&irq->refcount)) vgic_release_lpi_locked(dist, irq); } diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h index fe49fb56dc3c..cefddc9c621d 100644 --- a/include/kvm/arm_vgic.h +++ b/include/kvm/arm_vgic.h @@ -247,9 +247,6 @@ struct vgic_irq { * affinity reg (v3). */ - bool pending_release:1; /* Used for LPIs only, unreferenced IRQ - * pending a release */ - bool pending_latch:1; /* The pending latch state used to calculate * the pending state for both level * and edge triggered IRQs. */ -- cgit v1.2.3 From 21f12496fdd357ad4e1fcdd07dc80ab7378f7d24 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Carlos=20L=C3=B3pez?= Date: Wed, 15 Jul 2026 12:51:38 +0200 Subject: KVM: arm64: vgic: Mitigate potential LPI registration failure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Mitigate a potential failure when inserting a new LPI into the VGIC LPI xarray. When vgic_add_lpi() is preparing to register a new LPI, it pre-allocates an xarray entry using xa_reserve_irq(), so that it can later perform the insertion under the xarray lock without allocating. However, since xa_reserve_irq() is called before acquiring such lock, there is a potential race where xa_reserve_irq() observes a populated entry, thus not performing the allocation, and another CPU removes that entry before the xarray lock is grabbed to perform the insertion. CPU0 (Adding new LPI) CPU1 (Releasing LPI) ===================== =================== vgic_add_lpi() /* Entry populated, does not allocate */ xa_reserve_irq(.., intid, ..) vgic_release_deleted_lpis() xa_lock_irqsave() vgic_release_lpi_locked() xarray node freed --> __xa_erase(.., intid) xa_unlock_irqrestore() xa_lock_irqsave() xa_load(.., intid) == NULL vgic_try_get_irq_ref(NULL) == false __xa_store(.., intid, irq, 0) <-- xarray node was freed, gfp=0 cannot allocate, returns -ENOMEM This can happen e.g. if the guest issues a DISCARD while the LPI is still referenced from a vCPU's active-pending list (ap_list), and the same INTID is re-mapped via MAPTI. Mitigate this by passing GFP_NOWAIT to __xa_store(), so that the allocation can happen under the lock in the rare case that this condition is hit. Add __GFP_ACCOUNT as well to match xa_reserve_irq()'s flags. Reported-by: Sashiko Fixes: 1d6f83f60f79 ("KVM: arm64: vgic: Store LPIs in an xarray") Signed-off-by: Carlos López Link: https://patch.msgid.link/20260715105137.3973823-5-clopez@suse.de Signed-off-by: Marc Zyngier --- arch/arm64/kvm/vgic/vgic-its.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 5c5d1772147d..36ab3e492915 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -121,7 +121,8 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, * from the deferred release path, pending cleanup by * vgic_release_deleted_lpis(). Evict and free it if present. */ - oldirq = __xa_store(&dist->lpi_xa, intid, irq, 0); + oldirq = __xa_store(&dist->lpi_xa, intid, irq, + GFP_NOWAIT | __GFP_ACCOUNT); ret = xa_err(oldirq); if (ret) { xa_unlock_irqrestore(&dist->lpi_xa, flags); -- cgit v1.2.3 From bbece712cfc7f286b2908ac120dcf700279d87eb Mon Sep 17 00:00:00 2001 From: Vincent Donnefort Date: Wed, 15 Jul 2026 11:51:00 +0100 Subject: KVM: arm64: Fix hyp_trace clock disabling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fix the disable path in hyp_trace_clock_enable(), which fell through to re-initialize and reschedule the clock after cancelling the work. Return early instead. While at it, cleanup hyp_trace_clock::lock which is unused and hyp_trace_clock::running which is redundant: the trace_remote framework already serializes calls to the callback enable_tracing. Fixes: b22888917fa4 ("KVM: arm64: Sync boot clock with the nVHE/pKVM hyp") Signed-off-by: Vincent Donnefort Reviewed-by: Fuad Tabba (✓ DKIM/linux.dev) Link: https://patch.msgid.link/20260715105100.3178255-1-vdonnefort@google.com Signed-off-by: Marc Zyngier --- arch/arm64/kvm/hyp_trace.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kvm/hyp_trace.c b/arch/arm64/kvm/hyp_trace.c index 2411b4c32932..9bfa368dd841 100644 --- a/arch/arm64/kvm/hyp_trace.c +++ b/arch/arm64/kvm/hyp_trace.c @@ -37,8 +37,6 @@ static struct hyp_trace_clock { u32 shift; struct delayed_work work; struct completion ready; - struct mutex lock; - bool running; } hyp_clock; static void __hyp_clock_work(struct work_struct *work) @@ -110,12 +108,9 @@ static void hyp_trace_clock_enable(struct hyp_trace_clock *hyp_clock, bool enabl { struct system_time_snapshot snap; - if (hyp_clock->running == enable) - return; - if (!enable) { cancel_delayed_work_sync(&hyp_clock->work); - hyp_clock->running = false; + return; } ktime_get_snapshot_id(CLOCK_BOOTTIME, &snap); @@ -128,7 +123,6 @@ static void hyp_trace_clock_enable(struct hyp_trace_clock *hyp_clock, bool enabl INIT_DELAYED_WORK(&hyp_clock->work, __hyp_clock_work); schedule_delayed_work(&hyp_clock->work, msecs_to_jiffies(CLOCK_INIT_MS)); wait_for_completion(&hyp_clock->ready); - hyp_clock->running = true; } /* Access to this struct within the trace_remote_callbacks are protected by the trace_remote lock */ @@ -304,9 +298,15 @@ static void hyp_trace_unload(struct trace_buffer_desc *desc, void *priv) static int hyp_trace_enable_tracing(bool enable, void *priv) { + int ret; + hyp_trace_clock_enable(&hyp_clock, enable); - return kvm_call_hyp_nvhe(__tracing_enable, enable); + ret = kvm_call_hyp_nvhe(__tracing_enable, enable); + if (ret) + hyp_trace_clock_enable(&hyp_clock, !enable); + + return ret; } static int hyp_trace_swap_reader_page(unsigned int cpu, void *priv) -- cgit v1.2.3 From df7a9d376f7a388ecfacf8dfe0b5819ddfba6972 Mon Sep 17 00:00:00 2001 From: Vincent Donnefort Date: Fri, 10 Jul 2026 12:48:18 +0100 Subject: KVM: arm64: Fix potential leak in hyp_trace_buffer_alloc_bpages_backing In the very unlikely event of a failure in __map_hyp, the allocated backing pages are leaked in hyp_trace_buffer_alloc_bpages_backing(). Fix this by freeing the pages on error. Fixes: 3aed038aac8d ("KVM: arm64: Add trace remote for the nVHE/pKVM hyp") Reported-by: Sashiko Reviewed-by: Fuad Tabba Tested-by: Fuad Tabba Signed-off-by: Vincent Donnefort Link: https://patch.msgid.link/20260710114819.2689386-2-vdonnefort@google.com Signed-off-by: Marc Zyngier --- arch/arm64/kvm/hyp_trace.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp_trace.c b/arch/arm64/kvm/hyp_trace.c index 9bfa368dd841..aabf2989d70d 100644 --- a/arch/arm64/kvm/hyp_trace.c +++ b/arch/arm64/kvm/hyp_trace.c @@ -154,6 +154,7 @@ static int hyp_trace_buffer_alloc_bpages_backing(struct hyp_trace_buffer *trace_ int nr_bpages = (PAGE_ALIGN(size) / PAGE_SIZE) + 1; size_t backing_size; void *start; + int ret; backing_size = PAGE_ALIGN(sizeof(struct simple_buffer_page) * nr_bpages * num_possible_cpus()); @@ -162,10 +163,16 @@ static int hyp_trace_buffer_alloc_bpages_backing(struct hyp_trace_buffer *trace_ if (!start) return -ENOMEM; + ret = __map_hyp(start, backing_size); + if (ret) { + free_pages_exact(start, backing_size); + return ret; + } + trace_buffer->desc->bpages_backing_start = (unsigned long)start; trace_buffer->desc->bpages_backing_size = backing_size; - return __map_hyp(start, backing_size); + return ret; } static void hyp_trace_buffer_free_bpages_backing(struct hyp_trace_buffer *trace_buffer) -- cgit v1.2.3 From ca28278d10ec234592989ad370d58294b9d43e0f Mon Sep 17 00:00:00 2001 From: Vincent Donnefort Date: Fri, 10 Jul 2026 12:48:19 +0100 Subject: KVM: arm64: Fix hyp_trace_desc allocation size in hyp_trace_load() The footprint calculated for struct hyp_trace_desc sizes only trace_buffer_desc and do not take into account the other fields. It worked so far thanks to the follow-up PAGE_ALIGN(). Fix the descriptor size and while at it, enforce an overflow check after PAGE_ALIGN(). Reported-by: Sashiko Fixes: 3aed038aac8d ("KVM: arm64: Add trace remote for the nVHE/pKVM hyp") Signed-off-by: Vincent Donnefort Reviewed-by: Fuad Tabba Tested-by: Fuad Tabba Link: https://patch.msgid.link/20260710114819.2689386-3-vdonnefort@google.com Signed-off-by: Marc Zyngier --- arch/arm64/kvm/hyp_trace.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp_trace.c b/arch/arm64/kvm/hyp_trace.c index aabf2989d70d..1adfdc800187 100644 --- a/arch/arm64/kvm/hyp_trace.c +++ b/arch/arm64/kvm/hyp_trace.c @@ -229,18 +229,22 @@ static int hyp_trace_buffer_share_hyp(struct hyp_trace_buffer *trace_buffer) static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv) { struct hyp_trace_buffer *trace_buffer = priv; + size_t desc_size, tb_desc_size; struct hyp_trace_desc *desc; - size_t desc_size; int ret; if (WARN_ON(trace_buffer->desc)) return ERR_PTR(-EINVAL); - desc_size = trace_buffer_desc_size(size, num_possible_cpus()); + tb_desc_size = trace_buffer_desc_size(size, num_possible_cpus()); + desc_size = size_add(tb_desc_size, offsetof(struct hyp_trace_desc, trace_buffer_desc)); if (desc_size == SIZE_MAX) return ERR_PTR(-E2BIG); desc_size = PAGE_ALIGN(desc_size); + if (!desc_size) + return ERR_PTR(-E2BIG); + desc = (struct hyp_trace_desc *)alloc_pages_exact(desc_size, GFP_KERNEL); if (!desc) return ERR_PTR(-ENOMEM); @@ -256,7 +260,7 @@ static struct trace_buffer_desc *hyp_trace_load(unsigned long size, void *priv) if (ret) goto err_free_desc; - ret = trace_remote_alloc_buffer(&desc->trace_buffer_desc, desc_size, size, + ret = trace_remote_alloc_buffer(&desc->trace_buffer_desc, tb_desc_size, size, cpu_possible_mask); if (ret) goto err_free_backing; -- cgit v1.2.3 From e7821048e8d72a94b1fb7422f8b45aa374ff076f Mon Sep 17 00:00:00 2001 From: Vincent Donnefort Date: Wed, 17 Jun 2026 10:52:38 +0100 Subject: KVM: arm64: Add missing hyp_enter when trapping sysreg Add a missing hypervisor event call for hyp_enter on sysreg trapping, causing an unbalanced hyp_enter/hyp_exit. The enum hyp_enter_exit_reason is not ABI, so we can keep the ERET reasons at the end for clarity. Fixes: 696dfec22b8e ("KVM: arm64: Add hyp_enter/hyp_exit events to nVHE/pKVM hyp") Signed-off-by: Vincent Donnefort Reviewed-by: Fuad Tabba Tested-by: Fuad Tabba Link: https://patch.msgid.link/20260617095238.1530121-1-vdonnefort@google.com Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/kvm_hypevents.h | 1 + arch/arm64/kvm/hyp/nvhe/hyp-main.c | 1 + arch/arm64/kvm/hyp_trace.c | 1 + 3 files changed, 3 insertions(+) diff --git a/arch/arm64/include/asm/kvm_hypevents.h b/arch/arm64/include/asm/kvm_hypevents.h index 743c49bd878f..5f6e6789d121 100644 --- a/arch/arm64/include/asm/kvm_hypevents.h +++ b/arch/arm64/include/asm/kvm_hypevents.h @@ -12,6 +12,7 @@ enum hyp_enter_exit_reason { HYP_REASON_SMC, HYP_REASON_HVC, + HYP_REASON_SYS, HYP_REASON_PSCI, HYP_REASON_HOST_ABORT, HYP_REASON_GUEST_EXIT, diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index d3c69de698f4..d3df96ed8ba4 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -932,6 +932,7 @@ void handle_trap(struct kvm_cpu_context *host_ctxt) handle_host_mem_abort(host_ctxt); break; case ESR_ELx_EC_SYS64: + trace_hyp_enter(host_ctxt, HYP_REASON_SYS); if (handle_host_mte(esr)) break; fallthrough; diff --git a/arch/arm64/kvm/hyp_trace.c b/arch/arm64/kvm/hyp_trace.c index 1adfdc800187..9644c424819b 100644 --- a/arch/arm64/kvm/hyp_trace.c +++ b/arch/arm64/kvm/hyp_trace.c @@ -409,6 +409,7 @@ static const char *__hyp_enter_exit_reason_str(u8 reason) static const char strs[][12] = { "smc", "hvc", + "sys", "psci", "host_abort", "guest_exit", -- cgit v1.2.3 From 679d7201c1f09e37fa1c12ce28d84079c17fc87f Mon Sep 17 00:00:00 2001 From: Alexandru Elisei Date: Wed, 22 Jul 2026 10:03:54 +0100 Subject: KVM: arm64: Reject guest_memfd memslots when the VM has MTE The user cannot use MTE on VMAs created by mapping a guest_memfd file, as arch_calc_vm_flag_bits() does not set VM_MTE_ALLOWED. When creating a guest_memfd backed memslot, kvm_arch_prepare_memory_region() rejects the memslot if MTE is enabled for the VM and if guest_memfd has been mapped in a VMA that intersects the memslot. However, the documentation for KVM_SET_USER_MEMORY_REGION2 explicitly states that the only condition for userspace_addr is for it to be a legal userspace address, but the mapping is not required to be valid nor populated at memslot creation. If userspace sets userspace_addr to an address that hasn't been mapped, or if userspace_addr belongs to a VMA that isn't backed by the guest_memfd file, or if the VMA doesn't intersect the memslot, memslot creation is successful and KVM ends up with a VM with MTE and guest_memfd-backed memslots. The same happens if the order is reversed: when userspace enables MTE, KVM does not check if memslots backed by guest_memfd are already present. Fix both issues by rejecting guest_memfd-backed memslots when MTE is enabled, and by rejecting MTE when guest_memfd-backed memslots are already present. Fixes: 32e200bd6e44 ("KVM: arm64: Enable support for guest_memfd backed memory") Tested-by: Fuad Tabba Reviewed-by: Fuad Tabba Signed-off-by: Alexandru Elisei Link: https://patch.msgid.link/20260722090354.94245-1-alexandru.elisei@arm.com Signed-off-by: Marc Zyngier --- Documentation/virt/kvm/api.rst | 6 ++++++ arch/arm64/kvm/arm.c | 25 +++++++++++++++++++------ arch/arm64/kvm/mmu.c | 4 ++++ 3 files changed, 29 insertions(+), 6 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index a5f9ee92f43e..e3003a241d5b 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -8414,6 +8414,12 @@ When this capability is enabled all memory in memslots must be mapped as attempts to create a memslot with an invalid mmap will result in an -EINVAL return. +``guest_memfd``, even though it is an anonymous file, is not supported with MTE. +Attempting to create a memslot backed by ``guest_memfd`` when the MTE capability +is enabled, or attempting to enable the MTE capability after +``guest_memfd``-backed memslots have been created, will result in an -EINVAL +return. + When enabled the VMM may make use of the ``KVM_ARM_MTE_COPY_TAGS`` ioctl to perform a bulk copy of tags to/from the guest. diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 50adfff75be8..9a6c72a18672 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -149,14 +149,27 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm, set_bit(KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER, &kvm->arch.flags); break; - case KVM_CAP_ARM_MTE: - mutex_lock(&kvm->lock); - if (system_supports_mte() && !kvm->created_vcpus) { - r = 0; - set_bit(KVM_ARCH_FLAG_MTE_ENABLED, &kvm->arch.flags); + case KVM_CAP_ARM_MTE: { + struct kvm_memory_slot *memslot; + int bkt; + + guard(mutex)(&kvm->lock); + if (!system_supports_mte() || kvm->created_vcpus) + break; + + r = 0; + guard(mutex)(&kvm->slots_lock); + kvm_for_each_memslot(memslot, bkt, kvm_memslots(kvm)) { + if (kvm_slot_has_gmem(memslot)) { + r = -EINVAL; + break; + } } - mutex_unlock(&kvm->lock); + if (r == 0) + set_bit(KVM_ARCH_FLAG_MTE_ENABLED, &kvm->arch.flags); break; + + } case KVM_CAP_ARM_SYSTEM_SUSPEND: r = 0; set_bit(KVM_ARCH_FLAG_SYSTEM_SUSPEND_ENABLED, &kvm->arch.flags); diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 6c941aaa10c6..2d95203386ba 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -2652,6 +2652,10 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, if (kvm_slot_has_gmem(new) && !kvm_memslot_is_gmem_only(new)) return -EINVAL; + /* guest_memfd is incompatible with MTE. */ + if (kvm_slot_has_gmem(new) && kvm_has_mte(kvm)) + return -EINVAL; + hva = new->userspace_addr; reg_end = hva + (new->npages << PAGE_SHIFT); -- cgit v1.2.3 From 8fa01be5a6149404adb82c0979a78f6347edd3ef Mon Sep 17 00:00:00 2001 From: Farhan Ali Date: Thu, 23 Jul 2026 15:14:04 -0700 Subject: KVM: s390: pci: Reject adapter interrupt forwarding if already enabled The MPCIFC instruction doesn't allow registering adapter interrupts without first unregistering. So reject any request to enable interrupt forwarding if its already enabled for the zPCI device. This also fixes overwriting and thus leaking resources when the ioctl is called multiple times for the same device. Fixes: 3c5a1b6f0a18 ("KVM: s390: pci: provide routines for enabling/disabling interrupt forwarding") Cc: stable@vger.kernel.org Reviewed-by: Christian Borntraeger Reviewed-by: Matthew Rosato Signed-off-by: Farhan Ali Tested-by: Matthew Rosato Signed-off-by: Christian Borntraeger --- arch/s390/kvm/pci.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c index 720bb58cabe2..d2a11cdf6941 100644 --- a/arch/s390/kvm/pci.c +++ b/arch/s390/kvm/pci.c @@ -237,6 +237,10 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, if (zdev->gisa == 0) return -EINVAL; + /* AIF already enabled for the device */ + if (zdev->kzdev->fib.fmt0.aibv != 0) + return -EINVAL; + kvm = zdev->kzdev->kvm; msi_vecs = min_t(unsigned int, fib->fmt0.noi, zdev->max_msi); -- cgit v1.2.3 From 36f6999ecde3976731a8bfc0b8e667da6f593069 Mon Sep 17 00:00:00 2001 From: Farhan Ali Date: Thu, 23 Jul 2026 15:14:05 -0700 Subject: KVM: s390: pci: Fix memory accounting for pinned/unpinned pages The account_mem() and unaccount_mem() functions call get_uid() which increments the reference count of struct user_struct on every invocation. But we don't decrement the count by calling free_uid(). It also accounted/unaccounted the pages against the current->mm. But its possible the unaccount_mem() can be called from a different process context than the one that originally pinned the pages. Let's fix this by storing the pinning process user_struct and mm_struct when accounting for pinned pages, and subsequently free these resources when the pages are unpinned. Fixes: 3c5a1b6f0a18 ("KVM: s390: pci: provide routines for enabling/disabling interrupt forwarding") Cc: stable@vger.kernel.org Reviewed-by: Christian Borntraeger Reviewed-by: Matthew Rosato Signed-off-by: Farhan Ali Tested-by: Matthew Rosato [borntraeger@linux.ibm.com: Fixed whitespace] Signed-off-by: Christian Borntraeger --- arch/s390/kvm/pci.c | 43 ++++++++++++++++++++++++++++++++----------- arch/s390/kvm/pci.h | 2 ++ 2 files changed, 34 insertions(+), 11 deletions(-) diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c index d2a11cdf6941..0741aed442bc 100644 --- a/arch/s390/kvm/pci.c +++ b/arch/s390/kvm/pci.c @@ -190,33 +190,54 @@ static int kvm_zpci_clear_airq(struct zpci_dev *zdev) return cc ? -EIO : 0; } -static inline void unaccount_mem(unsigned long nr_pages) +static inline void unaccount_mem(struct kvm_zdev *kzdev, unsigned long nr_pages) { - struct user_struct *user = get_uid(current_user()); + struct user_struct *user = kzdev->user_account; + struct mm_struct *mm_account = kzdev->mm_account; - if (user) + if (user) { atomic_long_sub(nr_pages, &user->locked_vm); - if (current->mm) - atomic64_sub(nr_pages, ¤t->mm->pinned_vm); + free_uid(user); + kzdev->user_account = NULL; + } + + if (mm_account) { + atomic64_sub(nr_pages, &mm_account->pinned_vm); + mmdrop(mm_account); + kzdev->mm_account = NULL; + } } -static inline int account_mem(unsigned long nr_pages) +static inline int account_mem(struct kvm_zdev *kzdev, unsigned long nr_pages) { struct user_struct *user = get_uid(current_user()); unsigned long page_limit, cur_pages, new_pages; + int rc = 0; page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; cur_pages = atomic_long_read(&user->locked_vm); do { new_pages = cur_pages + nr_pages; - if (new_pages > page_limit) - return -ENOMEM; + if (new_pages > page_limit) { + rc = -ENOMEM; + goto out; + } } while (!atomic_long_try_cmpxchg(&user->locked_vm, &cur_pages, new_pages)); - atomic64_add(nr_pages, ¤t->mm->pinned_vm); + if (current->mm) { + mmgrab(current->mm); + atomic64_add(nr_pages, ¤t->mm->pinned_vm); + } + + kzdev->user_account = user; + kzdev->mm_account = current->mm; return 0; + +out: + free_uid(user); + return rc; } static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, @@ -279,7 +300,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, } /* Account for pinned pages, roll back on failure */ - if (account_mem(pcount)) + if (account_mem(zdev->kzdev, pcount)) goto unpin2; /* AISB must be allocated before we can fill in GAITE */ @@ -400,7 +421,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force) pcount++; } if (pcount > 0) - unaccount_mem(pcount); + unaccount_mem(kzdev, pcount); out: mutex_unlock(&aift->aift_lock); diff --git a/arch/s390/kvm/pci.h b/arch/s390/kvm/pci.h index ff0972dd5e71..fdf8c7bf4ed0 100644 --- a/arch/s390/kvm/pci.h +++ b/arch/s390/kvm/pci.h @@ -22,6 +22,8 @@ struct kvm_zdev { struct kvm *kvm; struct zpci_fib fib; struct list_head entry; + struct user_struct *user_account; + struct mm_struct *mm_account; }; struct zpci_gaite { -- cgit v1.2.3 From f86842e4d6c482300f4567f492d512c9ccf5bc4f Mon Sep 17 00:00:00 2001 From: Farhan Ali Date: Thu, 23 Jul 2026 15:14:06 -0700 Subject: KVM: s390: pci: Fix missing error codes and memory unaccounting In kvm_s390_pci_aif_enable() two error paths failed to set an error code, causing the function to return 0 on failure. It also failed to rollback memory accounting on failure. Fix both by propagating an error code on failure and calling unaccount_mem() in the cleanup path. Fixes: 3c5a1b6f0a18 ("KVM: s390: pci: provide routines for enabling/disabling interrupt forwarding") Cc: stable@vger.kernel.org Reviewed-by: Christian Borntraeger Reviewed-by: Matthew Rosato Signed-off-by: Farhan Ali Tested-by: Matthew Rosato Signed-off-by: Christian Borntraeger --- arch/s390/kvm/pci.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c index 0741aed442bc..36eb30953bb5 100644 --- a/arch/s390/kvm/pci.c +++ b/arch/s390/kvm/pci.c @@ -300,14 +300,17 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, } /* Account for pinned pages, roll back on failure */ - if (account_mem(zdev->kzdev, pcount)) + rc = account_mem(zdev->kzdev, pcount); + if (rc) goto unpin2; /* AISB must be allocated before we can fill in GAITE */ mutex_lock(&aift->aift_lock); bit = airq_iv_alloc_bit(aift->sbv); - if (bit == -1UL) + if (bit == -1UL) { + rc = -ENOMEM; goto unlock; + } zdev->aisb = bit; /* store the summary bit number */ zdev->aibv = airq_iv_create(msi_vecs, AIRQ_IV_DATA | AIRQ_IV_BITLOCK | @@ -351,6 +354,8 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, return rc; unlock: + if (pcount > 0) + unaccount_mem(zdev->kzdev, pcount); mutex_unlock(&aift->aift_lock); unpin2: if (fib->fmt0.sum == 1) -- cgit v1.2.3 From 8bf09b9b7d3232806df95f409581f8a9fd99a3fa Mon Sep 17 00:00:00 2001 From: Farhan Ali Date: Thu, 23 Jul 2026 15:14:07 -0700 Subject: KVM: s390: pci: Fix NULL dereference on AIBV allocation failure The airq_iv_create() can return NULL on failure, but the return value was never checked. If it fails, zdev->aibv will be NULL and fail when dereferenced in kvm_zpci_set_airq(). Add a NULL check and free the previously allocated AISB bit and zdev->aisb on failure. Fixes: 3c5a1b6f0a18 ("KVM: s390: pci: provide routines for enabling/disabling interrupt forwarding") Cc: stable@vger.kernel.org Reviewed-by: Christian Borntraeger Reviewed-by: Matthew Rosato Signed-off-by: Farhan Ali Tested-by: Matthew Rosato Signed-off-by: Christian Borntraeger --- arch/s390/kvm/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c index 36eb30953bb5..1eb127fc9f89 100644 --- a/arch/s390/kvm/pci.c +++ b/arch/s390/kvm/pci.c @@ -317,6 +317,11 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, AIRQ_IV_GUESTVEC, phys_to_virt(fib->fmt0.aibv)); + if (!zdev->aibv) { + rc = -ENOMEM; + goto free_aisb; + } + spin_lock_irq(&aift->gait_lock); gaite = aift->gait + zdev->aisb; @@ -353,6 +358,9 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, rc = kvm_zpci_set_airq(zdev); return rc; +free_aisb: + airq_iv_free_bit(aift->sbv, zdev->aisb); + zdev->aisb = 0; unlock: if (pcount > 0) unaccount_mem(zdev->kzdev, pcount); -- cgit v1.2.3 From 5580c9858f1e00f60191eb09c3add359836d60b6 Mon Sep 17 00:00:00 2001 From: Farhan Ali Date: Thu, 23 Jul 2026 15:14:08 -0700 Subject: KVM: s390: pci: Fix resource leak on IRQ registration failure Currently if kvm_zpci_set_airq() fails, kvm_s390_pci_aif_enable() returns the error code but doesn't do any resource cleanup thus leaking resources. Fix this by cleaning up all the resources such as the GAITE, AIBV, AISB and unpinning any pinned pages. While at it, remove dead code that stored FIB values that were never referenced. As part of the cleanup, we are also holding the aift_lock a bit longer, as we hold the lock while executing the MPCIFC instruction. Though this is not strictly necessary, it means we don't have to drop and re-acquire in the error case. Fixes: 3c5a1b6f0a18 ("KVM: s390: pci: provide routines for enabling/disabling interrupt forwarding") Cc: stable@vger.kernel.org Reviewed-by: Matthew Rosato Reviewed-by: Christian Borntraeger Signed-off-by: Farhan Ali Tested-by: Matthew Rosato Signed-off-by: Christian Borntraeger --- arch/s390/kvm/pci.c | 29 +++++++++++++++++++++-------- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c index 1eb127fc9f89..50f495bc8303 100644 --- a/arch/s390/kvm/pci.c +++ b/arch/s390/kvm/pci.c @@ -344,19 +344,32 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, aift->kzdev[zdev->aisb] = zdev->kzdev; spin_unlock_irq(&aift->gait_lock); - /* Update guest FIB for re-issue */ - fib->fmt0.aisbo = zdev->aisb & 63; - fib->fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8; - fib->fmt0.isc = gisc; - /* Save some guest fib values in the host for later use */ - zdev->kzdev->fib.fmt0.isc = fib->fmt0.isc; + zdev->kzdev->fib.fmt0.isc = gisc; zdev->kzdev->fib.fmt0.aibv = fib->fmt0.aibv; - mutex_unlock(&aift->aift_lock); /* Issue the clp to setup the irq now */ rc = kvm_zpci_set_airq(zdev); - return rc; + if (!rc) { + mutex_unlock(&aift->aift_lock); + return rc; + } + + /* Start cleanup */ + zdev->kzdev->fib.fmt0.isc = 0; + zdev->kzdev->fib.fmt0.aibv = 0; + + spin_lock_irq(&aift->gait_lock); + gaite->count--; + gaite->aisb = 0; + gaite->gisc = 0; + gaite->aisbo = 0; + gaite->gisa = 0; + aift->kzdev[zdev->aisb] = NULL; + spin_unlock_irq(&aift->gait_lock); + + airq_iv_release(zdev->aibv); + zdev->aibv = NULL; free_aisb: airq_iv_free_bit(aift->sbv, zdev->aisb); -- cgit v1.2.3 From 868d32ac72cba21c5c6d8a66a814b7c25a3a5c01 Mon Sep 17 00:00:00 2001 From: Farhan Ali Date: Thu, 23 Jul 2026 15:14:09 -0700 Subject: KVM: s390: pci: Validate AIBV and AISB before pinning guest pages The AIBV holds one bit per MSI-X vector for a given function. The size of the bit vector is derived from the NOI and the AIBVO. If the size of the AIBV exceeds a single page boundary, then reject the request as we cannot safely pin the guest AIBV. Similarly reject the request if the AISB address is not 8-byte aligned as the architecture requires doubleword alignment for the summary bit address. Since the AISBO can address up to 64 bits, the size of the AISB can only be 8 bytes for the function. This also ensures the AISB doesn't exceed a single page boundary. Fixes: 3c5a1b6f0a18 ("KVM: s390: pci: provide routines for enabling/disabling interrupt forwarding") Cc: stable@vger.kernel.org Reviewed-by: Christian Borntraeger Reviewed-by: Matthew Rosato Signed-off-by: Farhan Ali Tested-by: Matthew Rosato Signed-off-by: Christian Borntraeger --- arch/s390/kvm/pci.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/arch/s390/kvm/pci.c b/arch/s390/kvm/pci.c index 50f495bc8303..50f5ec79600e 100644 --- a/arch/s390/kvm/pci.c +++ b/arch/s390/kvm/pci.c @@ -244,7 +244,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, bool assist) { struct page *pages[1], *aibv_page, *aisb_page = NULL; - unsigned int msi_vecs, idx; + unsigned int msi_vecs, idx, size; struct zpci_gaite *gaite; unsigned long hva, bit; struct kvm *kvm; @@ -271,6 +271,14 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, return gisc; /* Replace AIBV address */ + size = BITS_TO_LONGS(msi_vecs + fib->fmt0.aibvo) * sizeof(unsigned long); + npages = DIV_ROUND_UP((fib->fmt0.aibv & ~PAGE_MASK) + size, PAGE_SIZE); + /* AIBV cannot span more than 1 page */ + if (npages > 1) { + rc = -EINVAL; + goto out; + } + idx = srcu_read_lock(&kvm->srcu); hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aibv)); npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, pages); @@ -286,6 +294,12 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib, /* Pin the guest AISB if one was specified */ if (fib->fmt0.sum == 1) { + /* AISB must be dword aligned */ + if (fib->fmt0.aisb & 0x7) { + rc = -EINVAL; + goto unpin1; + } + idx = srcu_read_lock(&kvm->srcu); hva = gfn_to_hva(kvm, gpa_to_gfn((gpa_t)fib->fmt0.aisb)); npages = pin_user_pages_fast(hva, 1, FOLL_WRITE | FOLL_LONGTERM, -- cgit v1.2.3 From 9972befc3e34ff8b6847198c84f11bfc312dde40 Mon Sep 17 00:00:00 2001 From: Jaehoon Kim Date: Fri, 24 Jul 2026 08:39:43 -0500 Subject: KVM: s390: Fall back to short-term pinning in MAP ioctl FOLL_LONGTERM pinning fails for some memory types, such as file-backed guest memory. As a result, kvm_s390_adapter_map() returns -EINVAL and irqfd adapter registration fails even though interrupt delivery could still work via the existing non-atomic path. When FOLL_LONGTERM pinning fails, verify that the page is accessible using a short-term pin instead. If the short-term pin succeeds, unpin the page and add a map entry with pinned=false to preserve MAP/UNMAP symmetry. The non-atomic irqfd path already performs short-term pinning for interrupt delivery, so this restores the previous behavior for memory that cannot be pinned long-term. get_map_info() is updated to return NULL for unpinned entries so that the atomic irqfd fast path falls back to the non-atomic path. kvm_s390_adapter_unmap() and kvm_s390_unmap_all_adapters() skip dirty marking and unpin for unpinned entries. Update Documentation/virt/kvm/devices/s390_flic.rst to reflect the new MAP/UNMAP behavior. Fixes: c9a568838086 ("KVM: s390: Add map/unmap ioctl and clean mappings post-guest") Signed-off-by: Jaehoon Kim Reviewed-by: Douglas Freimuth Reviewed-by: Matthew Rosato Signed-off-by: Christian Borntraeger --- Documentation/virt/kvm/devices/s390_flic.rst | 15 +++++++- arch/s390/include/asm/kvm_host.h | 5 +++ arch/s390/kvm/interrupt.c | 56 +++++++++++++++++++++------- 3 files changed, 60 insertions(+), 16 deletions(-) diff --git a/Documentation/virt/kvm/devices/s390_flic.rst b/Documentation/virt/kvm/devices/s390_flic.rst index b784f8016748..983c858b444a 100644 --- a/Documentation/virt/kvm/devices/s390_flic.rst +++ b/Documentation/virt/kvm/devices/s390_flic.rst @@ -112,9 +112,20 @@ Groups: mask or unmask the adapter, as specified in mask KVM_S390_IO_ADAPTER_MAP - This is now a no-op. The mapping is purely done by the irq route. + Map an adapter indicator or summary page for long-term pinning so that + interrupt injection can be performed in atomic context. If long-term + pinning is not possible (e.g. file-backed memory), the page is verified + via a short-term pin and the ioctl returns success; interrupt injection + will use the non-atomic irqfd path with short-term pinning on each + interrupt. In Secure Execution mode this is a no-op and the ioctl + returns success. + KVM_S390_IO_ADAPTER_UNMAP - This is now a no-op. The mapping is purely done by the irq route. + Unmap a previously mapped adapter indicator or summary page and release + the long-term pin. If the page was not long-term pinned (e.g. file-backed + memory), the map entry is removed and success is returned; if no prior + map entry exists, -ENOENT is returned. In Secure Execution mode this is + a no-op and the ioctl returns success. KVM_DEV_FLIC_AISM modify the adapter-interruption-suppression mode for a given isc if the diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h index eaa34c5bd3c1..c172f9b212d1 100644 --- a/arch/s390/include/asm/kvm_host.h +++ b/arch/s390/include/asm/kvm_host.h @@ -476,6 +476,11 @@ struct s390_map_info { __u64 guest_addr; __u64 addr; struct page *page; + /* + * True if the page is long-term pinned. False if long-term pinning + * failed and this entry exists only to preserve MAP/UNMAP symmetry. + */ + bool pinned; }; struct s390_io_adapter { diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index 9e3e6b0d72ad..7514d9e2403c 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -2520,8 +2520,22 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) map->addr = host_addr; map->page = pin_map_page(kvm, host_addr, FOLL_LONGTERM); if (!map->page) { - ret = -EINVAL; - goto out; + /* + * Long-term pinning may fail for memory types such as file-backed + * memory. Verify that short-term pinning succeeds so that the + * non-atomic irqfd path can handle interrupt injection. + */ + map->page = pin_map_page(kvm, host_addr, 0); + if (!map->page) { + ret = -EINVAL; + goto out; + } + unpin_user_page(map->page); + map->page = NULL; + map->pinned = false; + /* Add an entry to preserve MAP/UNMAP symmetry. */ + } else { + map->pinned = true; } spin_lock_irqsave(&adapter->maps_lock, flags); if (adapter->nr_maps < MAX_S390_ADAPTER_MAPS) { @@ -2532,7 +2546,7 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr) ret = -EINVAL; } spin_unlock_irqrestore(&adapter->maps_lock, flags); - if (ret) + if (ret && map->page) unpin_user_page(map->page); out: if (ret) @@ -2546,6 +2560,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) struct s390_map_info *map, *tmp, *map_to_free; struct page *map_page_to_put = NULL; u64 map_addr_to_mark = 0; + bool map_pinned = false; unsigned long flags; int found = 0, idx; @@ -2560,6 +2575,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) list_del(&map->list); map_page_to_put = map->page; map_addr_to_mark = map->guest_addr; + map_pinned = map->pinned; map_to_free = map; break; } @@ -2568,11 +2584,18 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr) if (found) { kfree(map_to_free); - idx = srcu_read_lock(&kvm->srcu); - mark_page_dirty(kvm, map_addr_to_mark >> PAGE_SHIFT); - set_page_dirty_lock(map_page_to_put); - srcu_read_unlock(&kvm->srcu, idx); - unpin_user_page(map_page_to_put); + if (map_pinned) { + /* + * Only long-term pinned pages need to be marked dirty + * and released. Fallback entries exist only for + * MAP/UNMAP symmetry. + */ + idx = srcu_read_lock(&kvm->srcu); + mark_page_dirty(kvm, map_addr_to_mark >> PAGE_SHIFT); + set_page_dirty_lock(map_page_to_put); + srcu_read_unlock(&kvm->srcu, idx); + unpin_user_page(map_page_to_put); + } } return found ? 0 : -ENOENT; @@ -2598,11 +2621,13 @@ void kvm_s390_unmap_all_adapters(struct kvm *kvm) list_for_each_entry_safe(map, tmp, &local_list, list) { list_del(&map->list); - idx = srcu_read_lock(&kvm->srcu); - mark_page_dirty(kvm, map->guest_addr >> PAGE_SHIFT); - set_page_dirty_lock(map->page); - srcu_read_unlock(&kvm->srcu, idx); - unpin_user_page(map->page); + if (map->pinned) { + idx = srcu_read_lock(&kvm->srcu); + mark_page_dirty(kvm, map->guest_addr >> PAGE_SHIFT); + set_page_dirty_lock(map->page); + srcu_read_unlock(&kvm->srcu, idx); + unpin_user_page(map->page); + } kfree(map); } } @@ -2929,8 +2954,11 @@ static struct s390_map_info *get_map_info(struct s390_io_adapter *adapter, return NULL; list_for_each_entry(map, &adapter->maps, list) { - if (map->addr == addr) + if (map->addr == addr) { + if (!map->pinned) + return NULL; return map; + } } return NULL; } -- cgit v1.2.3 From 0e65cd9e5d41c34f86b7c347967bedac54926041 Mon Sep 17 00:00:00 2001 From: Paolo Bonzini Date: Tue, 21 Jul 2026 18:31:49 +0200 Subject: KVM: VMX: add memory clobber to asm for VMX instructions VMCLEAR/VMREAD/VMWRITE/VMPTRLD access the internal VMCS cache, which is not visible to the compiler; without a memory clobber, the compiler can reorder them in troublesome ways because "asm volatile" and "asm goto" only protect against removal of the asm. For example, placing a VMWRITE before the corresponding VMCS pointer is loaded can lead to corruption. While none of this has been observed, it is better to prevent than cure. Likewise, INVEPT and INVVPID access the TLB and, even though in their case the effect is only visible to the next VMLAUNCH/VMRESUME, it is technically correct to add the clobber there too. So avoid any urge to special case them, and simply hardcode "memory" into the clobber list of vmx_asm1() and vmx_asm2(). __vmcs_readl() open-codes its own asm, so add the clobber there as well. Link: https://lore.kernel.org/kvm/CABgObfbL3t21yVeSwiLSjjOUER+rTYDPHYAH9YU4TWGRjx6XHg@mail.gmail.com/ Cc: Sean Christopherson Cc: stable@vger.kernel.org Signed-off-by: Paolo Bonzini --- arch/x86/kvm/vmx/vmx_ops.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx_ops.h b/arch/x86/kvm/vmx/vmx_ops.h index 81784befaaf4..9ffe03a0c223 100644 --- a/arch/x86/kvm/vmx/vmx_ops.h +++ b/arch/x86/kvm/vmx/vmx_ops.h @@ -101,7 +101,7 @@ static __always_inline unsigned long __vmcs_readl(unsigned long field) : [output] "=r" (value) : [field] "r" (field) - : "cc" + : "cc", "memory" : do_fail, do_exception); return value; @@ -145,7 +145,7 @@ do_exception: : ASM_CALL_CONSTRAINT, [output] "=&r" (value) : [field] "r" (field) - : "cc"); + : "cc", "memory"); return value; #endif /* CONFIG_CC_HAS_ASM_GOTO_OUTPUT */ @@ -192,7 +192,7 @@ do { \ asm goto("1: " __stringify(insn) " %0\n\t" \ "jna %l[error]\n\t" \ _ASM_EXTABLE(1b, %l[fault]) \ - : : op1 : "cc" : error, fault); \ + : : op1 : "cc", "memory" : error, fault); \ return; \ error: \ instrumentation_begin(); \ @@ -208,7 +208,7 @@ do { \ asm goto("1: " __stringify(insn) " %1, %0\n\t" \ "jna %l[error]\n\t" \ _ASM_EXTABLE(1b, %l[fault]) \ - : : op1, op2 : "cc" : error, fault); \ + : : op1, op2 : "cc", "memory" : error, fault);\ return; \ error: \ instrumentation_begin(); \ -- cgit v1.2.3 From 9910e835580fef3bef53b70241dd00c4bffad693 Mon Sep 17 00:00:00 2001 From: Weiming Shi Date: Mon, 27 Jul 2026 10:17:18 -0700 Subject: KVM: x86: Cancel delayed I/O APIC EOI handling before destroying vCPUs Cancel (and flush) the I/O APIC's delayed EOI handling work during the "pre VM destroy" phase, before vCPUs are destroyed, as processing the EOI broadcast will inject another IRQ if the line is asserted, i.e. will try to deliver an IRQ to the target vCPU(s). Canceling the work after vCPUs are destroyed leads to UAF if the delayed work is processed after vCPUs are destroyed. BUG: KASAN: slab-use-after-free in __kvm_irq_delivery_to_apic_fast+0x9bf/0xa20 arch/x86/kvm/lapic.c:1250 Read of size 8 at addr ffff8880499abea0 by task kworker/1:2/1218 CPU: 1 UID: 0 PID: 1218 Comm: kworker/1:2 Not tainted 7.1.0-rc7 #5 PREEMPT(lazy) Hardware name: QEMU Ubuntu 25.10 PC v2 (i440FX + PIIX, + 10.1 machine, 1996), BIOS 1.16.3-debian-1.16.3-2 04/01/2014 Workqueue: events kvm_ioapic_eoi_inject_work Call Trace: __dump_stack lib/dump_stack.c:94 dump_stack_lvl+0x100/0x190 lib/dump_stack.c:120 print_address_description mm/kasan/report.c:378 print_report+0x139/0x4ad mm/kasan/report.c:482 kasan_report+0xe4/0x1d0 mm/kasan/report.c:595 __kvm_irq_delivery_to_apic_fast+0x9bf/0xa20 arch/x86/kvm/lapic.c:1250 __kvm_irq_delivery_to_apic+0xd8/0xbf0 arch/x86/kvm/lapic.c:1345 kvm_irq_delivery_to_apic arch/x86/kvm/lapic.h:129 ioapic_service+0x308/0x590 arch/x86/kvm/ioapic.c:492 kvm_ioapic_eoi_inject_work+0x13c/0x190 arch/x86/kvm/ioapic.c:532 process_one_work+0xa59/0x19a0 kernel/workqueue.c:3314 process_scheduled_works kernel/workqueue.c:3397 worker_thread+0x5eb/0xe50 kernel/workqueue.c:3478 kthread+0x370/0x450 kernel/kthread.c:436 ret_from_fork+0x72b/0xd30 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 Note, the VM is unreachable once kvm_destroy_vm() starts, and scheduling new work via kvm_ioapic_send_eoi() can only be done via KVM_RUN, i.e. requires a live vCPU. Alternatively, KVM could simply destroy the I/O APIC during the "pre" phase of VM destruction, but that gets more than a bit sketchy as KVM expects the I/O APIC to exist if ioapic_in_kernel() is true, and nested virtualization in particular has a bad habit of touching VM-scope state during vCPU destruction. E.g. attempting to free the PIC during the pre phase would lead to a NULL pointer dereference in kvm_cpu_has_extint(), and it's not hard to imagine the I/O APIC having a similar flaw. Fixes: 17bcd7144263 ("KVM: x86: Free vCPUs before freeing VM state") Reported-by: Reported-by: Zhong Wang Reported-by: Xuanqing Shi Cc: stable@vger.kernel.org Signed-off-by: Weiming Shi Co-developed-by: Sean Christopherson Signed-off-by: Sean Christopherson Message-ID: <20260727171718.543491-1-seanjc@google.com> Signed-off-by: Paolo Bonzini --- arch/x86/kvm/x86.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index afcac1042947..47cb9eba113b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -13429,9 +13429,15 @@ void kvm_arch_pre_destroy_vm(struct kvm *kvm) * iterating over vCPUs in a different task while vCPUs are being freed * is unsafe, i.e. will lead to use-after-free. The PIT also needs to * be stopped before IRQ routing is freed. + * + * Do NOT free the in-kernel PIC or I/O APIC here (but as above, make + * sure to flush any background work), as KVM expects interrupt routing + * structures to be valid until vCPUs are destroyed. */ #ifdef CONFIG_KVM_IOAPIC kvm_free_pit(kvm); + if (kvm->arch.vioapic) + cancel_delayed_work_sync(&kvm->arch.vioapic->eoi_inject); #endif kvm_mmu_pre_destroy_vm(kvm); -- cgit v1.2.3 From 0f38453cdb2e17566ccb7c0f3dabd5bd21caca26 Mon Sep 17 00:00:00 2001 From: Jinu Kim Date: Tue, 21 Jul 2026 19:35:11 +0900 Subject: KVM: x86/mmu: Check write tracking in all address spaces kvm_gfn_is_write_tracked() checks only the supplied memslot, but page tracking is per-address-space and shadow pages are shared across all address spaces. With SMM, a GFN can therefore be write-tracked in one address space and appear untracked through the other. Check the supplied slot first, then the slot for the other address space. This ensures all callers honor write tracking regardless of the active address space. In particular, it prevents mmu_try_to_unsync_pages() from marking an upper-level shadow page unsync and eventually triggering the BUG in pte_list_remove(). Fixes: 699023e23965 ("KVM: x86: add SMM to the MMU role, support SMRAM address space") Assisted-by: Codex:GPT-5 Signed-off-by: Jinu Kim Message-ID: <20260721103512.2136240-2-kimjw04271234@gmail.com> [invert direction of the conditional. - Paolo] Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/page_track.c | 32 ++++++++++++++++++++++++++------ 1 file changed, 26 insertions(+), 6 deletions(-) diff --git a/arch/x86/kvm/mmu/page_track.c b/arch/x86/kvm/mmu/page_track.c index 1b17b12393a8..7e8195a311bb 100644 --- a/arch/x86/kvm/mmu/page_track.c +++ b/arch/x86/kvm/mmu/page_track.c @@ -130,13 +130,23 @@ void __kvm_write_track_remove_gfn(struct kvm *kvm, kvm_mmu_gfn_allow_lpage(slot, gfn); } -/* - * check if the corresponding access on the specified guest page is tracked. - */ +static bool __kvm_gfn_is_write_tracked(const struct kvm_memory_slot *slot, + gfn_t gfn) +{ + int index; + + if (!slot) + return false; + + index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K); + return !!READ_ONCE(slot->arch.gfn_write_track[index]); +} + +/* check if write access is tracked on the specified guest page. */ bool kvm_gfn_is_write_tracked(struct kvm *kvm, const struct kvm_memory_slot *slot, gfn_t gfn) { - int index; + const struct kvm_memory_slot *other_slot; if (!slot) return false; @@ -144,8 +154,18 @@ bool kvm_gfn_is_write_tracked(struct kvm *kvm, if (!kvm_page_track_write_tracking_enabled(kvm)) return false; - index = gfn_to_index(gfn, slot->base_gfn, PG_LEVEL_4K); - return !!READ_ONCE(slot->arch.gfn_write_track[index]); + BUILD_BUG_ON(KVM_MAX_NR_ADDRESS_SPACES > 2); + + if (__kvm_gfn_is_write_tracked(slot, gfn)) + return true; + + if (kvm_arch_nr_memslot_as_ids(kvm) > 1) { + other_slot = __gfn_to_memslot(__kvm_memslots(kvm, slot->as_id ^ 1), gfn); + if (__kvm_gfn_is_write_tracked(other_slot, gfn)) + return true; + } + + return false; } #ifdef CONFIG_KVM_EXTERNAL_WRITE_TRACKING -- cgit v1.2.3 From 2e8a2c1b03068d76782343446f1b2114ae2ee0bd Mon Sep 17 00:00:00 2001 From: Jinu Kim Date: Tue, 21 Jul 2026 19:35:12 +0900 Subject: KVM: x86/mmu: Check all address spaces before skipping unsync mmu_try_to_unsync_pages() skips the shadow-page lookup when the supplied memslot allows a hugepage, because a shadow page would disallow hugepages. But hugepage metadata is per-address-space while shadow pages are shared across all address spaces. With SMM, the other address space can therefore have a shadow page even when the supplied memslot allows a hugepage. Check the corresponding memslot in the other address space before taking the fast path. Skip the shadow-page lookup only when all address spaces allow a hugepage. Fixes: b3ae3ceb5569 ("KVM: x86/mmu: KVM: x86/mmu: Skip unsync when large pages are allowed") Assisted-by: Codex:GPT-5 Signed-off-by: Jinu Kim [invert direction of the conditional. - Paolo] Message-ID: <20260721103512.2136240-3-kimjw04271234@gmail.com> Signed-off-by: Paolo Bonzini --- arch/x86/kvm/mmu/mmu.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 22cf222d3033..66e69d2a41b3 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -722,6 +722,26 @@ static struct kvm_lpage_info *lpage_info_slot(gfn_t gfn, return &slot->arch.lpage_info[level - 2][idx]; } +static bool kvm_gfn_is_lpage_allowed(struct kvm *kvm, + const struct kvm_memory_slot *slot, + gfn_t gfn, int level) +{ + const struct kvm_memory_slot *other_slot; + + BUILD_BUG_ON(KVM_MAX_NR_ADDRESS_SPACES > 2); + + if (lpage_info_slot(gfn, slot, level)->disallow_lpage) + return false; + + if (kvm_arch_nr_memslot_as_ids(kvm) > 1) { + other_slot = __gfn_to_memslot(__kvm_memslots(kvm, slot->as_id ^ 1), gfn); + if (other_slot && lpage_info_slot(gfn, other_slot, level)->disallow_lpage) + return false; + } + + return true; +} + /* * The most significant bit in disallow_lpage tracks whether or not memory * attributes are mixed, i.e. not identical for all gfns at the current level. @@ -2968,7 +2988,7 @@ int mmu_try_to_unsync_pages(struct kvm *kvm, const struct kvm_memory_slot *slot, * write-protected (see above), thus if the gfn can be mapped with a * hugepage and isn't write-tracked, it can't have a shadow page. */ - if (!lpage_info_slot(gfn, slot, PG_LEVEL_2M)->disallow_lpage) + if (kvm_gfn_is_lpage_allowed(kvm, slot, gfn, PG_LEVEL_2M)) return 0; /* -- cgit v1.2.3 From 7d3aae206663c4e006b25a1c7a20a4029e67da76 Mon Sep 17 00:00:00 2001 From: Sean Christopherson Date: Fri, 10 Jul 2026 09:20:51 -0700 Subject: KVM: SVM: Update x2APIC MSR intercepts if AVIC is inhibited while L2 is active Always update x2APIC MSR intercepts for L1 when AVIC is deactivated, even if L2 is active and KVM is using a separate MSR bitmap to run L2. If AVIC is fully enabled prior to running L2, and is then inhibited while L2 is active (for a VM-scoped inhibit), then KVM will run L1 with AVIC disabled, but with x2APIC MSR intercepts disabled, i.e. will allow L1 to read most of the host's APIC state, send arbitrary interrupts, change task priority, and ultimately trivially DoS the host. E.g. sending a self-IPI in L1 on HYPERV_REENLIGHTENMENT_VECTOR, 0xee, with CONFIG_HYPERV=n in the host kernel as a "safe" PoC, yields: Spurious interrupt (vector 0xee) on CPU#425. Acked And hacking KVM to abuse kvm_set_posted_intr_wakeup_handler() to register a handler and WARN on POSTED_INTR_WAKEUP_VECTOR yields: ------------[ cut here ]------------ WARNING: arch/x86/kvm/svm/svm.c:5594 at pi_wakeup_handler+0x9/0x10 [kvm_amd], CPU#156: nested_x2apic_t/316940 CPU: 156 UID: 0 PID: 316940 Comm: nested_x2apic_t Tainted: G S U Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER Hardware name: Google Astoria-Turin/astoria, BIOS 0.20260209.0-0 02/09/2026 RIP: 0010:pi_wakeup_handler+0x9/0x10 [kvm_am