Merge tag 'drm-next-2026-08-20' of https://gitlab.freedesktop.org/drm/kernel

Pull drm updates from Dave Airlie:
 "Highlights:

   - dmemcg eviction support is good for low VRAM things like Steam
     Machine

   - AMD adds gfx6-8 modifier support for older GPUs that enables a
     bunch of wayland stuff

   - i915/xe has some new hw support but also a lot of display
     refactoring

  Everything:

  perf:
   - export perf_allow_ APIs for xe

  udmabuf:
   - remove default size limit of 64MB

  rust:
   - i/o rework (signed tag from driver-core tree)
   - add registration guard and registration data
   - fix unbounded lifetimes in ioctl handler args
   - fix a drm_dev_register race
   - gem_shmem: add DmaResvGuard helper
   - gpuvm: require send/sync for driver data
       - implement send/sync for GpuVaAlloc and GpuVmBo
       - add SmContext lifetime
   - rename dma_handle to dma_address
   - change pci_sriov_get_totalvfs return to unsigned int

  core:
   - create drm_of_get_panel_orientation
   - send per-connector hotplug events
   - add thunderbolt UBHR tunneling support

  connector:
   - add color format property

  dmem:
   - introduce a peak file
   - accept one region per limit
   - add dmemcg support for eviction

  gpusvm:
   - reorg code to give drivers more flexibility

  atomic:
   - add create_state callback and helper
   - add documentation on atomic commit lifetime

  buddy:
   - add per-order free
   - add used block scoreboard
   - fix UAF
   - test buffer clearance on resume
   - add phys_addr->block helper

  gem:
   - drop DRIVER_GEM_GPUVA flag

  ttm:
   - be more aggressive allocating below protection limit

  sched:
   - add test suite for concurrent job submissions

  hdmi:
   - hook the color format property in helpers

  mipi-dsi:
   - add MIPI_DSI_MODE_DSC_ALL_SLICES_IN_PKT

  bridge:
   - add atomic create callbacks
   - drop atomic reset
   - display-connector: don't autoenable HPD IRQ
   - trigger initial HPD for DP
   - ti-sn65dsi83: remove NO_HFP and NO_HBP mode flags
   - analogix_dp: switch to DP link training helpers

  dp:
   - add support for DSC max delta BPP

  edid:
   - parse panel type from DisplayID 2.x Display Parameters

  sysfb:
   - improve panel, stride, framebuffer size validation

  panel:
   - implement ref counting for struct drm_panel
   - himax-hx83121a: add backlight regulator support
   - novatek-nt36672a: Inline panel init sequences
   - visionox-vtdr6130: enable DSC
   - novatek-nt37801: Use mipi_dsi_*_multi() functions
   - samsung-s6d16d0: Fix prepare error handling
   - support Novatek NT36536 plus DT bindings
   - sofef00: fix backlight updates
   - osd101t2587: use mipi_dsi_*_multi interface
   - panel-edp: adjust timing for AUO displays
   - panel-lvds: support Opto Logic SCX1001511GGC49
   - panel-simple: support Kyocera tcg070wvlq
   - panel-edp: quirks
       - AUO B116XAT04.3, CMN N116BCP-EA2, CSW MNB601LS1-8
       - BOE NV116WH2-M30, BOE NT116WHM-N21, BOE NV116FH1-M31
       - BOE NV116FH1-M30, NV140FHM-N5B, TM156VDXP25
       - BOE NE160QDM-NY1, MB116AS01
   - new:
       - Samsung ATNA40HQ08-0, Anbernic TD4310
       - Chipone ICNA35XX, Ilitek ILI9488
       - Ilitek ILI7807S, Renesas R63419
       - MNE001BS6-2, MNF601BS4-1, Sharp LQ120P1JX51

  virtio:
   - add support for save/restore virtio_gpu_objects
   - abort vq wait on device removal

  amdgpu:
   - add color format DRM property
   - initial compute pipe reset support
   - add GFX 6-8 modifier support
   - initial DCN 6.0.0 support
   - dmemcg eviction support
   - improved boundary checking for bios parsing
   - RAS updates and rework
   - VCN secure submission fixes
   - 8K panel fix
   - Display KUNIT tests
   - parse panel type from DisplayID
   - Align IP discovery to pci device lifetime
   - SOC15 register macro cleanups
   - UVD memory placement fixes
   - GFX9 mode2 reset fixes
   - drop unnecessary BUG/BUG_ON
   - GFX8 soft reset rework
   - enable soft reset on GFX8
   - PSP/SMU 15.0.9 update
   - VI ASPM fix
   - userq fixes
   - amdgpu_vm_get_task_info_pasid lifetime fix
   - DC CACP support
   - change system_unbound_wq with system_dfl_wq
   - Loosen VFCT bios parsing to deal with pci=realloc
   - SI/SMU7 AC/DC switch fix
   - VM fence handling fix
   - GEM close optimisation
   - Apple Studio Display fixes
   - DC FRL fixes

  amdkfd:
   - initial compute pipe reset support
   - allow applications to opt out of sigbus on fatal errors
   - improve CRIU boundary checks
   - MQD handling rework
   - move TBA/TMA from system to device memory
   - avoid topology-lock in kfd_mmap
   - SVM eviction fixes

  radeon:
   - fix unset CONFIG_ACPI build

  i915:
   - Novalake (NVL display version 35) timing generator enabling
   - NVL DC3CO enabling
   - enable UBHR link rates on thunderbolt tunnels
   - Reduce Xe3+ PM demand peak bandwidth
   - enable pipe DMC error interrupts for display 30+
   - add kunit tests for DP link config selection
   - refactor and document DP link recovery
   - i915/xe driver display probe/remove/suspend/resume/shutdown cleanup
     and unification
   - i915/xe display runtime PM unified
   - Break i915 and xe panic dependency on struct intel_framebuffer
   - Streamline Pre/Post-CSC LUT loops
   - drop TGL DC3DO support
   - CDCLK santization
   - fix HDMI scrambling enable
   - fix phys bo pread/pwrite with offset
   - add missing nospec on parallel submit slot
   - fix some NULL derefs

  xe:
   - drop force_execlist module param
   - gate observation streams with perf_allow_cpu
   - skip FORCE_WC and vm_bound check for external dma-bufs
   - dmemcg eviction support
   - remove unused NVL-S GuC
   - TLB invalidation improvements
   - NVL-S updated PCI-IDs and w/a
   - madvise: optimise invalidation path
   - fix infinite gt-reset loop in timeout recovery
   - update TTM device benefical_order
   - wait on external BO kernel fences in exec ioctl
   - add/use more KLV helpers
   - sriov: disable display in admin only PF mode
   - add RAS GPU health indicator
   - optimise TTM populate for DONTNEED BO
   - drop force_probe for NVL-s
   - add debugfs for pcode info

  amdxdna:
   - disable device buffer export

  nova:
   - build nova-core/nova-drm from drivers/gpu
   - export nova-core rust symbols (workaround)
   - GSP boot process consolidation
   - Boot GSP with vGPU enabled
   - TLV firmware image format support
   - Hopper/Blackwell fixes and cleanups
   - I/O projection adoption

  tyr:
   - firmware loading and MCU boot
   - add generic slot manager + MMU
   - GPU VM support ARM64 LPAE page tables
   - add kernel buffer object for internal allocations
   - add parser for Mali CSF
   - add MCU booting

  nouveau:
   - race fixes
   - check instmem iomapping at first use
   - add dmemcg support
   - expose NVDEC channels
   - add scanline position/head state support for GSP

  qxl:
   - convert simple encoder to regular

  ethosu:
   - add perf counter support

  etnaviv:
   - force flush on power register ops

  msm:
   - support DSC configuration with slice_per_pkt > 1

  mxsfb:
   - fix disable sequence

  panthor:
   - support sparse mappings

  rockchip:
   - switch away from simple helpers
   - support YUV background color
   - fix layer config timeout
   - add edp support for rk3576
   - add batch command submission function

  rocket:
   - error handling and NULL ptr deref fixes

  sun4i:
   - switch away from simple helpers

  imagination:
   - mark BXM-4-64 MC1 as support

  host1x:
   - support tegra264

  tegra:
   - add DSI for tegra 20/30

  v3d:
   - reduce PM runtime autosuspend delay
   - scheduler fixes and refactoring
   - deprecate v3d 3.3 and 4.1
   - validate CPU job query boundaries

  hibmc:
   - improve plane format handling
   - switch to gem shmem

  mediatek:
   - cec: correct compat for mt7623-8167?

  exynos:
   - remove simple dependency
   - add error handling to encoder paths
   - take i2c adapter module reference"

* tag 'drm-next-2026-08-20' of https://gitlab.freedesktop.org/drm/kernel: (2074 commits)
  drm/xe/mcr: Take vcs1/vecs1 into account for first media slice
  drm/xe: Fix a bug in pc_adjust_freq_bounds()
  drm/xe: Fix xe_device_probe() failure
  drm/xe/drm_ras: Move has_drm_ras check to drm_ras layer
  drm/xe/ras: Fix boot-time ras error processing
  drm/amd/display: make DC_RUN_WITH_PREEMPTION_ENABLED misuse a build error
  drm/amd/pm: silence uninitialized variable warnings
  drm/amdgpu: skip BOs being torn down during GTT recovery
  drm/amdgpu: Reject UVD message with invalid number of h265 refs
  drm/amdgpu: keep PRT mappings off the vm_bo state lists
  drm/amdgpu: fix nbif 6.3.1 l1 low power not functional
  drm/amd/display: fix BT.2020 YCbCr output CSC matrices for DCE
  drm/amd/display: fix BT.2020 YCbCr limited output CSC matrix
  drm/amdgpu: Implement insert_end for VCE 3
  drm/amdgpu: Fix UVD min buffer sizes
  drm/amdgpu: Fix UVD decode image min size calculation
  drm/amdgpu: Fix UVD dpb min size calculation for H264
  drm/amdgpu: Reject UVD message with dimensions above 4096
  drm/amdgpu: check ASPM on the dGPU host link
  drm/radeon: fix autosuspend cleanup during teardown
  ...

16 files changed, 1162 insertions, 325 deletions

diff --git a/rust/kernel/alloc/kvec.rs b/rust/kernel/alloc/kvec.rs
index f7af62835aa8..c7546b9da4fa 100644
--- a/rust/kernel/alloc/kvec.rs
+++ b/rust/kernel/alloc/kvec.rs
@@ -9,6 +9,7 @@ use super::{
         Vmalloc,
         VmallocPageIter, //
     },
+    flags::__GFP_ZERO,
     layout::ArrayLayout,
     AllocError,
     Allocator,
@@ -51,6 +52,8 @@ use core::{
     }, //
 };
 
+use pin_init::Zeroable;
+
 mod errors;
 pub use self::errors::{InsertError, PushError, RemoveError};
 
@@ -532,6 +535,30 @@ where
         Ok(v)
     }
 
+    /// Creates a new [`Vec`] with `n` zero-initialized elements.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// let v = KVec::<u32>::zeroed(20, GFP_KERNEL)?;
+    ///
+    /// assert!(v.iter().all(|&x| x == 0));
+    /// # Ok::<(), Error>(())
+    /// ```
+    pub fn zeroed(n: usize, flags: Flags) -> Result<Self, AllocError>
+    where
+        T: Zeroable,
+    {
+        let mut v = Self::with_capacity(n, flags | __GFP_ZERO)?;
+
+        // SAFETY:
+        // - `n <= capacity - len`: `with_capacity(n)` guarantees capacity >= n, len is 0.
+        // - All elements in `[0, n)` are initialized: `__GFP_ZERO` zeroes the allocation,
+        //   and `T: Zeroable` guarantees all-zeroes is a valid bit pattern.
+        unsafe { v.inc_len(n) };
+        Ok(v)
+    }
+
     /// Creates a `Vec<T, A>` from a pointer, a length and a capacity using the allocator `A`.
     ///
     /// # Examples
diff --git a/rust/kernel/dma.rs b/rust/kernel/dma.rs
index 9a0bf745614c..2ce09f8e90c6 100644
--- a/rust/kernel/dma.rs
+++ b/rust/kernel/dma.rs
@@ -585,7 +585,7 @@ impl<T: AsBytes + FromBytes + KnownSize + ?Sized> From<CoherentBox<T>> for Coher
 /// # Invariants
 ///
 /// - For the lifetime of an instance of [`Coherent`], the `cpu_addr` is a valid pointer
-///   to an allocated region of coherent memory and `dma_handle` is the DMA address base of the
+///   to an allocated region of coherent memory and `dma_addr` is the DMA address base of the
 ///   region.
 /// - The size in bytes of the allocation is equal to size information via pointer.
 // TODO
@@ -602,7 +602,7 @@ impl<T: AsBytes + FromBytes + KnownSize + ?Sized> From<CoherentBox<T>> for Coher
 // entire `Coherent` including the allocated memory itself.
 pub struct Coherent<T: KnownSize + ?Sized> {
     dev: ARef<device::Device>,
-    dma_handle: DmaAddress,
+    dma_addr: DmaAddress,
     cpu_addr: NonNull<T>,
     dma_attrs: Attrs,
 }
@@ -627,11 +627,10 @@ impl<T: KnownSize + ?Sized> Coherent<T> {
         self.cpu_addr.as_ptr()
     }
 
-    /// Returns a DMA handle which may be given to the device as the DMA address base of
-    /// the region.
+    /// Returns a DMA address which may be given to the device as the base of the region.
     #[inline]
-    pub fn dma_handle(&self) -> DmaAddress {
-        self.dma_handle
+    pub fn dma_address(&self) -> DmaAddress {
+        self.dma_addr
     }
 
     /// Returns a reference to the data in the region.
@@ -678,13 +677,13 @@ impl<T: AsBytes + FromBytes> Coherent<T> {
             );
         }
 
-        let mut dma_handle = 0;
+        let mut dma_addr = 0;
         // SAFETY: Device pointer is guaranteed as valid by the type invariant on `Device`.
         let addr = unsafe {
             bindings::dma_alloc_attrs(
                 dev.as_raw(),
                 core::mem::size_of::<T>(),
-                &mut dma_handle,
+                &mut dma_addr,
                 gfp_flags.as_raw(),
                 dma_attrs.as_raw(),
             )
@@ -696,7 +695,7 @@ impl<T: AsBytes + FromBytes> Coherent<T> {
         // - We also hold a refcounted reference to the device.
         Ok(Self {
             dev: dev.into(),
-            dma_handle,
+            dma_addr,
             cpu_addr,
             dma_attrs,
         })
@@ -795,13 +794,13 @@ impl<T: AsBytes + FromBytes> Coherent<T> {
         }
 
         let size = core::mem::size_of::<T>().checked_mul(len).ok_or(ENOMEM)?;
-        let mut dma_handle = 0;
+        let mut dma_addr = 0;
         // SAFETY: Device pointer is guaranteed as valid by the type invariant on `Device`.
         let addr = unsafe {
             bindings::dma_alloc_attrs(
                 dev.as_raw(),
                 size,
-                &mut dma_handle,
+                &mut dma_addr,
                 gfp_flags.as_raw(),
                 dma_attrs.as_raw(),
             )
@@ -813,7 +812,7 @@ impl<T: AsBytes + FromBytes> Coherent<T> {
         // - We also hold a refcounted reference to the device.
         Ok(Coherent {
             dev: dev.into(),
-            dma_handle,
+            dma_addr,
             cpu_addr,
             dma_attrs,
         })
@@ -927,14 +926,14 @@ impl<T: KnownSize + ?Sized> Drop for Coherent<T> {
     fn drop(&mut self) {
         let size = T::size(self.cpu_addr.as_ptr());
         // SAFETY: Device pointer is guaranteed as valid by the type invariant on `Device`.
-        // The cpu address, and the dma handle are valid due to the type invariants on
+        // The cpu address, and the dma address are valid due to the type invariants on
         // `Coherent`.
         unsafe {
             bindings::dma_free_attrs(
                 self.dev.as_raw(),
                 size,
                 self.cpu_addr.as_ptr().cast(),
-                self.dma_handle,
+                self.dma_addr,
                 self.dma_attrs.as_raw(),
             )
         }
@@ -993,13 +992,13 @@ impl<T: KnownSize + AsBytes + ?Sized> debugfs::BinaryWriter for Coherent<T> {
 ///
 /// - `cpu_handle` holds the opaque handle returned by `dma_alloc_attrs` with
 ///   `DMA_ATTR_NO_KERNEL_MAPPING` set, and is only valid for passing back to `dma_free_attrs`.
-/// - `dma_handle` is the corresponding bus address for device DMA.
+/// - `dma_addr` is the corresponding bus address for device DMA.
 /// - `size` is the allocation size in bytes as passed to `dma_alloc_attrs`.
 /// - `dma_attrs` contains the attributes used for the allocation, always including
 ///   `DMA_ATTR_NO_KERNEL_MAPPING`.
 pub struct CoherentHandle {
     dev: ARef<device::Device>,
-    dma_handle: DmaAddress,
+    dma_addr: DmaAddress,
     cpu_handle: NonNull<c_void>,
     size: usize,
     dma_attrs: Attrs,
@@ -1023,13 +1022,13 @@ impl CoherentHandle {
         }
 
         let dma_attrs = dma_attrs | Attrs(bindings::DMA_ATTR_NO_KERNEL_MAPPING);
-        let mut dma_handle = 0;
+        let mut dma_addr = 0;
         // SAFETY: `dev.as_raw()` is valid by the type invariant on `device::Device`.
         let cpu_handle = unsafe {
             bindings::dma_alloc_attrs(
                 dev.as_raw(),
                 size,
-                &mut dma_handle,
+                &mut dma_addr,
                 gfp_flags.as_raw(),
                 dma_attrs.as_raw(),
             )
@@ -1038,11 +1037,11 @@ impl CoherentHandle {
         let cpu_handle = NonNull::new(cpu_handle).ok_or(ENOMEM)?;
 
         // INVARIANT: `cpu_handle` is the opaque handle from a successful `dma_alloc_attrs` call
-        // with `DMA_ATTR_NO_KERNEL_MAPPING`, `dma_handle` is the corresponding DMA address,
+        // with `DMA_ATTR_NO_KERNEL_MAPPING`, `dma_addr` is the corresponding DMA address,
         // and we hold a refcounted reference to the device.
         Ok(Self {
             dev: dev.into(),
-            dma_handle,
+            dma_addr,
             cpu_handle,
             size,
             dma_attrs,
@@ -1059,12 +1058,12 @@ impl CoherentHandle {
         Self::alloc_with_attrs(dev, size, gfp_flags, Attrs(0))
     }
 
-    /// Returns the DMA handle for this allocation.
+    /// Returns the DMA address for this allocation.
     ///
     /// This address can be programmed into device hardware for DMA access.
     #[inline]
-    pub fn dma_handle(&self) -> DmaAddress {
-        self.dma_handle
+    pub fn dma_address(&self) -> DmaAddress {
+        self.dma_addr
     }
 
     /// Returns the size in bytes of this allocation.
@@ -1083,28 +1082,29 @@ impl Drop for CoherentHandle {
                 self.dev.as_raw(),
                 self.size,
                 self.cpu_handle.as_ptr(),
-                self.dma_handle,
+                self.dma_addr,
                 self.dma_attrs.as_raw(),
             )
         }
     }
 }
 
-// SAFETY: `CoherentHandle` only holds a device reference, a DMA handle, an opaque CPU handle,
+// SAFETY: `CoherentHandle` only holds a device reference, a DMA address, an opaque CPU handle,
 // and a size. None of these are tied to a specific thread.
 unsafe impl Send for CoherentHandle {}
 
 // SAFETY: `CoherentHandle` provides no CPU access to the underlying allocation. The only
-// operations on `&CoherentHandle` are reading the DMA handle and size, both of which are
+// operations on `&CoherentHandle` are reading the DMA address and size, both of which are
 // plain `Copy` values.
 unsafe impl Sync for CoherentHandle {}
 
 /// View type for `Coherent`.
 ///
-/// This is same as [`SysMem`] but with additional information that allows handing out a DMA handle.
+/// This is same as [`SysMem`] but with additional information that allows handing out a DMA
+/// address.
 pub struct CoherentView<'a, T: ?Sized> {
     cpu_addr: SysMem<'a, T>,
-    dma_handle: DmaAddress,
+    dma_addr: DmaAddress,
 }
 
 impl<T: ?Sized> Copy for CoherentView<'_, T> {}
@@ -1116,16 +1116,16 @@ impl<T: ?Sized> Clone for CoherentView<'_, T> {
 }
 
 impl<'a, T: ?Sized> CoherentView<'a, T> {
-    /// Erase the DMA handle information and obtain a [`SysMem`] view of the same memory region.
+    /// Erase the DMA address information and obtain a [`SysMem`] view of the same memory region.
     #[inline]
     pub fn as_sys_mem(self) -> SysMem<'a, T> {
         self.cpu_addr
     }
 
-    /// Returns a DMA handle which may be given to the device as the DMA address base of the region.
+    /// Returns the DMA address which may be given to the device as base of the region.
     #[inline]
-    pub fn dma_handle(self) -> DmaAddress {
-        self.dma_handle
+    pub fn dma_address(self) -> DmaAddress {
+        self.dma_addr
     }
 
     /// Returns a reference to the data in the region.
@@ -1178,9 +1178,9 @@ impl IoBackend for CoherentIoBackend {
     ) -> Self::View<'a, U> {
         let offset = ptr.addr() - view.cpu_addr.as_ptr().addr();
         // CAST: The offset DMA address can never overflow.
-        let dma_handle = view.dma_handle + offset as DmaAddress;
+        let dma_addr = view.dma_addr + offset as DmaAddress;
         CoherentView {
-            dma_handle,
+            dma_addr,
             // SAFETY: Per safety requirement.
             cpu_addr: unsafe { SysMemBackend::project_view(view.cpu_addr, ptr) },
         }
@@ -1245,7 +1245,7 @@ impl<'a, T: ?Sized + KnownSize> IoBase<'a> for &'a Coherent<T> {
         CoherentView {
             // SAFETY: `cpu_addr` is valid and aligned kernel accessible memory.
             cpu_addr: unsafe { SysMem::new(self.cpu_addr.as_ptr()) },
-            dma_handle: self.dma_handle,
+            dma_addr: self.dma_addr,
         }
     }
 }
diff --git a/rust/kernel/drm/device.rs b/rust/kernel/drm/device.rs
index 81f9f7e59817..6b88ade28e24 100644
--- a/rust/kernel/drm/device.rs
+++ b/rust/kernel/drm/device.rs
@@ -32,6 +32,7 @@ use crate::{
 };
 use core::{
     alloc::Layout,
+    cell::UnsafeCell,
     marker::PhantomData,
     mem,
     ops::Deref,
@@ -74,66 +75,59 @@ macro_rules! drm_legacy_fields {
 
 /// A trait implemented by all possible contexts a [`Device`] can be used in.
 ///
-/// Setting up a new [`Device`] is a multi-stage process. Each step of the process that a user
-/// interacts with in Rust has a respective [`DeviceContext`] typestate. For example,
-/// `Device<T, Registered>` would be a [`Device`] that reached the [`Registered`] [`DeviceContext`].
+/// A [`Device`] can be in one of the following contexts:
 ///
-/// Each stage of this process is described below:
+/// - [`Normal`]: The general-purpose, reference-counted context. A [`Device`] in this context may
+///   or may not be registered with userspace.
+/// - [`Ioctl`]: The device has been registered with userspace at some point; used in ioctl
+///   dispatch context.
+/// - [`Registered`]: The device is currently registered with userspace and the parent bus device
+///   is bound.
 ///
-/// ```text
-///        1                     2                         3
-/// +--------------+   +------------------+   +-----------------------+
-/// |Device created| → |Device initialized| → |Registered w/ userspace|
-/// +--------------+   +------------------+   +-----------------------+
-///    (Uninit)                                      (Registered)
-/// ```
-///
-/// 1. The [`Device`] is in the [`Uninit`] context and is not guaranteed to be initialized or
-///    registered with userspace. Only a limited subset of DRM core functionality is available.
-/// 2. The [`Device`] is guaranteed to be fully initialized, but is not guaranteed to be registered
-///    with userspace. All DRM core functionality which doesn't interact with userspace is
-///    available. We currently don't have a context for representing this.
-/// 3. The [`Device`] is guaranteed to be fully initialized, and is guaranteed to have been
-///    registered with userspace at some point - thus putting it in the [`Registered`] context.
+/// Both `Device<T, Ioctl>` and `Device<T, Registered>` dereference to `Device<T>` ([`Normal`]),
+/// so any method available on a [`Normal`] device is also available in the other contexts.
+pub trait DeviceContext: Sealed + Send + Sync + 'static {}
+
+/// The general-purpose, reference-counted [`DeviceContext`].
 ///
-/// An important caveat of [`DeviceContext`] which must be kept in mind: when used as a typestate
-/// for a reference type, it can only guarantee that a [`Device`] reached a particular stage in the
-/// initialization process _at the time the reference was taken_. No guarantee is made in regards to
-/// what stage of the process the [`Device`] is currently in. This means for instance that a
-/// `&Device<T, Uninit>` may actually be registered with userspace, it just wasn't known to be
-/// registered at the time the reference was taken.
-pub trait DeviceContext: Sealed + Send + Sync {}
-
-/// The [`DeviceContext`] of a [`Device`] that was registered with userspace at some point.
+/// A [`Device`] in this context may or may not be registered with userspace. This context is used
+/// for reference-counted device handles and during device setup via [`UnregisteredDevice`].
 ///
-/// This represents a [`Device`] which is guaranteed to have been registered with userspace at
-/// some point in time. Such a DRM device is guaranteed to have been fully-initialized.
+/// [`AlwaysRefCounted`] is only implemented for `Device<T, Normal>`, making this the required
+/// context for [`ARef`]-based device handles.
+pub struct Normal;
+
+impl Sealed for Normal {}
+impl DeviceContext for Normal {}
+
+/// The [`DeviceContext`] of a [`Device`] that is currently registered with userspace.
 ///
-/// Note: A device in this context is not guaranteed to remain registered with userspace for its
-/// entire lifetime, as this is impossible to guarantee at compile-time.
+/// A [`Device`] in this context is guaranteed to be registered and its parent bus device is
+/// guaranteed to be bound. This is enforced at runtime by [`RegistrationGuard`], which holds a
+/// `drm_dev_enter()` / `drm_dev_exit()` SRCU critical section.
 ///
 /// # Invariants
 ///
-/// A [`Device`] in this [`DeviceContext`] is guaranteed to have been registered with userspace
-/// at some point in time.
+/// The parent bus device is bound for the duration of any reference to a `Device<T, Registered>`.
 pub struct Registered;
 
 impl Sealed for Registered {}
 impl DeviceContext for Registered {}
 
-/// The [`DeviceContext`] of a [`Device`] that may be unregistered and partly uninitialized.
+/// The [`DeviceContext`] of a [`Device`] that has been registered with userspace previously.
 ///
-/// A [`Device`] in this context is only guaranteed to be partly initialized, and may or may not
-/// be registered with userspace. Thus operations which depend on the [`Device`] being fully
-/// initialized, or which depend on the [`Device`] being registered with userspace are not
-/// available through this [`DeviceContext`].
+/// A [`Device`] in this context has been registered at some point, but may be concurrently
+/// unregistering or already unregistered. `drm_dev_enter()` can guard against this, ensuring the
+/// device remains registered for the duration of the critical section.
 ///
-/// A [`Device`] in this context can be used to create a
-/// [`Registration`](drm::driver::Registration).
-pub struct Uninit;
+/// # Invariants
+///
+/// A [`Device`] in this context has been registered with userspace via `drm_dev_register()` at
+/// some point.
+pub struct Ioctl;
 
-impl Sealed for Uninit {}
-impl DeviceContext for Uninit {}
+impl Sealed for Ioctl {}
+impl DeviceContext for Ioctl {}
 
 /// A [`Device`] which is known at compile-time to be unregistered with userspace.
 ///
@@ -147,10 +141,10 @@ impl DeviceContext for Uninit {}
 ///
 /// The device in `self.0` is guaranteed to be a newly created [`Device`] that has not yet been
 /// registered with userspace until this type is dropped.
-pub struct UnregisteredDevice<T: drm::Driver>(ARef<Device<T, Uninit>>, NotThreadSafe);
+pub struct UnregisteredDevice<T: drm::Driver>(ARef<Device<T, Normal>>, NotThreadSafe);
 
 impl<T: drm::Driver> Deref for UnregisteredDevice<T> {
-    type Target = Device<T, Uninit>;
+    type Target = Device<T, Normal>;
 
     fn deref(&self) -> &Self::Target {
         &self.0
@@ -178,15 +172,13 @@ impl<T: drm::Driver> UnregisteredDevice<T> {
         master_drop: None,
         debugfs_init: None,
 
-        // Ignore the Uninit DeviceContext below. It is only provided because it is required by the
-        // compiler, and it is not actually used by these functions.
-        gem_create_object: T::Object::<Uninit>::ALLOC_OPS.gem_create_object,
-        prime_handle_to_fd: T::Object::<Uninit>::ALLOC_OPS.prime_handle_to_fd,
-        prime_fd_to_handle: T::Object::<Uninit>::ALLOC_OPS.prime_fd_to_handle,
-        gem_prime_import: T::Object::<Uninit>::ALLOC_OPS.gem_prime_import,
-        gem_prime_import_sg_table: T::Object::<Uninit>::ALLOC_OPS.gem_prime_import_sg_table,
-        dumb_create: T::Object::<Uninit>::ALLOC_OPS.dumb_create,
-        dumb_map_offset: T::Object::<Uninit>::ALLOC_OPS.dumb_map_offset,
+        gem_create_object: T::Object::ALLOC_OPS.gem_create_object,
+        prime_handle_to_fd: T::Object::ALLOC_OPS.prime_handle_to_fd,
+        prime_fd_to_handle: T::Object::ALLOC_OPS.prime_fd_to_handle,
+        gem_prime_import: T::Object::ALLOC_OPS.gem_prime_import,
+        gem_prime_import_sg_table: T::Object::ALLOC_OPS.gem_prime_import_sg_table,
+        dumb_create: T::Object::ALLOC_OPS.dumb_create,
+        dumb_map_offset: T::Object::ALLOC_OPS.dumb_map_offset,
 
         show_fdinfo: None,
         fbdev_probe: None,
@@ -209,10 +201,13 @@ impl<T: drm::Driver> UnregisteredDevice<T> {
     /// Create a new `UnregisteredDevice` for a `drm::Driver`.
     ///
     /// This can be used to create a [`Registration`](kernel::drm::Registration).
-    pub fn new(dev: &device::Device, data: impl PinInit<T::Data, Error>) -> Result<Self> {
+    pub fn new(
+        dev: &T::ParentDevice<device::Bound>,
+        data: impl PinInit<T::Data, Error>,
+    ) -> Result<Self> {
         // `__drm_dev_alloc` uses `kmalloc()` to allocate memory, hence ensure a `kmalloc()`
         // compatible `Layout`.
-        let layout = Kmalloc::aligned_layout(Layout::new::<Device<T, Uninit>>());
+        let layout = Kmalloc::aligned_layout(Layout::new::<Device<T, Normal>>());
 
         // Use a temporary vtable without a `release` callback until `data` is initialized, so
         // init failure can release the DRM device without dropping uninitialized fields.
@@ -224,12 +219,12 @@ impl<T: drm::Driver> UnregisteredDevice<T> {
         // SAFETY:
         // - `alloc_vtable` reference remains valid until no longer used,
         // - `dev` is valid by its type invarants,
-        let raw_drm: *mut Device<T, Uninit> = unsafe {
+        let raw_drm: *mut Device<T, Normal> = unsafe {
             bindings::__drm_dev_alloc(
-                dev.as_raw(),
+                dev.as_ref().as_raw(),
                 &alloc_vtable,
                 layout.size(),
-                mem::offset_of!(Device<T, Uninit>, dev),
+                mem::offset_of!(Device<T, Normal>, dev),
             )
         }
         .cast();
@@ -254,6 +249,9 @@ impl<T: drm::Driver> UnregisteredDevice<T> {
         // SAFETY: `drm_dev` is still private to this function.
         unsafe { (*drm_dev).driver = const { &Self::VTABLE } };
 
+        // SAFETY: `raw_drm` is valid; no concurrent access before registration.
+        unsafe { (*raw_drm.as_ptr()).registration_data = UnsafeCell::new(NonNull::dangling()) };
+
         // SAFETY: The reference count is one, and now we take ownership of that reference as a
         // `drm::Device`.
         // INVARIANT: We just created the device above, but have yet to call `drm_dev_register`.
@@ -265,16 +263,8 @@ impl<T: drm::Driver> UnregisteredDevice<T> {
 
 /// A typed DRM device with a specific [`drm::Driver`] implementation and [`DeviceContext`].
 ///
-/// Since DRM devices can be used before being fully initialized and registered with userspace, `C`
-/// represents the furthest [`DeviceContext`] we can guarantee that this [`Device`] has reached.
-///
-/// Keep in mind: this means that an unregistered device can still have the registration state
-/// [`Registered`] as long as it was registered with userspace once in the past, and that the
-/// behavior of such a device is still well-defined. Additionally, a device with the registration
-/// state [`Uninit`] simply does not have a guaranteed registration state at compile time, and could
-/// be either registered or unregistered. Since there is no way to guarantee a long-lived reference
-/// to an unregistered device would remain unregistered, we do not provide a [`DeviceContext`] for
-/// this.
+/// A device in the [`Registered`] context is currently registered with userspace and its parent
+/// bus device is bound. The [`Normal`] context is the general-purpose, reference-counted context.
 ///
 /// # Invariants
 ///
@@ -282,9 +272,10 @@ impl<T: drm::Driver> UnregisteredDevice<T> {
 /// * The data layout of `Self` remains the same across all implementations of `C`.
 /// * Any invariants for `C` also apply.
 #[repr(C)]
-pub struct Device<T: drm::Driver, C: DeviceContext = Registered> {
+pub struct Device<T: drm::Driver, C: DeviceContext = Normal> {
     dev: Opaque<bindings::drm_device>,
     data: T::Data,
+    pub(super) registration_data: UnsafeCell<NonNull<T::RegistrationData<'static>>>,
     _ctx: PhantomData<C>,
 }
 
@@ -353,7 +344,111 @@ impl<T: drm::Driver, C: DeviceContext> Device<T, C> {
     }
 }
 
-impl<T: drm::Driver, C: DeviceContext> Deref for Device<T, C> {
+impl<T: drm::Driver> Device<T, Ioctl> {
+    /// Guard against the parent bus device being unbound.
+    ///
+    /// Returns a [`RegistrationGuard`] if the device has not been unplugged, [`None`] otherwise.
+    ///
+    /// While [`RegistrationGuard`] is held the parent device is guaranteed to be bound.
+    #[must_use]
+    pub fn registration_guard(&self) -> Option<RegistrationGuard<'_, T>> {
+        let mut idx: i32 = 0;
+        // SAFETY: `self.as_raw()` is a valid pointer to a `struct drm_device`.
+        if unsafe { bindings::drm_dev_enter(self.as_raw(), &mut idx) } {
+            // INVARIANT:
+            // - `idx` is the SRCU index from the successful `drm_dev_enter()` above.
+            // - The parent bus device is bound: `drm_dev_enter()` succeeded, meaning
+            //   `drm_dev_unplug()` has not completed; since it is only called from
+            //   `Registration::drop()` during parent unbind, the parent is still bound.
+            Some(RegistrationGuard {
+                // SAFETY: See INVARIANT above; the `Registered` context invariant holds.
+                dev: unsafe { self.assume_ctx() },
+                idx,
+                _not_send: NotThreadSafe,
+            })
+        } else {
+            None
+        }
+    }
+}
+
+/// A guard proving the DRM device is registered and the parent bus device is bound.
+///
+/// The guard dereferences to [`Device<T, Registered>`], providing access to the DRM device with
+/// the guarantee that the parent bus device is bound for the entire duration of the critical
+/// section.
+///
+/// Internally this is backed by a `drm_dev_enter()` / `drm_dev_exit()` SRCU critical section.
+///
+/// # Invariants
+///
+/// - `idx` is the SRCU read lock index returned by a successful `drm_dev_enter()` call.
+/// - The parent bus device of `dev` is bound for the lifetime of this guard.
+#[must_use]
+pub struct RegistrationGuard<'a, T: drm::Driver> {
+    dev: &'a Device<T, Registered>,
+    idx: i32,
+    _not_send: NotThreadSafe,
+}
+
+impl<T: drm::Driver> Device<T, Registered> {
+    /// Returns a reference to the registration data with lifetime shortened from `'static`.
+    ///
+    /// # Safety
+    ///
+    /// The returned reference must not be exposed to code that can choose a concrete lifetime for
+    /// it, as that would be unsound for types that are invariant over their lifetime parameter
+    /// (e.g. it must be passed through an HRTB-bounded closure).
+    #[inline]
+    unsafe fn registration_data_unchecked(&self) -> &T::RegistrationData<'_> {
+        // SAFETY:
+        // - `Registered` guarantees the parent bus device is bound, hence the pointer is valid.
+        // - The pointer cast from `Of<'static>` to `Of<'_>` is layout-compatible since lifetimes
+        //   are erased at runtime.
+        // - Caller guarantees the reference is only used behind an HRTB, making the lifetime
+        //   shortening sound regardless of variance.
+        unsafe { (*self.registration_data.get()).cast::<_>().as_ref() }
+    }
+
+    /// Access the registration data through a closure, with the lifetime tied to the closure
+    /// scope.
+    ///
+    /// The data is owned by [`Registration`](drm::Registration) and is guaranteed to remain valid
+    /// as long as the device is registered, since [`Registration`](drm::Registration)'s `drop`
+    /// calls `drm_dev_unplug()` which waits for all `drm_dev_enter()` critical sections to
+    /// complete.
+    #[inline]
+    pub fn registration_data_with<R, F>(&self, f: F) -> R
+    where
+        F: for<'a> FnOnce(&'a T::RegistrationData<'a>) -> R,
+    {
+        // SAFETY: `Registered` guarantees the device is registered and the parent bus device is
+        // bound. The closure's HRTB `for<'a>` prevents the caller from smuggling in references
+        // with a concrete short lifetime, satisfying the lifetime requirement of
+        // `registration_data_unchecked`.
+        f(unsafe { self.registration_data_unchecked() })
+    }
+}
+
+impl<T: drm::Driver> Deref for RegistrationGuard<'_, T> {
+    type Target = Device<T, Registered>;
+
+    #[inline]
+