| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Initialize the hba->rpmbs list in ufshcd_alloc_host() to prevent NULL
pointer dereference in the device teardown path if ufs_rpmb_probe()
fails.
Fixes: b06b8c421485 ("scsi: ufs: core: Add OP-TEE based RPMB driver for UFS devices")
Co-developed-by: Jiazi Li <jiazi.li@transsion.com>
Signed-off-by: Jiazi Li <jiazi.li@transsion.com>
Signed-off-by: Ao Sun <ao.sun@transsion.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Link: https://patch.msgid.link/20260723034440.217-1-ao.sun@transsion.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
threaded IRQ handler"
There have been multiple reports of performance regressions caused by
commit 3c7ac40d7322 ("scsi: ufs: core: Delegate the interrupt service
routine to a threaded IRQ handler"). Hence this revert.
This patch reverts most of the following commits:
* 3c7ac40d7322 ("scsi: ufs: core: Delegate the interrupt service
routine to a threaded IRQ handler")
* 6475cfb81fc4 ("scsi: ufs: core: Avoid IRQ thread wakeup during active
UIC command")
This patch preserves the following commits:
* 034d319c8899 ("scsi: ufs: core: Fix interrupt handling for MCQ Mode")
* eabcac808ca3 ("scsi: ufs: core: Fix IRQ lock inversion for the SCSI
host lock")
Cc: Neil Armstrong <neil.armstrong@linaro.org>
Cc: 孙魁 (Kui Sun) <kui.sun@unisoc.com>
Cc: André Draszik <andre.draszik@linaro.org>
Cc: Gregory CLEMENT <gregory.clement@bootlin.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Fixes: 3c7ac40d7322 ("scsi: ufs: core: Delegate the interrupt service routine to a threaded IRQ handler")
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Tested-by: André Draszik <andre.draszik@linaro.org> # on Pixel 6
Reviewed-by: André Draszik <andre.draszik@linaro.org>
Link: https://patch.msgid.link/b70eb60a01f971bed68c42c5b555929db5f835df.1784135511.git.bvanassche@acm.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
UFS RTC support schedules ufs_rtc_update_work to periodically update the
device RTC. The work can issue query commands and access the UFS host
controller.
A previous change moved the RTC work cancellation before the PRE_CHANGE
vendor suspend callback to close a race in the common suspend path.
However, the active-active path jumps directly to vops_suspend after
flushing exception handling work and therefore bypasses the
cancellation.
If the RTC work runs while the vendor suspend callback is gating or
otherwise changing hardware state, it can access the controller during
suspend and trigger an SError.
Cancel the RTC work before entering the vendor suspend callback in the
active-active path. Since this path now cancels the work, move the RTC
work scheduling outside the device and link state restoration block in
the resume path. This restarts RTC updates after an active-active
suspend and resume cycle.
Fixes: b0bd84c39289 ("scsi: ufs: core: Fix SError in ufshcd_rtc_work() during UFS suspend")
Signed-off-by: Guangshuo Li <lgs201920130244@gmail.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260714172726.1736967-1-lgs201920130244@gmail.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
The trace events in drivers/ufs/core/ufs_trace.h were converted to take
a pointer to the hba structure as an argument for the tracepoint and
then in TP_printk() the printing of the dev_name from the ring buffer
was converted to using the dev dereferenced pointer from the hba saved
pointer.
This is not allowed as the TP_printk() is executed at the time the trace
event is read from /sys/kernel/tracing/trace file. That can happen
literally, seconds, minutes, hours, weeks, days, or even months later!
There is no guarantee that the hba pointer will still exist by the time
it is dereferenced when the "trace" file is read.
Instead, save the device name from the hba pointer at the time the
tracepoint is called and place it into the ring buffer event. Then the
TP_printk() can read the name directly from the ring buffer and remove
the possibility that it will read a freed pointer and crash the kernel.
This was detected when testing the trace event code that looks for
TP_printk() parameters doing illegal derferences[1]
[1] https://lore.kernel.org/all/20260630184836.74d477b6@gandalf.local.home/
Cc: stable@vger.kernel.org
Fixes: 583e518e7100 ("scsi: ufs: core: Add hba parameter to trace events")
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260630185412.283c26c5@gandalf.local.home
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Pull in outstanding commits from 7.1 branch.
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
A PM START STOP sent from the UFS well-known LU resume path can race
with SCSI EH:
The "wl resume" task flow is:
__ufshcd_wl_resume()
ufshcd_set_dev_pwr_mode(UFS_ACTIVE_PWR_MODE)
ufshcd_execute_start_stop()
scsi_execute_cmd()
blk_execute_rq <-- wait
scsi_check_passthrough() <-- may retry START STOP
If the first START STOP time out, SCSI EH may already recover the link and
reset the device before scsi_execute_cmd() returns:
scsi_timeout()
scsi_eh_scmd_add()
scsi_error_handler()
scsi_unjam_host()
scsi_eh_ready_devs()
scsi_eh_host_reset()
ufshcd_eh_host_reset_handler()
if (hba->pm_op_in_progress)
ufshcd_link_recovery()
ufshcd_device_reset()
ufshcd_host_reset_and_restore()
...
scsi_eh_flush_done_q() <-- wakeup "wl resume" task
... <-- host still in SHOST_RECOVERY
scsi_restart_operations()
A later passthrough retry can then run while the host is still in
SHOST_RECOVERY and hit the SCMD_FAIL_IF_RECOVERING path:
scsi_queue_rq()
if (scsi_host_in_recovery(shost) &&
cmd->flags & SCMD_FAIL_IF_RECOVERING)
return BLK_STS_OFFLINE
That retry completes with DID_ERROR or DID_NO_CONNECT even though EH may
already have restored the device to an operational ACTIVE state.
Handle these PM timeouts directly from ufshcd_eh_timed_out() instead.
After ufshcd_link_recovery(), complete the timed-out command immediately
if it has not been completed already.
For regular SCSI commands, complete them with DID_REQUEUE to match the
existing MCQ force-completion semantics and allow scsi_execute_cmd() to
retry if needed. For reserved internal device-management commands,
finish the request with DID_TIME_OUT without calling
ufshcd_release_scsi_cmd() since those commands use different resource
lifetime rules.
The system_suspending flag is no longer needed because PM command
timeout handling now uses pm_op_in_progress.
Fixes: b8c3a7bac9b6 ("scsi: ufs: Have midlayer retry start stop errors")
Signed-off-by: Hongjie Fang <hongjiefang@asrmicro.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260605112034.3802540-1-hongjiefang@asrmicro.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
ufshcd_variant_hba_init/exit() check 'if (!hba->vops)' before calling
vops wrappers, but the wrappers already do NULL check internally. Remove
the redundant checks. Also remove ufshcd_variant_hba_exit() entirely
since it only wraps ufshcd_vops_exit() with no added value.
Signed-off-by: Chanwoo Lee <cw9316.lee@samsung.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260529061623.301291-1-cw9316.lee@samsung.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
ufshcd_vops_exit(), ufshcd_vops_setup_task_mgmt(), and
ufshcd_vops_hibern8_notify() use 'return hba->vops->xxx()' while other
void vops wrappers call without return. Remove the unnecessary return
keywords for consistency.
Signed-off-by: Chanwoo Lee <cw9316.lee@samsung.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260529061503.301182-1-cw9316.lee@samsung.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
In ufshcd_transfer_rsp_status(), the default case of the inner switch
statement prints the UPIU response code when an unexpected response is
received. However, the code was printing 'result' variable which is
always 0 at that point, making the error message useless for debugging.
Fix this by printing the actual UPIU response code returned by
ufshcd_get_req_rsp().
Fixes: 08108d31129a ("scsi: ufs: Improve type safety")
Signed-off-by: Chanwoo Lee <cw9316.lee@samsung.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260527092134.275887-1-cw9316.lee@samsung.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
ufshcd_tag_to_cmd() may return NULL if no command is associated with the
given tag. However, several callers dereference the returned cmd pointer
via scsi_cmd_priv() without checking for NULL first, leading to a
potential NULL pointer dereference.
Fix this by adding NULL checks for cmd before calling scsi_cmd_priv()
and moving the lrbp initialization after the NULL check.
Signed-off-by: Chanwoo Lee <cw9316.lee@samsung.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260529010739.295391-1-cw9316.lee@samsung.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
ufshcd_validate_link_params(), added by commit e72323f3b09f ("scsi: ufs:
core: Configure only active lanes during link"), is called
unconditionally from ufshcd_link_startup() and fails link startup with
-ENOLINK when the connected lane count read from the device differs from
hba->lanes_per_direction.
lanes_per_direction is only set by ufshcd-pltfrm (default 2, or the
"lanes-per-direction" devicetree property); ufshcd-pci controllers
(e.g. Intel) leave it 0. As the device always reports >= 1 connected
lanes, the check can never match and link startup always fails.
Reproduced with QEMU's UFS device.
Skip the check when lanes_per_direction is unset: with no expected value
to validate against, restore the behaviour from before that commit.
Fixes: e72323f3b09f ("scsi: ufs: core: Configure only active lanes during link")
Signed-off-by: Daejun Park <daejun7.park@samsung.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Link: https://patch.msgid.link/20260520070009epcms2p6542f3abb7660839e9d8140b3f2f145c3@epcms2p6
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Use cached values in ufshcd_add_uic_command_trace() instead of calling
readl() when tracing command submission (UFS_CMD_SEND).
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260519212135.3130556-4-bvanassche@acm.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
According to the UFSHCI standard, the lowest byte of UIC argument 2 is
an output value. Additionally, ufshcd_uic_cmd_compl() is based on the
assumption that the lowest byte of UIC argument 2 is zero. Hence,
complain if the result byte is set when a UIC command is submitted.
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260519212135.3130556-3-bvanassche@acm.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
The implementation of the two functions ufshcd_get_uic_cmd_result() and
ufshcd_get_dme_attr_val() is very short. Additionally, both functions
only have one caller. Inline both functions to make the code shorter.
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260519212135.3130556-2-bvanassche@acm.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Add a quirk to support TX Equalization Training (EQTR) using Adapt
L0L1L2L3 length which is larger than what is allowed by M-PHY spec ver
6.0.
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Ziqi Chen <ziqi.chen@oss.qualcomm.com>
Link: https://patch.msgid.link/20260501131641.826258-2-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
The number of connected lanes detected during UFS link startup can be
fewer than the lanes specified in the device tree. The current driver
logic attempts to configure all lanes defined in the device tree,
regardless of their actual availability. This mismatch may cause
failures during power mode changes.
Hence, Add a check during link startup to ensure that only the lanes
actually discovered are considered valid. If a mismatch is detected,
fail the initialization early, preventing the driver from entering an
unsupported configuration that could cause power mode transition
failures.
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Shawn Lin <shawn.lin@rock-chips.com>
Reviewed-by: Manivannan Sadhasivam <mani@kernel.org>
Signed-off-by: Palash Kambar <palash.kambar@oss.qualcomm.com>
Link: https://patch.msgid.link/20260423102023.3779489-2-palash.kambar@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Add support for UFS v5.0 JEDEC attributes qTxEQGnSettings and
wTxEQGnSettingsExt to enable persistent storage and retrieval of
optimal TX Equalization settings.
This provides a fast-path for TX Equalization by reusing previously
stored optimal settings, avoiding TX Equalization Training (EQTR)
procedures during subsequent Power Mode changes.
When no valid TX Equalization settings are found, fall back to full TX
EQTR procedures and optionally save the results for future use.
The validity of one set of TX Equalization settings is indicated by
Bit[15] in wTxEQGnSettingsExt.
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260424151420.111675-3-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Introduce a new generic function ufshcd_query_attr_qword() to handle
quad-word (64-bit) UFS attribute operations. This consolidates the
handling of 64-bit attributes which was previously scattered across
multiple specialized functions.
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260424151420.111675-2-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
According to the UFS spec, the bRefClkFreq attribute can only be written
when both sub-links are in LS-MODE. However, in HS LSS mode with
resetmode = HS_MODE, if the UFS device's default bRefClkFreq value
differs from the host controller's dev_ref_clk_freq setting, the write
operation will fail.
To fix this issue, introduce ufshcd_get_op_mode() function to detect the
current link operational mode. Call ufshcd_set_dev_ref_clk() only when
both sub-links are in LS-MODE to ensure the attribute can be written
successfully.
Signed-off-by: Wang Shuaiwei <wangshuaiwei1@xiaomi.com>
Link: https://patch.msgid.link/20260414033718.1459540-1-wangshuaiwei1@xiaomi.com
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Kioxia has another product that does not support the qTimestamp
attribute.
Signed-off-by: Aaron Kling <webgeek1234@gmail.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260403-thgjfjt0e25baip-no-timestamp-v1-1-1ddb34225133@gmail.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Add the include directives and forward declarations that are missing
from the UFS core header files. This prevents compilation failures if
include directives are reordered.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260401202506.1445324-4-bvanassche@acm.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Nothing in the ufshcd-crypto.h header file depends on the ufshcd-priv.h
header file. Hence, stop including that header file. This include
directive was introduced by commit 4bc26113c603 ("scsi: ufs: Split the
ufshcd.h header file").
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260401202506.1445324-3-bvanassche@acm.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Document the aspects of ufshcd_mcq_compl_all_cqes_lock() that are
nontrivial in a comment block above this function.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260401202506.1445324-2-bvanassche@acm.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Pull in fixes to resolve mpi3mr merge conflict.
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Can Guo <can.guo@oss.qualcomm.com> says:
Hi,
The UFS v5.0 and UFSHCI v5.0 standards have published, introducing support
for HS-G6 (46.6 Gbps per lane) through the new UniPro V3.0 interconnect
layer and M-PHY V6.0 physical layer specifications. To achieve reliable
operation at these higher speeds, UniPro V3.0 introduces TX Equalization
and Pre-Coding mechanisms that are essential for signal integrity.
This patch series implements TX Equalization support in the UFS core
driver as specified in UFSHCI v5.0, along with the necessary vendor
operations and a reference implementation for Qualcomm UFS host
controllers.
Background
==========
TX Equalization is a signal conditioning technique that compensates for
channel impairments at high data rates (HS-G4 through HS-G6). It works
by adjusting two key parameters:
- PreShoot: Pre-emphasis applied before the main signal transition
- DeEmphasis: De-emphasis applied after the main signal transition
UniPro V3.0 defines TX Equalization Training (EQTR) procedure to
automatically discover optimal TX Equalization settings. The EQTR
procedure:
1. Starts from the most reliable link state (HS-G1)
2. Iterates through all possible PreShoot and DeEmphasis combinations
3. Evaluates signal quality using Figure of Merit (FOM) measurements
4. Selects the best settings for both host and device TX lanes
For HS-G6, Pre-Coding is also introduced to further improve signal
quality. Pre-Coding must be enabled on both transmitter and receiver
when the RX_FOM indicates it is required.
Implementation Overview
=======================
The implementation follows the UFSHCI v5.0 specification and consists of:
Core Infrastructure (Patches 1-6):
- New vops callback negotiate_pwr_mode() to allow vendors to negotiate
power mode parameters before applying TX Equalization settings
- Support for HS-G6 gear enumeration
- Complete TX EQTR procedure implementation in ufs-txeq.c
- Debugfs interface for TX Equalization parameter inspection and manual
retraining
- Module parameters for adaptive TX Equalization control
Qualcomm Implementation (Patches 7-11):
- PHY-specific configurations for TX EQTR procedure
- Vendor-specific FOM measurement support
- TX Equalization settings application
- Enable TX Equalization for HW version 0x7 and onwards
The implementation is designed to be vendor-agnostic, with platform-
specific details handled through the vops callbacks. Other vendors can
add support by implementing the three new vops:
- tx_eqtr_notify(): Called before/after TX EQTR for vendor setup
- apply_tx_eqtr_settings(): Apply vendor-specific PHY configurations
- get_rx_fom(): Retrieve vendor-specific FOM measurements if needed
Module Parameters
=================
The implementation provides several module parameters for flexibility:
- use_adaptive_txeq: Enable/disable adaptive TX Equalization (default: false)
- adaptive_txeq_gear: Minimum gear for adaptive TX EQ (default: HS-G6)
- use_txeq_presets: Use only the 8 standaird presets (default: false)
- txeq_presets_selected[]: Select specific presets for EQTR
Testing
=======
This patch series has been tested on Qualcomm platforms with UFS 5.0
devices, validating:
- Successful TX EQTR completion for HS-G6
- Proper FOM evaluation and optimal settings selection
- Pre-Coding enablement for HS-G6
- Power mode changes with TX Equalization settings applied
- Report of TX Equalization settings via debugfs entries
- Report of TX EQTR histories via debug entries (see next section)
- Re-training TX Equalization via debugfs entry
Example of TX EQTR history
==========================
Device TX EQTR record summary -
Target Power Mode: HS-G6, Rate-B
Most recent record index: 2
Most recent record timestamp: 219573378 us
TX Lane 0 FOM - PreShoot\DeEmphasis
\ 0 1 2 3 4 5 6 7
0 50 70 65 - - - - x
1 x x x x x x x x
2 100 90 70 - - - - x
3 x x x x x x x x
4 95 90 - - - - - x
5 - - - - - - - x
6 x x x x x x x x
7 x x x x x x x x
TX Lane 1 FOM - PreShoot\DeEmphasis
\ 0 1 2 3 4 5 6 7
0 50 70 60 - - - - x
1 x x x x x x x x
2 100 80 65 - - - - x
3 x x x x x x x x
4 95 85 - - - - - x
5 - - - - - - - x
6 x x x x x x x x
7 x x x x x x x x
Patch Structure
===============
Patches 1-3: Preparatory changes for power mode negotiation and HS-G6
Patch 4: Core TX Equalization and EQTR implementation
Patches 5-7: Debugfs support for TX Equalization
Patches 8-12: Qualcomm vendor implementation
Next
====
One more series has been developed to enhance TX Equalization support,
which will be submitted for review after this series is accepted:
- Provide board specific (static) TX Equalization settings from DTS
- Parse static TX Equalization settings from DTS if provided
- Apply static TX Equalization settings if use_adaptive_txeq is disabled
- Add support for UFS v5.0 attributes qTxEQGnSettings & wTxEQGnSettingsExt
- Enable persistent storage and retrieval of optimal TX Equalization settings
Link: https://patch.msgid.link/20260325152154.1604082-1-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
On some platforms, host's M-PHY RX_FOM Attribute always reads 0, meaning SW
cannot rely on Figure of Merit (FOM) to identify the optimal TX
Equalization settings for device's TX Lanes. Implement the vops
ufs_qcom_get_rx_fom() such that SW can utilize the UFS Eye Opening Monitor
(EOM) to evaluate the TX Equalization settings for device's TX Lanes.
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-11-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Drastic environmental changes, such as significant temperature shifts, can
impact link signal integrity. In such cases, retraining TX Equalization is
necessary to compensate for these environmental changes.
Add a debugfs entry, 'tx_eq_ctrl', to allow userspace to manually trigger
the TX Equalization training (EQTR) procedure and apply the identified
optimal settings on the fly. These entries are created on a per-gear basis
for High Speed Gear 4 (HS-G4) and above, as TX EQTR is not supported for
lower gears.
The 'tx_eq_ctrl' entry currently accepts the 'retrain' command to initiate
the procedure. The interface is designed to be scalable to support
additional commands in the future.
Reading the 'tx_eq_ctrl' entry provides a usage hint to the user, ensuring
the interface is self-documenting.
The ufshcd's debugfs folder structure will look like below:
/sys/kernel/debug/ufshcd/*ufs*/
|--tx_eq_hs_gear1/
| |--device_tx_eq_params
| |--host_tx_eq_params
|--tx_eq_hs_gear2/
|--tx_eq_hs_gear3/
|--tx_eq_hs_gear4/
|--tx_eq_hs_gear5/
|--tx_eq_hs_gear6/
|--device_tx_eq_params
|--device_tx_eqtr_record
|--host_tx_eq_params
|--host_tx_eqtr_record
|--tx_eq_ctrl
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-8-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
In preparation for supporting TX Equalization refreshing, introduce helper
functions to safely pause and resume command processing.
ufshcd_pause_command_processing() ensures the host is in a quiescent state
by stopping the block layer tagset, acquiring the necessary locks
(scan_mutex and clk_scaling_lock), and waiting for any in-flight commands
to complete within a specified timeout.
ufshcd_resume_command_processing() restores the host to its previous
operational state by reversing these steps in the correct order.
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-7-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Add debugfs support for UFS TX Equalization and UFS TX Equalization
Training (EQTR) to facilitate runtime inspection of link quality. These
entries allow developers to monitor and optimize TX Equalization parameters
and EQTR records during live operation.
The debugfs entries are organized on a per-gear basis under the HBA's
debugfs root. Since TX EQTR is only defined for High Speed Gear 4 (HS-G4)
and above, EQTR-related entries are explicitly excluded for HS-G1 through
HS-G3 to avoid exposing unsupported attributes.
The ufshcd's debugfs folder structure will look like below:
/sys/kernel/debug/ufshcd/*ufs*/
|--tx_eq_hs_gear1/
| |--device_tx_eq_params
| |--host_tx_eq_params
|--tx_eq_hs_gear2/
|--tx_eq_hs_gear3/
|--tx_eq_hs_gear4/
|--tx_eq_hs_gear5/
|--tx_eq_hs_gear6/
|--device_tx_eq_params
|--device_tx_eqtr_record
|--host_tx_eq_params
|--host_tx_eqtr_record
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-6-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
MIPI Unipro3.0 introduced PA_TxEQGnSetting and PA_PreCodeEn attributes for
TX Equalization and Pre-Coding. It is Host Software's responsibility to
configure these attributes for both host and device before initiating Power
Mode Change to High-Speed Gears.
MIPI Unipro3.0 also introduced TX Equalization Training (EQTR) to identify
optimal TX Equalization settings for use by both Host's and Device's
UniPro. TX EQTR shall be initiated from the most reliable High-Speed Gear
(HS-G1) targeting High-Speed Gears (HS-G4 to HS-G6).
Implement TX Equalization configuration and TX EQTR procedure as defined
in UFSHCI v5.0 specification. The TX EQTR procedure determines the optimal
TX Equalization settings by iterating through all possible PreShoot and
DeEmphasis combinations and selecting the best combinations for both Host
and Device based on Figure of Merit (FOM) evaluation.
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-5-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Currently, callers must manually toggle hba->force_pmc before and after
calling ufshcd_config_pwr_mode() to force a Power Mode change. Introduce
enum ufshcd_pmc_policy and refactor ufshcd_config_pwr_mode() to accept
pmc_policy as a parameter to force a Power Mode change.
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-3-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Most vendor specific implemenations of vops pwr_change_notify(PRE_CHANGE)
are fulfilling two things at once:
- Vendor specific target power mode negotiation
- Vendor specific power mode change preparation
When TX Equalization is added into consideration, before power mode change
to a target power mode, TX Equalization Training (EQTR) needs be done for
that target power mode. In addition, UFSHCI spec requires to start TX EQTR
from HS-G1 (the most reliable High Speed Gear).
Adding TX EQTR before pwr_change_notify(PRE_CHANGE) is not applicable
because we don't know the negotiated power mode yet.
Adding TX EQTR post pwr_change_notify(PRE_CHANGE) is inappropriate because
pwr_change_notify(PRE_CHANGE) has finished preparation for a power mode
change to negotiated power mode, yet we are changing power mode to HS-G1
for TX EQTR.
Add a new vops negotiate_pwr_mode() so that vendor specific power mode
negotiation can be fulfilled in its vendor specific implementations. Later
on, TX EQTR can be added post vops negotiate_pwr_mode() and before vops
pwr_change_notify(PRE_CHANGE).
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-2-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Add support for handling aggregation-based interrupts when operating in MCQ
mode.
In legacy interrupt mode, an IE.IAGES is triggered when the counter or
timer threshold is reached. To manage this, the handler now resets the
aggregation counter and timer by writing to the MCQIACRy.CTR register.
Since the register layout of MCQIACRy is identical to the existing UTRIACR
register, this implementation reuses the previously defined bitfield masks
to maintain consistency and reduce code duplication.
Extend ufshcd_handle_mcq_cq_events() with a boolean iag parameter. If set,
the handler resets the MCQ IAG counter and timer.
Define MCQ_IAG_EVENT_STATUS (0x200000) and include it in
UFSHCD_ENABLE_MCQ_INTRS to ensure the interrupt is unmasked during
initialization.
Signed-off-by: Vamshi Gajjela <vamshigajjela@google.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260310190308.2474956-1-vamshigajjela@google.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
On some platforms, the VCC regulator has a slow ramp-up time. Add a delay
after enabling VCC to ensure voltage has fully stabilized before we enable
the clocks.
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Ed Tsai <ed.tsai@mediatek.com>
Link: https://patch.msgid.link/20260310005230.4001904-4-ed.tsai@mediatek.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
In __ufshcd_wl_suspend(), cancel_delayed_work_sync() is called to cancel
the UFS RTC work, but it is placed after ufshcd_vops_suspend(hba, pm_op,
POST_CHANGE). This creates a race condition where ufshcd_rtc_work() can
still be running while ufshcd_vops_suspend() is executing. When
UFSHCD_CAP_CLK_GATING is not supported, the condition
!hba->clk_gating.active_reqs is always true, causing ufshcd_update_rtc()
to be executed. Since ufshcd_vops_suspend() typically performs clock
gating operations, executing ufshcd_update_rtc() at that moment triggers
an SError. The kernel panic trace is as follows:
Kernel panic - not syncing: Asynchronous SError Interrupt
Call trace:
dump_backtrace+0xec/0x128
show_stack+0x18/0x28
dump_stack_lvl+0x40/0xa0
dump_stack+0x18/0x24
panic+0x148/0x374
nmi_panic+0x3c/0x8c
arm64_serror_panic+0x64/0x8c
do_serror+0xc4/0xc8
el1h_64_error_handler+0x34/0x4c
el1h_64_error+0x68/0x6c
el1_interrupt+0x20/0x58
el1h_64_irq_handler+0x18/0x24
el1h_64_irq+0x68/0x6c
ktime_get+0xc4/0x12c
ufshcd_mcq_sq_stop+0x4c/0xec
ufshcd_mcq_sq_cleanup+0x64/0x1dc
ufshcd_clear_cmd+0x38/0x134
ufshcd_issue_dev_cmd+0x298/0x4d0
ufshcd_exec_dev_cmd+0x1a4/0x1c4
ufshcd_query_attr+0xbc/0x19c
ufshcd_rtc_work+0x10c/0x1c8
process_scheduled_works+0x1c4/0x45c
worker_thread+0x32c/0x3e8
kthread+0x120/0x1d8
ret_from_fork+0x10/0x20
Fix this by moving cancel_delayed_work_sync() before the call to
ufshcd_vops_suspend(hba, pm_op, PRE_CHANGE), ensuring the UFS RTC work is
fully completed or cancelled at that point.
Cc: Bean Huo <beanhuo@iokpp.de>
Fixes: 6bf999e0eb41 ("scsi: ufs: core: Add UFS RTC support")
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Wang Shuaiwei <wangshuaiwei1@xiaomi.com>
Link: https://patch.msgid.link/20260307035128.3419687-1-wangshuaiwei1@xiaomi.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Only return IRQ_WAKE_THREAD when MCQ and ESI are not enabled and no UIC
command is active. The default UIC command timeout is 500ms, Using threaded
IRQs during an active UIC command increases the risk of timeout due to
possible preemption by other system IRQs.
Signed-off-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260306054419.3816557-1-peter.wang@mediatek.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
The UniPro stack manages to repair many potential Link problems without the
need to notify the Application Layer. Repair mechanisms of the stack
include L2 re-transmission and successful handling of PA_INIT.req.
Nevertheless, any successful repair sequence requires Link bandwidth that
is no longer vailable for the Application. Therefore, it may be useful for
an Application to understand how often such repair attempts are made.
The DME implements Quality of Service monitoring using a simple counting
scheme, counting error events and comparing them against the number of
correctly received or transmitted bytes. When the error counter exceeds a
programmed threshold before the byte counter overflows, a DME_QoS.ind is
issued to the Application and both counters are reset. When the byte
counter overflows before the error counter has reached the programmed
threshold, both counters are reset without triggering a DME_QoS.ind.
The DME provides Link quality monitoring for the following purposes:
1. Detection of re-occurring repaired fatal error conditions on the Link
(PA_INIT loop). This kind of detection is useful if capabilities
exchanged between local and peer permit a potential operation at a
higher M-PHY Gear, but the physical interconnect between local and peer
Device does not, or, after Line quality degradation, no longer satisfies
channel characteristics.
2. Detection of degraded inbound or outbound Link quality, to allow an
Application to issue an ADAPT sequence for a Link running in HS-G4 or
higher HS Gears. This kind of detection is used to monitor a slowly
degrading Link quality, e.g., one being affected by temperature and
voltage variations, against the expected M-PHY bit error rate.
Userspace can configure and enable UniPro QoS via UniPro QoS Attributes
(via UFS BSG) and get notified by dme_qos_notification without polling
UniPro QoS Status attribute. The dme_qos_notification attribute is a
bitfield with the following bit assignments:
Bit Description
=== ======================================
0 DME QoS Monitor has been reset by host
1 QoS from TX is detected
2 QoS from RX is detected
3 QoS from PA_INIT is detected
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260305110856.959211-2-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
According to JESD223F, the maximum number of queues (MAXQ) is 32. When MCQ
is enabled and ESI is disabled, nr_hw_queues=32 causes a shift overflow
problem.
Fix this by using 64-bit intermediate values to handle the nr_hw_queues=32
case safely.
Signed-off-by: wangshuaiwei <wangshuaiwei1@xiaomi.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260224063228.50112-1-wangshuaiwei1@xiaomi.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
The UFSHCI 4.1 specification introduces a new completion queue (CQ) entry
format, allowing the tag to be obtained directly.
Signed-off-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260210071834.1837878-1-peter.wang@mediatek.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
It is difficult to debug situations where an MCQ command timeout occurs,
the corresponding CQ tag response is received, but the request is not
completed. Add a one-line log to indicate when the CQ entry is abnormal.
Signed-off-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260210070837.1820710-3-peter.wang@mediatek.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
It is difficult to debug when a UIC command timeout occurs simultaneously
with a UIC command complete interrupt. Currently, we only see the timeout
log without any debug information, making it unclear whether the UFS device
failed to respond or the host entered an incorrect state. Add a one-line
log to cover this situation.
Signed-off-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260210070837.1820710-2-peter.wang@mediatek.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
Move the link recovery trigger from ufshcd_uic_pwr_ctrl() to
__ufshcd_wl_resume(). Ensure link recovery is only attempted when hibern8
exit fails during resume, not during hibern8 enter in suspend. Improve
error handling and prevent unnecessary link recovery attempts.
Fixes: 35dabf4503b9 ("scsi: ufs: core: Use link recovery when h8 exit fails during runtime resume")
Signed-off-by: Peter Wang <peter.wang@mediatek.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260223103906.2533654-1-peter.wang@mediatek.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
|
|
ufshcd_add_command_trace()
The kernel log indicates a crash in ufshcd_add_command_trace, due to a NULL
pointer dereference when accessing hwq->id. This can happen if
ufshcd_mcq_req_to_hwq() returns NULL.
This p |
