Merge patch series "scsi: ufs: Add TX Equalization support for UFS 5.0"

Can Guo <can.guo@oss.qualcomm.com> says:

Hi,

The UFS v5.0 and UFSHCI v5.0 standards have published, introducing support
for HS-G6 (46.6 Gbps per lane) through the new UniPro V3.0 interconnect
layer and M-PHY V6.0 physical layer specifications. To achieve reliable
operation at these higher speeds, UniPro V3.0 introduces TX Equalization
and Pre-Coding mechanisms that are essential for signal integrity.

This patch series implements TX Equalization support in the UFS core
driver as specified in UFSHCI v5.0, along with the necessary vendor
operations and a reference implementation for Qualcomm UFS host
controllers.

Background
==========

TX Equalization is a signal conditioning technique that compensates for
channel impairments at high data rates (HS-G4 through HS-G6). It works
by adjusting two key parameters:

- PreShoot: Pre-emphasis applied before the main signal transition
- DeEmphasis: De-emphasis applied after the main signal transition

UniPro V3.0 defines TX Equalization Training (EQTR) procedure to
automatically discover optimal TX Equalization settings. The EQTR
procedure:

1. Starts from the most reliable link state (HS-G1)
2. Iterates through all possible PreShoot and DeEmphasis combinations
3. Evaluates signal quality using Figure of Merit (FOM) measurements
4. Selects the best settings for both host and device TX lanes

For HS-G6, Pre-Coding is also introduced to further improve signal
quality. Pre-Coding must be enabled on both transmitter and receiver
when the RX_FOM indicates it is required.

Implementation Overview
=======================

The implementation follows the UFSHCI v5.0 specification and consists of:

Core Infrastructure (Patches 1-6):
- New vops callback negotiate_pwr_mode() to allow vendors to negotiate
  power mode parameters before applying TX Equalization settings
- Support for HS-G6 gear enumeration
- Complete TX EQTR procedure implementation in ufs-txeq.c
- Debugfs interface for TX Equalization parameter inspection and manual
  retraining
- Module parameters for adaptive TX Equalization control

Qualcomm Implementation (Patches 7-11):
- PHY-specific configurations for TX EQTR procedure
- Vendor-specific FOM measurement support
- TX Equalization settings application
- Enable TX Equalization for HW version 0x7 and onwards

The implementation is designed to be vendor-agnostic, with platform-
specific details handled through the vops callbacks. Other vendors can
add support by implementing the three new vops:

- tx_eqtr_notify(): Called before/after TX EQTR for vendor setup
- apply_tx_eqtr_settings(): Apply vendor-specific PHY configurations
- get_rx_fom(): Retrieve vendor-specific FOM measurements if needed

Module Parameters
=================

The implementation provides several module parameters for flexibility:

- use_adaptive_txeq: Enable/disable adaptive TX Equalization (default: false)
- adaptive_txeq_gear: Minimum gear for adaptive TX EQ (default: HS-G6)
- use_txeq_presets: Use only the 8 standaird presets (default: false)
- txeq_presets_selected[]: Select specific presets for EQTR

Testing
=======

This patch series has been tested on Qualcomm platforms with UFS 5.0
devices, validating:

- Successful TX EQTR completion for HS-G6
- Proper FOM evaluation and optimal settings selection
- Pre-Coding enablement for HS-G6
- Power mode changes with TX Equalization settings applied
- Report of TX Equalization settings via debugfs entries
- Report of TX EQTR histories via debug entries (see next section)
- Re-training TX Equalization via debugfs entry

Example of TX EQTR history
==========================

Device TX EQTR record summary -
Target Power Mode: HS-G6, Rate-B
Most recent record index: 2
Most recent record timestamp: 219573378 us

TX Lane 0 FOM - PreShoot\DeEmphasis
\       0        1        2        3        4        5        6        7
0      50       70       65        -        -        -        -        x
1       x        x        x        x        x        x        x        x
2     100       90       70        -        -        -        -        x
3       x        x        x        x        x        x        x        x
4      95       90        -        -        -        -        -        x
5       -        -        -        -        -        -        -        x
6       x        x        x        x        x        x        x        x
7       x        x        x        x        x        x        x        x

TX Lane 1 FOM - PreShoot\DeEmphasis
\       0        1        2        3        4        5        6        7
0      50       70       60        -        -        -        -        x
1       x        x        x        x        x        x        x        x
2     100       80       65        -        -        -        -        x
3       x        x        x        x        x        x        x        x
4      95       85        -        -        -        -        -        x
5       -        -        -        -        -        -        -        x
6       x        x        x        x        x        x        x        x
7       x        x        x        x        x        x        x        x

Patch Structure
===============

Patches 1-3: Preparatory changes for power mode negotiation and HS-G6
Patch 4: Core TX Equalization and EQTR implementation
Patches 5-7: Debugfs support for TX Equalization
Patches 8-12: Qualcomm vendor implementation

Next
====

One more series has been developed to enhance TX Equalization support,
which will be submitted for review after this series is accepted:

- Provide board specific (static) TX Equalization settings from DTS
- Parse static TX Equalization settings from DTS if provided
- Apply static TX Equalization settings if use_adaptive_txeq is disabled
- Add support for UFS v5.0 attributes qTxEQGnSettings & wTxEQGnSettingsExt
- Enable persistent storage and retrieval of optimal TX Equalization settings

Link: https://patch.msgid.link/20260325152154.1604082-1-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>

15 files changed, 2779 insertions, 115 deletions

diff --git a/drivers/ufs/core/Makefile b/drivers/ufs/core/Makefile
index 51e1867e524e..ce7d16d2cf35 100644
--- a/drivers/ufs/core/Makefile
+++ b/drivers/ufs/core/Makefile
@@ -1,7 +1,7 @@
 # SPDX-License-Identifier: GPL-2.0
 
 obj-$(CONFIG_SCSI_UFSHCD)		+= ufshcd-core.o
-ufshcd-core-y				+= ufshcd.o ufs-sysfs.o ufs-mcq.o
+ufshcd-core-y				+= ufshcd.o ufs-sysfs.o ufs-mcq.o ufs-txeq.o
 ufshcd-core-$(CONFIG_RPMB)		+= ufs-rpmb.o
 ufshcd-core-$(CONFIG_DEBUG_FS)		+= ufs-debugfs.o
 ufshcd-core-$(CONFIG_SCSI_UFS_BSG)	+= ufs_bsg.o
diff --git a/drivers/ufs/core/ufs-debugfs.c b/drivers/ufs/core/ufs-debugfs.c
index e3baed6c70bd..e3dd81d6fe82 100644
--- a/drivers/ufs/core/ufs-debugfs.c
+++ b/drivers/ufs/core/ufs-debugfs.c
@@ -209,6 +209,265 @@ static const struct ufs_debugfs_attr ufs_attrs[] = {
 	{ }
 };
 
+static int ufs_tx_eq_params_show(struct seq_file *s, void *data)
+{
+	const char *file_name = s->file->f_path.dentry->d_name.name;
+	u32 gear = (u32)(uintptr_t)s->file->f_inode->i_private;
+	struct ufs_hba *hba = hba_from_file(s->file);
+	struct ufshcd_tx_eq_settings *settings;
+	struct ufs_pa_layer_attr *pwr_info;
+	struct ufshcd_tx_eq_params *params;
+	u32 rate = hba->pwr_info.hs_rate;
+	u32 num_lanes;
+	int lane;
+
+	if (!ufshcd_is_tx_eq_supported(hba))
+		return -EOPNOTSUPP;
+
+	if (gear < UFS_HS_G1 || gear > UFS_HS_GEAR_MAX) {
+		seq_printf(s, "Invalid gear selected: %u\n", gear);
+		return 0;
+	}
+
+	if (!hba->max_pwr_info.is_valid) {
+		seq_puts(s, "Max power info is invalid\n");
+		return 0;
+	}
+
+	pwr_info = &hba->max_pwr_info.info;
+	params = &hba->tx_eq_params[gear - 1];
+	if (!params->is_valid) {
+		seq_printf(s, "TX EQ params are invalid for HS-G%u, Rate-%s\n",
+			   gear, ufs_hs_rate_to_str(rate));
+		return 0;
+	}
+
+	if (strcmp(file_name, "host_tx_eq_params") == 0) {
+		settings = params->host;
+		num_lanes = pwr_info->lane_tx;
+		seq_printf(s, "Host TX EQ PreShoot Cap: 0x%02x, DeEmphasis Cap: 0x%02x\n",
+			   hba->host_preshoot_cap, hba->host_deemphasis_cap);
+	} else if (strcmp(file_name, "device_tx_eq_params") == 0) {
+		settings = params->device;
+		num_lanes = pwr_info->lane_rx;
+		seq_printf(s, "Device TX EQ PreShoot Cap: 0x%02x, DeEmphasis Cap: 0x%02x\n",
+			   hba->device_preshoot_cap, hba->device_deemphasis_cap);
+	} else {
+		return -ENOENT;
+	}
+
+	seq_printf(s, "TX EQ setting for HS-G%u, Rate-%s:\n", gear,
+		   ufs_hs_rate_to_str(rate));
+	for (lane = 0; lane < num_lanes; lane++)
+		seq_printf(s, "TX Lane %d - PreShoot: %d, DeEmphasis: %d, Pre-Coding %senabled\n",
+			   lane, settings[lane].preshoot,
+			   settings[lane].deemphasis,
+			   settings[lane].precode_en ? "" : "not ");
+
+	return 0;
+}
+
+static int ufs_tx_eq_params_open(struct inode *inode, struct file *file)
+{
+	return single_open(file, ufs_tx_eq_params_show, inode->i_private);
+}
+
+static const struct file_operations ufs_tx_eq_params_fops = {
+	.owner		= THIS_MODULE,
+	.open		= ufs_tx_eq_params_open,
+	.read		= seq_read,
+	.llseek		= seq_lseek,
+	.release	= single_release,
+};
+
+static const struct ufs_debugfs_attr ufs_tx_eq_attrs[] = {
+	{ "host_tx_eq_params", 0400, &ufs_tx_eq_params_fops },
+	{ "device_tx_eq_params", 0400, &ufs_tx_eq_params_fops },
+	{ }
+};
+
+static int ufs_tx_eqtr_record_show(struct seq_file *s, void *data)
+{
+	const char *file_name = s->file->f_path.dentry->d_name.name;
+	u8 (*fom_array)[TX_HS_NUM_PRESHOOT][TX_HS_NUM_DEEMPHASIS];
+	u32 gear = (u32)(uintptr_t)s->file->f_inode->i_private;
+	unsigned long preshoot_bitmap, deemphasis_bitmap;
+	struct ufs_hba *hba = hba_from_file(s->file);
+	struct ufs_pa_layer_attr *pwr_info;
+	struct ufshcd_tx_eq_params *params;
+	struct ufshcd_tx_eqtr_record *rec;
+	u32 rate = hba->pwr_info.hs_rate;
+	u8 preshoot, deemphasis;
+	u32 num_lanes;
+	char name[32];
+	int lane;
+
+	if (!ufshcd_is_tx_eq_supported(hba))
+		return -EOPNOTSUPP;
+
+	if (gear < UFS_HS_G1 || gear > UFS_HS_GEAR_MAX) {
+		seq_printf(s, "Invalid gear selected: %u\n", gear);
+		return 0;
+	}
+
+	if (!hba->max_pwr_info.is_valid) {
+		seq_puts(s, "Max power info is invalid\n");
+		return 0;
+	}
+
+	pwr_info = &hba->max_pwr_info.info;
+	params = &hba->tx_eq_params[gear - 1];
+	if (!params->is_valid) {
+		seq_printf(s, "TX EQ params are invalid for HS-G%u, Rate-%s\n",
+			   gear, ufs_hs_rate_to_str(rate));
+		return 0;
+	}
+
+	rec = params->eqtr_record;
+	if (!rec || !rec->last_record_index) {
+		seq_printf(s, "No TX EQTR records found for HS-G%u, Rate-%s.\n",
+			   gear, ufs_hs_rate_to_str(rate));
+		return 0;
+	}
+
+	if (strcmp(file_name, "host_tx_eqtr_record") == 0) {
+		preshoot_bitmap = (hba->host_preshoot_cap << 0x1) | 0x1;
+		deemphasis_bitmap = (hba->host_deemphasis_cap << 0x1) | 0x1;
+		num_lanes = pwr_info->lane_tx;
+		fom_array = rec->host_fom;
+		snprintf(name, sizeof(name), "%s", "Host");
+	} else if (strcmp(file_name, "device_tx_eqtr_record") == 0) {
+		preshoot_bitmap = (hba->device_preshoot_cap << 0x1) | 0x1;
+		deemphasis_bitmap = (hba->device_deemphasis_cap << 0x1) | 0x1;
+		num_lanes = pwr_info->lane_rx;
+		fom_array = rec->device_fom;
+		snprintf(name, sizeof(name), "%s", "Device");
+	} else {
+		return -ENOENT;
+	}
+
+	seq_printf(s, "%s TX EQTR record summary -\n", name);
+	seq_printf(s, "Target Power Mode: HS-G%u, Rate-%s\n", gear,
+		   ufs_hs_rate_to_str(rate));
+	seq_printf(s, "Most recent record index: %d\n",
+		   rec->last_record_index);
+	seq_printf(s, "Most recent record timestamp: %llu us\n",
+		   ktime_to_us(rec->last_record_ts));
+
+	for (lane = 0; lane < num_lanes; lane++) {
+		seq_printf(s, "\nTX Lane %d FOM - %s\n", lane, "PreShoot\\DeEmphasis");
+		seq_puts(s, "\\");
+		/* Print DeEmphasis header as X-axis. */
+		for (deemphasis = 0; deemphasis < TX_HS_NUM_DEEMPHASIS; deemphasis++)
+			seq_printf(s, "%8d%s", deemphasis, " ");
+		seq_puts(s, "\n");
+		/* Print matrix rows with PreShoot as Y-axis. */
+		for (preshoot = 0; preshoot < TX_HS_NUM_PRESHOOT; preshoot++) {
+			seq_printf(s, "%d", preshoot);
+			for (deemphasis = 0; deemphasis < TX_HS_NUM_DEEMPHASIS; deemphasis++) {
+				if (test_bit(preshoot, &preshoot_bitmap) &&
+				    test_bit(deemphasis, &deemphasis_bitmap)) {
+					u8 fom = fom_array[lane][preshoot][deemphasis];
+					u8 fom_val = fom & RX_FOM_VALUE_MASK;
+					bool precode_en = fom & RX_FOM_PRECODING_EN_BIT;
+
+					if (ufshcd_is_txeq_presets_used(hba) &&
+					    !ufshcd_is_txeq_preset_selected(preshoot, deemphasis))
+						seq_printf(s, "%8s%s", "-", " ");
+					else
+						seq_printf(s, "%8u%s", fom_val,
+							   precode_en ? "*" : " ");
+				} else {
+					seq_printf(s, "%8s%s", "x", " ");
+				}
+			}
+			seq_puts(s, "\n");
+		}
+	}
+
+	return 0;
+}
+
+static int ufs_tx_eqtr_record_open(struct inode *inode, struct file *file)
+{
+	return single_open(file, ufs_tx_eqtr_record_show, inode->i_private);
+}
+
+static const struct file_operations ufs_tx_eqtr_record_fops = {
+	.owner		= THIS_MODULE,
+	.open		= ufs_tx_eqtr_record_open,
+	.read		= seq_read,
+	.llseek		= seq_lseek,
+	.release	= single_release,
+};
+
+static ssize_t ufs_tx_eq_ctrl_write(struct file *file, const char __user *buf,
+				    size_t count, loff_t *ppos)
+{
+	u32 gear = (u32)(uintptr_t)file->f_inode->i_private;
+	struct ufs_hba *hba = hba_from_file(file);
+	char kbuf[32];
+	int ret;
+
+	if (count >= sizeof(kbuf))
+		return -EINVAL;
+
+	if (copy_from_user(kbuf, buf, count))
+		return -EFAULT;
+
+	if (!ufshcd_is_tx_eq_supported(hba))
+		return -EOPNOTSUPP;
+
+	if (hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL ||
+	    !hba->max_pwr_info.is_valid)
+		return -EBUSY;
+
+	if (!hba->ufs_device_wlun)
+		return -ENODEV;
+
+	kbuf[count] = '\0';
+
+	if (sysfs_streq(kbuf, "retrain")) {
+		ret = ufs_debugfs_get_user_access(hba);
+		if (ret)
+			return ret;
+		ret = ufshcd_retrain_tx_eq(hba, gear);
+		ufs_debugfs_put_user_access(hba);
+	} else {
+		/* Unknown operation */
+		return -EINVAL;
+	}
+
+	return ret ? ret : count;
+}
+
+static int ufs_tx_eq_ctrl_show(struct seq_file *s, void *data)
+{
+	seq_puts(s, "write 'retrain' to retrain TX Equalization settings\n");
+	return 0;
+}
+
+static int ufs_tx_eq_ctrl_open(struct inode *inode, struct file *file)
+{
+	return single_open(file, ufs_tx_eq_ctrl_show, inode->i_private);
+}
+
+static const struct file_operations ufs_tx_eq_ctrl_fops = {
+	.owner		= THIS_MODULE,
+	.open		= ufs_tx_eq_ctrl_open,
+	.read		= seq_read,
+	.llseek		= seq_lseek,
+	.write		= ufs_tx_eq_ctrl_write,
+	.release	= single_release,
+};
+
+static const struct ufs_debugfs_attr ufs_tx_eqtr_attrs[] = {
+	{ "host_tx_eqtr_record", 0400, &ufs_tx_eqtr_record_fops },
+	{ "device_tx_eqtr_record", 0400, &ufs_tx_eqtr_record_fops },
+	{ "tx_eq_ctrl", 0600, &ufs_tx_eq_ctrl_fops },
+	{ }
+};
+
 void ufs_debugfs_hba_init(struct ufs_hba *hba)
 {
 	const struct ufs_debugfs_attr *attr;
@@ -230,6 +489,37 @@ void ufs_debugfs_hba_init(struct ufs_hba *hba)
 			    hba, &ee_usr_mask_fops);
 	debugfs_create_u32("exception_event_rate_limit_ms", 0600, hba->debugfs_root,
 			   &hba->debugfs_ee_rate_limit_ms);
+
+	if (!(hba->caps & UFSHCD_CAP_TX_EQUALIZATION))
+		return;
+
+	for (u32 gear = UFS_HS_G1; gear <= UFS_HS_GEAR_MAX; gear++) {
+		struct dentry *txeq_dir;
+		char name[32];
+
+		snprintf(name, sizeof(name), "tx_eq_hs_gear%d", gear);
+		txeq_dir = debugfs_create_dir(name, hba->debugfs_root);
+		if (IS_ERR_OR_NULL(txeq_dir))
+			return;
+
+		d_inode(txeq_dir)->i_private = hba;
+
+		/* Create files for TX Equalization parameters */
+		for (attr = ufs_tx_eq_attrs; attr->name; attr++)
+			debugfs_create_file(attr->name, attr->mode, txeq_dir,
+					    (void *)(uintptr_t)gear,
+					    attr->fops);
+
+		/* TX EQTR is supported for HS-G4 and higher Gears */
+		if (gear < UFS_HS_G4)
+			continue;
+
+		/* Create files for TX EQTR related attributes */
+		for (attr = ufs_tx_eqtr_attrs; attr->name; attr++)
+			debugfs_create_file(attr->name, attr->mode, txeq_dir,
+					    (void *)(uintptr_t)gear,
+					    attr->fops);
+	}
 }
 
 void ufs_debugfs_hba_exit(struct ufs_hba *hba)
diff --git a/drivers/ufs/core/ufs-txeq.c b/drivers/ufs/core/ufs-txeq.c
new file mode 100644
index 000000000000..b2dc89124353
--- /dev/null
+++ b/drivers/ufs/core/ufs-txeq.c
@@ -0,0 +1,1293 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2026 Qualcomm Technologies, Inc.
+ *
+ * Author:
+ *	Can Guo <can.guo@oss.qualcomm.com>
+ */
+
+#include <linux/bitops.h>
+#include <linux/delay.h>
+#include <linux/errno.h>
+#include <linux/kernel.h>
+#include <ufs/ufshcd.h>
+#include <ufs/unipro.h>
+#include "ufshcd-priv.h"
+
+static bool use_adaptive_txeq;
+module_param(use_adaptive_txeq, bool, 0644);
+MODULE_PARM_DESC(use_adaptive_txeq, "Find and apply optimal TX Equalization settings before changing Power Mode (default: false)");
+
+static int txeq_gear_set(const char *val, const struct kernel_param *kp)
+{
+	return param_set_uint_minmax(val, kp, UFS_HS_G1, UFS_HS_GEAR_MAX);
+}
+
+static const struct kernel_param_ops txeq_gear_ops = {
+	.set = txeq_gear_set,
+	.get = param_get_uint,
+};
+
+static unsigned int adaptive_txeq_gear = UFS_HS_G6;
+module_param_cb(adaptive_txeq_gear, &txeq_gear_ops, &adaptive_txeq_gear, 0644);
+MODULE_PARM_DESC(adaptive_txeq_gear, "For HS-Gear[n] and above, adaptive txeq shall be used");
+
+static bool use_txeq_presets;
+module_param(use_txeq_presets, bool, 0644);
+MODULE_PARM_DESC(use_txeq_presets, "Use only the 8 TX Equalization Presets (pre-defined Pre-Shoot & De-Emphasis combinations) for TX EQTR (default: false)");
+
+static bool txeq_presets_selected[UFS_TX_EQ_PRESET_MAX] = {[0 ... (UFS_TX_EQ_PRESET_MAX - 1)] = 1};
+module_param_array(txeq_presets_selected, bool, NULL, 0644);
+MODULE_PARM_DESC(txeq_presets_selected, "Use only the selected Presets out of the 8 TX Equalization Presets for TX EQTR");
+
+/*
+ * ufs_tx_eq_preset - Table of minimum required list of presets.
+ *
+ * A HS-G6 capable M-TX shall support the presets defined in M-PHY v6.0 spec.
+ * Preset	Pre-Shoot(dB)	De-Emphasis(dB)
+ * P0		0.0		0.0
+ * P1		0.0		0.8
+ * P2		0.0		1.6
+ * P3		0.8		0.0
+ * P4		1.6		0.0
+ * P5		0.8		0.8
+ * P6		0.8		1.6
+ * P7		1.6		0.8
+ */
+static const struct __ufs_tx_eq_preset {
+	u8 preshoot;
+	u8 deemphasis;
+} ufs_tx_eq_preset[UFS_TX_EQ_PRESET_MAX] = {
+	[UFS_TX_EQ_PRESET_P0] = {UFS_TX_HS_PRESHOOT_DB_0P0, UFS_TX_HS_DEEMPHASIS_DB_0P0},
+	[UFS_TX_EQ_PRESET_P1] = {UFS_TX_HS_PRESHOOT_DB_0P0, UFS_TX_HS_DEEMPHASIS_DB_0P8},
+	[UFS_TX_EQ_PRESET_P2] = {UFS_TX_HS_PRESHOOT_DB_0P0, UFS_TX_HS_DEEMPHASIS_DB_1P6},
+	[UFS_TX_EQ_PRESET_P3] = {UFS_TX_HS_PRESHOOT_DB_0P8, UFS_TX_HS_DEEMPHASIS_DB_0P0},
+	[UFS_TX_EQ_PRESET_P4] = {UFS_TX_HS_PRESHOOT_DB_1P6, UFS_TX_HS_DEEMPHASIS_DB_0P0},
+	[UFS_TX_EQ_PRESET_P5] = {UFS_TX_HS_PRESHOOT_DB_0P8, UFS_TX_HS_DEEMPHASIS_DB_0P8},
+	[UFS_TX_EQ_PRESET_P6] = {UFS_TX_HS_PRESHOOT_DB_0P8, UFS_TX_HS_DEEMPHASIS_DB_1P6},
+	[UFS_TX_EQ_PRESET_P7] = {UFS_TX_HS_PRESHOOT_DB_1P6, UFS_TX_HS_DEEMPHASIS_DB_0P8},
+};
+
+/*
+ * pa_peer_rx_adapt_initial - Table of UniPro PA_PeerRxHSGnAdaptInitial
+ * attribute IDs for High Speed (HS) Gears.
+ *
+ * This table maps HS Gears to their respective UniPro PA_PeerRxHSGnAdaptInitial
+ * attribute IDs. Entries for Gears 1-3 are 0 (unsupported).
+ */
+static const u32 pa_peer_rx_adapt_initial[UFS_HS_GEAR_MAX] = {
+	0,
+	0,
+	0,
+	PA_PEERRXHSG4ADAPTINITIAL,
+	PA_PEERRXHSG5ADAPTINITIAL,
+	PA_PEERRXHSG6ADAPTINITIALL0L3
+};
+
+/*
+ * rx_adapt_initial_cap - Table of M-PHY RX_HS_Gn_ADAPT_INITIAL_Capability
+ * attribute IDs for High Speed (HS) Gears.
+ *
+ * This table maps HS Gears to their respective M-PHY
+ * RX_HS_Gn_ADAPT_INITIAL_Capability attribute IDs. Entries for Gears 1-3 are 0
+ * (unsupported).
+ */
+static const u32 rx_adapt_initial_cap[UFS_HS_GEAR_MAX] = {
+	0,
+	0,
+	0,
+	RX_HS_G4_ADAPT_INITIAL_CAP,
+	RX_HS_G5_ADAPT_INITIAL_CAP,
+	RX_HS_G6_ADAPT_INITIAL_CAP
+};
+
+/*
+ * pa_tx_eq_setting - Table of UniPro PA_TxEQGnSetting attribute IDs for High
+ * Speed (HS) Gears.
+ *
+ * This table maps HS Gears to their respective UniPro PA_TxEQGnSetting
+ * attribute IDs.
+ */
+static const u32 pa_tx_eq_setting[UFS_HS_GEAR_MAX] = {
+	PA_TXEQG1SETTING,
+	PA_TXEQG2SETTING,
+	PA_TXEQG3SETTING,
+	PA_TXEQG4SETTING,
+	PA_TXEQG5SETTING,
+	PA_TXEQG6SETTING
+};
+
+/**
+ * ufshcd_configure_precoding - Configure Pre-Coding for all active lanes
+ * @hba: per adapter instance
+ * @params: TX EQ parameters data structure
+ *
+ * Bit[7] in RX_FOM indicates that the receiver needs to enable Pre-Coding when
+ * set. Pre-Coding must be enabled on both the transmitter and receiver to
+ * ensure proper operation.
+ *
+ * Returns 0 on success, non-zero error code otherwise
+ */
+static int ufshcd_configure_precoding(struct ufs_hba *hba,
+				      struct ufshcd_tx_eq_params *params)
+{
+	struct ufs_pa_layer_attr *pwr_info = &hba->max_pwr_info.info;
+	u32 local_precode_en = 0;
+	u32 peer_precode_en = 0;
+	int lane, ret;
+
+	/* Enable Pre-Coding for Host's TX & Device's RX pair */
+	for (lane = 0; lane < pwr_info->lane_tx; lane++) {
+		if (params->host[lane].precode_en) {
+			local_precode_en |= PRECODEEN_TX_BIT(lane);
+			peer_precode_en |= PRECODEEN_RX_BIT(lane);
+		}
+	}
+
+	/* Enable Pre-Coding for Device's TX & Host's RX pair */
+	for (lane = 0; lane < pwr_info->lane_rx; lane++) {
+		if (params->device[lane].precode_en) {
+			peer_precode_en |= PRECODEEN_TX_BIT(lane);
+			local_precode_en |= PRECODEEN_RX_BIT(lane);
+		}
+	}
+
+	if (!local_precode_en && !peer_precode_en) {
+		dev_dbg(hba->dev, "Pre-Coding is not required for Host and Device\n");
+		return 0;
+	}
+
+	/* Set local PA_PreCodeEn */
+	ret = ufshcd_dme_set(hba, UIC_ARG_MIB(PA_PRECODEEN), local_precode_en);
+	if (ret) {
+		dev_err(hba->dev, "Failed to set local PA_PreCodeEn: %d\n", ret);
+		return ret;
+	}
+
+	/* Set peer PA_PreCodeEn */
+	ret = ufshcd_dme_peer_set(hba, UIC_ARG_MIB(PA_PRECODEEN), peer_precode_en);
+	if (ret) {
+		dev_err(hba->dev, "Failed to set peer PA_PreCodeEn: %d\n", ret);
+		return ret;
+	}
+
+	dev_dbg(hba->dev, "Local PA_PreCodeEn: 0x%02x, Peer PA_PreCodeEn: 0x%02x\n",
+		local_precode_en, peer_precode_en);
+
+	return 0;
+}
+
+void ufshcd_print_tx_eq_params(struct ufs_hba *hba)
+{
+	struct ufs_pa_layer_attr *pwr_info = &hba->max_pwr_info.info;
+	struct ufshcd_tx_eq_params *params;
+	u32 gear = hba->pwr_info.gear_tx;
+	int lane;
+
+	if (!ufshcd_is_tx_eq_supported(hba))
+		return;
+
+	if (gear < UFS_HS_G1 || gear > UFS_HS_GEAR_MAX)
+		return;
+
+	params = &hba->tx_eq_params[gear - 1];
+	if (!params->is_valid || !params->is_applied)
+		return;
+
+	for (lane = 0; lane < pwr_info->lane_tx; lane++)
+		dev_dbg(hba->dev, "Host TX Lane %d: PreShoot %u, DeEmphasis %u, FOM %u, PreCodeEn %d\n",
+			lane, params->host[lane].preshoot,
+			params->host[lane].deemphasis,
+			params->host[lane].fom_val,
+			params->host[lane].precode_en);
+
+	for (lane = 0; lane < pwr_info->lane_rx; lane++)
+		dev_dbg(hba->dev, "Device TX Lane %d: PreShoot %u, DeEmphasis %u, FOM %u, PreCodeEn %d\n",
+			lane, params->device[lane].preshoot,
+			params->device[lane].deemphasis,
+			params->device[lane].fom_val,
+			params->device[lane].precode_en);
+}
+
+static inline u32
+ufshcd_compose_tx_eq_setting(struct ufshcd_tx_eq_settings *settings,
+			     int num_lanes)
+{
+	u32 setting = 0;
+	int lane;
+
+	for (lane = 0; lane < num_lanes; lane++, settings++) {
+		setting |= TX_HS_PRESHOOT_BITS(lane, settings->preshoot);
+		setting |= TX_HS_DEEMPHASIS_BITS(lane, settings->deemphasis);
+	}
+
+	return setting;
+}
+
+/**
+ * ufshcd_apply_tx_eq_settings - Apply TX Equalization settings for target gear
+ * @hba: per adapter instance
+ * @params: TX EQ parameters data structure
+ * @gear: target gear
+ *
+ * Returns 0 on success, negative error code otherwise
+ */
+int ufshcd_apply_tx_eq_settings(struct ufs_hba *hba,
+				struct ufshcd_tx_eq_params *params, u32 gear)
+{
+	struct ufs_pa_layer_attr *pwr_info = &hba->max_pwr_info.info;
+	u32 setting;
+	int ret;
+
+	/* Compose settings for Host's TX Lanes */
+	setting = ufshcd_compose_tx_eq_setting(params->host, pwr_info->lane_tx);
+	ret = ufshcd_dme_set(hba, UIC_ARG_MIB(pa_tx_eq_setting[gear - 1]), setting);
+	if (ret)
+		return ret;
+
+	/* Compose settings for Device's TX Lanes */
+	setting = ufshcd_compose_tx_eq_setting(params->device, pwr_info->lane_rx);
+	ret = ufshcd_dme_peer_set(hba, UIC_ARG_MIB(pa_tx_eq_setting[gear - 1]), setting);
+	if (ret)
+		return ret;
+
+	/* Configure Pre-Coding */
+	if (gear >= UFS_HS_G6) {
+		ret = ufshcd_configure_precoding(hba, params);
+		if (ret) {
+			dev_err(hba->dev, "Failed to configure pre-coding: %d\n", ret);
+			return ret;
+		}
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(ufshcd_apply_tx_eq_settings);
+
+/**
+ * ufshcd_evaluate_tx_eqtr_fom - Evaluate TX EQTR FOM results
+ * @hba: per adapter instance
+ * @pwr_mode: target power mode containing gear and rate information
+ * @eqtr_data: TX EQTR data structure
+ * @h_iter: host TX EQTR iterator data structure
+ * @d_iter: device TX EQTR iterator data structure
+ *
+ * Evaluate TX EQTR FOM results, update host and device TX EQTR data accordingy
+ * if FOM have been improved compared to previous iteration, and record TX EQTR
+ * FOM results.
+ */
+static void ufshcd_evaluate_tx_eqtr_fom(struct ufs_hba *hba,
+					struct ufs_pa_layer_attr *pwr_mode,
+					struct ufshcd_tx_eqtr_data *eqtr_data,
+					struct tx_eqtr_iter *h_iter,
+					struct tx_eqtr_iter *d_iter)
+{
+	u8 preshoot, deemphasis, fom_value;
+	bool precode_en;
+	int lane;
+
+	for (lane = 0; h_iter->is_updated && lane < pwr_mode->lane_tx; lane++) {
+		preshoot = h_iter->preshoot;
+		deemphasis = h_iter->deemphasis;
+		fom_value = h_iter->fom[lane] & RX_FOM_VALUE_MASK;
+		precode_en = h_iter->fom[lane] & RX_FOM_PRECODING_EN_BIT;
+
+		/* Record host TX EQTR FOM */
+		eqtr_data->host_fom[lane][preshoot][deemphasis] = h_iter->fom[lane];
+
+		/* Check if FOM has been improved for host's TX Lanes */
+		if (fom_value > eqtr_data->host[lane].fom_val) {
+			eqtr_data->host[lane].preshoot = preshoot;
+			eqtr_data->host[lane].deemphasis = deemphasis;
+			eqtr_data->host[lane].fom_val = fom_value;
+			eqtr_data->host[lane].precode_en = precode_en;
+		}
+
+		dev_dbg(hba->dev, "TX EQTR: Host TX Lane %d: PreShoot %u, DeEmphasis %u, FOM value %u, PreCodeEn %d\n",
+			lane, preshoot, deemphasis, fom_value, precode_en);
+	}
+
+	for (lane = 0; d_iter->is_updated && lane < pwr_mode->lane_rx; lane++) {
+		preshoot = d_iter->preshoot;
+		deemphasis = d_iter->deemphasis;
+		fom_value = d_iter->fom[lane] & RX_FOM_VALUE_MASK;
+		precode_en = d_iter->fom[lane] & RX_FOM_PRECODING_EN_BIT;
+
+		/* Record device TX EQTR FOM */
+		eqtr_data->device_fom[lane][preshoot][deemphasis] = d_iter->fom[lane];
+
+		/* Check if FOM has been improved for Device's TX Lanes */
+		if (fom_value > eqtr_data->device[lane].fom_val) {
+			eqtr_data->device[lane].preshoot = preshoot;
+			eqtr_data->device[lane].deemphasis = deemphasis;
+			eqtr_data->device[lane].fom_val = fom_value;
+			eqtr_data->device[lane].precode_en = precode_en;
+		}
+
+		dev_dbg(hba->dev, "TX EQTR: Device TX Lane %d: PreShoot %u, DeEmphasis %u, FOM value %u, PreCodeEn %d\n",
+			lane, preshoot, deemphasis, fom_value, precode_en);
+	}
+}
+
+/**
+ * ufshcd_get_rx_fom - Get Figure of Merit (FOM) for both sides
+ * @hba: per adapter instance
+ * @pwr_mode: target power mode containing gear and rate information
+ * @h_iter: host TX EQTR iterator data structure
+ * @d_iter: device TX EQTR iterator data structure
+ *
+ * Returns 0 on success, negative error code otherwise
+ */
+static int ufshcd_get_rx_fom(struct ufs_hba *hba,
+			     struct ufs_pa_layer_attr *pwr_mode,
+			     struct tx_eqtr_iter *h_iter,
+			     struct tx_eqtr_iter *d_iter)
+{
+	int lane, ret;
+	u32 fom;
+
+	/* Get FOM of host's TX lanes from device's RX_FOM. */
+	for (lane = 0; lane < pwr_mode->lane_tx; lane++) {
+		ret = ufshcd_dme_peer_get(hba, UIC_ARG_MIB_SEL(RX_FOM,
+					  UIC_ARG_MPHY_RX_GEN_SEL_INDEX(lane)),
+					  &fom);
+		if (ret)
+			return ret;
+
+		h_iter->fom[lane] = (u8)fom;
+	}
+
+	/* Get FOM of device's TX lanes from host's RX_FOM. */
+	for (lane = 0; lane < pwr_mode->lane_rx; lane++) {
+		ret = ufshcd_dme_get(hba, UIC_ARG_MIB_SEL(RX_FOM,
+				     UIC_ARG_MPHY_RX_GEN_SEL_INDEX(lane)),
+				     &fom);
+		if (ret)
+			return ret;
+
+		d_iter->fom[lane] = (u8)fom;
+	}
+
+	ret = ufshcd_vops_get_rx_fom(hba, pwr_mode, h_iter, d_iter);
+	if (ret)
+		dev_err(hba->dev, "Failed to get FOM via vops: %d\n", ret);
+
+	return ret;
+}
+
+bool ufshcd_is_txeq_presets_used(struct ufs_hba *hba)
+{
+	return use_txeq_presets;
+}
+
+bool ufshcd_is_txeq_preset_selected(u8 preshoot, u8 deemphasis)
+{
+	int i;
+
+	for (i = 0; i < UFS_TX_EQ_PRESET_MAX; i++) {
+		if (!txeq_presets_selected[i])
+			continue;
+
+		if (preshoot == ufs_tx_eq_preset[i].preshoot &&
+		    deemphasis == ufs_tx_eq_preset[i].deemphasis)
+			return true;
+	}
+
+	return false;
+}
+
+/**
+ * tx_eqtr_iter_try_update - Try to update a TX EQTR iterator
+ * @iter: TX EQTR iterator data structure
+ * @preshoot: PreShoot value
+ * @deemphasis: DeEmphasis value
+ *
+ * This function validates whether the provided PreShoot and DeEmphasis
+ * combination can be used or not. If yes, it updates the TX EQTR iterator with
+ * the provided PreShoot and DeEmphasis, it also sets the is_updated flag
+ * to indicate the iterator has been updated.
+ */
+static void tx_eqtr_iter_try_update(struct tx_eqtr_iter *iter,
+				    u8 preshoot, u8 deemphasis)
+{
+	if (!test_bit(preshoot, &iter->preshoot_bitmap) ||
+	    !test_bit(deemphasis, &iter->deemphasis_bitmap) ||
+	    (use_txeq_presets && !ufshcd_is_txeq_preset_selected(preshoot, deemphasis))) {
+		iter->is_updated = false;
+		return;
+	}
+
+	iter->preshoot = preshoot;
+	iter->deemphasis = deemphasis;
+	iter->is_updated = true;
+}
+
+/**
+ * tx_eqtr_iter_update() - Update host and deviceTX EQTR iterators
+ * @preshoot: PreShoot value
+ * @deemphasis: DeEmphasis value
+ * @h_iter: Host TX EQTR iterator data structure
+ * @d_iter: Device TX EQTR iterator data structure
+ *
+ * Updates host and device TX Equalization training iterators with the
+ * provided PreShoot and DeEmphasis.
+ *
+ * Return: true if host and/or device TX Equalization training iterator has
+ * been updated to the provided PreShoot and DeEmphasis, false otherwise.
+ */
+static bool tx_eqtr_iter_update(u8 preshoot, u8 deemphasis,
+				struct tx_eqtr_iter *h_iter,
+				struct tx_eqtr_iter *d_iter)
+{
+	tx_eqtr_iter_try_update(h_iter, preshoot, deemphasis);
+	tx_eqtr_iter_try_update(d_iter, preshoot, deemphasis);
+
+	return h_iter->is_updated || d_iter->is_updated;
+}
+
+/**
+ * ufshcd_tx_eqtr_iter_init - Initialize host and device TX EQTR iterators
+ * @hba: per adapter instance
+ * @h_iter: host TX EQTR iterator data structure
+ * @d_iter: device TX EQTR iterator data structure
+ *
+ * This function initializes the TX EQTR iterator structures for both host and
+ * device by reading their TX equalization capabilities. The capabilities are
+ * cached in the hba structure to avoid redundant DME operations in subsequent
+ * calls. In the TX EQTR procedure, the iterator structures are updated by
+ * tx_eqtr_iter_update() to systematically iterate through supported TX
+ * Equalization setting combinations.
+ *
+ * Returns 0 on success, negative error code otherwise
+ */
+static int ufshcd_tx_eqtr_iter_init(struct ufs_hba *hba,
+				    struct tx_eqtr_iter *h_iter,
+				    struct tx_eqtr_iter *d_iter)
+{
+	u32 cap;
+	int ret;
+
+	if (!hba->host_preshoot_cap) {
+		ret = ufshcd_dme_get(hba, UIC_ARG_MIB(TX_HS_PRESHOOT_SETTING_CAP), &cap);
+		if (ret)
+			return ret;
+
+		hba->host_preshoot_cap = cap & TX_EQTR_CAP_MASK;
+	}
+
+	if (!hba->host_deemphasis_cap) {
+		ret = ufshcd_dme_get(hba, UIC_ARG_MIB(TX_HS_DEEMPHASIS_SETTING_CAP), &cap);
+		if (ret)
+			return ret;
+
+		hba->host_deemphasis_cap = cap & TX_EQTR_CAP_MASK;
+	}
+
+	if (!hba->device_preshoot_cap) {
+		ret = ufshcd_dme_peer_get(hba, UIC_ARG_MIB(TX_HS_PRESHOOT_SETTING_CAP), &cap);
+		if (ret)
+			return ret;
+
+		hba->device_preshoot_cap = cap & TX_EQTR_CAP_MASK;
+	}
+
+	if (!hba->device_deemphasis_cap) {
+		ret = ufshcd_dme_peer_get(hba, UIC_ARG_MIB(TX_HS_DEEMPHASIS_SETTING_CAP), &cap);
+		if (ret)
+			return ret;
+
+		hba->device_deemphasis_cap = cap & TX_EQTR_CAP_MASK;
+	}
+
+	/*
+	 * Support PreShoot & DeEmphasis of value 0 is mandatory, hence they are
+	 * not reflected in PreShoot/DeEmphasis capabilities. Left shift the
+	 * capability bitmap by 1 and set bit[0] to reflect value 0 is
+	 * supported, such that test_bit() can be used later for convenience.
+	 */
+	h_iter->preshoot_bitmap = (hba->host_preshoot_cap << 0x1) | 0x1;
+	h_iter->deemphasis_bitmap = (hba->host_deemphasis_cap <<