From patchwork Thu Jun 24 21:33:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pali_Roh=C3=A1r?= X-Patchwork-Id: 1496833 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=je0lJSUk; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4G9tgG5SbSz9sRf for ; Fri, 25 Jun 2021 07:34:02 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232705AbhFXVgV (ORCPT ); Thu, 24 Jun 2021 17:36:21 -0400 Received: from mail.kernel.org ([198.145.29.99]:51854 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232591AbhFXVgU (ORCPT ); Thu, 24 Jun 2021 17:36:20 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id D3A7F613B7; Thu, 24 Jun 2021 21:34:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1624570441; bh=ticgXE7CjNhWfVymrLf5bGBpQYvv4IhRaSX2oXSsd1I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=je0lJSUkjWr6LfhmkY1QEILZbvNQwe8wmPRM/TrXmK1AjiIKFHuB5dK1TjC7vQ2H7 Q+UUhqcw18blwKk+JBNE3j0MipA4pwbaSgSAdQFmLp3kvMHKKS4ddIwQQcwamz0iVN nxxI57PuJUXjYyGchYvhyN1SQmiVVMPbpfY5XDkpBuYg6nbc3mzQWK5y02BX8c88nr OVCSxio5AaC3KGaYE71Wy3udYilJBd7MCUobi0VF1BINrS/Bpvim0YyMtsJ67auJrt SRXRGezrr+rG7w29mktUwxw9KHlWRSrBqBV+7p7YFPcAhSVPlhZFzf0+IJ62u5NJDl Up2vvC69POWNQ== Received: by pali.im (Postfix) id 0E1EB96D; Thu, 24 Jun 2021 23:33:59 +0200 (CEST) From: =?utf-8?q?Pali_Roh=C3=A1r?= To: Lorenzo Pieralisi , Thomas Petazzoni , Bjorn Helgaas Cc: =?utf-8?q?Marek_Beh=C3=BAn?= , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RESEND PATCH 1/3] PCI: aardvark: Fix checking for PIO Non-posted Request Date: Thu, 24 Jun 2021 23:33:43 +0200 Message-Id: <20210624213345.3617-2-pali@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210624213345.3617-1-pali@kernel.org> References: <20210624213345.3617-1-pali@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org PIO_NON_POSTED_REQ for PIO_STAT register is incorrectly defined. Bit 10 in register PIO_STAT indicates the response is to a non-posted request. Signed-off-by: Pali Rohár Reviewed-by: Marek Behún Cc: stable@vger.kernel.org --- drivers/pci/controller/pci-aardvark.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c index e3f5e7ab7606..2f8380a1f84f 100644 --- a/drivers/pci/controller/pci-aardvark.c +++ b/drivers/pci/controller/pci-aardvark.c @@ -57,7 +57,7 @@ #define PIO_COMPLETION_STATUS_UR 1 #define PIO_COMPLETION_STATUS_CRS 2 #define PIO_COMPLETION_STATUS_CA 4 -#define PIO_NON_POSTED_REQ BIT(0) +#define PIO_NON_POSTED_REQ BIT(10) #define PIO_ADDR_LS (PIO_BASE_ADDR + 0x8) #define PIO_ADDR_MS (PIO_BASE_ADDR + 0xc) #define PIO_WR_DATA (PIO_BASE_ADDR + 0x10) From patchwork Thu Jun 24 21:33:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pali_Roh=C3=A1r?= X-Patchwork-Id: 1496834 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=GpFVCcAb; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4G9tgJ0MrXz9sWw for ; Fri, 25 Jun 2021 07:34:04 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232684AbhFXVgV (ORCPT ); Thu, 24 Jun 2021 17:36:21 -0400 Received: from mail.kernel.org ([198.145.29.99]:51864 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232677AbhFXVgU (ORCPT ); Thu, 24 Jun 2021 17:36:20 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id E362E613C3; Thu, 24 Jun 2021 21:34:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1624570441; bh=Fu4/t9pMjXwvsBTR1oTGCXyaBjHyoagLyBJfrUrRITM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GpFVCcAbGR4zcsWlck2egQvyCOOeASI/B8zSUTp69atLhimEY8pYZlvo1OT7I8gIk NgZjPhImZvva8OkleAEER4VUBTLqByhbxDdtzXoRD1UqjLkFdVnvq7mD+NWN599NPQ 66xw/tQMcjfY7mdpzQbXrVE3AShG2Lki+cZwTzy88yFpVG/fjmIMR9YsHVeWmyHddU J1g/8ZAh9C5v5PkT48l2abmi94imL1L3LS6oE6b2kEgADgpzXl1FAE/xGVEsJfexpz NzZ9FNqsLYPjcDtPoyKTBy2rKm/tgdCdMdGLbmQbbhTg1rFy95TLJSRtixi6/wKsUC ynue32kge/i5w== Received: by pali.im (Postfix) id A284B52D; Thu, 24 Jun 2021 23:34:00 +0200 (CEST) From: =?utf-8?q?Pali_Roh=C3=A1r?= To: Lorenzo Pieralisi , Thomas Petazzoni , Bjorn Helgaas Cc: =?utf-8?q?Marek_Beh=C3=BAn?= , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RESEND PATCH 2/3] PCI: aardvark: Fix checking for PIO status Date: Thu, 24 Jun 2021 23:33:44 +0200 Message-Id: <20210624213345.3617-3-pali@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210624213345.3617-1-pali@kernel.org> References: <20210624213345.3617-1-pali@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org From: Evan Wang There is an issue that when PCIe switch is connected to an Armada 3700 board, there will be lots of warnings about PIO errors when reading the config space. According to Aardvark PIO read and write sequence in HW specification, the current way to check PIO status has the following issues: 1) For PIO read operation, it reports the error message, which should be avoided according to HW specification. 2) For PIO read and write operations, it only checks PIO operation complete status, which is not enough, and error status should also be checked. This patch aligns the code with Aardvark PIO read and write sequence in HW specification on PIO status check and fix the warnings when reading config space. This patch also returns Completion Retry Status value when the read request timeout, to give the caller a chance to send the request again instead of failing. Signed-off-by: Evan Wang Reviewed-by: Victor Gu Tested-by: Victor Gu [pali: Return CRS also after timeout] Signed-off-by: Pali Rohár Reviewed-by: Marek Behún Cc: stable@vger.kernel.org # b1bd5714472c ("PCI: aardvark: Indicate error in 'val' when config read fails") --- drivers/pci/controller/pci-aardvark.c | 93 +++++++++++++++++++++++---- 1 file changed, 80 insertions(+), 13 deletions(-) diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c index 2f8380a1f84f..a37ba86f1b2d 100644 --- a/drivers/pci/controller/pci-aardvark.c +++ b/drivers/pci/controller/pci-aardvark.c @@ -58,6 +58,7 @@ #define PIO_COMPLETION_STATUS_CRS 2 #define PIO_COMPLETION_STATUS_CA 4 #define PIO_NON_POSTED_REQ BIT(10) +#define PIO_ERR_STATUS BIT(11) #define PIO_ADDR_LS (PIO_BASE_ADDR + 0x8) #define PIO_ADDR_MS (PIO_BASE_ADDR + 0xc) #define PIO_WR_DATA (PIO_BASE_ADDR + 0x10) @@ -176,6 +177,9 @@ #define MSI_IRQ_NUM 32 +#define CFG_RD_UR_VAL 0xffffffff +#define CFG_RD_CRS_VAL 0xffff0001 + struct advk_pcie { struct platform_device *pdev; void __iomem *base; @@ -461,7 +465,7 @@ static void advk_pcie_setup_hw(struct advk_pcie *pcie) advk_writel(pcie, reg, PCIE_CORE_CMD_STATUS_REG); } -static void advk_pcie_check_pio_status(struct advk_pcie *pcie) +static int advk_pcie_check_pio_status(struct advk_pcie *pcie, u32 *val) { struct device *dev = &pcie->pdev->dev; u32 reg; @@ -472,15 +476,50 @@ static void advk_pcie_check_pio_status(struct advk_pcie *pcie) status = (reg & PIO_COMPLETION_STATUS_MASK) >> PIO_COMPLETION_STATUS_SHIFT; - if (!status) - return; - + /* + * According to HW spec, the PIO status check sequence as below: + * 1) even if COMPLETION_STATUS(bit9:7) indicates successful, + * it still needs to check Error Status(bit11), only when this bit + * indicates no error happen, the operation is successful. + * 2) value Unsupported Request(1) of COMPLETION_STATUS(bit9:7) only + * means a PIO write error, and for PIO read it is successful with + * a read value of 0xFFFFFFFF. + * 3) value Completion Retry Status(CRS) of COMPLETION_STATUS(bit9:7) + * only means a PIO write error, and for PIO read it is successful + * with a read value of 0xFFFF0001. + * 4) value Completer Abort (CA) of COMPLETION_STATUS(bit9:7) means + * error for both PIO read and PIO write operation. + * 5) other errors are indicated as 'unknown'. + */ switch (status) { + case PIO_COMPLETION_STATUS_OK: + if (reg & PIO_ERR_STATUS) { + strcomp_status = "COMP_ERR"; + break; + } + /* Get the read result */ + if (val) + *val = advk_readl(pcie, PIO_RD_DATA); + /* No error */ + strcomp_status = NULL; + break; case PIO_COMPLETION_STATUS_UR: - strcomp_status = "UR"; + if (val) { + /* For reading, UR is not an error status */ + *val = CFG_RD_UR_VAL; + strcomp_status = NULL; + } else { + strcomp_status = "UR"; + } break; case PIO_COMPLETION_STATUS_CRS: - strcomp_status = "CRS"; + if (val) { + /* For reading, CRS is not an error status */ + *val = CFG_RD_CRS_VAL; + strcomp_status = NULL; + } else { + strcomp_status = "CRS"; + } break; case PIO_COMPLETION_STATUS_CA: strcomp_status = "CA"; @@ -490,6 +529,9 @@ static void advk_pcie_check_pio_status(struct advk_pcie *pcie) break; } + if (!strcomp_status) + return 0; + if (reg & PIO_NON_POSTED_REQ) str_posted = "Non-posted"; else @@ -497,6 +539,8 @@ static void advk_pcie_check_pio_status(struct advk_pcie *pcie) dev_err(dev, "%s PIO Response Status: %s, %#x @ %#x\n", str_posted, strcomp_status, reg, advk_readl(pcie, PIO_ADDR_LS)); + + return -EFAULT; } static int advk_pcie_wait_pio(struct advk_pcie *pcie) @@ -703,8 +747,17 @@ static int advk_pcie_rd_conf(struct pci_bus *bus, u32 devfn, size, val); if (advk_pcie_pio_is_running(pcie)) { - *val = 0xffffffff; - return PCIBIOS_SET_FAILED; + /* + * For PCI_VENDOR_ID register, return Completion Retry Status + * so caller tries to issue the request again insted of failing + */ + if (where == PCI_VENDOR_ID) { + *val = CFG_RD_CRS_VAL; + return PCIBIOS_SUCCESSFUL; + } else { + *val = 0xffffffff; + return PCIBIOS_SET_FAILED; + } } /* Program the control register */ @@ -729,15 +782,27 @@ static int advk_pcie_rd_conf(struct pci_bus *bus, u32 devfn, advk_writel(pcie, 1, PIO_START); ret = advk_pcie_wait_pio(pcie); + if (ret < 0) { + /* + * For PCI_VENDOR_ID register, return Completion Retry Status + * so caller tries to issue the request again instead of failing + */ + if (where == PCI_VENDOR_ID) { + *val = CFG_RD_CRS_VAL; + return PCIBIOS_SUCCESSFUL; + } else { + *val = 0xffffffff; + return PCIBIOS_SET_FAILED; + } + } + + /* Check PIO status and get the read result */ + ret = advk_pcie_check_pio_status(pcie, val); if (ret < 0) { *val = 0xffffffff; return PCIBIOS_SET_FAILED; } - advk_pcie_check_pio_status(pcie); - - /* Get the read result */ - *val = advk_readl(pcie, PIO_RD_DATA); if (size == 1) *val = (*val >> (8 * (where & 3))) & 0xff; else if (size == 2) @@ -801,7 +866,9 @@ static int advk_pcie_wr_conf(struct pci_bus *bus, u32 devfn, if (ret < 0) return PCIBIOS_SET_FAILED; - advk_pcie_check_pio_status(pcie); + ret = advk_pcie_check_pio_status(pcie, NULL); + if (ret < 0) + return PCIBIOS_SET_FAILED; return PCIBIOS_SUCCESSFUL; } From patchwork Thu Jun 24 21:33:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Pali_Roh=C3=A1r?= X-Patchwork-Id: 1496835 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=fmmttLlB; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4G9tgL36Ykz9sWc for ; Fri, 25 Jun 2021 07:34:06 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232776AbhFXVgX (ORCPT ); Thu, 24 Jun 2021 17:36:23 -0400 Received: from mail.kernel.org ([198.145.29.99]:51896 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232760AbhFXVgW (ORCPT ); Thu, 24 Jun 2021 17:36:22 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 9E239613C8; Thu, 24 Jun 2021 21:34:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1624570442; bh=IdVDiP6+1WGD69KpQ4ecO/tfD5bBRFQpZYGTE4VGzkg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fmmttLlBHOX/lXoEZQuJ0H2dwBDmAXPubMF8us5jmhTju1TPEsnnwfgb0M3VRmUIU XfpkBR4bA7bjYxAMGMFI4yvyauaWGLBU9m6pcByF7EU9y5zxYkm64uJQZPkloicoSV /dh99vPYceqmaZLiYiWJkwkeolU8eWZPxF9HuCQ+xnhgNTN2MUoe6T7Gq7uYXI9sXF MInstLluHiBQ0xQeeurgCLCwP3cq0lXI47ZnD8wj6Z1iu2xpVzHGjYy1Mqq6MGgfOo PaTZABsDo+le9spomSJU75yYTK/aTBubEjM8fXvs19yqhjPg8Z0CUxJAVvx5ZXSKYq 4Kmo9E31uus9g== Received: by pali.im (Postfix) id 6019552D; Thu, 24 Jun 2021 23:34:02 +0200 (CEST) From: =?utf-8?q?Pali_Roh=C3=A1r?= To: Lorenzo Pieralisi , Thomas Petazzoni , Bjorn Helgaas Cc: =?utf-8?q?Marek_Beh=C3=BAn?= , linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RESEND PATCH 3/3] PCI: aardvark: Increase polling delay to 1.5s while waiting for PIO response Date: Thu, 24 Jun 2021 23:33:45 +0200 Message-Id: <20210624213345.3617-4-pali@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210624213345.3617-1-pali@kernel.org> References: <20210624213345.3617-1-pali@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Measurements in different conditions showed that aardvark hardware PIO response can take up to 1.44s. Increase wait timeout from 1ms to 1.5s to ensure that we do not miss responses from hardware. After 1.44s hardware returns errors (e.g. Completer abort). The previous two patches fixed checking for PIO status, so now we can use it to also catch errors which are reported by hardware after 1.44s. After applying this patch, kernel can detect and print PIO errors to dmesg: [ 6.879999] advk-pcie d0070000.pcie: Non-posted PIO Response Status: CA, 0xe00 @ 0x100004 [ 6.896436] advk-pcie d0070000.pcie: Posted PIO Response Status: COMP_ERR, 0x804 @ 0x100004 [ 6.913049] advk-pcie d0070000.pcie: Posted PIO Response Status: COMP_ERR, 0x804 @ 0x100010 [ 6.929663] advk-pcie d0070000.pcie: Non-posted PIO Response Status: CA, 0xe00 @ 0x100010 [ 6.953558] advk-pcie d0070000.pcie: Posted PIO Response Status: COMP_ERR, 0x804 @ 0x100014 [ 6.970170] advk-pcie d0070000.pcie: Non-posted PIO Response Status: CA, 0xe00 @ 0x100014 [ 6.994328] advk-pcie d0070000.pcie: Posted PIO Response Status: COMP_ERR, 0x804 @ 0x100004 Without this patch kernel prints only a generic error to dmesg: [ 5.246847] advk-pcie d0070000.pcie: config read/write timed out Signed-off-by: Pali Rohár Reviewed-by: Marek Behún Cc: stable@vger.kernel.org # 7fbcb5da811b ("PCI: aardvark: Don't rely on jiffies while holding spinlock") --- drivers/pci/controller/pci-aardvark.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c index a37ba86f1b2d..3f3c72927afb 100644 --- a/drivers/pci/controller/pci-aardvark.c +++ b/drivers/pci/controller/pci-aardvark.c @@ -166,7 +166,7 @@ #define PCIE_CONFIG_WR_TYPE0 0xa #define PCIE_CONFIG_WR_TYPE1 0xb -#define PIO_RETRY_CNT 500 +#define PIO_RETRY_CNT 750000 /* 1.5 s */ #define PIO_RETRY_DELAY 2 /* 2 us*/ #define LINK_WAIT_MAX_RETRIES 10