From patchwork Mon Oct 26 04:18:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Chan X-Patchwork-Id: 1387451 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=broadcom.com header.i=@broadcom.com header.a=rsa-sha256 header.s=google header.b=RrXZpceO; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4CKM5p4y30z9sTD for ; Mon, 26 Oct 2020 15:18:38 +1100 (AEDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1421001AbgJZESi (ORCPT ); Mon, 26 Oct 2020 00:18:38 -0400 Received: from mail-pj1-f43.google.com ([209.85.216.43]:33655 "EHLO mail-pj1-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1420823AbgJZESg (ORCPT ); Mon, 26 Oct 2020 00:18:36 -0400 Received: by mail-pj1-f43.google.com with SMTP id k8so1189565pjd.0 for ; Sun, 25 Oct 2020 21:18:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=AUW8DaQudLFeTlc624ikva6Dlpr0wexoXiqMWFoNacM=; b=RrXZpceOCQX7hJqRQM0MsO22THzjHm1Kj4HrOwfWxy8m3alXblO/ntUXSPU7jDhpeG XhVIN7D6GrdCqdnxhhbgnfeR5UojvKq14drjbNM2O3IatNLtt1yO6xBz/Kc2DxLlrxjN RMSXfvCwFn+yCGtFda1AygIYxpVclpcn7uyKU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=AUW8DaQudLFeTlc624ikva6Dlpr0wexoXiqMWFoNacM=; b=aoO36QoB4bFn3qtHiZiJjpnUEastjF/C3XDxhwAehn0+eyHeIEESQz3JXQXv9OYCdc PknWszcySxTj8Z6bbW6k1VZA57/SbefDsop0wp8e/KQsDsrbiHUdvUf527P/qbW3YGp2 aLdy6ASweYIR3Wd6fFZk20EBHuINOAxBiNjKWG/dKQh0175Gk42yKI/h+TL9bCsc6tJj 4/bxndpyINoK0KBUcoYxVHtLQzYvU8TDxgUtKqnikCk381fTC3l+CW1zc9AFYbBr/A08 3AtCJyYOUgssSq78faIcvTkhxLPqPThwNJMONZeTqebDxac3A5+atNw5lWdvYPHD6Kbz Dn1A== X-Gm-Message-State: AOAM532jTN/N0utAsLU8JBhKPKaUwB1ZbtVtOqjXZY1FZDWmWX0y1H/s 7P7EOwWRER91cGwnOLVX5n2/oA== X-Google-Smtp-Source: ABdhPJzjo48rS0+EgiscZ33qCnfhkepuKuu67jFaY8eN2beAbhFDh5Ds4gUR19sXpn8CO1nk+cLTaA== X-Received: by 2002:a17:90a:348e:: with SMTP id p14mr15276150pjb.75.1603685914768; Sun, 25 Oct 2020 21:18:34 -0700 (PDT) Received: from localhost.swdvt.lab.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id 10sm11505835pjt.50.2020.10.25.21.18.33 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Oct 2020 21:18:33 -0700 (PDT) From: Michael Chan To: kuba@kernel.org Cc: netdev@vger.kernel.org, gospo@broadcom.com, Vasundhara Volam Subject: [PATCH net 3/5] bnxt_en: Re-write PCI BARs after PCI fatal error. Date: Mon, 26 Oct 2020 00:18:19 -0400 Message-Id: <1603685901-17917-4-git-send-email-michael.chan@broadcom.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1603685901-17917-1-git-send-email-michael.chan@broadcom.com> References: <1603685901-17917-1-git-send-email-michael.chan@broadcom.com> Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Vasundhara Volam When a PCIe fatal error occurs, the internal latched BAR addresses in the chip get reset even though the BAR register values in config space are retained. pci_restore_state() will not rewrite the BAR addresses if the BAR address values are valid, causing the chip's internal BAR addresses to stay invalid. So we need to zero the BAR registers during PCIe fatal error to force pci_restore_state() to restore the BAR addresses. These write cycles to the BAR registers will cause the proper BAR addresses to latch internally. Fixes: 6316ea6db93d ("bnxt_en: Enable AER support.") Signed-off-by: Vasundhara Volam Signed-off-by: Michael Chan --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 19 ++++++++++++++++++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 1 + 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 7be232018015..8012386b4a0f 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -12852,6 +12852,9 @@ static pci_ers_result_t bnxt_io_error_detected(struct pci_dev *pdev, return PCI_ERS_RESULT_DISCONNECT; } + if (state == pci_channel_io_frozen) + set_bit(BNXT_STATE_PCI_CHANNEL_IO_FROZEN, &bp->state); + if (netif_running(netdev)) bnxt_close(netdev); @@ -12878,7 +12881,7 @@ static pci_ers_result_t bnxt_io_slot_reset(struct pci_dev *pdev) { struct net_device *netdev = pci_get_drvdata(pdev); struct bnxt *bp = netdev_priv(netdev); - int err = 0; + int err = 0, off; pci_ers_result_t result = PCI_ERS_RESULT_DISCONNECT; netdev_info(bp->dev, "PCI Slot Reset\n"); @@ -12890,6 +12893,20 @@ static pci_ers_result_t bnxt_io_slot_reset(struct pci_dev *pdev) "Cannot re-enable PCI device after reset.\n"); } else { pci_set_master(pdev); + /* Upon fatal error, our device internal logic that latches to + * BAR value is getting reset and will restore only upon + * rewritting the BARs. + * + * As pci_restore_state() does not re-write the BARs if the + * value is same as saved value earlier, driver needs to + * write the BARs to 0 to force restore, in case of fatal error. + */ + if (test_and_clear_bit(BNXT_STATE_PCI_CHANNEL_IO_FROZEN, + &bp->state)) { + for (off = PCI_BASE_ADDRESS_0; + off <= PCI_BASE_ADDRESS_5; off += 4) + pci_write_config_dword(bp->pdev, off, 0); + } pci_restore_state(pdev); pci_save_state(pdev); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 21ef1c21f602..47b3c3127879 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1781,6 +1781,7 @@ struct bnxt { #define BNXT_STATE_ABORT_ERR 5 #define BNXT_STATE_FW_FATAL_COND 6 #define BNXT_STATE_DRV_REGISTERED 7 +#define BNXT_STATE_PCI_CHANNEL_IO_FROZEN 8 #define BNXT_NO_FW_ACCESS(bp) \ (test_bit(BNXT_STATE_FW_FATAL_COND, &(bp)->state) || \