From patchwork Mon Sep 28 01:06:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jon Derrick X-Patchwork-Id: 1372243 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4C04c00Tfcz9sTC for ; Mon, 28 Sep 2020 11:26:24 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726421AbgI1B0I (ORCPT ); Sun, 27 Sep 2020 21:26:08 -0400 Received: from mga17.intel.com ([192.55.52.151]:64441 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726409AbgI1B0I (ORCPT ); Sun, 27 Sep 2020 21:26:08 -0400 IronPort-SDR: zpfYRArASu6XaUezuyRIHuBCWCxdFNcxNJU/96qiJXBvvP36OHAYAKvLMbH5scvk6JDetLbUSW PSVg3ZEc5PCg== X-IronPort-AV: E=McAfee;i="6000,8403,9757"; a="141939476" X-IronPort-AV: E=Sophos;i="5.77,312,1596524400"; d="scan'208";a="141939476" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Sep 2020 18:26:23 -0700 IronPort-SDR: 97q8wsth1ijdCrQtfVzMUbssknhglp8wwkIh5U76AoMCcssH/ZG9TXFg2shaAZZoDAUslv6pHh HbW5CFyEzlVg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,312,1596524400"; d="scan'208";a="456639876" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.116.36]) by orsmga004.jf.intel.com with ESMTP; 27 Sep 2020 18:26:22 -0700 From: Jon Derrick To: Cc: Lorenzo Pieralisi , Bjorn Helgaas , Andrzej Jakowski , Dave Fugate , Jon Derrick Subject: [PATCH 1/3] PCI: Create helper to release/restore bridge resources Date: Sun, 27 Sep 2020 21:06:07 -0400 Message-Id: <20200928010609.5375-2-jonathan.derrick@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20200928010609.5375-1-jonathan.derrick@intel.com> References: <20200928010609.5375-1-jonathan.derrick@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Moves bridge release and restore code into a common helper. No functional changes. Signed-off-by: Jon Derrick --- drivers/pci/setup-bus.c | 49 +++++++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 3951e02b7ded..f22502e8e6e6 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -2047,6 +2047,33 @@ static void pci_bridge_distribute_available_resources(struct pci_dev *bridge, available_mmio_pref); } +static void release_and_restore_resources(struct list_head *head) +{ + struct pci_dev_resource *dev_res; + + list_for_each_entry(dev_res, head, list) + pci_bus_release_bridge_resources(dev_res->dev->bus, + dev_res->flags & PCI_RES_TYPE_MASK, + whole_subtree); + + /* Restore size and flags */ + list_for_each_entry(dev_res, head, list) { + struct resource *res = dev_res->res; + int idx; + + res->start = dev_res->start; + res->end = dev_res->end; + res->flags = dev_res->flags; + + if (pci_is_bridge(dev_res->dev)) { + idx = res - &dev_res->dev->resource[0]; + if (idx >= PCI_BRIDGE_RESOURCES && + idx <= PCI_BRIDGE_RESOURCE_END) + res->flags = 0; + } + } +} + void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge) { struct pci_bus *parent = bridge->subordinate; @@ -2088,27 +2115,7 @@ void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge) * Try to release leaf bridge's resources that aren't big enough * to contain child device resources. */ - list_for_each_entry(fail_res, &fail_head, list) - pci_bus_release_bridge_resources(fail_res->dev->bus, - fail_res->flags & PCI_RES_TYPE_MASK, - whole_subtree); - - /* Restore size and flags */ - list_for_each_entry(fail_res, &fail_head, list) { - struct resource *res = fail_res->res; - int idx; - - res->start = fail_res->start; - res->end = fail_res->end; - res->flags = fail_res->flags; - - if (pci_is_bridge(fail_res->dev)) { - idx = res - &fail_res->dev->resource[0]; - if (idx >= PCI_BRIDGE_RESOURCES && - idx <= PCI_BRIDGE_RESOURCE_END) - res->flags = 0; - } - } + release_and_restore_resources(&fail_head); free_list(&fail_head); goto again; From patchwork Mon Sep 28 01:06:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jon Derrick X-Patchwork-Id: 1372244 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4C04c0529Mz9sTQ for ; Mon, 28 Sep 2020 11:26:24 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726440AbgI1B0J (ORCPT ); Sun, 27 Sep 2020 21:26:09 -0400 Received: from mga17.intel.com ([192.55.52.151]:64441 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726409AbgI1B0I (ORCPT ); Sun, 27 Sep 2020 21:26:08 -0400 IronPort-SDR: NscM44WksQD1InfeZeYwO+qDLZiP7iaY6iT/VSDSgfJzLWq2Ee826XCWxCElBJCfiYVez3TnlQ h8MXVE+ecnVA== X-IronPort-AV: E=McAfee;i="6000,8403,9757"; a="141939478" X-IronPort-AV: E=Sophos;i="5.77,312,1596524400"; d="scan'208";a="141939478" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Sep 2020 18:26:23 -0700 IronPort-SDR: /otH0sgCXg2q4brE7zz1N9lx9pMbMW8X/gu/8gsZ3M3or6UBnz0ha5lqNowAcXr8BiB3i8i8Ry K9hOGDqqfhDw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,312,1596524400"; d="scan'208";a="456639887" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.116.36]) by orsmga004.jf.intel.com with ESMTP; 27 Sep 2020 18:26:23 -0700 From: Jon Derrick To: Cc: Lorenzo Pieralisi , Bjorn Helgaas , Andrzej Jakowski , Dave Fugate , Jon Derrick Subject: [PATCH 2/3] PCI: Introduce a minimizing assignment algorithm Date: Sun, 27 Sep 2020 21:06:08 -0400 Message-Id: <20200928010609.5375-3-jonathan.derrick@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20200928010609.5375-1-jonathan.derrick@intel.com> References: <20200928010609.5375-1-jonathan.derrick@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Some PCI domains have limited resources that get exhausted by hotplug resource domains. VMD subdevice domains, for example, tend to support only 32MB MMIO, of which the decoable address space is split between prefetchable and non-prefetchable windows using existing resource assignment algorithms. In addition to these limitations, hotplug bridges require additional resource reservations as specified by default or module parameters "pci=hp{io,mmio,mmiopref}size, further exhausting the domain resources prior to full domain assignment. Introduce a minimizing assignment algorithm which starts with the default or user-requested hotplug resource values, tries with minimal hotplug resource values, and lastly tries no hotplug resource values. Signed-off-by: Jon Derrick --- drivers/pci/setup-bus.c | 98 ++++++++++++++++++++++++++++++++++++++--- include/linux/pci.h | 2 + 2 files changed, 95 insertions(+), 5 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index f22502e8e6e6..7beb4f37660b 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1200,6 +1200,35 @@ static void pci_bus_size_cardbus(struct pci_bus *bus, ; } +enum { + PCI_SIZING_VARIANT_DEFAULT, + PCI_SIZING_VARIANT_NOHOTPLUG, + PCI_SIZING_VARIANT_MINIMUM, + PCI_NUM_SIZING_VARIANTS, +}; + +static void hotplug_sizes(int sizing_variant, resource_size_t *io, + resource_size_t *mmio, resource_size_t *pref) +{ + switch (sizing_variant) { + case PCI_SIZING_VARIANT_MINIMUM: + *io = 0; + *mmio = 0; + *pref = 0; + break; + case PCI_SIZING_VARIANT_NOHOTPLUG: + *io = 256; + *mmio = 1 << 20; + *pref = 1 << 20; + break; + case PCI_SIZING_VARIANT_DEFAULT: + default: + *io = pci_hotplug_io_size; + *mmio = pci_hotplug_mmio_size; + *pref = pci_hotplug_mmio_pref_size; + } +} + void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) { struct pci_dev *dev; @@ -1248,11 +1277,11 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) case PCI_HEADER_TYPE_BRIDGE: pci_bridge_check_ranges(bus); - if (bus->self->is_hotplug_bridge) { - additional_io_size = pci_hotplug_io_size; - additional_mmio_size = pci_hotplug_mmio_size; - additional_mmio_pref_size = pci_hotplug_mmio_pref_size; - } + if (bus->self->is_hotplug_bridge) + hotplug_sizes(bus->self->sizing_variant, + &additional_io_size, + &additional_mmio_size, + &additional_mmio_pref_size); /* Fall through */ default: pbus_size_io(bus, realloc_head ? 0 : additional_io_size, @@ -2247,3 +2276,62 @@ void pci_assign_unassigned_bus_resources(struct pci_bus *bus) BUG_ON(!list_empty(&add_list)); } EXPORT_SYMBOL_GPL(pci_assign_unassigned_bus_resources); + +static int __set_sizing_variant(struct pci_dev *dev, void *data) +{ + if (dev->is_hotplug_bridge) + dev->sizing_variant = *((int *) data); + + return 0; +} + +static void release_bridge_resources(struct pci_bus *bus) +{ + struct resource *res; + struct pci_dev *dev; + int i; + + list_for_each_entry(dev, &bus->devices, bus_list) { + if (dev->subordinate) { + for (i = PCI_BRIDGE_RESOURCES; i < PCI_BRIDGE_RESOURCE_END; i++) + reset_resource(&dev->resource[i]); + + release_bridge_resources(dev->subordinate); + } + + if (pci_is_root_bus(bus)) + continue; + + pci_bus_for_each_resource(bus, res, i) + reset_resource(res); + } +} + +void pci_bus_assign_resources_fallback_sizing(struct pci_bus *bus) +{ + LIST_HEAD(fail_head); + int i = 0; + + pci_walk_bus(bus, __set_sizing_variant, &i); + __pci_bus_assign_resources(bus, NULL, &fail_head); + + if (list_empty(&fail_head)) + return; + + for (i = 0; i < PCI_NUM_SIZING_VARIANTS; i++) { + pci_walk_bus(bus, __set_sizing_variant, &i); + + down_read(&pci_bus_sem); + __pci_bus_size_bridges(bus, NULL); + up_read(&pci_bus_sem); + + __pci_bus_assign_resources(bus, NULL, &fail_head); + if (list_empty(&fail_head)) + return; + + release_and_restore_resources(&fail_head); + release_bridge_resources(bus); + free_list(&fail_head); + } +} +EXPORT_SYMBOL(pci_bus_assign_resources_fallback_sizing); diff --git a/include/linux/pci.h b/include/linux/pci.h index 801e9ad0d57e..72ae11d3b5ea 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -424,6 +424,7 @@ struct pci_dev { unsigned int is_hotplug_bridge:1; unsigned int shpc_managed:1; /* SHPC owned by shpchp */ unsigned int is_thunderbolt:1; /* Thunderbolt controller */ + unsigned int sizing_variant:2; /* normal, minimum, no hotplug */ /* * Devices marked being untrusted are the ones that can potentially * execute DMA attacks and similar. They are typically connected @@ -1299,6 +1300,7 @@ void pci_assign_unassigned_resources(void); void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge); void pci_assign_unassigned_bus_resources(struct pci_bus *bus); void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus); +void pci_bus_assign_resources_fallback_sizing(struct pci_bus *bus); int pci_reassign_bridge_resources(struct pci_dev *bridge, unsigned long type); void pdev_enable_device(struct pci_dev *); int pci_enable_resources(struct pci_dev *, int mask); From patchwork Mon Sep 28 01:06:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jon Derrick X-Patchwork-Id: 1372245 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=intel.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4C04c12Zndz9sSG for ; Mon, 28 Sep 2020 11:26:25 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726442AbgI1B0J (ORCPT ); Sun, 27 Sep 2020 21:26:09 -0400 Received: from mga17.intel.com ([192.55.52.151]:64441 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726409AbgI1B0J (ORCPT ); Sun, 27 Sep 2020 21:26:09 -0400 IronPort-SDR: iGAfyU/ZKBTVYpQ74sC9WowQUygYtaignH+w/QD/KtVozshbUFk895uEvVBj96ceMZCy05XBNZ MB7hkkSvd59g== X-IronPort-AV: E=McAfee;i="6000,8403,9757"; a="141939480" X-IronPort-AV: E=Sophos;i="5.77,312,1596524400"; d="scan'208";a="141939480" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Sep 2020 18:26:24 -0700 IronPort-SDR: 42i1DP5Sb2AtqkUyWUrxGWSArXnTG/7ce/dVVHGewZ1hRjxp40YgS5rfGHO6pKGF3vapqTMmPM CKm3d+Leflnw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,312,1596524400"; d="scan'208";a="456639894" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.116.36]) by orsmga004.jf.intel.com with ESMTP; 27 Sep 2020 18:26:23 -0700 From: Jon Derrick To: Cc: Lorenzo Pieralisi , Bjorn Helgaas , Andrzej Jakowski , Dave Fugate , Jon Derrick Subject: [PATCH 3/3] PCI: vmd: Wire up VMD for fallback resource assignment Date: Sun, 27 Sep 2020 21:06:09 -0400 Message-Id: <20200928010609.5375-4-jonathan.derrick@intel.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20200928010609.5375-1-jonathan.derrick@intel.com> References: <20200928010609.5375-1-jonathan.derrick@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The VMD subdevice domain would prefer all devices be assigned resources and working rather than a few or none assigned, but with valid hotplug bridge resources. The resource assignment fallback algorithm works best for these requirements. Signed-off-by: Jon Derrick --- drivers/pci/controller/vmd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c index fdc1a206f73e..4debc547c813 100644 --- a/drivers/pci/controller/vmd.c +++ b/drivers/pci/controller/vmd.c @@ -777,7 +777,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features) dev_set_msi_domain(&vmd->bus->dev, vmd->irq_domain); pci_scan_child_bus(vmd->bus); - pci_assign_unassigned_bus_resources(vmd->bus); + pci_bus_assign_resources_fallback_sizing(vmd->bus); /* * VMD root buses are virtual and don't return true on pci_is_pcie()