From patchwork Mon May 18 18:24:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292714 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnVr33Y1z9t0P; Tue, 19 May 2020 04:25:08 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakRk-00052Q-Ev; Mon, 18 May 2020 18:25:04 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRi-00051d-HU for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:02 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRi-0008Kv-62 for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:02 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 1/12] s390/pci: Improve handling of unset UID Date: Mon, 18 May 2020 20:24:27 +0200 Message-Id: <20200518182438.392269-2-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Niklas Schnelle BugLink: https://bugs.launchpad.net/bugs/1874056 When UID checking is enabled a UID value of 0 is invalid and can not be set by the user. On z/VM it is however used to indicate an unset UID. Until now, this lead to the behavior that one PCI function could be attached with UID 0 after which z/VM would prohibit further attachment. Now if the user then turns off UID checking in z/VM the user could seemingly attach additional PCI functions that would however not show up in Linux as that would not be informed of the change in UID checking mode. This is unexpected and confusing and lead to bug reports against Linux. Instead now, if we encounter an unset UID value of 0 treat it as indicating that UID checking was turned off, switch to automatic domain allocation, and warn the user of the possible misconfiguration. Signed-off-by: Niklas Schnelle Reviewed-by: Peter Oberparleiter Signed-off-by: Vasily Gorbik (backported from commit 7a11c67a1ff9b0231eaaaa6a28294776d55b569a) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 3 +++ arch/s390/pci/pci.c | 8 ++++++++ arch/s390/pci/pci_clp.c | 2 +- 3 files changed, 12 insertions(+), 1 deletion(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 7850e8c8c79a..abce7aa6c8cb 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -190,6 +190,9 @@ int clp_enable_fh(struct zpci_dev *, u8); int clp_disable_fh(struct zpci_dev *); int clp_get_state(u32 fid, enum zpci_state *state); +/* UID */ +void update_uid_checking(bool new); + /* IOMMU Interface */ int zpci_init_iommu(struct zpci_dev *zdev); void zpci_destroy_iommu(struct zpci_dev *zdev); diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 0af46683dd66..c0f80e896d7b 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -662,6 +662,13 @@ static int zpci_alloc_domain(struct zpci_dev *zdev) if (zpci_unique_uid) { zdev->domain = (u16) zdev->uid; + if (zdev->domain == 0) { + pr_warn("UID checking is active but no UID is set for PCI function %08x, so automatic domain allocation is used instead\n", + zdev->fid); + update_uid_checking(false); + goto auto_allocate; + } + if (test_bit(zdev->domain, zpci_domain)) { spin_unlock(&zpci_domain_lock); pr_err("Adding PCI function %08x failed because domain %04x is already assigned\n", @@ -673,6 +680,7 @@ static int zpci_alloc_domain(struct zpci_dev *zdev) spin_unlock(&zpci_domain_lock); return 0; } +auto_allocate: /* * We can always auto allocate domains below ZPCI_NR_DEVICES. * There is either a free domain or we have reached the maximum in diff --git a/arch/s390/pci/pci_clp.c b/arch/s390/pci/pci_clp.c index 281e0dd4c614..cfc9ae9df85d 100644 --- a/arch/s390/pci/pci_clp.c +++ b/arch/s390/pci/pci_clp.c @@ -24,7 +24,7 @@ bool zpci_unique_uid; -static void update_uid_checking(bool new) +void update_uid_checking(bool new) { if (zpci_unique_uid != new) zpci_dbg(1, "uid checking:%d\n", new); From patchwork Mon May 18 18:24:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292719 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnVx60ktz9t79; Tue, 19 May 2020 04:25:10 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakRm-00053W-LC; Mon, 18 May 2020 18:25:06 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRk-00052E-BE for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:04 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRk-0008Kv-31 for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:04 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 2/12] s390/pci: embedding hotplug_slot in zdev Date: Mon, 18 May 2020 20:24:28 +0200 Message-Id: <20200518182438.392269-3-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 Embedding the hotplug_slot in zdev structure allows to greatly simplify the hotplug handling by eliminating the handling of the slot_list. Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit 035f212fa7f21035537cf6dea620fe5653191eb6) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 2 + drivers/pci/hotplug/s390_pci_hpc.c | 99 +++++++++--------------------- 2 files changed, 31 insertions(+), 70 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index abce7aa6c8cb..3e95403d210f 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -100,6 +101,7 @@ struct s390_domain; struct zpci_dev { struct pci_bus *bus; struct list_head entry; /* list of all zpci_devices, needed for hotplug, etc. */ + struct hotplug_slot hotplug_slot; enum zpci_state state; u32 fid; /* function ID, used by sclp */ diff --git a/drivers/pci/hotplug/s390_pci_hpc.c b/drivers/pci/hotplug/s390_pci_hpc.c index 30ee72268790..39295d88f670 100644 --- a/drivers/pci/hotplug/s390_pci_hpc.c +++ b/drivers/pci/hotplug/s390_pci_hpc.c @@ -19,7 +19,6 @@ #include #define SLOT_NAME_SIZE 10 -static LIST_HEAD(s390_hotplug_slot_list); static int zpci_fn_configured(enum zpci_state state) { @@ -27,97 +26,86 @@ static int zpci_fn_configured(enum zpci_state state) state == ZPCI_FN_STATE_ONLINE; } -/* - * struct slot - slot information for each *physical* slot - */ -struct slot { - struct list_head slot_list; - struct hotplug_slot hotplug_slot; - struct zpci_dev *zdev; -}; - -static inline struct slot *to_slot(struct hotplug_slot *hotplug_slot) +static inline int zdev_configure(struct zpci_dev *zdev) { - return container_of(hotplug_slot, struct slot, hotplug_slot); -} - -static inline int slot_configure(struct slot *slot) -{ - int ret = sclp_pci_configure(slot->zdev->fid); + int ret = sclp_pci_configure(zdev->fid); - zpci_dbg(3, "conf fid:%x, rc:%d\n", slot->zdev->fid, ret); + zpci_dbg(3, "conf fid:%x, rc:%d\n", zdev->fid, ret); if (!ret) - slot->zdev->state = ZPCI_FN_STATE_CONFIGURED; + zdev->state = ZPCI_FN_STATE_CONFIGURED; return ret; } -static inline int slot_deconfigure(struct slot *slot) +static inline int zdev_deconfigure(struct zpci_dev *zdev) { - int ret = sclp_pci_deconfigure(slot->zdev->fid); + int ret = sclp_pci_deconfigure(zdev->fid); - zpci_dbg(3, "deconf fid:%x, rc:%d\n", slot->zdev->fid, ret); + zpci_dbg(3, "deconf fid:%x, rc:%d\n", zdev->fid, ret); if (!ret) - slot->zdev->state = ZPCI_FN_STATE_STANDBY; + zdev->state = ZPCI_FN_STATE_STANDBY; return ret; } static int enable_slot(struct hotplug_slot *hotplug_slot) { - struct slot *slot = to_slot(hotplug_slot); + struct zpci_dev *zdev = container_of(hotplug_slot, struct zpci_dev, + hotplug_slot); int rc; - if (slot->zdev->state != ZPCI_FN_STATE_STANDBY) + if (zdev->state != ZPCI_FN_STATE_STANDBY) return -EIO; - rc = slot_configure(slot); + rc = zdev_configure(zdev); if (rc) return rc; - rc = zpci_enable_device(slot->zdev); + rc = zpci_enable_device(zdev); if (rc) goto out_deconfigure; - pci_scan_slot(slot->zdev->bus, ZPCI_DEVFN); + pci_scan_slot(zdev->bus, ZPCI_DEVFN); pci_lock_rescan_remove(); - pci_bus_add_devices(slot->zdev->bus); + pci_bus_add_devices(zdev->bus); pci_unlock_rescan_remove(); return rc; out_deconfigure: - slot_deconfigure(slot); + zdev_deconfigure(zdev); return rc; } static int disable_slot(struct hotplug_slot *hotplug_slot) { - struct slot *slot = to_slot(hotplug_slot); + struct zpci_dev *zdev = container_of(hotplug_slot, struct zpci_dev, + hotplug_slot); struct pci_dev *pdev; int rc; - if (!zpci_fn_configured(slot->zdev->state)) + if (!zpci_fn_configured(zdev->state)) return -EIO; - pdev = pci_get_slot(slot->zdev->bus, ZPCI_DEVFN); + pdev = pci_get_slot(zdev->bus, ZPCI_DEVFN); if (pdev) { pci_stop_and_remove_bus_device_locked(pdev); pci_dev_put(pdev); } - rc = zpci_disable_device(slot->zdev); + rc = zpci_disable_device(zdev); if (rc) return rc; - return slot_deconfigure(slot); + return zdev_deconfigure(zdev); } static int get_power_status(struct hotplug_slot *hotplug_slot, u8 *value) { - struct slot *slot = to_slot(hotplug_slot); + struct zpci_dev *zdev = container_of(hotplug_slot, struct zpci_dev, + hotplug_slot); - switch (slot->zdev->state) { + switch (zdev->state) { case ZPCI_FN_STATE_STANDBY: *value = 0; break; @@ -145,44 +133,15 @@ static const struct hotplug_slot_ops s390_hotplug_slot_ops = { int zpci_init_slot(struct zpci_dev *zdev) { char name[SLOT_NAME_SIZE]; - struct slot *slot; - int rc; - if (!zdev) - return 0; - - slot = kzalloc(sizeof(*slot), GFP_KERNEL); - if (!slot) - goto error; - - slot->zdev = zdev; - slot->hotplug_slot.ops = &s390_hotplug_slot_ops; + zdev->hotplug_slot.ops = &s390_hotplug_slot_ops; snprintf(name, SLOT_NAME_SIZE, "%08x", zdev->fid); - rc = pci_hp_register(&slot->hotplug_slot, zdev->bus, - ZPCI_DEVFN, name); - if (rc) - goto error_reg; - - list_add(&slot->slot_list, &s390_hotplug_slot_list); - return 0; - -error_reg: - kfree(slot); -error: - return -ENOMEM; + return pci_hp_register(&zdev->hotplug_slot, zdev->bus, + ZPCI_DEVFN, name); } void zpci_exit_slot(struct zpci_dev *zdev) { - struct slot *slot, *next; - - list_for_each_entry_safe(slot, next, &s390_hotplug_slot_list, - slot_list) { - if (slot->zdev != zdev) - continue; - list_del(&slot->slot_list); - pci_hp_deregister(&slot->hotplug_slot); - kfree(slot); - } + pci_hp_deregister(&zdev->hotplug_slot); } From patchwork Mon May 18 18:24:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292716 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnVx6TGpz9vd1; Tue, 19 May 2020 04:25:12 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakRn-00054R-Rw; Mon, 18 May 2020 18:25:07 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRm-00053B-59 for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:06 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRl-0008Kv-RQ for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:05 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 3/12] s390/pci: Expose new port attribute for PCIe functions Date: Mon, 18 May 2020 20:24:29 +0200 Message-Id: <20200518182438.392269-4-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Alexander Schmidt BugLink: https://bugs.launchpad.net/bugs/1874056 Add SysFS attribute that provides the port number for PCI functions representing a single port of a multi-port device. Signed-off-by: Alexander Schmidt Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit e6ab7490ffaed83d6581f512e66c7c8cc6f58c2d) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 1 + arch/s390/include/asm/pci_clp.h | 3 ++- arch/s390/pci/pci_clp.c | 1 + arch/s390/pci/pci_sysfs.c | 2 ++ 4 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 3e95403d210f..0fdec3f44e23 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -111,6 +111,7 @@ struct zpci_dev { u8 pfgid; /* function group ID */ u8 pft; /* pci function type */ u16 domain; + u8 port; struct mutex lock; u8 pfip[CLP_PFIP_NR_SEGMENTS]; /* pci function internal path */ diff --git a/arch/s390/include/asm/pci_clp.h b/arch/s390/include/asm/pci_clp.h index 50359172cc48..a7a157bf9784 100644 --- a/arch/s390/include/asm/pci_clp.h +++ b/arch/s390/include/asm/pci_clp.h @@ -102,7 +102,8 @@ struct clp_rsp_query_pci { u16 pchid; __le32 bar[PCI_BAR_COUNT]; u8 pfip[CLP_PFIP_NR_SEGMENTS]; /* pci function internal path */ - u32 : 16; + u16 : 12; + u16 port : 4; u8 fmb_len; u8 pft; /* pci function type */ u64 sdma; /* start dma as */ diff --git a/arch/s390/pci/pci_clp.c b/arch/s390/pci/pci_clp.c index cfc9ae9df85d..193e5b0d0296 100644 --- a/arch/s390/pci/pci_clp.c +++ b/arch/s390/pci/pci_clp.c @@ -155,6 +155,7 @@ static int clp_store_query_pci_fn(struct zpci_dev *zdev, zdev->pfgid = response->pfgid; zdev->pft = response->pft; zdev->vfn = response->vfn; + zdev->port = response->port; zdev->uid = response->uid; zdev->fmb_length = sizeof(u32) * response->fmb_len; diff --git a/arch/s390/pci/pci_sysfs.c b/arch/s390/pci/pci_sysfs.c index 215f17437a4f..8ea8d04ed86d 100644 --- a/arch/s390/pci/pci_sysfs.c +++ b/arch/s390/pci/pci_sysfs.c @@ -33,6 +33,7 @@ zpci_attr(pchid, "0x%04x\n", pchid); zpci_attr(pfgid, "0x%02x\n", pfgid); zpci_attr(vfn, "0x%04x\n", vfn); zpci_attr(pft, "0x%02x\n", pft); +zpci_attr(port, "%d\n", port); zpci_attr(uid, "0x%x\n", uid); zpci_attr(segment0, "0x%02x\n", pfip[0]); zpci_attr(segment1, "0x%02x\n", pfip[1]); @@ -142,6 +143,7 @@ static struct attribute *zpci_dev_attrs[] = { &dev_attr_pchid.attr, &dev_attr_pfgid.attr, &dev_attr_pft.attr, + &dev_attr_port.attr, &dev_attr_vfn.attr, &dev_attr_uid.attr, &dev_attr_recover.attr, From patchwork Mon May 18 18:24:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292718 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnVz2gSdz9t0P; Tue, 19 May 2020 04:25:14 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakRq-00056S-6W; Mon, 18 May 2020 18:25:10 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRn-00054J-Rj for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:07 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRn-0008Kv-I1 for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:07 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 4/12] s390/pci: adaptation of iommu to multifunction Date: Mon, 18 May 2020 20:24:30 +0200 Message-Id: <20200518182438.392269-5-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 In the future the bus sysdata may not directly point to the zpci_dev. In preparation of upcoming patches let us abstract the access to the zpci_dev from the device inside the pci device. Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit d08d6f5d75242ceb410efbdf650efecc40d68c2d) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 5 +++++ drivers/iommu/s390-iommu.c | 8 ++++---- 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 0fdec3f44e23..3ee949a83778 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -234,6 +234,11 @@ static inline struct zpci_dev *to_zpci(struct pci_dev *pdev) return pdev->sysdata; } +static inline struct zpci_dev *to_zpci_dev(struct device *dev) +{ + return to_zpci(to_pci_dev(dev)); +} + struct zpci_dev *get_zdev_by_fid(u32); /* DMA */ diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 3b0b18e23187..57ed2ea794a1 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -87,7 +87,7 @@ static int s390_iommu_attach_device(struct iommu_domain *domain, struct device *dev) { struct s390_domain *s390_domain = to_s390_domain(domain); - struct zpci_dev *zdev = to_pci_dev(dev)->sysdata; + struct zpci_dev *zdev = to_zpci_dev(dev); struct s390_domain_device *domain_device; unsigned long flags; int rc; @@ -139,7 +139,7 @@ static void s390_iommu_detach_device(struct iommu_domain *domain, struct device *dev) { struct s390_domain *s390_domain = to_s390_domain(domain); - struct zpci_dev *zdev = to_pci_dev(dev)->sysdata; + struct zpci_dev *zdev = to_zpci_dev(dev); struct s390_domain_device *domain_device, *tmp; unsigned long flags; int found = 0; @@ -169,7 +169,7 @@ static void s390_iommu_detach_device(struct iommu_domain *domain, static int s390_iommu_add_device(struct device *dev) { struct iommu_group *group = iommu_group_get_for_dev(dev); - struct zpci_dev *zdev = to_pci_dev(dev)->sysdata; + struct zpci_dev *zdev = to_zpci_dev(dev); if (IS_ERR(group)) return PTR_ERR(group); @@ -182,7 +182,7 @@ static int s390_iommu_add_device(struct device *dev) static void s390_iommu_remove_device(struct device *dev) { - struct zpci_dev *zdev = to_pci_dev(dev)->sysdata; + struct zpci_dev *zdev = to_zpci_dev(dev); struct iommu_domain *domain; /* From patchwork Mon May 18 18:24:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292717 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnVz2ghqzB2xQ; Tue, 19 May 2020 04:25:15 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakRr-00056z-0q; Mon, 18 May 2020 18:25:11 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRp-00055T-8p for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:09 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRo-0008Kv-V4 for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:09 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 5/12] s390/pci: define kernel parameters for PCI multifunction Date: Mon, 18 May 2020 20:24:31 +0200 Message-Id: <20200518182438.392269-6-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 Using PCI multifunctions in S390 is a new feature we may want to ignore to continue provide the same topology as in the past to userland even if the configuration supports exposing the topology of a multi-Function device. A new boolean parameters allows to overwrite the kernel pci configuration: - pci=norid when on, disallow the use a new firmware field, RID, which provides the PCI :. part of the PCI address. To be used in the following patches and satisfy the checkpatch.pl the variable is exposed in pci.h Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit 6cf17f9a67c124aa4739b79709008d942635b975) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 1 + arch/s390/pci/pci.c | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 3ee949a83778..5dfcb8db5828 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -171,6 +171,7 @@ static inline bool zdev_enabled(struct zpci_dev *zdev) extern const struct attribute_group *zpci_attr_groups[]; extern unsigned int s390_pci_force_floating __initdata; +extern unsigned int s390_pci_no_rid; /* ----------------------------------------------------------------------------- Prototypes diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index c0f80e896d7b..1675fef70690 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -47,6 +47,8 @@ static unsigned int zpci_num_domains_allocated; min(((unsigned long) ZPCI_NR_DEVICES * PCI_BAR_COUNT / 2), \ ZPCI_IOMAP_MAX_ENTRIES) +unsigned int s390_pci_no_rid; + static DEFINE_SPINLOCK(zpci_iomap_lock); static unsigned long *zpci_iomap_bitmap; struct zpci_iomap_entry *zpci_iomap_start; @@ -886,6 +888,10 @@ char * __init pcibios_setup(char *str) s390_pci_force_floating = 1; return NULL; } + if (!strcmp(str, "norid")) { + s390_pci_no_rid = 1; + return NULL; + } return str; } From patchwork Mon May 18 18:24:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292720 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnW555zYz9sVf; Tue, 19 May 2020 04:25:21 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakRw-0005Bc-Nj; Mon, 18 May 2020 18:25:16 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRr-00056j-27 for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:11 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRq-0008Kv-8x for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:10 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 6/12] s390/pci: define RID and RID available Date: Mon, 18 May 2020 20:24:32 +0200 Message-Id: <20200518182438.392269-7-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 Firmware provides the bus/devfn part of the PCI addresses of a zPCI function inside the new field RID of the CLP query PCI function with a bit to know if this field is available to use. Let's add these fields to the clp_rsp_query_pci structure, add corresponding fields to zdev and initialize them. Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit c9a1752b84f1a8f73187c116ff0514b2ab24d878) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 3 +++ arch/s390/include/asm/pci_clp.h | 9 +++++++-- arch/s390/pci/pci_clp.c | 3 +++ 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 5dfcb8db5828..671323481602 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -112,6 +112,9 @@ struct zpci_dev { u8 pft; /* pci function type */ u16 domain; u8 port; + u8 rid_available : 1; + u8 reserved : 7; + unsigned int devfn; /* DEVFN part of the RID*/ struct mutex lock; u8 pfip[CLP_PFIP_NR_SEGMENTS]; /* pci function internal path */ diff --git a/arch/s390/include/asm/pci_clp.h b/arch/s390/include/asm/pci_clp.h index a7a157bf9784..4391ce82230f 100644 --- a/arch/s390/include/asm/pci_clp.h +++ b/arch/s390/include/asm/pci_clp.h @@ -93,7 +93,9 @@ struct clp_req_query_pci { struct clp_rsp_query_pci { struct clp_rsp_hdr hdr; u16 vfn; /* virtual fn number */ - u16 : 6; + u16 : 3; + u16 rid_avail : 1; + u16 : 2; u16 mio_addr_avail : 1; u16 util_str_avail : 1; /* utility string available? */ u16 pfgid : 8; /* pci function group id */ @@ -108,7 +110,10 @@ struct clp_rsp_query_pci { u8 pft; /* pci function type */ u64 sdma; /* start dma as */ u64 edma; /* end dma as */ - u32 reserved[11]; +#define ZPCI_RID_MASK_DEVFN 0x00ff + u16 rid; /* BUS/DEVFN PCI address */ + u16 reserved0; + u32 reserved[10]; u32 uid; /* user defined id */ u8 util_str[CLP_UTIL_STR_LEN]; /* utility string */ u32 reserved2[16]; diff --git a/arch/s390/pci/pci_clp.c b/arch/s390/pci/pci_clp.c index 193e5b0d0296..f7dfcd9f8b9e 100644 --- a/arch/s390/pci/pci_clp.c +++ b/arch/s390/pci/pci_clp.c @@ -158,6 +158,9 @@ static int clp_store_query_pci_fn(struct zpci_dev *zdev, zdev->port = response->port; zdev->uid = response->uid; zdev->fmb_length = sizeof(u32) * response->fmb_len; + zdev->rid_available = response->rid_avail; + if (!s390_pci_no_rid && zdev->rid_available) + zdev->devfn = response->rid & ZPCI_RID_MASK_DEVFN; memcpy(zdev->pfip, response->pfip, sizeof(zdev->pfip)); if (response->util_str_avail) { From patchwork Mon May 18 18:24:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292721 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnW755C0z9sdn; Tue, 19 May 2020 04:25:23 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakRy-0005Cj-E2; Mon, 18 May 2020 18:25:18 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRs-000582-2R for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:12 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRr-0008Kv-Ct for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:11 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 7/12] s390/pci: create zPCI bus Date: Mon, 18 May 2020 20:24:33 +0200 Message-Id: <20200518182438.392269-8-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 The zPCI bus is in charge to handle common zPCI resources for zPCI devices. Creating the zPCI bus, the PCI bus, the zPCI devices and the PCI devices and hotplug slots done in a specific order: - PCI hotplug slot creation needs a PCI bus - PCI bus needs a PCI domain which is reported by the pci_domain_nr() when setting up the host bridge - PCI domain is set from the zPCI with devfn 0 this is necessary to have a reproducible enumeration Therefore we can not create devices or hotplug slots for any PCI device associated with a zPCI device before having discovered the function zero of the bus. The discovery and initialization of devices can be done at several points in the code: - On Events, serialized in a thread context - On initialization, in the kernel init thread context - When powering on the hotplug slot, in a user thread context The removal of devices and their parent bus may also be done on events or for devices when powering down the slot. To guarantee the existence of the bus and devices until they are no more needed we use kref in zPCI bus and introduce a reference count in the zPCI devices. In this patch the zPCI bus still only accept a device with a devfn 0. Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit 05bc1be6db4b2683bbf5b9394a75d0fb3acfcede) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 26 ++++- arch/s390/pci/Makefile | 3 +- arch/s390/pci/pci.c | 169 ++++++++++++----------------- arch/s390/pci/pci_bus.c | 147 +++++++++++++++++++++++++ arch/s390/pci/pci_bus.h | 30 +++++ arch/s390/pci/pci_event.c | 17 ++- arch/s390/pci/pci_sysfs.c | 2 +- drivers/pci/hotplug/s390_pci_hpc.c | 11 +- 8 files changed, 287 insertions(+), 118 deletions(-) create mode 100644 arch/s390/pci/pci_bus.c create mode 100644 arch/s390/pci/pci_bus.h diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 671323481602..3295514a466a 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -31,6 +31,12 @@ int pci_proc_domain(struct pci_bus *); #define ZPCI_NR_DEVICES CONFIG_PCI_NR_FUNCTIONS #define ZPCI_DOMAIN_BITMAP_SIZE (1 << 16) +#ifdef PCI +#if (ZPCI_NR_DEVICES > ZPCI_DOMAIN_BITMAP_SIZE) +# error ZPCI_NR_DEVICES can not be bigger than ZPCI_DOMAIN_BITMAP_SIZE +#endif +#endif /* PCI */ + /* PCI Function Controls */ #define ZPCI_FC_FN_ENABLED 0x80 #define ZPCI_FC_ERROR 0x40 @@ -97,10 +103,23 @@ struct zpci_bar_struct { struct s390_domain; +#define ZPCI_FUNCTIONS_PER_BUS 256 +struct zpci_bus { + struct kref kref; + struct pci_bus *bus; + struct zpci_dev *function[ZPCI_FUNCTIONS_PER_BUS]; + struct list_head resources; + struct list_head bus_next; + int pchid; + int domain_nr; + enum pci_bus_speed max_bus_speed; +}; + /* Private data per function */ struct zpci_dev { - struct pci_bus *bus; + struct zpci_bus *zbus; struct list_head entry; /* list of all zpci_devices, needed for hotplug, etc. */ + struct kref kref; struct hotplug_slot hotplug_slot; enum zpci_state state; @@ -110,7 +129,6 @@ struct zpci_dev { u16 pchid; /* physical channel ID */ u8 pfgid; /* function group ID */ u8 pft; /* pci function type */ - u16 domain; u8 port; u8 rid_available : 1; u8 reserved : 7; @@ -235,7 +253,9 @@ static inline void zpci_exit_slot(struct zpci_dev *zdev) {} /* Helpers */ static inline struct zpci_dev *to_zpci(struct pci_dev *pdev) { - return pdev->sysdata; + struct zpci_bus *zbus = pdev->sysdata; + + return zbus->function[ZPCI_DEVFN]; } static inline struct zpci_dev *to_zpci_dev(struct device *dev) diff --git a/arch/s390/pci/Makefile b/arch/s390/pci/Makefile index 748626a33028..b4e3c84772a1 100644 --- a/arch/s390/pci/Makefile +++ b/arch/s390/pci/Makefile @@ -4,4 +4,5 @@ # obj-$(CONFIG_PCI) += pci.o pci_irq.o pci_dma.o pci_clp.o pci_sysfs.o \ - pci_event.o pci_debug.o pci_insn.o pci_mmio.o + pci_event.o pci_debug.o pci_insn.o pci_mmio.o \ + pci_bus.o diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 1675fef70690..6405b1c16349 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -35,13 +35,14 @@ #include #include +#include "pci_bus.h" + /* list of all detected zpci devices */ static LIST_HEAD(zpci_list); static DEFINE_SPINLOCK(zpci_list_lock); static DECLARE_BITMAP(zpci_domain, ZPCI_DOMAIN_BITMAP_SIZE); static DEFINE_SPINLOCK(zpci_domain_lock); -static unsigned int zpci_num_domains_allocated; #define ZPCI_IOMAP_ENTRIES \ min(((unsigned long) ZPCI_NR_DEVICES * PCI_BAR_COUNT / 2), \ @@ -89,17 +90,12 @@ void zpci_remove_reserved_devices(void) spin_unlock(&zpci_list_lock); list_for_each_entry_safe(zdev, tmp, &remove, entry) - zpci_remove_device(zdev); -} - -static struct zpci_dev *get_zdev_by_bus(struct pci_bus *bus) -{ - return (bus && bus->sysdata) ? (struct zpci_dev *) bus->sysdata : NULL; + zpci_zdev_put(zdev); } int pci_domain_nr(struct pci_bus *bus) { - return ((struct zpci_dev *) bus->sysdata)->domain; + return ((struct zpci_bus *) bus->sysdata)->domain_nr; } EXPORT_SYMBOL_GPL(pci_domain_nr); @@ -507,15 +503,15 @@ static struct resource *__alloc_res(struct zpci_dev *zdev, unsigned long start, return r; } -static int zpci_setup_bus_resources(struct zpci_dev *zdev, - struct list_head *resources) +int zpci_setup_bus_resources(struct zpci_dev *zdev, + struct list_head *resources) { unsigned long addr, size, flags; struct resource *res; int i, entry; snprintf(zdev->res_name, sizeof(zdev->res_name), - "PCI Bus %04x:%02x", zdev->domain, ZPCI_BUS_NR); + "PCI Bus %04x:%02x", zdev->uid, ZPCI_BUS_NR); for (i = 0; i < PCI_BAR_COUNT; i++) { if (!zdev->bars[i].size) @@ -652,98 +648,53 @@ struct dev_pm_ops pcibios_pm_ops = { }; #endif /* CONFIG_HIBERNATE_CALLBACKS */ -static int zpci_alloc_domain(struct zpci_dev *zdev) +static int __zpci_register_domain(int domain) { spin_lock(&zpci_domain_lock); - if (zpci_num_domains_allocated > (ZPCI_NR_DEVICES - 1)) { + if (test_bit(domain, zpci_domain)) { spin_unlock(&zpci_domain_lock); - pr_err("Adding PCI function %08x failed because the configured limit of %d is reached\n", - zdev->fid, ZPCI_NR_DEVICES); - return -ENOSPC; + pr_err("Domain %04x is already assigned\n", domain); + return -EEXIST; } + set_bit(domain, zpci_domain); + spin_unlock(&zpci_domain_lock); + return domain; +} - if (zpci_unique_uid) { - zdev->domain = (u16) zdev->uid; - if (zdev->domain == 0) { - pr_warn("UID checking is active but no UID is set for PCI function %08x, so automatic domain allocation is used instead\n", - zdev->fid); - update_uid_checking(false); - goto auto_allocate; - } +static int __zpci_alloc_domain(void) +{ + int domain; - if (test_bit(zdev->domain, zpci_domain)) { - spin_unlock(&zpci_domain_lock); - pr_err("Adding PCI function %08x failed because domain %04x is already assigned\n", - zdev->fid, zdev->domain); - return -EEXIST; - } - set_bit(zdev->domain, zpci_domain); - zpci_num_domains_allocated++; - spin_unlock(&zpci_domain_lock); - return 0; - } -auto_allocate: + spin_lock(&zpci_domain_lock); /* * We can always auto allocate domains below ZPCI_NR_DEVICES. * There is either a free domain or we have reached the maximum in * which case we would have bailed earlier. */ - zdev->domain = find_first_zero_bit(zpci_domain, ZPCI_NR_DEVICES); - set_bit(zdev->domain, zpci_domain); - zpci_num_domains_allocated++; + domain = find_first_zero_bit(zpci_domain, ZPCI_NR_DEVICES); + set_bit(domain, zpci_domain); spin_unlock(&zpci_domain_lock); - return 0; + return domain; } -static void zpci_free_domain(struct zpci_dev *zdev) +int zpci_alloc_domain(int domain) { - spin_lock(&zpci_domain_lock); - clear_bit(zdev->domain, zpci_domain); - zpci_num_domains_allocated--; - spin_unlock(&zpci_domain_lock); + if (zpci_unique_uid) { + if (domain) + return __zpci_register_domain(domain); + pr_warn("UID checking was active but no UID is provided: switching to automatic domain allocation\n"); + update_uid_checking(false); + } + return __zpci_alloc_domain(); } -void pcibios_remove_bus(struct pci_bus *bus) +void zpci_free_domain(int domain) { - struct zpci_dev *zdev = get_zdev_by_bus(bus); - - zpci_exit_slot(zdev); - zpci_cleanup_bus_resources(zdev); - zpci_destroy_iommu(zdev); - zpci_free_domain(zdev); - - spin_lock(&zpci_list_lock); - list_del(&zdev->entry); - spin_unlock(&zpci_list_lock); - - zpci_dbg(3, "rem fid:%x\n", zdev->fid); - kfree(zdev); + spin_lock(&zpci_domain_lock); + clear_bit(domain, zpci_domain); + spin_unlock(&zpci_domain_lock); } -static int zpci_scan_bus(struct zpci_dev *zdev) -{ - LIST_HEAD(resources); - int ret; - - ret = zpci_setup_bus_resources(zdev, &resources); - if (ret) - goto error; - - zdev->bus = pci_scan_root_bus(NULL, ZPCI_BUS_NR, &pci_root_ops, - zdev, &resources); - if (!zdev->bus) { - ret = -EIO; - goto error; - } - zdev->bus->max_bus_speed = zdev->max_bus_speed; - pci_bus_add_devices(zdev->bus); - return 0; - -error: - zpci_cleanup_bus_resources(zdev); - pci_free_resource_list(&resources); - return ret; -} int zpci_enable_device(struct zpci_dev *zdev) { @@ -778,13 +729,15 @@ int zpci_create_device(struct zpci_dev *zdev) { int rc; - rc = zpci_alloc_domain(zdev); - if (rc) - goto out; + kref_init(&zdev->kref); + + spin_lock(&zpci_list_lock); + list_add_tail(&zdev->entry, &zpci_list); + spin_unlock(&zpci_list_lock); rc = zpci_init_iommu(zdev); if (rc) - goto out_free; + goto out; mutex_init(&zdev->lock); if (zdev->state == ZPCI_FN_STATE_CONFIGURED) { @@ -792,16 +745,12 @@ int zpci_create_device(struct zpci_dev *zdev) if (rc) goto out_destroy_iommu; } - rc = zpci_scan_bus(zdev); + + rc = zpci_bus_device_register(zdev, &pci_root_ops); if (rc) goto out_disable; - spin_lock(&zpci_list_lock); - list_add_tail(&zdev->entry, &zpci_list); - spin_unlock(&zpci_list_lock); - zpci_init_slot(zdev); - return 0; out_disable: @@ -809,19 +758,39 @@ int zpci_create_device(struct zpci_dev *zdev) zpci_disable_device(zdev); out_destroy_iommu: zpci_destroy_iommu(zdev); -out_free: - zpci_free_domain(zdev); out: + spin_lock(&zpci_list_lock); + list_del(&zdev->entry); + spin_unlock(&zpci_list_lock); return rc; } -void zpci_remove_device(struct zpci_dev *zdev) +void zpci_release_device(struct kref *kref) { - if (!zdev->bus) - return; + struct zpci_dev *zdev = container_of(kref, struct zpci_dev, kref); - pci_stop_root_bus(zdev->bus); - pci_remove_root_bus(zdev->bus); + switch (zdev->state) { + case ZPCI_FN_STATE_ONLINE: + case ZPCI_FN_STATE_CONFIGURED: + zpci_disable_device(zdev); + fallthrough; + case ZPCI_FN_STATE_STANDBY: + if (zdev->zbus) { + zpci_exit_slot(zdev); + zpci_cleanup_bus_resources(zdev); + zpci_bus_device_unregister(zdev); + zpci_destroy_iommu(zdev); + } + fallthrough; + default: + break; + } + + spin_lock(&zpci_list_lock); + list_del(&zdev->entry); + spin_unlock(&zpci_list_lock); + zpci_dbg(3, "rem fid:%x\n", zdev->fid); + kfree(zdev); } int zpci_report_error(struct pci_dev *pdev, diff --git a/arch/s390/pci/pci_bus.c b/arch/s390/pci/pci_bus.c new file mode 100644 index 000000000000..e1565b8537de --- /dev/null +++ b/arch/s390/pci/pci_bus.c @@ -0,0 +1,147 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright IBM Corp. 2020 + * + * Author(s): + * Pierre Morel + * + */ + +#define KMSG_COMPONENT "zpci" +#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "pci_bus.h" + +static LIST_HEAD(zbus_list); +static DEFINE_SPINLOCK(zbus_list_lock); +static int zpci_nb_devices; + +/* zpci_bus_scan + * @zbus: the zbus holding the zdevices + * @ops: the pci operations + * + * The domain number must be set before pci_scan_root_bus is called. + * This function can be called once the domain is known, hence + * when the function_0 is dicovered. + */ +static int zpci_bus_scan(struct zpci_bus *zbus, int domain, struct pci_ops *ops) +{ + struct pci_bus *bus; + int rc; + + rc = zpci_alloc_domain(domain); + if (rc < 0) + return rc; + zbus->domain_nr = rc; + + bus = pci_scan_root_bus(NULL, ZPCI_BUS_NR, ops, zbus, &zbus->resources); + if (!bus) { + zpci_free_domain(zbus->domain_nr); + return -EFAULT; + } + + zbus->bus = bus; + pci_bus_add_devices(bus); + return 0; +} + +static void zpci_bus_release(struct kref *kref) +{ + struct zpci_bus *zbus = container_of(kref, struct zpci_bus, kref); + + pci_lock_rescan_remove(); + pci_stop_root_bus(zbus->bus); + + zpci_free_domain(zbus->domain_nr); + pci_free_resource_list(&zbus->resources); + + pci_remove_root_bus(zbus->bus); + pci_unlock_rescan_remove(); + + spin_lock(&zbus_list_lock); + list_del(&zbus->bus_next); + spin_unlock(&zbus_list_lock); + kfree(zbus); +} + +static void zpci_bus_put(struct zpci_bus *zbus) +{ + kref_put(&zbus->kref, zpci_bus_release); +} + +static struct zpci_bus *zpci_bus_alloc(int pchid) +{ + struct zpci_bus *zbus; + + zbus = kzalloc(sizeof(*zbus), GFP_KERNEL); + if (!zbus) + return NULL; + + zbus->pchid = pchid; + INIT_LIST_HEAD(&zbus->bus_next); + spin_lock(&zbus_list_lock); + list_add_tail(&zbus->bus_next, &zbus_list); + spin_unlock(&zbus_list_lock); + + kref_init(&zbus->kref); + INIT_LIST_HEAD(&zbus->resources); + + return zbus; +} + +int zpci_bus_device_register(struct zpci_dev *zdev, struct pci_ops *ops) +{ + struct zpci_bus *zbus; + int rc; + + if (zpci_nb_devices == ZPCI_NR_DEVICES) { + pr_warn("Adding PCI function %08x failed because the configured limit of %d is reached\n", + zdev->fid, ZPCI_NR_DEVICES); + return -ENOSPC; + } + zpci_nb_devices++; + + if (zdev->devfn != ZPCI_DEVFN) + return -EINVAL; + + zbus = zpci_bus_alloc(zdev->pchid); + if (!zbus) + return -ENOMEM; + + zdev->zbus = zbus; + zbus->function[ZPCI_DEVFN] = zdev; + + zpci_setup_bus_resources(zdev, &zbus->resources); + zbus->max_bus_speed = zdev->max_bus_speed; + + rc = zpci_bus_scan(zbus, (u16)zdev->uid, ops); + if (!rc) + return 0; + + pr_err("Adding PCI function %08x failed\n", zdev->fid); + zdev->zbus = NULL; + zpci_bus_put(zbus); + return rc; +} + +void zpci_bus_device_unregister(struct zpci_dev *zdev) +{ + struct zpci_bus *zbus = zdev->zbus; + + zpci_nb_devices--; + zbus->function[ZPCI_DEVFN] = NULL; + zpci_bus_put(zbus); +} diff --git a/arch/s390/pci/pci_bus.h b/arch/s390/pci/pci_bus.h new file mode 100644 index 000000000000..c6aff42cc2cf --- /dev/null +++ b/arch/s390/pci/pci_bus.h @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright IBM Corp. 2020 + * + * Author(s): + * Pierre Morel + * + */ + +int zpci_bus_device_register(struct zpci_dev *zdev, struct pci_ops *ops); +void zpci_bus_device_unregister(struct zpci_dev *zdev); +int zpci_bus_init(void); + +void zpci_release_device(struct kref *kref); +static inline void zpci_zdev_put(struct zpci_dev *zdev) +{ + kref_put(&zdev->kref, zpci_release_device); +} + +int zpci_alloc_domain(int domain); +void zpci_free_domain(int domain); +int zpci_setup_bus_resources(struct zpci_dev *zdev, + struct list_head *resources); + +static inline struct zpci_dev *get_zdev_by_bus(struct pci_bus *bus) +{ + struct zpci_bus *zbus = bus->sysdata; + + return zbus->function[ZPCI_DEVFN]; +} diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index 8d6ee4af4230..d39e9299e133 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -14,6 +14,8 @@ #include #include +#include "pci_bus.h" + /* Content Code Description for PCI Function Error */ struct zpci_ccdf_err { u32 reserved1; @@ -53,7 +55,7 @@ static void __zpci_event_error(struct zpci_ccdf_err *ccdf) zpci_err_hex(ccdf, sizeof(*ccdf)); if (zdev) - pdev = pci_get_slot(zdev->bus, ZPCI_DEVFN); + pdev = pci_get_slot(zdev->zbus->bus, ZPCI_DEVFN); pr_err("%s: Event 0x%x reports an error for PCI function 0x%x\n", pdev ? pci_name(pdev) : "n/a", ccdf->pec, ccdf->fid); @@ -78,11 +80,9 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) enum zpci_state state; int ret; - if (zdev) - pdev = pci_get_slot(zdev->bus, ZPCI_DEVFN); + if (zdev && zdev->zbus && zdev->zbus->bus) + pdev = pci_get_slot(zdev->zbus->bus, ZPCI_DEVFN); - pr_info("%s: Event 0x%x reconfigured PCI function 0x%x\n", - pdev ? pci_name(pdev) : "n/a", ccdf->pec, ccdf->fid); zpci_err("avail CCDF:\n"); zpci_err_hex(ccdf, sizeof(*ccdf)); @@ -102,7 +102,7 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) if (ret) break; pci_lock_rescan_remove(); - pci_rescan_bus(zdev->bus); + pci_rescan_bus(zdev->zbus->bus); pci_unlock_rescan_remove(); break; case 0x0302: /* Reserved -> Standby */ @@ -140,7 +140,7 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) zdev->state = ZPCI_FN_STATE_STANDBY; if (!clp_get_state(ccdf->fid, &state) && state == ZPCI_FN_STATE_RESERVED) { - zpci_remove_device(zdev); + zpci_zdev_put(zdev); } break; case 0x0306: /* 0x308 or 0x302 for multiple devices */ @@ -149,12 +149,11 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) case 0x0308: /* Standby -> Reserved */ if (!zdev) break; - zpci_remove_device(zdev); + zpci_zdev_put(zdev); break; default: break; } - pci_dev_put(pdev); } void zpci_event_availability(void *data) diff --git a/arch/s390/pci/pci_sysfs.c b/arch/s390/pci/pci_sysfs.c index 8ea8d04ed86d..5c028bee91b9 100644 --- a/arch/s390/pci/pci_sysfs.c +++ b/arch/s390/pci/pci_sysfs.c @@ -89,7 +89,7 @@ static ssize_t recover_store(struct device *dev, struct device_attribute *attr, ret = zpci_enable_device(zdev); if (ret) goto out; - pci_rescan_bus(zdev->bus); + pci_rescan_bus(zdev->zbus->bus); } out: pci_unlock_rescan_remove(); diff --git a/drivers/pci/hotplug/s390_pci_hpc.c b/drivers/pci/hotplug/s390_pci_hpc.c index 39295d88f670..a9c9f05fe54b 100644 --- a/drivers/pci/hotplug/s390_pci_hpc.c +++ b/drivers/pci/hotplug/s390_pci_hpc.c @@ -52,6 +52,7 @@ static int enable_slot(struct hotplug_slot *hotplug_slot) { struct zpci_dev *zdev = container_of(hotplug_slot, struct zpci_dev, hotplug_slot); + struct zpci_bus *zbus = zdev->zbus; int rc; if (zdev->state != ZPCI_FN_STATE_STANDBY) @@ -65,9 +66,9 @@ static int enable_slot(struct hotplug_slot *hotplug_slot) if (rc) goto out_deconfigure; - pci_scan_slot(zdev->bus, ZPCI_DEVFN); + pci_scan_slot(zbus->bus, ZPCI_DEVFN); pci_lock_rescan_remove(); - pci_bus_add_devices(zdev->bus); + pci_bus_add_devices(zbus->bus); pci_unlock_rescan_remove(); return rc; @@ -82,12 +83,13 @@ static int disable_slot(struct hotplug_slot *hotplug_slot) struct zpci_dev *zdev = container_of(hotplug_slot, struct zpci_dev, hotplug_slot); struct pci_dev *pdev; + struct zpci_bus *zbus = zdev->zbus; int rc; if (!zpci_fn_configured(zdev->state)) return -EIO; - pdev = pci_get_slot(zdev->bus, ZPCI_DEVFN); + pdev = pci_get_slot(zbus->bus, ZPCI_DEVFN); if (pdev) { pci_stop_and_remove_bus_device_locked(pdev); pci_dev_put(pdev); @@ -133,11 +135,12 @@ static const struct hotplug_slot_ops s390_hotplug_slot_ops = { int zpci_init_slot(struct zpci_dev *zdev) { char name[SLOT_NAME_SIZE]; + struct zpci_bus *zbus = zdev->zbus; zdev->hotplug_slot.ops = &s390_hotplug_slot_ops; snprintf(name, SLOT_NAME_SIZE, "%08x", zdev->fid); - return pci_hp_register(&zdev->hotplug_slot, zdev->bus, + return pci_hp_register(&zdev->hotplug_slot, zbus->bus, ZPCI_DEVFN, name); } From patchwork Mon May 18 18:24:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292722 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnW94Ngqz9sVf; Tue, 19 May 2020 04:25:25 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakS0-0005Dx-6C; Mon, 18 May 2020 18:25:20 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRt-00058o-EG for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:13 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRs-0008Kv-IB for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:12 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 8/12] s390/pci: adapt events for zbus Date: Mon, 18 May 2020 20:24:34 +0200 Message-Id: <20200518182438.392269-9-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 Simplify the event handling. Set the zpci state explicitly. Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit f606b3ef47c9f874af605323099663a10f691b24) Signed-off-by: Frank Heimes --- arch/s390/pci/pci_event.c | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index d39e9299e133..c296214f0a19 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -89,25 +89,19 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) switch (ccdf->pec) { case 0x0301: /* Reserved|Standby -> Configured */ if (!zdev) { - ret = clp_add_pci_device(ccdf->fid, ccdf->fh, 0); - if (ret) - break; - zdev = get_zdev_by_fid(ccdf->fid); - } - if (!zdev || zdev->state != ZPCI_FN_STATE_STANDBY) + ret = clp_add_pci_device(ccdf->fid, ccdf->fh, 1); break; - zdev->state = ZPCI_FN_STATE_CONFIGURED; + } zdev->fh = ccdf->fh; - ret = zpci_enable_device(zdev); - if (ret) - break; - pci_lock_rescan_remove(); - pci_rescan_bus(zdev->zbus->bus); - pci_unlock_rescan_remove(); + zdev->state = ZPCI_FN_STATE_CONFIGURED; + zpci_create_device(zdev); break; case 0x0302: /* Reserved -> Standby */ - if (!zdev) + if (!zdev) { clp_add_pci_device(ccdf->fid, ccdf->fh, 0); + break; + } + zdev->fh = ccdf->fh; break; case 0x0303: /* Deconfiguration requested */ if (!zdev) @@ -135,8 +129,6 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) pci_stop_and_remove_bus_device_locked(pdev); } - zdev->fh = ccdf->fh; - zpci_disable_device(zdev); zdev->state = ZPCI_FN_STATE_STANDBY; if (!clp_get_state(ccdf->fid, &state) && state == ZPCI_FN_STATE_RESERVED) { From patchwork Mon May 18 18:24:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292723 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnWD2KcDz9sfR; Tue, 19 May 2020 04:25:28 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakS2-0005FL-Bb; Mon, 18 May 2020 18:25:22 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRu-00059h-0W for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:14 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRt-0008Kv-Lw for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:13 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 9/12] s390/pci: Handling multifunctions Date: Mon, 18 May 2020 20:24:35 +0200 Message-Id: <20200518182438.392269-10-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 We allow multiple functions on a single bus. We suppress the ZPCI_DEVFN definition and replace its occurences with zpci->devfn. We verify the number of device during the registration. There can never be more domains in use than existing devices, so we do not need to verify the count of domain after having verified the count of devices. Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit 44510d6fa0c00aa90b80075caa6b313b25927475) Signed-off-by: Frank Heimes --- arch/s390/include/asm/pci.h | 8 +- arch/s390/pci/pci.c | 39 ++++---- arch/s390/pci/pci_bus.c | 148 +++++++++++++++++++++++++---- arch/s390/pci/pci_bus.h | 5 +- arch/s390/pci/pci_event.c | 4 +- drivers/pci/hotplug/s390_pci_hpc.c | 6 +- 6 files changed, 159 insertions(+), 51 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 3295514a466a..4bbd2c79ec4a 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -25,7 +25,6 @@ int pci_domain_nr(struct pci_bus *); int pci_proc_domain(struct pci_bus *); #define ZPCI_BUS_NR 0 /* default bus number */ -#define ZPCI_DEVFN 0 /* default device number */ #define ZPCI_NR_DMA_SPACES 1 #define ZPCI_NR_DEVICES CONFIG_PCI_NR_FUNCTIONS @@ -112,6 +111,7 @@ struct zpci_bus { struct list_head bus_next; int pchid; int domain_nr; + bool multifunction; enum pci_bus_speed max_bus_speed; }; @@ -119,6 +119,7 @@ struct zpci_bus { struct zpci_dev { struct zpci_bus *zbus; struct list_head entry; /* list of all zpci_devices, needed for hotplug, etc. */ + struct list_head bus_next; struct kref kref; struct hotplug_slot hotplug_slot; @@ -131,7 +132,8 @@ struct zpci_dev { u8 pft; /* pci function type */ u8 port; u8 rid_available : 1; - u8 reserved : 7; + u8 has_hp_slot : 1; + u8 reserved : 6; unsigned int devfn; /* DEVFN part of the RID*/ struct mutex lock; @@ -255,7 +257,7 @@ static inline struct zpci_dev *to_zpci(struct pci_dev *pdev) { struct zpci_bus *zbus = pdev->sysdata; - return zbus->function[ZPCI_DEVFN]; + return zbus->function[pdev->devfn]; } static inline struct zpci_dev *to_zpci_dev(struct device *dev) diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 6405b1c16349..ec7240cf20a7 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -370,29 +370,17 @@ EXPORT_SYMBOL(pci_iounmap); static int pci_read(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 *val) { - struct zpci_dev *zdev = get_zdev_by_bus(bus); - int ret; + struct zpci_dev *zdev = get_zdev_by_bus(bus, devfn); - if (!zdev || devfn != ZPCI_DEVFN) - ret = -ENODEV; - else - ret = zpci_cfg_load(zdev, where, val, size); - - return ret; + return (zdev) ? zpci_cfg_load(zdev, where, val, size) : -ENODEV; } static int pci_write(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 val) { - struct zpci_dev *zdev = get_zdev_by_bus(bus); - int ret; + struct zpci_dev *zdev = get_zdev_by_bus(bus, devfn); - if (!zdev || devfn != ZPCI_DEVFN) - ret = -ENODEV; - else - ret = zpci_cfg_store(zdev, where, val, size); - - return ret; + return (zdev) ? zpci_cfg_store(zdev, where, val, size) : -ENODEV; } static struct pci_ops pci_root_ops = { @@ -750,12 +738,12 @@ int zpci_create_device(struct zpci_dev *zdev) if (rc) goto out_disable; - zpci_init_slot(zdev); return 0; out_disable: if (zdev->state == ZPCI_FN_STATE_ONLINE) zpci_disable_device(zdev); + out_destroy_iommu: zpci_destroy_iommu(zdev); out: @@ -769,18 +757,25 @@ void zpci_release_device(struct kref *kref) { struct zpci_dev *zdev = container_of(kref, struct zpci_dev, kref); + if (zdev->zbus->bus) { + struct pci_dev *pdev; + + pdev = pci_get_slot(zdev->zbus->bus, zdev->devfn); + if (pdev) + pci_stop_and_remove_bus_device_locked(pdev); + } + switch (zdev->state) { case ZPCI_FN_STATE_ONLINE: case ZPCI_FN_STATE_CONFIGURED: zpci_disable_device(zdev); fallthrough; case ZPCI_FN_STATE_STANDBY: - if (zdev->zbus) { + if (zdev->has_hp_slot) zpci_exit_slot(zdev); - zpci_cleanup_bus_resources(zdev); - zpci_bus_device_unregister(zdev); - zpci_destroy_iommu(zdev); - } + zpci_cleanup_bus_resources(zdev); + zpci_bus_device_unregister(zdev); + zpci_destroy_iommu(zdev); fallthrough; default: break; diff --git a/arch/s390/pci/pci_bus.c b/arch/s390/pci/pci_bus.c index e1565b8537de..8b76b88b0840 100644 --- a/arch/s390/pci/pci_bus.c +++ b/arch/s390/pci/pci_bus.c @@ -62,14 +62,16 @@ static void zpci_bus_release(struct kref *kref) { struct zpci_bus *zbus = container_of(kref, struct zpci_bus, kref); - pci_lock_rescan_remove(); - pci_stop_root_bus(zbus->bus); + if (zbus->bus) { + pci_lock_rescan_remove(); + pci_stop_root_bus(zbus->bus); - zpci_free_domain(zbus->domain_nr); - pci_free_resource_list(&zbus->resources); + zpci_free_domain(zbus->domain_nr); + pci_free_resource_list(&zbus->resources); - pci_remove_root_bus(zbus->bus); - pci_unlock_rescan_remove(); + pci_remove_root_bus(zbus->bus); + pci_unlock_rescan_remove(); + } spin_lock(&zbus_list_lock); list_del(&zbus->bus_next); @@ -82,6 +84,23 @@ static void zpci_bus_put(struct zpci_bus *zbus) kref_put(&zbus->kref, zpci_bus_release); } +static struct zpci_bus *zpci_bus_get(int pchid) +{ + struct zpci_bus *zbus; + + spin_lock(&zbus_list_lock); + list_for_each_entry(zbus, &zbus_list, bus_next) { + if (pchid == zbus->pchid) { + kref_get(&zbus->kref); + goto out_unlock; + } + } + zbus = NULL; +out_unlock: + spin_unlock(&zbus_list_lock); + return zbus; +} + static struct zpci_bus *zpci_bus_alloc(int pchid) { struct zpci_bus *zbus; @@ -102,11 +121,62 @@ static struct zpci_bus *zpci_bus_alloc(int pchid) return zbus; } -int zpci_bus_device_register(struct zpci_dev *zdev, struct pci_ops *ops) +static int zpci_bus_add_device(struct zpci_bus *zbus, struct zpci_dev *zdev) { - struct zpci_bus *zbus; + struct pci_bus *bus; + struct resource_entry *window, *n; + struct resource *res; + struct pci_dev *pdev; int rc; + bus = zbus->bus; + if (!bus) + return -EINVAL; + + pdev = pci_get_slot(bus, zdev->devfn); + if (pdev) { + /* Device is already known. */ + pci_dev_put(pdev); + return 0; + } + + rc = zpci_init_slot(zdev); + if (rc) + return rc; + zdev->has_hp_slot = 1; + + resource_list_for_each_entry_safe(window, n, &zbus->resources) { + res = window->res; + pci_bus_add_resource(bus, res, 0); + } + + pdev = pci_scan_single_device(bus, zdev->devfn); + if (pdev) { + pdev->multifunction = 1; + pci_bus_add_device(pdev); + } + + return 0; +} + +static void zpci_bus_add_devices(struct zpci_bus *zbus) +{ + int i; + + for (i = 1; i < ZPCI_FUNCTIONS_PER_BUS; i++) + if (zbus->function[i]) + zpci_bus_add_device(zbus, zbus->function[i]); + + pci_lock_rescan_remove(); + pci_bus_add_devices(zbus->bus); + pci_unlock_rescan_remove(); +} + +int zpci_bus_device_register(struct zpci_dev *zdev, struct pci_ops *ops) +{ + struct zpci_bus *zbus = NULL; + int rc = -EBADF; + if (zpci_nb_devices == ZPCI_NR_DEVICES) { pr_warn("Adding PCI function %08x failed because the configured limit of %d is reached\n", zdev->fid, ZPCI_NR_DEVICES); @@ -114,25 +184,65 @@ int zpci_bus_device_register(struct zpci_dev *zdev, struct pci_ops *ops) } zpci_nb_devices++; - if (zdev->devfn != ZPCI_DEVFN) + if (zdev->devfn >= ZPCI_FUNCTIONS_PER_BUS) return -EINVAL; - zbus = zpci_bus_alloc(zdev->pchid); - if (!zbus) - return -ENOMEM; + if (!s390_pci_no_rid && zdev->rid_available) + zbus = zpci_bus_get(zdev->pchid); + + if (!zbus) { + zbus = zpci_bus_alloc(zdev->pchid); + if (!zbus) + return -ENOMEM; + } zdev->zbus = zbus; - zbus->function[ZPCI_DEVFN] = zdev; + if (zbus->function[zdev->devfn]) { + pr_err("devfn %04x is already assigned\n", zdev->devfn); + goto error; /* rc already set */ + } + zbus->function[zdev->devfn] = zdev; zpci_setup_bus_resources(zdev, &zbus->resources); - zbus->max_bus_speed = zdev->max_bus_speed; - rc = zpci_bus_scan(zbus, (u16)zdev->uid, ops); - if (!rc) - return 0; + if (zbus->bus) { + if (!zbus->multifunction) { + WARN_ONCE(1, "zbus is not multifunction\n"); + goto error_bus; + } + if (!zdev->rid_available) { + WARN_ONCE(1, "rid_available not set for multifunction\n"); + goto error_bus; + } + rc = zpci_bus_add_device(zbus, zdev); + if (rc) + goto error_bus; + } else if (zdev->devfn == 0) { + if (zbus->multifunction && !zdev->rid_available) { + WARN_ONCE(1, "rid_available not set on function 0 for multifunction\n"); + goto error_bus; + } + rc = zpci_bus_scan(zbus, (u16)zdev->uid, ops); + if (rc) + goto error_bus; + zpci_bus_add_devices(zbus); + rc = zpci_init_slot(zdev); + if (rc) + goto error_bus; + zdev->has_hp_slot = 1; + zbus->multifunction = zdev->rid_available; + zbus->max_bus_speed = zdev->max_bus_speed; + } else { + zbus->multifunction = 1; + } + return 0; + +error_bus: + zpci_nb_devices--; + zbus->function[zdev->devfn] = NULL; +error: pr_err("Adding PCI function %08x failed\n", zdev->fid); - zdev->zbus = NULL; zpci_bus_put(zbus); return rc; } @@ -142,6 +252,6 @@ void zpci_bus_device_unregister(struct zpci_dev *zdev) struct zpci_bus *zbus = zdev->zbus; zpci_nb_devices--; - zbus->function[ZPCI_DEVFN] = NULL; + zbus->function[zdev->devfn] = NULL; zpci_bus_put(zbus); } diff --git a/arch/s390/pci/pci_bus.h b/arch/s390/pci/pci_bus.h index c6aff42cc2cf..89be3c354b7b 100644 --- a/arch/s390/pci/pci_bus.h +++ b/arch/s390/pci/pci_bus.h @@ -22,9 +22,10 @@ void zpci_free_domain(int domain); int zpci_setup_bus_resources(struct zpci_dev *zdev, struct list_head *resources); -static inline struct zpci_dev *get_zdev_by_bus(struct pci_bus *bus) +static inline struct zpci_dev *get_zdev_by_bus(struct pci_bus *bus, + unsigned int devfn) { struct zpci_bus *zbus = bus->sysdata; - return zbus->function[ZPCI_DEVFN]; + return (devfn >= ZPCI_FUNCTIONS_PER_BUS) ? NULL : zbus->function[devfn]; } diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index c296214f0a19..08e1d619398e 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -55,7 +55,7 @@ static void __zpci_event_error(struct zpci_ccdf_err *ccdf) zpci_err_hex(ccdf, sizeof(*ccdf)); if (zdev) - pdev = pci_get_slot(zdev->zbus->bus, ZPCI_DEVFN); + pdev = pci_get_slot(zdev->zbus->bus, zdev->devfn); pr_err("%s: Event 0x%x reports an error for PCI function 0x%x\n", pdev ? pci_name(pdev) : "n/a", ccdf->pec, ccdf->fid); @@ -81,7 +81,7 @@ static void __zpci_event_availability(struct zpci_ccdf_avail *ccdf) int ret; if (zdev && zdev->zbus && zdev->zbus->bus) - pdev = pci_get_slot(zdev->zbus->bus, ZPCI_DEVFN); + pdev = pci_get_slot(zdev->zbus->bus, zdev->devfn); zpci_err("avail CCDF:\n"); zpci_err_hex(ccdf, sizeof(*ccdf)); diff --git a/drivers/pci/hotplug/s390_pci_hpc.c b/drivers/pci/hotplug/s390_pci_hpc.c index a9c9f05fe54b..1579ba895edf 100644 --- a/drivers/pci/hotplug/s390_pci_hpc.c +++ b/drivers/pci/hotplug/s390_pci_hpc.c @@ -66,7 +66,7 @@ static int enable_slot(struct hotplug_slot *hotplug_slot) if (rc) goto out_deconfigure; - pci_scan_slot(zbus->bus, ZPCI_DEVFN); + pci_scan_slot(zbus->bus, zdev->devfn); pci_lock_rescan_remove(); pci_bus_add_devices(zbus->bus); pci_unlock_rescan_remove(); @@ -89,7 +89,7 @@ static int disable_slot(struct hotplug_slot *hotplug_slot) if (!zpci_fn_configured(zdev->state)) return -EIO; - pdev = pci_get_slot(zbus->bus, ZPCI_DEVFN); + pdev = pci_get_slot(zbus->bus, zdev->devfn); if (pdev) { pci_stop_and_remove_bus_device_locked(pdev); pci_dev_put(pdev); @@ -141,7 +141,7 @@ int zpci_init_slot(struct zpci_dev *zdev) snprintf(name, SLOT_NAME_SIZE, "%08x", zdev->fid); return pci_hp_register(&zdev->hotplug_slot, zbus->bus, - ZPCI_DEVFN, name); + zdev->devfn, name); } void zpci_exit_slot(struct zpci_dev *zdev) From patchwork Mon May 18 18:24:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292724 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnWG2GFvz9sdn; Tue, 19 May 2020 04:25:30 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakS4-0005H4-Jf; Mon, 18 May 2020 18:25:24 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRv-0005AY-8Z for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:15 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRu-0008Kv-NV for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:14 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 10/12] s390/pci: Do not disable PF when VFs exist Date: Mon, 18 May 2020 20:24:36 +0200 Message-Id: <20200518182438.392269-11-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 The Physical function should not be disabled until no virtual functions depends on it. Let's force the user to first use echo 0 > sriov_numfs before allowing to disable the PF with echo 0 > power. Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit 53dd462ac4dc3fc61ee90ad03d96202e17589156) Signed-off-by: Frank Heimes --- drivers/pci/hotplug/s390_pci_hpc.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/pci/hotplug/s390_pci_hpc.c b/drivers/pci/hotplug/s390_pci_hpc.c index 1579ba895edf..b59f84918fe0 100644 --- a/drivers/pci/hotplug/s390_pci_hpc.c +++ b/drivers/pci/hotplug/s390_pci_hpc.c @@ -91,6 +91,9 @@ static int disable_slot(struct hotplug_slot *hotplug_slot) pdev = pci_get_slot(zbus->bus, zdev->devfn); if (pdev) { + if (pci_num_vf(pdev)) + return -EBUSY; + pci_stop_and_remove_bus_device_locked(pdev); pci_dev_put(pdev); } From patchwork Mon May 18 18:24:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292726 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnWM1HVCz9sTS; Tue, 19 May 2020 04:25:35 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakS9-0005LX-JS; Mon, 18 May 2020 18:25:29 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRx-0005BI-5S for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:17 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRv-0008Kv-PA for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:15 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 11/12] s390/pci: Documentation for zPCI Date: Mon, 18 May 2020 20:24:37 +0200 Message-Id: <20200518182438.392269-12-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 There are changes in the usage of PCI for the user: - new kernel parameter - modification of the way functions are enumerated Let's document these. Signed-off-by: Pierre Morel Signed-off-by: Vasily Gorbik (backported from commit de267a7c71ba6be7857da0185871759067513d9c) Signed-off-by: Frank Heimes --- .../admin-guide/kernel-parameters.txt | 2 + Documentation/s390/index.rst | 1 + Documentation/s390/pci.rst | 126 ++++++++++++++++++ MAINTAINERS | 1 + 4 files changed, 130 insertions(+) create mode 100644 Documentation/s390/pci.rst diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index dc93d6b94d82..8b70fafbb18d 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3587,6 +3587,8 @@ may put more devices in an IOMMU group. force_floating [S390] Force usage of floating interrupts. nomio [S390] Do not use MIO instructions. + norid [S390] ignore the RID field and force use of + one PCI domain per PCI function pcie_aspm= [PCIE] Forcibly enable or disable PCIe Active State Power Management. diff --git a/Documentation/s390/index.rst b/Documentation/s390/index.rst index f7af2061e406..cf71df5776b4 100644 --- a/Documentation/s390/index.rst +++ b/Documentation/s390/index.rst @@ -15,6 +15,7 @@ s390 Architecture vfio-ccw zfcpdump common_io + pci text_files diff --git a/Documentation/s390/pci.rst b/Documentation/s390/pci.rst new file mode 100644 index 000000000000..75e043d4da85 --- /dev/null +++ b/Documentation/s390/pci.rst @@ -0,0 +1,126 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========= +S/390 PCI +========= + +Authors: + - Pierre Morel + +Copyright, IBM Corp. 2020 + + +command line parameters and debugfs entries +=========================================== + +Command line parameters +----------------------- + +* nomio + + Do not use MIO instructions. + +* norid + + Ignore the RID field and force use of one PCI domain per PCI function. + +debugfs entries +--------------- + +* /sys/kernel/debug/s390dbf/pci_*/ (S/390 debug feature) + + Some views generated by the debug feature to hold various debug outputs. + + - /sys/kernel/debug/s390dbf/pci_msg/sprintf + Messages from the processing of PCI events like machine check handling + and setting of global functionality like UID checking. + + The level of logging can be changed to be more or less verbose by piping to + /sys/kernel/debug/s390dbf/pci_*/level a number between 0 and 6; see the + documentation on the S/390 debug feature (Documentation/s390/s390dbf.rst) + for details. + +Sysfs entries +============= + +Specific entries, or entries specificities for zPCI functions. + +* /sys/bus/pci/slots/XXXXXXXX + + The slot entries are setup using the FID (Function Identifier) of the + PCI function. + + - /sys/bus/pci/slots/XXXXXXXX/power + + A physical function currently supporting virtual function can not be + powered-off until all virtual-function have been removed with + echo 0 > /sys/bus/pci/devices/XXXX:XX:XX.X/sriov_numvf + +* /sys/bus/pci/devices/XXXX:XX:XX.X/ + + - function_id + zPCI function identifier unique for the complete Z System. + It define uniquely a function in the system. + + - function_handle + Low level identifier used for a configured PCI function. + It may be useful for debuging. + + - pchid + Model dependent location of the I/O adapter. + + - pfgid + PCI Function Group ID, functions sharing identical functionality + are using a common identifier. + A PCI group defines interrupts, IOMMU, IOTLB and DMA specifics. + + - vfn + The Virtual Function Number, from 1 to N for virtual functions. + 0 for physical functions. + + - pft + PCI function type specifies the type of the PCI function. + + - port + The port correspond to the physical port the function is attached to. + It also gives an indication on the physical function a virtual function + is attached to. + + - uid + The UID, Unique Identifier is defined when configuring a LPAR and is + unique inside an LPAR. + + - pfip/segmentX + The segments are used to determine the isolation of a function. + They corresponds to the physical path to the function. + The more the segment are different the more the functions are isolated. + +Enumeration and hotplug +======================= + +The PCI address is made of 4 parts: domain, bus, device and function, +like in DDDD:BB:dd.f + +* When not using multi-functions (norid is set or firmware does not support + multi-functions) + + - There is only one function per domain. + + - the domain is set from the zPCI function's UID as defined during the + LPAR creation. + + - Addresses look like DDDD:00:00.0 + +* When using multi-functions (norid parameter is not set), there are some + change in the way zPCI functions are addressed: + + - There is still only one bus per domain. + + - There can be up to 256 functions per bus. + + - The domain part of the address of all functions of all functions for + a multi-Function device is set from the zPCI function's UID as defined + in the LPAR creation for the function zero. + + - New functions will only be ready to be used after the function zero + (the function with devfn 0) has been enumerated. diff --git a/MAINTAINERS b/MAINTAINERS index 6cb5a8e601fb..c7c1e182a3d1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -14177,6 +14177,7 @@ W: http://www.ibm.com/developerworks/linux/linux390/ S: Supported F: arch/s390/pci/ F: drivers/pci/hotplug/s390_pci_hpc.c +F: Documentation/s390/pci.rst S390 VFIO-CCW DRIVER M: Cornelia Huck From patchwork Mon May 18 18:24:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Frank Heimes X-Patchwork-Id: 1292727 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49QnWN5Gz0z9shm; Tue, 19 May 2020 04:25:36 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1jakSC-0005OT-Cx; Mon, 18 May 2020 18:25:32 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRx-0005Br-Rk for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:17 +0000 Received: from 2.general.fheimes.uk.vpn ([10.172.194.67] helo=T570.fritz.box) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1jakRw-0008Kv-Og for kernel-team@lists.ubuntu.com; Mon, 18 May 2020 18:25:16 +0000 From: frank.heimes@canonical.com To: kernel-team@lists.ubuntu.com Subject: [SRU][F][PATCH 12/12] s390/pci: removes wrong PCI multifunction assignment Date: Mon, 18 May 2020 20:24:38 +0200 Message-Id: <20200518182438.392269-13-frank.heimes@canonical.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200518182438.392269-1-frank.heimes@canonical.com> References: <20200518182438.392269-1-frank.heimes@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Pierre Morel BugLink: https://bugs.launchpad.net/bugs/1874056 The assignment of the PCI device multifunction attribute is set during the PCI device probe. There is no need to set it here. Let's do it right and remove this assignment. Signed-off-by: Pierre Morel Reviewed-by: Niklas Schnelle Signed-off-by: Vasily Gorbik (backported from commit d1379279f2d6b407bd08324a170cb21928e69854) Signed-off-by: Frank Heimes --- arch/s390/pci/pci_bus.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/s390/pci/pci_bus.c b/arch/s390/pci/pci_bus.c index 8b76b88b0840..fdc36f948ab2 100644 --- a/arch/s390/pci/pci_bus.c +++ b/arch/s390/pci/pci_bus.c @@ -151,10 +151,8 @@ static int zpci_bus_add_device(struct zpci_bus *zbus, struct zpci_dev *zdev) } pdev = pci_scan_single_device(bus, zdev->devfn); - if (pdev) { - pdev->multifunction = 1; + if (pdev) pci_bus_add_device(pdev); - } return 0; }