From patchwork Thu May 5 08:38:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 1626788 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=EMmk08qo; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4Kv6YW35Qgz9sCq for ; Thu, 5 May 2022 18:38:55 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349794AbiEEImc (ORCPT ); Thu, 5 May 2022 04:42:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349784AbiEEImb (ORCPT ); Thu, 5 May 2022 04:42:31 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3A3249C8B; Thu, 5 May 2022 01:38:52 -0700 (PDT) Received: from pps.filterd (m0098394.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2457xWJq019540; Thu, 5 May 2022 08:38:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=eQMrxDwS4o5AY3NeYTPbXuHkF5e1oKNwRgLoIiLPsKE=; b=EMmk08qo0me1lfMvCA+nFjtST0Cj78zlgM4iM1eUuxiFzIa2gvHSu6DXsOPJODaBhLsn r7XLHzw4a1R6UQQa53ddiiA3FEHhFwvoPlaiuaoqtly8gv23kK2PrrcnnjdsY3ZlkSpL tnsu6JsGJqWbbTm0PXq0NtMcGp69HfooazWLFIFraF3je5ksXRdXV8fDwxL/L2BnYp+p ey1z3nWI5QR8xe/qy2ggAZjIXLpCD9Vg9bKvAuJVFA495hYqgjmO+L0vBHrnDmJOGlPn WpQVm2Khl92pbML3mTYNx2d8obysj/qEcSrYoOF0k8Y+DKdeJdy03ehIiSEvnuA5GzsQ +Q== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3fv7w5betn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:46 +0000 Received: from m0098394.ppops.net (m0098394.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2458KvMw017230; Thu, 5 May 2022 08:38:45 GMT Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3fv7w5bet0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:45 +0000 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2458YW0F028380; Thu, 5 May 2022 08:38:43 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma06ams.nl.ibm.com with ESMTP id 3frvcj6w5b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:43 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2458cels50790688 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 May 2022 08:38:40 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 00C6E11C04A; Thu, 5 May 2022 08:38:40 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B0B8411C050; Thu, 5 May 2022 08:38:39 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 5 May 2022 08:38:39 +0000 (GMT) From: Niklas Schnelle To: Bjorn Helgaas , Jan Kiszka , Matthew Rosato , Pierre Morel Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, linux-pci@vger.kernel.org Subject: [PATCH RESEND v5 1/4] PCI: Clean up pci_scan_slot() Date: Thu, 5 May 2022 10:38:36 +0200 Message-Id: <20220505083839.1315705-2-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220505083839.1315705-1-schnelle@linux.ibm.com> References: <20220505083839.1315705-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: H3T2bmtFXG4-hx5o3DsuI46mqvzrRfVL X-Proofpoint-ORIG-GUID: MhCCQx5Kko8KoCD-vcw7CGz62AaUiKAO X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-05_02,2022-05-05_01,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 impostorscore=0 spamscore=0 clxscore=1015 malwarescore=0 adultscore=0 priorityscore=1501 suspectscore=0 bulkscore=0 mlxlogscore=812 phishscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2205050059 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org While determining the next PCI function is factored out of pci_scan_slot() into next_fn() the former still handles the first function as a special case. This duplicates the code from the scan loop. Furthermore the non ARI branch of next_fn() is generally hard to understand and especially the check for multifunction devices is hidden in the handling of NULL devices for non-contiguous multifunction. It also signals that no further functions need to be scanned by returning 0 via wraparound and this is a valid function number. Improve upon this by transforming the conditions in next_fn() to be easier to understand. By changing next_fn() to return -ENODEV instead of 0 when there is no next function we can then handle the initial function inside the loop and deduplicate the shared handling. This also makes it more explicit that only function 0 must exist. No functional change is intended. Cc: Jan Kiszka Signed-off-by: Niklas Schnelle --- drivers/pci/probe.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 17a969942d37..78aa1bccab2f 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2579,8 +2579,7 @@ struct pci_dev *pci_scan_single_device(struct pci_bus *bus, int devfn) } EXPORT_SYMBOL(pci_scan_single_device); -static unsigned int next_fn(struct pci_bus *bus, struct pci_dev *dev, - unsigned int fn) +static int next_fn(struct pci_bus *bus, struct pci_dev *dev, int fn) { int pos; u16 cap = 0; @@ -2588,24 +2587,26 @@ static unsigned int next_fn(struct pci_bus *bus, struct pci_dev *dev, if (pci_ari_enabled(bus)) { if (!dev) - return 0; + return -ENODEV; pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ARI); if (!pos) - return 0; + return -ENODEV; pci_read_config_word(dev, pos + PCI_ARI_CAP, &cap); next_fn = PCI_ARI_CAP_NFN(cap); if (next_fn <= fn) - return 0; /* protect against malformed list */ + return -ENODEV; /* protect against malformed list */ return next_fn; } + if (fn >= 7) + return -ENODEV; - /* dev may be NULL for non-contiguous multifunction devices */ - if (!dev || dev->multifunction) - return (fn + 1) % 8; + /* only multifunction devices may have more functions */ + if (dev && !dev->multifunction) + return -ENODEV; - return 0; + return fn + 1; } static int only_one_child(struct pci_bus *bus) @@ -2643,26 +2644,25 @@ static int only_one_child(struct pci_bus *bus) */ int pci_scan_slot(struct pci_bus *bus, int devfn) { - unsigned int fn, nr = 0; struct pci_dev *dev; + int fn = 0, nr = 0; if (only_one_child(bus) && (devfn > 0)) return 0; /* Already scanned the entire slot */ - dev = pci_scan_single_device(bus, devfn); - if (!dev) - return 0; - if (!pci_dev_is_added(dev)) - nr++; - - for (fn = next_fn(bus, dev, 0); fn > 0; fn = next_fn(bus, dev, fn)) { + do { dev = pci_scan_single_device(bus, devfn + fn); if (dev) { if (!pci_dev_is_added(dev)) nr++; - dev->multifunction = 1; + if (fn > 0) + dev->multifunction = 1; + } else if (fn == 0) { + /* function 0 is required */ + break; } - } + fn = next_fn(bus, dev, fn); + } while (fn >= 0); /* Only one slot has PCIe device */ if (bus->self && nr) From patchwork Thu May 5 08:38:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 1626785 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=OrxRrq9q; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4Kv6YT1GZrz9sBB for ; Thu, 5 May 2022 18:38:53 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236021AbiEEIm3 (ORCPT ); Thu, 5 May 2022 04:42:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53618 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234120AbiEEIm2 (ORCPT ); Thu, 5 May 2022 04:42:28 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1643947AC2; Thu, 5 May 2022 01:38:49 -0700 (PDT) Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2457Xj3H016263; Thu, 5 May 2022 08:38:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : mime-version; s=pp1; bh=BUKnHMMtJlhw8wLxcVR6t/uKudxySSsONcFK+URS3a0=; b=OrxRrq9qgudzRy/XjXooJffvCGFRQCmBA/bGALyBm9F2GtpAPLrsW94o8SE5Frv15+Xc pHY1Rnir4BgDD+R+8wlvOJ47Ql6iz9QDJmLg0js2ox9etTkERugpZI5866f6nckYVLDr wJL2V0Gg6Oijmfff5hcClA1AAlBUYruzpZ2+i4+Qpgnyt+QY/HhrZiUrFsMwK8Eam0Ap YX9fH+U//cX2N66tn4Sp++QCXRF5izZHmS3jkciqX2+PqeveN3EwYYlFoHYaqgJqeOdg dCvd7097dLhe/SL0F0MPXEnOFV+tWTv6cwmD3PkHtDg5CYIcIgzdV3xqNHFlVSaNgl0+ 8w== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3fvaa111yt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:45 +0000 Received: from m0098416.ppops.net (m0098416.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2458Biqe000563; Thu, 5 May 2022 08:38:45 GMT Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3fvaa111ye-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:45 +0000 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2458ZCGN007839; Thu, 5 May 2022 08:38:43 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma03ams.nl.ibm.com with ESMTP id 3ftp7fuqys-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:43 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2458ceJn50790692 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 May 2022 08:38:40 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 50BFC11C04A; Thu, 5 May 2022 08:38:40 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0CB9D11C04C; Thu, 5 May 2022 08:38:40 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 5 May 2022 08:38:39 +0000 (GMT) From: Niklas Schnelle To: Bjorn Helgaas , Jan Kiszka , Matthew Rosato , Pierre Morel Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, linux-pci@vger.kernel.org Subject: [PATCH RESEND v5 2/4] PCI: Move jailhouse's isolated function handling to pci_scan_slot() Date: Thu, 5 May 2022 10:38:37 +0200 Message-Id: <20220505083839.1315705-3-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220505083839.1315705-1-schnelle@linux.ibm.com> References: <20220505083839.1315705-1-schnelle@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 7_okw5VO9nMZ0QZry_mfWJucIibowslR X-Proofpoint-GUID: zFULyoTFeGyMfuWO1xX_FUtobJfcJOgZ X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-05_02,2022-05-04_02,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 suspectscore=0 phishscore=0 lowpriorityscore=0 priorityscore=1501 bulkscore=0 adultscore=0 spamscore=0 mlxlogscore=999 mlxscore=0 impostorscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2205050055 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The special case of the jailhouse hypervisor passing through individual PCI functions handles scanning for PCI functions even if function 0 does not exist. Currently this is done with an extra loop duplicating the one in pci_scan_slot(). By incorporating the check for jailhouse_paravirt() into pci_scan_slot() we can instead do this as part of the normal slot scan. Note that with the assignment of dev->multifunction gated by fn > 0 we set dev->multifunction unconditionally for all functions if function 0 is missing just as in the existing jailhouse loop. The only functional change is that we now call pcie_aspm_init_link_state() for these functions but this already happened if function 0 was passed through and should not be a problem. Cc: Jan Kiszka Link: https://lore.kernel.org/linux-pci/20220408224514.GA353445@bhelgaas/ Suggested-by: Bjorn Helgaas Signed-off-by: Niklas Schnelle --- drivers/pci/probe.c | 30 ++++++++++-------------------- 1 file changed, 10 insertions(+), 20 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 78aa1bccab2f..3029edc68ff7 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2658,8 +2658,13 @@ int pci_scan_slot(struct pci_bus *bus, int devfn) if (fn > 0) dev->multifunction = 1; } else if (fn == 0) { - /* function 0 is required */ - break; + /* + * function 0 is required unless we are running on + * a hypervisor which passes through individual PCI + * functions. + */ + if (!jailhouse_paravirt()) + break; } fn = next_fn(bus, dev, fn); } while (fn >= 0); @@ -2858,29 +2863,14 @@ static unsigned int pci_scan_child_bus_extend(struct pci_bus *bus, { unsigned int used_buses, normal_bridges = 0, hotplug_bridges = 0; unsigned int start = bus->busn_res.start; - unsigned int devfn, fn, cmax, max = start; + unsigned int devfn, cmax, max = start; struct pci_dev *dev; - int nr_devs; dev_dbg(&bus->dev, "scanning bus\n"); /* Go find them, Rover! */ - for (devfn = 0; devfn < 256; devfn += 8) { - nr_devs = pci_scan_slot(bus, devfn); - - /* - * The Jailhouse hypervisor may pass individual functions of a - * multi-function device to a guest without passing function 0. - * Look for them as well. - */ - if (jailhouse_paravirt() && nr_devs == 0) { - for (fn = 1; fn < 8; fn++) { - dev = pci_scan_single_device(bus, devfn + fn); - if (dev) - dev->multifunction = 1; - } - } - } + for (devfn = 0; devfn < 256; devfn += 8) + pci_scan_slot(bus, devfn); /* Reserve buses for SR-IOV capability */ used_buses = pci_iov_bus_range(bus); From patchwork Thu May 5 08:38:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 1626786 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=rzQQIlyG; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4Kv6YT6hPBz9sBB for ; Thu, 5 May 2022 18:38:53 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349741AbiEEIma (ORCPT ); Thu, 5 May 2022 04:42:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346179AbiEEIm3 (ORCPT ); Thu, 5 May 2022 04:42:29 -0400 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A449B49C80; Thu, 5 May 2022 01:38:50 -0700 (PDT) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2458E8Vh003693; Thu, 5 May 2022 08:38:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=M1zvcLMG92FcPS3g1DuhwhZILf+zrGpIaNaaLno10YU=; b=rzQQIlyG9QLWACBzo0S5tel4b9OaKNqPSFSNk0mlIp5hfZk8b6RrGCTZLPMaxTpDnyqC qXeNYp+kr5HUOG7DGEK3iLIuuOaCnd8noyqZC1KP1CpoUdqH6xDb/Z2mvSB6rHZVLu7M Ehiy7SOUDafIRQvq+US98tywR1VZ+/zUaPWe8+xKlOFo/ZGlgFU3V3Wy1EbB53bYDWve 9ujceayJHbZIZXg5zYh38gLoy5b8vXb+02eRG+GAQccBTLp9lbQ0YkF42oyrax4pKa9e 8OBa4LyCHK93NFiMqyK/9clKopDVFSi6+Op2BAq0fgjzw/QlMbIol98BdqjGfqcFLEUv bQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3fvavw8c8u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:46 +0000 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2458Ltni013170; Thu, 5 May 2022 08:38:45 GMT Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3fvavw8c8f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:45 +0000 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2458ZARJ015579; Thu, 5 May 2022 08:38:43 GMT Received: from b06cxnps4076.portsmouth.uk.ibm.com (d06relay13.portsmouth.uk.ibm.com [9.149.109.198]) by ppma01fra.de.ibm.com with ESMTP id 3fuxwkrnna-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:43 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2458cecG48038174 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 May 2022 08:38:40 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9FF5211C04A; Thu, 5 May 2022 08:38:40 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5C01011C050; Thu, 5 May 2022 08:38:40 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 5 May 2022 08:38:40 +0000 (GMT) From: Niklas Schnelle To: Bjorn Helgaas , Jan Kiszka , Matthew Rosato , Pierre Morel Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, linux-pci@vger.kernel.org Subject: [PATCH RESEND v5 3/4] PCI: Extend isolated function probing to s390 Date: Thu, 5 May 2022 10:38:38 +0200 Message-Id: <20220505083839.1315705-4-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220505083839.1315705-1-schnelle@linux.ibm.com> References: <20220505083839.1315705-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: NneUbkkqHQCD1bqXHyEtMSnl21YNPwNm X-Proofpoint-ORIG-GUID: yM2KTLT7oIWxQms6Ex6487p6gnFJ_iHR X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-05_02,2022-05-04_02,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 priorityscore=1501 spamscore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 impostorscore=0 clxscore=1015 mlxscore=0 mlxlogscore=725 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2205050055 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Like the jailhouse hypervisor s390's PCI architecture allows passing isolated PCI functions to an OS instance. As of now this is was not utilized even with multi-function support as the s390 PCI code makes sure that only virtual PCI busses including a function with devfn 0 are presented to the PCI subsystem. A subsequent change will remove this restriction. Allow probing such functions by replacing the existing check for jailhouse_paravirt() with a new hypervisor_isolated_pci_functions() helper. Cc: Jan Kiszka Signed-off-by: Niklas Schnelle --- drivers/pci/probe.c | 2 +- include/linux/hypervisor.h | 8 ++++++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 3029edc68ff7..4e27cc929095 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2663,7 +2663,7 @@ int pci_scan_slot(struct pci_bus *bus, int devfn) * a hypervisor which passes through individual PCI * functions. */ - if (!jailhouse_paravirt()) + if (!hypervisor_isolated_pci_functions()) break; } fn = next_fn(bus, dev, fn); diff --git a/include/linux/hypervisor.h b/include/linux/hypervisor.h index fc08b433c856..33b1c0482aac 100644 --- a/include/linux/hypervisor.h +++ b/include/linux/hypervisor.h @@ -32,4 +32,12 @@ static inline bool jailhouse_paravirt(void) #endif /* !CONFIG_X86 */ +static inline bool hypervisor_isolated_pci_functions(void) +{ + if (IS_ENABLED(CONFIG_S390)) + return true; + else + return jailhouse_paravirt(); +} + #endif /* __LINUX_HYPEVISOR_H */ From patchwork Thu May 5 08:38:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Niklas Schnelle X-Patchwork-Id: 1626787 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=ZWLJSCm4; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=2620:137:e000::1:20; helo=out1.vger.email; envelope-from=linux-pci-owner@vger.kernel.org; receiver=) Received: from out1.vger.email (out1.vger.email [IPv6:2620:137:e000::1:20]) by bilbo.ozlabs.org (Postfix) with ESMTP id 4Kv6YW0NWLz9sBB for ; Thu, 5 May 2022 18:38:55 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239396AbiEEImb (ORCPT ); Thu, 5 May 2022 04:42:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53636 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349368AbiEEIm3 (ORCPT ); Thu, 5 May 2022 04:42:29 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDD4449C88; Thu, 5 May 2022 01:38:50 -0700 (PDT) Received: from pps.filterd (m0098420.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2457qFpb013405; Thu, 5 May 2022 08:38:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=VKd5vFUiEKfuRcGuSWKwzFHHhQa5vtGA/lkq5v9/Zc8=; b=ZWLJSCm4MoiyHrMbzszFKRbxWFmGrctHhx/ZJfnPiHGIzj0yJY1/ElxSrP+lRCIrkB8i ebhaVY894JT2QTBMtpTYrl5luf8VFDYuJYYnWuWeAbVTRpYSSFyuvEi7HXzF+xm+EDb+ 49ignSPnsldSzwgIpnUSp+pL6NmHqBXQmP4VWpGRNogjSilZp2jsUbJimxIMk4fWuhxC UFWxvWXeUsQqxErYPVRDQ2J8JDy8aDbmRnEHSLfBbqqQtvdN6UoOancYrUo0chtWw9XS bax8qhu10l3XynLb59Ik62Phjs9N+GpQ4uhXq0iWANw3QK9p0TK5YCAs5ZrHBA59iPnz Xw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3fvajtrrvu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:46 +0000 Received: from m0098420.ppops.net (m0098420.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 2458bJFO021792; Thu, 5 May 2022 08:38:46 GMT Received: from ppma03fra.de.ibm.com (6b.4a.5195.ip4.static.sl-reverse.com [149.81.74.107]) by mx0b-001b2d01.pphosted.com (PPS) with ESMTPS id 3fvajtrrvh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:46 +0000 Received: from pps.filterd (ppma03fra.de.ibm.com [127.0.0.1]) by ppma03fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2458YV4x014660; Thu, 5 May 2022 08:38:44 GMT Received: from b06avi18878370.portsmouth.uk.ibm.com (b06avi18878370.portsmouth.uk.ibm.com [9.149.26.194]) by ppma03fra.de.ibm.com with ESMTP id 3fscdk4wym-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 05 May 2022 08:38:44 +0000 Received: from d06av25.portsmouth.uk.ibm.com (d06av25.portsmouth.uk.ibm.com [9.149.105.61]) by b06avi18878370.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2458cao823921060 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 5 May 2022 08:38:36 GMT Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EE94911C04A; Thu, 5 May 2022 08:38:40 +0000 (GMT) Received: from d06av25.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AB7D011C04C; Thu, 5 May 2022 08:38:40 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av25.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 5 May 2022 08:38:40 +0000 (GMT) From: Niklas Schnelle To: Bjorn Helgaas , Jan Kiszka , Matthew Rosato , Pierre Morel Cc: linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, linux-pci@vger.kernel.org Subject: [PATCH RESEND v5 4/4] s390/pci: allow zPCI zbus without a function zero Date: Thu, 5 May 2022 10:38:39 +0200 Message-Id: <20220505083839.1315705-5-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220505083839.1315705-1-schnelle@linux.ibm.com> References: <20220505083839.1315705-1-schnelle@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: IpSXq_jPtXQ6x3HR3qPs-TFOY4OL8BO- X-Proofpoint-GUID: V6jJxyV-HUK-5mpmTHwZBSNtk5Hfm5bt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-05-05_02,2022-05-04_02,2022-02-23_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 impostorscore=0 bulkscore=0 priorityscore=1501 mlxscore=0 lowpriorityscore=0 phishscore=0 clxscore=1015 mlxlogscore=940 malwarescore=0 spamscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000 definitions=main-2205050055 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Currently the zPCI code block PCI bus creation and probing of a zPCI zbus unless there is a PCI function with devfn 0. This is always the case for the PCI functions with hidden RID but may keep PCI functions from a multi-function PCI device with RID information invisible until the function 0 becomes visible. Worse as a PCI bus is necessary to even present a PCI hotplug slot even that remains invisible. With the probing of these so called isolated PCI functions enabled for s390 in common code this restriction is no longer necessary. On network cards with multiple ports and a PF per port this also allows using each port on its own while still providing the physical PCI topology information in the devfn needed to associate VFs with their parent PF. Signed-off-by: Niklas Schnelle --- arch/s390/pci/pci_bus.c | 82 ++++++++++------------------------------- 1 file changed, 20 insertions(+), 62 deletions(-) diff --git a/arch/s390/pci/pci_bus.c b/arch/s390/pci/pci_bus.c index 5d77acbd1c87..6a8da1b742ae 100644 --- a/arch/s390/pci/pci_bus.c +++ b/arch/s390/pci/pci_bus.c @@ -145,9 +145,6 @@ int zpci_bus_scan_bus(struct zpci_bus *zbus) struct zpci_dev *zdev; int devfn, rc, ret = 0; - if (!zbus->function[0]) - return 0; - for (devfn = 0; devfn < ZPCI_FUNCTIONS_PER_BUS; devfn++) { zdev = zbus->function[devfn]; if (zdev && zdev->state == ZPCI_FN_STATE_CONFIGURED) { @@ -184,26 +181,26 @@ void zpci_bus_scan_busses(void) /* zpci_bus_create_pci_bus - Create the PCI bus associated with this zbus * @zbus: the zbus holding the zdevices - * @f0: function 0 of the bus + * @fr: PCI root function that will determine the bus's domain, and bus speeed * @ops: the pci operations * - * Function zero is taken as a parameter as this is used to determine the - * domain, multifunction property and maximum bus speed of the entire bus. + * The PCI function @fr determines the domain (its UID), multifunction property + * and maximum bus speed of the entire bus. * * Return: 0 on success, an error code otherwise */ -static int zpci_bus_create_pci_bus(struct zpci_bus *zbus, struct zpci_dev *f0, struct pci_ops *ops) +static int zpci_bus_create_pci_bus(struct zpci_bus *zbus, struct zpci_dev *fr, struct pci_ops *ops) { struct pci_bus *bus; int domain; - domain = zpci_alloc_domain((u16)f0->uid); + domain = zpci_alloc_domain((u16)fr->uid); if (domain < 0) return domain; zbus->domain_nr = domain; - zbus->multifunction = f0->rid_available; - zbus->max_bus_speed = f0->max_bus_speed; + zbus->multifunction = fr->rid_available; + zbus->max_bus_speed = fr->max_bus_speed; /* * Note that the zbus->resources are taken over and zbus->resources @@ -303,47 +300,6 @@ void pcibios_bus_add_device(struct pci_dev *pdev) } } -/* zpci_bus_create_hotplug_slots - Add hotplug slot(s) for device added to bus - * @zdev: the zPCI device that was newly added - * - * Add the hotplug slot(s) for the newly added PCI function. Normally this is - * simply the slot for the function itself. If however we are adding the - * function 0 on a zbus, it might be that we already registered functions on - * that zbus but could not create their hotplug slots yet so add those now too. - * - * Return: 0 on success, an error code otherwise - */ -static int zpci_bus_create_hotplug_slots(struct zpci_dev *zdev) -{ - struct zpci_bus *zbus = zdev->zbus; - int devfn, rc = 0; - - rc = zpci_init_slot(zdev); - if (rc) - return rc; - zdev->has_hp_slot = 1; - - if (zdev->devfn == 0 && zbus->multifunction) { - /* Now that function 0 is there we can finally create the - * hotplug slots for those functions with devfn != 0 that have - * been parked in zbus->function[] waiting for us to be able to - * create the PCI bus. - */ - for (devfn = 1; devfn < ZPCI_FUNCTIONS_PER_BUS; devfn++) { - zdev = zbus->function[devfn]; - if (zdev && !zdev->has_hp_slot) { - rc = zpci_init_slot(zdev); - if (rc) - return rc; - zdev->has_hp_slot = 1; - } - } - - } - - return rc; -} - static int zpci_bus_add_device(struct zpci_bus *zbus, struct zpci_dev *zdev) { int rc = -EINVAL; @@ -352,21 +308,19 @@ static int zpci_bus_add_device(struct zpci_bus *zbus, struct zpci_dev *zdev) pr_err("devfn %04x is already assigned\n", zdev->devfn); return rc; } + zdev->zbus = zbus; zbus->function[zdev->devfn] = zdev; zpci_nb_devices++; - if (zbus->bus) { - if (zbus->multifunction && !zdev->rid_available) { - WARN_ONCE(1, "rid_available not set for multifunction\n"); - goto error; - } - - zpci_bus_create_hotplug_slots(zdev); - } else { - /* Hotplug slot will be created once function 0 appears */ - zbus->multifunction = 1; + if (zbus->multifunction && !zdev->rid_available) { + WARN_ONCE(1, "rid_available not set for multifunction\n"); + goto error; } + rc = zpci_init_slot(zdev); + if (rc) + goto error; + zdev->has_hp_slot = 1; return 0; @@ -400,7 +354,11 @@ int zpci_bus_device_register(struct zpci_dev *zdev, struct pci_ops *ops) return -ENOMEM; } - if (zdev->devfn == 0) { + if (!zbus->bus) { + /* The UID of the first PCI function registered with a zpci_bus + * is used as the domain number for that bus. Currently there + * is exactly one zpci_bus per domain. + */ rc = zpci_bus_create_pci_bus(zbus, zdev, ops); if (rc) goto error;