From patchwork Fri Mar 1 14:30:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 1050957 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 44CKcP4LCnz9sBF for ; Mon, 4 Mar 2019 10:48:53 +1100 (AEDT) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=yadro.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=yadro.com header.i=@yadro.com header.b="CZiaAI9Q"; dkim-atps=neutral Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 44CKcP2MdQzDqHY for ; Mon, 4 Mar 2019 10:48:53 +1100 (AEDT) X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=yadro.com (client-ip=89.207.88.251; helo=mta-01.yadro.com; envelope-from=s.miroshnichenko@yadro.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=yadro.com Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=yadro.com header.i=@yadro.com header.b="CZiaAI9Q"; dkim-atps=neutral Received: from mta-01.yadro.com (mta-01.yadro.com [89.207.88.251]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 449sKT1s0PzDqQL for ; Sat, 2 Mar 2019 01:30:53 +1100 (AEDT) Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 6A25041974; Fri, 1 Mar 2019 14:30:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1551450648; x=1553265049; bh=3SpedOBNf/p1tLD1QIPZlyYPqF1gCB74jgy Vrys/7h8=; b=CZiaAI9QyKzmZUDvX4bf1TAGg9DS4MBJtyTvlbAuS8YqdEyvCmY q+BQO2DYT9/EKTwRCjsb8U2A3tOcLAMvTPzzabpFNGJlqEwGbDQIXxc0wKu/3rJY ggsRtKv3Zo1BBz9ns+p78luHk9SSEIFZlJ34zyPk1nA77h81zN4VuzDc= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qOj0Rzt4VSSm; Fri, 1 Mar 2019 17:30:48 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 316FB41976; Fri, 1 Mar 2019 17:30:48 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Fri, 1 Mar 2019 17:30:47 +0300 From: Sergey Miroshnichenko To: Date: Fri, 1 Mar 2019 17:30:33 +0300 Message-ID: <20190301143038.23325-2-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190301143038.23325-1-s.miroshnichenko@yadro.com> References: <20190301143038.23325-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) X-Mailman-Approved-At: Mon, 04 Mar 2019 10:48:37 +1100 Subject: [Skiboot] [PATCH RFC 1/6] core/pci: Add functions to find accessible devices by bdfn X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Stewart Smith , linux@yadro.com Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" During the PCIe re-enumeration by an OS the .bdfn field of the struct pci_device may not correspond to its device's actual address for a while, so the pci_find_dev() will return an invalid descriptor in this case. Some code may rely on this behavior, so this patch introduces a new function to return a descriptor for addressable devices only. This new function will help in discovering hotplugged devices. Here's a synthetic (but possible) scenario where pci_find_dev_safe() is different from pci_find_dev(): I. Before the re-enumeration: +-[0021:00]---00.0-[01-04]--+-00.0-[02]-- +-01.0-[03]----00.0 Device Z1 +-02.0-[04]-- - pci_find_dev(0021:03:00.0) returns correct struct pci_device *pd for Device Z1; - pci_find_dev_safe(0021:03:00.0) returns the same pd. II. Re-enumeration, step 1: _____ Dropped the bus numbers by writing zeros / +-[0021:00]---00.0-[00-00]--+-00.0-[02]-- +-01.0-[03]----00.0 Device Z1 +-02.0-[04]-- - Device Z1 is unreachable (bus 1 is not assigned), but pci_find_dev(0021:03:00.0) returns its pd; - pci_find_dev_safe(0021:03:00.0) returns NULL. III. Re-enumeration, step 2: __ Update the bus numbers: primary <- 0, / secondary <- 1, subordinate <- ff / +-[0021:00]---00.0-[01-ff]--+-00.0-[02]-- +-01.0-[03]----00.0 Device Z1 +-02.0-[04]-- - By coincidence, pci_find_dev(0021:03:00.0) returns pd for Device Z1; - pci_find_dev_safe(0021:03:00.0) returns pd for Device Z1, which is correct by coincidence. IV. Re-enumeration, step 3: ___ Drop the bus numbers / +-[0021:00]---00.0-[01-ff]--+-00.0-[00-00]-- +-01.0-[00-00]----00.0 Device Z1 +-02.0-[00-00]-- - Device Z1 is unreachable (downstream 0021:01:01.0 has no bus numbers assigned), but pci_find_dev(0021:03:00.0) returns its uncorrected pd; - pci_find_dev_safe(0021:03:00.0) returns NULL. V. Re-enumeration, step 4: Set the bus numbers __ _ A phantom of Device Z1 \ / from pci_find_dev(03:00.0) +-[0021:00]---00.0-[01-ff]--+-00.0-[02-06]-- - +-01.0-[00-00]----00.0 Device Z1 +-02.0-[00-00]-- - pci_find_dev(0021:03:00.0) returns a phantom of the Device Z1 for wrong address; - pci_find_dev_safe(0021:03:00.0) returns NULL. VI. Re-enumeration, step 5: +-[0021:00]---00.0-[01-ff]--+-00.0-[02-06]-- -+-01.0-[07-0b]----00.0 Device Z1 / +-02.0-[00-00]-- Set the bus numbers -- - pci_find_dev(0021:03:00.0) returns a phantom of the Device Z1; - pci_find_dev_safe(0021:03:00.0) returns NULL; - pci_find_dev_safe(0021:07:00.0) returns pd for the Device Z1, but its .bdfn field needs to be corrected (by the following patch). VI. After the re-enumeration: +-[0021:00]---00.0-[01-10]--+-00.0-[02-06]-- +-01.0-[07-0b]----00.0 Device Z1 +-02.0-[0c-10]-- Even though the .bdfn field is not correct at the moment, it is clear that this is a previously known device, not a hotplugged one. Signed-off-by: Sergey Miroshnichenko --- core/pci.c | 83 +++++++++++++++++++++++++++++++++++++++++++++++++++ include/pci.h | 2 ++ 2 files changed, 85 insertions(+) diff --git a/core/pci.c b/core/pci.c index 454b5010..dac4e853 100644 --- a/core/pci.c +++ b/core/pci.c @@ -1867,6 +1867,89 @@ struct pci_device *pci_find_dev(struct phb *phb, uint16_t bdfn) return pci_walk_dev(phb, NULL, __pci_find_dev, &bdfn); } +static struct pci_device *__pci_find_parent(struct pci_device *pd, uint8_t bus) +{ + struct pci_device *child; + + if (!pd) + return NULL; + + if (pd->secondary_bus == bus) + return pd; + + if (bus < pd->secondary_bus || bus > pd->subordinate_bus) + return NULL; + + list_for_each(&pd->children, child, link) { + if (child->secondary_bus == bus) + return child; + } + + list_for_each(&pd->children, child, link) { + struct pci_device *found = __pci_find_parent(child, bus); + + if (found) + return found; + } + + return NULL; +} + +struct pci_device *pci_find_parent_dev(struct phb *phb, uint16_t bdfn) +{ + struct pci_device *pd; + uint8_t bus = (bdfn >> 8); + + if (!phb) + return NULL; + + list_for_each(&phb->devices, pd, link) { + struct pci_device *found = __pci_find_parent(pd, bus); + + if (found) + return found; + } + + return NULL; +} + +struct pci_device *pci_find_dev_safe(struct phb *phb, uint16_t bdfn) +{ + struct pci_device *parent; + struct pci_device *child; + + if (!phb) + return NULL; + + if (!bdfn) { + struct pci_device *pd; + + list_for_each(&phb->devices, pd, link) { + if (pd->bdfn == bdfn) + return pd; + } + } + + parent = pci_find_parent_dev(phb, bdfn); + if (!parent) + return NULL; + + list_for_each(&parent->children, child, link) { + if ((child->bdfn & 0xff) == (bdfn & 0xff)) { + if (child->bdfn != bdfn) { + PCIERR(phb, bdfn, "pci_device has invalid bdfn field %04x:%02x:%02x.%d\n", + phb->opal_id, + child->bdfn >> 8, + (child->bdfn >> 3) & 0x1f, + child->bdfn & 0x7); + } + return child; + } + } + + return NULL; +} + static int __pci_restore_bridge_buses(struct phb *phb, struct pci_device *pd, void *data __unused) diff --git a/include/pci.h b/include/pci.h index b49641ca..3262a4bd 100644 --- a/include/pci.h +++ b/include/pci.h @@ -473,6 +473,8 @@ extern struct pci_device *pci_walk_dev(struct phb *phb, void *), void *userdata); extern struct pci_device *pci_find_dev(struct phb *phb, uint16_t bdfn); +extern struct pci_device *pci_find_dev_safe(struct phb *phb, uint16_t bdfn); +extern struct pci_device *pci_find_parent_dev(struct phb *phb, uint16_t bdfn); extern void pci_restore_bridge_buses(struct phb *phb, struct pci_device *pd); extern struct pci_cfg_reg_filter *pci_find_cfg_reg_filter(struct pci_device *pd, uint32_t start, uint32_t len);