From patchwork Wed Aug 5 03:04:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Leonardo_Br=C3=A1s?= X-Patchwork-Id: 1341066 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BLxR828PGz9sR4 for ; Wed, 5 Aug 2020 13:08:52 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=tsDrwsn6; dkim-atps=neutral Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4BLxR764tCzDqcm for ; Wed, 5 Aug 2020 13:08:51 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::843; helo=mail-qt1-x843.google.com; envelope-from=leobras.c@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=tsDrwsn6; dkim-atps=neutral Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4BLxLw1N5nzDqRj for ; Wed, 5 Aug 2020 13:05:12 +1000 (AEST) Received: by mail-qt1-x843.google.com with SMTP id x12so24163176qtp.1 for ; Tue, 04 Aug 2020 20:05:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KZZhp6G/Nvdr8CubS5lc2IdWicGU//2bZ5AuIAz3WYY=; b=tsDrwsn6dw9H9iUUR3XvrWEQiOYRfexsj6w/btAB7+ZzsHuM2fIRTkEUdV2w0KaSFC h+IX6i/BpNognd2NP9nnDY28By5W++Z9D1wKijXnSEaKeFvsTyoAQiSC0GISyOn6VkO6 /izKrL2Lw4lhEwm6j0911VfV/BfOxL8Vu4qTPLxz8EnxznS8EDelcL5ChjBKc0CIhK6a 50HRa/GfTvzfXdeWKBrxjr9rpKlMUeXzixQoKOKKe2vjT43XaBcKASUqXJsFWR23r4Uo c3RAWn0op7UzU+dPXtz4YMyCP/dtAAUsvsAZkbdQ36dh7Ap8Fj2FoLUVktHDcd6czgfh eN3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KZZhp6G/Nvdr8CubS5lc2IdWicGU//2bZ5AuIAz3WYY=; b=sQ/GE+RK+ktUUPz6S5m+0h1mgSSTrgoBtGepQqYmoyA26i+QwaUIlWvgTenJqzeWV9 6rnOTWtWAET+oHPK7beTCeyW+p7pQ4RKqs1o3JjajpYHPrcVov1f6WniKbBl012TB9Mj dMCalQPbswkPF0BBc7xarAMdekrXaRsK8p/mRh29BN0W3ZlBDVyE9SPN2D+5FYyWxsl2 K+LZ1X88IJtTYX4idZ0s1RcElfqZqjmidBjz04YPndKmYaDue5sLAoPO6zn7AcEKogmx mxsMZuB5rxdPVnFJXKSYI0C5j1rrN8AjMvU+SODI9V1bIcRGL3aPanxvpdZf61ZWKSJ8 VggA== X-Gm-Message-State: AOAM530X3TZuu8KDz7dWp9bekIq5cuLqdUPJIoxc9YHYmNO1D3RaVhzi 52QspL9tPA7KBOYFzpHyj/o= X-Google-Smtp-Source: ABdhPJwEJlDXMiqIxjJ8NWBXOAzyEnPfjVEOVS1TpGiB9pkJhcOwymUcwPehjylHnqE+u6vPhAyQIA== X-Received: by 2002:ac8:4643:: with SMTP id f3mr1275598qto.128.1596596709387; Tue, 04 Aug 2020 20:05:09 -0700 (PDT) Received: from LeoBras.ibmuc.com (179-125-154-168.dynamic.desktop.com.br. [179.125.154.168]) by smtp.gmail.com with ESMTPSA id n4sm869946qtr.73.2020.08.04.20.05.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 20:05:08 -0700 (PDT) From: Leonardo Bras To: Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Leonardo Bras , Alexey Kardashevskiy , Thiago Jung Bauermann , Ram Pai , Brian King , Murilo Fossa Vicentini , David Dai Subject: [PATCH v5 1/4] powerpc/pseries/iommu: Create defines for operations in ibm, ddw-applicable Date: Wed, 5 Aug 2020 00:04:52 -0300 Message-Id: <20200805030455.123024-2-leobras.c@gmail.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200805030455.123024-1-leobras.c@gmail.com> References: <20200805030455.123024-1-leobras.c@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Create defines to help handling ibm,ddw-applicable values, avoiding confusion about the index of given operations. Signed-off-by: Leonardo Bras Tested-by: David Dai Reviewed-by: Alexey Kardashevskiy --- arch/powerpc/platforms/pseries/iommu.c | 43 ++++++++++++++++---------- 1 file changed, 26 insertions(+), 17 deletions(-) diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c index 6d47b4a3ce39..ac0d6376bdad 100644 --- a/arch/powerpc/platforms/pseries/iommu.c +++ b/arch/powerpc/platforms/pseries/iommu.c @@ -39,6 +39,14 @@ #include "pseries.h" +enum { + DDW_QUERY_PE_DMA_WIN = 0, + DDW_CREATE_PE_DMA_WIN = 1, + DDW_REMOVE_PE_DMA_WIN = 2, + + DDW_APPLICABLE_SIZE +}; + static struct iommu_table_group *iommu_pseries_alloc_group(int node) { struct iommu_table_group *table_group; @@ -771,12 +779,12 @@ static void remove_ddw(struct device_node *np, bool remove_prop) { struct dynamic_dma_window_prop *dwp; struct property *win64; - u32 ddw_avail[3]; + u32 ddw_avail[DDW_APPLICABLE_SIZE]; u64 liobn; int ret = 0; ret = of_property_read_u32_array(np, "ibm,ddw-applicable", - &ddw_avail[0], 3); + &ddw_avail[0], DDW_APPLICABLE_SIZE); win64 = of_find_property(np, DIRECT64_PROPNAME, NULL); if (!win64) @@ -798,15 +806,15 @@ static void remove_ddw(struct device_node *np, bool remove_prop) pr_debug("%pOF successfully cleared tces in window.\n", np); - ret = rtas_call(ddw_avail[2], 1, 1, NULL, liobn); + ret = rtas_call(ddw_avail[DDW_REMOVE_PE_DMA_WIN], 1, 1, NULL, liobn); if (ret) pr_warn("%pOF: failed to remove direct window: rtas returned " "%d to ibm,remove-pe-dma-window(%x) %llx\n", - np, ret, ddw_avail[2], liobn); + np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN], liobn); else pr_debug("%pOF: successfully removed direct window: rtas returned " "%d to ibm,remove-pe-dma-window(%x) %llx\n", - np, ret, ddw_avail[2], liobn); + np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN], liobn); delprop: if (remove_prop) @@ -889,11 +897,11 @@ static int query_ddw(struct pci_dev *dev, const u32 *ddw_avail, buid = pdn->phb->buid; cfg_addr = ((pdn->busno << 16) | (pdn->devfn << 8)); - ret = rtas_call(ddw_avail[0], 3, 5, (u32 *)query, - cfg_addr, BUID_HI(buid), BUID_LO(buid)); + ret = rtas_call(ddw_avail[DDW_QUERY_PE_DMA_WIN], 3, 5, (u32 *)query, + cfg_addr, BUID_HI(buid), BUID_LO(buid)); dev_info(&dev->dev, "ibm,query-pe-dma-windows(%x) %x %x %x" - " returned %d\n", ddw_avail[0], cfg_addr, BUID_HI(buid), - BUID_LO(buid), ret); + " returned %d\n", ddw_avail[DDW_QUERY_PE_DMA_WIN], cfg_addr, + BUID_HI(buid), BUID_LO(buid), ret); return ret; } @@ -920,15 +928,16 @@ static int create_ddw(struct pci_dev *dev, const u32 *ddw_avail, do { /* extra outputs are LIOBN and dma-addr (hi, lo) */ - ret = rtas_call(ddw_avail[1], 5, 4, (u32 *)create, - cfg_addr, BUID_HI(buid), BUID_LO(buid), - page_shift, window_shift); + ret = rtas_call(ddw_avail[DDW_CREATE_PE_DMA_WIN], 5, 4, + (u32 *)create, cfg_addr, BUID_HI(buid), + BUID_LO(buid), page_shift, window_shift); } while (rtas_busy_delay(ret)); dev_info(&dev->dev, "ibm,create-pe-dma-window(%x) %x %x %x %x %x returned %d " - "(liobn = 0x%x starting addr = %x %x)\n", ddw_avail[1], - cfg_addr, BUID_HI(buid), BUID_LO(buid), page_shift, - window_shift, ret, create->liobn, create->addr_hi, create->addr_lo); + "(liobn = 0x%x starting addr = %x %x)\n", + ddw_avail[DDW_CREATE_PE_DMA_WIN], cfg_addr, BUID_HI(buid), + BUID_LO(buid), page_shift, window_shift, ret, create->liobn, + create->addr_hi, create->addr_lo); return ret; } @@ -996,7 +1005,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) int page_shift; u64 dma_addr, max_addr; struct device_node *dn; - u32 ddw_avail[3]; + u32 ddw_avail[DDW_APPLICABLE_SIZE]; struct direct_window *window; struct property *win64; struct dynamic_dma_window_prop *ddwprop; @@ -1029,7 +1038,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) * the property is actually in the parent, not the PE */ ret = of_property_read_u32_array(pdn, "ibm,ddw-applicable", - &ddw_avail[0], 3); + &ddw_avail[0], DDW_APPLICABLE_SIZE); if (ret) goto out_failed; From patchwork Wed Aug 5 03:04:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Leonardo_Br=C3=A1s?= X-Patchwork-Id: 1341067 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BLxTH49XVz9s1x for ; Wed, 5 Aug 2020 13:10:43 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=cvRnM4uU; dkim-atps=neutral Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4BLxTH1k0PzDqPQ for ; Wed, 5 Aug 2020 13:10:43 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::f42; helo=mail-qv1-xf42.google.com; envelope-from=leobras.c@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=cvRnM4uU; dkim-atps=neutral Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4BLxM03fBCzDqRD for ; Wed, 5 Aug 2020 13:05:16 +1000 (AEST) Received: by mail-qv1-xf42.google.com with SMTP id j10so13422224qvo.13 for ; Tue, 04 Aug 2020 20:05:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FJwtXKryPct6ANmQ36Ytc2bYA9QUUsVGIRklUOOsfAw=; b=cvRnM4uUQvxlFcLUYZy2XEHeQVXsEUJxQQ4g9jIZ8R+hslzosSFNqwKSJ6nGGQ6fOL 0Y6LL3oD4znV3PRumEGOEvrjG1mWWNp5lwKuU3ZrEE4PNIQF+S6l2PkqU9XJf/7rnada IYcHsGjxnoBSeHoBqpNvRN1gmEHTYR+0/YkEVGikNa+8bMPcz8BADA1ulm/oOda10+Ow N4ODO/+3/0tX0FGpypH8MTtb/gbM/3dcTsg2k6zbSeV2FKmbg8fA5PvHhkB+uggiY9/w 3aNudLCLubMetQBk0avstYeOxGHiHiZCWXiM9nY6e4sucz2GCvoKfcaeXHtChpeOJxDB cxjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FJwtXKryPct6ANmQ36Ytc2bYA9QUUsVGIRklUOOsfAw=; b=NRO5zE986tSQ04C8YZDKVT9M7GGI14+9CIwnmAFhYvJPPxXnDrZGc44cQ4rYwLW5Wd Di5PRCIWWZZ3URYd1aqNG0CLdqGylbJd9w+6VWJF4cUE7xlMqDpyBhHHct/1YWMSOPLR l41yhw/QQU46Yq7BoSOJCngcuEZ3I6QtQ9gj/kqZr/C72hr3fPgrLX9mRam5KMmB6fJp 2nwSsDYOBe0OoFGvgNG3M9vGeYQ3VjExBW+LEboQRj1wgDNDn9JK6sIS2L38mlcKnQz3 Irrb5j2bkEmeA87gizmVp4pqdWxixV9ySP/lCdkdV77aLCzkCwwccQw1rK8H3nnq1Jka HVDA== X-Gm-Message-State: AOAM533DephR7yf+hYsBBcBiH9keTIFG0vwrHIiZEXyDT1sCeKkrxbhR uP//eQmm70qn8N2ggvtK1VFFSgeV X-Google-Smtp-Source: ABdhPJx8TXNGRdQ0ehIfJO2zJu6RAMRLpLNY5hqAh3lThLKPdZlULgxiiL7SHArVhaEze4tacaStfA== X-Received: by 2002:a0c:ff02:: with SMTP id w2mr1523304qvt.206.1596596713467; Tue, 04 Aug 2020 20:05:13 -0700 (PDT) Received: from LeoBras.ibmuc.com (179-125-154-168.dynamic.desktop.com.br. [179.125.154.168]) by smtp.gmail.com with ESMTPSA id n4sm869946qtr.73.2020.08.04.20.05.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 20:05:12 -0700 (PDT) From: Leonardo Bras To: Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Leonardo Bras , Alexey Kardashevskiy , Thiago Jung Bauermann , Ram Pai , Brian King , Murilo Fossa Vicentini , David Dai Subject: [PATCH v5 2/4] powerpc/pseries/iommu: Update call to ibm, query-pe-dma-windows Date: Wed, 5 Aug 2020 00:04:53 -0300 Message-Id: <20200805030455.123024-3-leobras.c@gmail.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200805030455.123024-1-leobras.c@gmail.com> References: <20200805030455.123024-1-leobras.c@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" From LoPAR level 2.8, "ibm,ddw-extensions" index 3 can make the number of outputs from "ibm,query-pe-dma-windows" go from 5 to 6. This change of output size is meant to expand the address size of largest_available_block PE TCE from 32-bit to 64-bit, which ends up shifting page_size and migration_capable. This ends up requiring the update of ddw_query_response->largest_available_block from u32 to u64, and manually assigning the values from the buffer into this struct, according to output size. Also, a routine was created for helping reading the ddw extensions as suggested by LoPAR: First reading the size of the extension array from index 0, checking if the property exists, and then returning it's value. Signed-off-by: Leonardo Bras Tested-by: David Dai Reviewed-by: Alexey Kardashevskiy --- arch/powerpc/platforms/pseries/iommu.c | 91 +++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 10 deletions(-) diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c index ac0d6376bdad..1a933c4e8bba 100644 --- a/arch/powerpc/platforms/pseries/iommu.c +++ b/arch/powerpc/platforms/pseries/iommu.c @@ -47,6 +47,12 @@ enum { DDW_APPLICABLE_SIZE }; +enum { + DDW_EXT_SIZE = 0, + DDW_EXT_RESET_DMA_WIN = 1, + DDW_EXT_QUERY_OUT_SIZE = 2 +}; + static struct iommu_table_group *iommu_pseries_alloc_group(int node) { struct iommu_table_group *table_group; @@ -342,7 +348,7 @@ struct direct_window { /* Dynamic DMA Window support */ struct ddw_query_response { u32 windows_available; - u32 largest_available_block; + u64 largest_available_block; u32 page_size; u32 migration_capable; }; @@ -877,14 +883,62 @@ static int find_existing_ddw_windows(void) } machine_arch_initcall(pseries, find_existing_ddw_windows); +/** + * ddw_read_ext - Get the value of an DDW extension + * @np: device node from which the extension value is to be read. + * @extnum: index number of the extension. + * @value: pointer to return value, modified when extension is available. + * + * Checks if "ibm,ddw-extensions" exists for this node, and get the value + * on index 'extnum'. + * It can be used only to check if a property exists, passing value == NULL. + * + * Returns: + * 0 if extension successfully read + * -EINVAL if the "ibm,ddw-extensions" does not exist, + * -ENODATA if "ibm,ddw-extensions" does not have a value, and + * -EOVERFLOW if "ibm,ddw-extensions" does not contain this extension. + */ +static inline int ddw_read_ext(const struct device_node *np, int extnum, + u32 *value) +{ + static const char propname[] = "ibm,ddw-extensions"; + u32 count; + int ret; + + ret = of_property_read_u32_index(np, propname, DDW_EXT_SIZE, &count); + if (ret) + return ret; + + if (count < extnum) + return -EOVERFLOW; + + if (!value) + value = &count; + + return of_property_read_u32_index(np, propname, extnum, value); +} + static int query_ddw(struct pci_dev *dev, const u32 *ddw_avail, - struct ddw_query_response *query) + struct ddw_query_response *query, + struct device_node *parent) { struct device_node *dn; struct pci_dn *pdn; - u32 cfg_addr; + u32 cfg_addr, ext_query, query_out[5]; u64 buid; - int ret; + int ret, out_sz; + + /* + * From LoPAR level 2.8, "ibm,ddw-extensions" index 3 can rule how many + * output parameters ibm,query-pe-dma-windows will have, ranging from + * 5 to 6. + */ + ret = ddw_read_ext(parent, DDW_EXT_QUERY_OUT_SIZE, &ext_query); + if (!ret && ext_query == 1) + out_sz = 6; + else + out_sz = 5; /* * Get the config address and phb buid of the PE window. @@ -897,11 +951,28 @@ static int query_ddw(struct pci_dev *dev, const u32 *ddw_avail, buid = pdn->phb->buid; cfg_addr = ((pdn->busno << 16) | (pdn->devfn << 8)); - ret = rtas_call(ddw_avail[DDW_QUERY_PE_DMA_WIN], 3, 5, (u32 *)query, + ret = rtas_call(ddw_avail[DDW_QUERY_PE_DMA_WIN], 3, out_sz, query_out, cfg_addr, BUID_HI(buid), BUID_LO(buid)); - dev_info(&dev->dev, "ibm,query-pe-dma-windows(%x) %x %x %x" - " returned %d\n", ddw_avail[DDW_QUERY_PE_DMA_WIN], cfg_addr, - BUID_HI(buid), BUID_LO(buid), ret); + dev_info(&dev->dev, "ibm,query-pe-dma-windows(%x) %x %x %x returned %d\n", + ddw_avail[DDW_QUERY_PE_DMA_WIN], cfg_addr, BUID_HI(buid), + BUID_LO(buid), ret); + + switch (out_sz) { + case 5: + query->windows_available = query_out[0]; + query->largest_available_block = query_out[1]; + query->page_size = query_out[2]; + query->migration_capable = query_out[3]; + break; + case 6: + query->windows_available = query_out[0]; + query->largest_available_block = ((u64)query_out[1] << 32) | + query_out[2]; + query->page_size = query_out[3]; + query->migration_capable = query_out[4]; + break; + } + return ret; } @@ -1049,7 +1120,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) * of page sizes: supported and supported for migrate-dma. */ dn = pci_device_to_OF_node(dev); - ret = query_ddw(dev, ddw_avail, &query); + ret = query_ddw(dev, ddw_avail, &query, pdn); if (ret != 0) goto out_failed; @@ -1077,7 +1148,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) /* check largest block * page size > max memory hotplug addr */ max_addr = ddw_memory_hotplug_max(); if (query.largest_available_block < (max_addr >> page_shift)) { - dev_dbg(&dev->dev, "can't map partition max 0x%llx with %u " + dev_dbg(&dev->dev, "can't map partition max 0x%llx with %llu " "%llu-sized pages\n", max_addr, query.largest_available_block, 1ULL << page_shift); goto out_failed; From patchwork Wed Aug 5 03:04:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Leonardo_Br=C3=A1s?= X-Patchwork-Id: 1341068 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BLxW64x2sz9s1x for ; Wed, 5 Aug 2020 13:12:18 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=o2qN9dqv; dkim-atps=neutral Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4BLxW636C1zDqZT for ; Wed, 5 Aug 2020 13:12:18 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::841; helo=mail-qt1-x841.google.com; envelope-from=leobras.c@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=o2qN9dqv; dkim-atps=neutral Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4BLxM442YYzDqbx for ; Wed, 5 Aug 2020 13:05:20 +1000 (AEST) Received: by mail-qt1-x841.google.com with SMTP id 6so32703587qtt.0 for ; Tue, 04 Aug 2020 20:05:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=h1QuQfa0AlBLlK+lh3SXPgl8VOAIvXOvZyydTehW8ec=; b=o2qN9dqvLp4T354UyM8u0kWQ7Uvuy3vmSsowvT/0MMculB6VqMhApKgSGyfpCtYG2M NhNoi0gXz883M9RAr0NnuidSwWwI+8xN4QlcUZqjEoyvcXT+q6YY3LG6KCCSAiJcx8ns B7YMHbyWt7I2G6SkjNczbtgYroHOApmihm0LMgD9J1aD9tMiDX5JNpsE1944BAXyuVJ5 XB2bdcjruuXwppSSkOahNGGnmOcQeP4P3NoXtf1nNlPto6rGudq7bG70XFgg6ZgQl8+b ZhlihaGLN716kq4DkmAEEkgmSlS3RQq3b1aw/YQ4oqWwUsI2XGHWbLU+EuDcdUXT5tn4 qsWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h1QuQfa0AlBLlK+lh3SXPgl8VOAIvXOvZyydTehW8ec=; b=BuDlvZ+7kRmnPx0m3fy0ROfsbbzn9BRNo48H18xLCet7/ZMzm0bHK78jbhiVIAwPkc F1llztx+EVnV13I7pJb7QIOMGXrrnTonVRteAeiP4EM8PTPe1lzH6p4uLn26cwT5/SwG K5UqPpHPBj6mO8y7Ou416IB7/8cyQC1SD17RzgEC7rOBSqO4UDjDkaixZfF8o5Ylp7AV EMQWkV4jGlqJ4ju8QntLiYHBGjZmQ+AGk3ih+H38NUo7w/hRlfhWlSSBr7MJfnarsmz9 QHet58guRsLvA/JzE1Od71SxBtdssnhNEMbA7TghptosUF06TXJKbg4LDTjEoPynzIzN ZdZA== X-Gm-Message-State: AOAM533uLci06TaOxVHODFaKhP4/Q36w/P9LQwhRqc7cvsTf8cHHqVe3 FYPqmlQWPU8W+kMHqrFIONU= X-Google-Smtp-Source: ABdhPJxarL3aPQiMgRMk8JODPnNh5wb3Bb4G09CsPOZAMdsyeMcw/hNTOILb64gKI1xr8dUFndQ29g== X-Received: by 2002:aed:3b57:: with SMTP id q23mr1315880qte.150.1596596717840; Tue, 04 Aug 2020 20:05:17 -0700 (PDT) Received: from LeoBras.ibmuc.com (179-125-154-168.dynamic.desktop.com.br. [179.125.154.168]) by smtp.gmail.com with ESMTPSA id n4sm869946qtr.73.2020.08.04.20.05.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 20:05:17 -0700 (PDT) From: Leonardo Bras To: Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Leonardo Bras , Alexey Kardashevskiy , Thiago Jung Bauermann , Ram Pai , Brian King , Murilo Fossa Vicentini , David Dai Subject: [PATCH v5 3/4] powerpc/pseries/iommu: Move window-removing part of remove_ddw into remove_dma_window Date: Wed, 5 Aug 2020 00:04:54 -0300 Message-Id: <20200805030455.123024-4-leobras.c@gmail.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200805030455.123024-1-leobras.c@gmail.com> References: <20200805030455.123024-1-leobras.c@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Move the window-removing part of remove_ddw into a new function (remove_dma_window), so it can be used to remove other DMA windows. It's useful for removing DMA windows that don't create DIRECT64_PROPNAME property, like the default DMA window from the device, which uses "ibm,dma-window". Signed-off-by: Leonardo Bras Tested-by: David Dai Reviewed-by: Alexey Kardashevskiy --- arch/powerpc/platforms/pseries/iommu.c | 45 +++++++++++++++----------- 1 file changed, 27 insertions(+), 18 deletions(-) diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c index 1a933c4e8bba..4e33147825cc 100644 --- a/arch/powerpc/platforms/pseries/iommu.c +++ b/arch/powerpc/platforms/pseries/iommu.c @@ -781,25 +781,14 @@ static int __init disable_ddw_setup(char *str) early_param("disable_ddw", disable_ddw_setup); -static void remove_ddw(struct device_node *np, bool remove_prop) +static void remove_dma_window(struct device_node *np, u32 *ddw_avail, + struct property *win) { struct dynamic_dma_window_prop *dwp; - struct property *win64; - u32 ddw_avail[DDW_APPLICABLE_SIZE]; u64 liobn; - int ret = 0; - - ret = of_property_read_u32_array(np, "ibm,ddw-applicable", - &ddw_avail[0], DDW_APPLICABLE_SIZE); - - win64 = of_find_property(np, DIRECT64_PROPNAME, NULL); - if (!win64) - return; - - if (ret || win64->length < sizeof(*dwp)) - goto delprop; + int ret; - dwp = win64->value; + dwp = win->value; liobn = (u64)be32_to_cpu(dwp->liobn); /* clear the whole window, note the arg is in kernel pages */ @@ -821,10 +810,30 @@ static void remove_ddw(struct device_node *np, bool remove_prop) pr_debug("%pOF: successfully removed direct window: rtas returned " "%d to ibm,remove-pe-dma-window(%x) %llx\n", np, ret, ddw_avail[DDW_REMOVE_PE_DMA_WIN], liobn); +} + +static void remove_ddw(struct device_node *np, bool remove_prop) +{ + struct property *win; + u32 ddw_avail[DDW_APPLICABLE_SIZE]; + int ret = 0; + + ret = of_property_read_u32_array(np, "ibm,ddw-applicable", + &ddw_avail[0], DDW_APPLICABLE_SIZE); + if (ret) + return; + + win = of_find_property(np, DIRECT64_PROPNAME, NULL); + if (!win) + return; + + if (win->length >= sizeof(struct dynamic_dma_window_prop)) + remove_dma_window(np, ddw_avail, win); + + if (!remove_prop) + return; -delprop: - if (remove_prop) - ret = of_remove_property(np, win64); + ret = of_remove_property(np, win); if (ret) pr_warn("%pOF: failed to remove direct window property: %d\n", np, ret); From patchwork Wed Aug 5 03:04:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Leonardo_Br=C3=A1s?= X-Patchwork-Id: 1341069 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4BLxYG2Nr4z9s1x for ; Wed, 5 Aug 2020 13:14:10 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=SDr30cOr; dkim-atps=neutral Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4BLxYG1LxfzDqcq for ; Wed, 5 Aug 2020 13:14:10 +1000 (AEST) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::843; helo=mail-qt1-x843.google.com; envelope-from=leobras.c@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=SDr30cOr; dkim-atps=neutral Received: from mail-qt1-x843.google.com (mail-qt1-x843.google.com [IPv6:2607:f8b0:4864:20::843]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4BLxM93XFnzDqZp for ; Wed, 5 Aug 2020 13:05:25 +1000 (AEST) Received: by mail-qt1-x843.google.com with SMTP id h21so26282271qtp.11 for ; Tue, 04 Aug 2020 20:05:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FHVfOCrTYMIyp/yDwtpHYrzGhqVPdI+0ppzTBFsHaGA=; b=SDr30cOrPPrZYzCmHSo4oEPMGN3I5/rZsla9OZMOm9PntE/t1BJHvFMlXcRk6+mjZD Kncqs0o0wYrUJ6ull6q1DDNHqUKZdUCsDq997qoQTmh9OqIMEzjnNB2ZviJiCQ0CU7H5 MC49G2KZ59LLpJOCQvGQZugxqR60osZDVU4wjOycCAaOClvspfKizKF97OguRvHv29ms crJlF1rPmmMGQzqb1Y8xf+RmwLLP4dKeO5xvoHFyEDLqVcgX50dryHE6ryOz4/oBK6HA UqesV4EUjDyulEMRlGs4Kmwd4/bXPTQ090k4f6VucU6g9Qyb3fdHpAI8co3277hsPbCJ OMXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FHVfOCrTYMIyp/yDwtpHYrzGhqVPdI+0ppzTBFsHaGA=; b=Ddi32czYNLJsYRKOoPyni+UCSnYbx3UQhnos6qpZKiKd6CW0OOY/SnEtKd0KqqB5Ov Zp1mxnA0iTUoRw1YUN2jZXGcLp6o8/TtgC6ATJVkbCYiLzBmF8OdIKL0P0oBYYDnWI5N UqCk7yxwjGPlP+SeRNze09bDj91J+z7GjY0yWTX0OMZe41N2vEuk8sIt3sckyRL8BNZ/ N5yOiyLzvjwKtqhyDuDPycGvdfJ/bTv5A86q7UR+bBk1rLJdSVO2k8SFX2GvfPXudZ9w h9lU2RPuQgdmykXLo/qLm3+ynNfcFlsZSoWzNoCJfVNqdfDBZehIoK/uEOkOObS4lA5S N74g== X-Gm-Message-State: AOAM532S072RvCsVe/Jgbi2qJzRgfErI/WHdFosKiOxBCbkSp/CjnJ5B 7Ououq2yzHMTVmhh/v7tQ5U= X-Google-Smtp-Source: ABdhPJzNtehPgt1cATjgfHbP2JS1QBrBnY1eNrCAtHYIpCI11soyyzLkI8wwF29/EsW7yYjTEPmXdQ== X-Received: by 2002:ac8:4741:: with SMTP id k1mr1259519qtp.41.1596596722106; Tue, 04 Aug 2020 20:05:22 -0700 (PDT) Received: from LeoBras.ibmuc.com (179-125-154-168.dynamic.desktop.com.br. [179.125.154.168]) by smtp.gmail.com with ESMTPSA id n4sm869946qtr.73.2020.08.04.20.05.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Aug 2020 20:05:21 -0700 (PDT) From: Leonardo Bras To: Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , Leonardo Bras , Alexey Kardashevskiy , Thiago Jung Bauermann , Ram Pai , Brian King , Murilo Fossa Vicentini , David Dai Subject: [PATCH v5 4/4] powerpc/pseries/iommu: Allow bigger 64bit window by removing default DMA window Date: Wed, 5 Aug 2020 00:04:55 -0300 Message-Id: <20200805030455.123024-5-leobras.c@gmail.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20200805030455.123024-1-leobras.c@gmail.com> References: <20200805030455.123024-1-leobras.c@gmail.com> MIME-Version: 1.0 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" On LoPAR "DMA Window Manipulation Calls", it's recommended to remove the default DMA window for the device, before attempting to configure a DDW, in order to make the maximum resources available for the next DDW to be created. This is a requirement for using DDW on devices in which hypervisor allows only one DMA window. If setting up a new DDW fails anywhere after the removal of this default DMA window, it's needed to restore the default DMA window. For this, an implementation of ibm,reset-pe-dma-windows rtas call is needed: Platforms supporting the DDW option starting with LoPAR level 2.7 implement ibm,ddw-extensions. The first extension available (index 2) carries the token for ibm,reset-pe-dma-windows rtas call, which is used to restore the default DMA window for a device, if it has been deleted. It does so by resetting the TCE table allocation for the PE to it's boot time value, available in "ibm,dma-window" device tree node. Signed-off-by: Leonardo Bras Tested-by: David Dai Reviewed-by: Alexey Kardashevskiy --- arch/powerpc/platforms/pseries/iommu.c | 73 +++++++++++++++++++++++--- 1 file changed, 66 insertions(+), 7 deletions(-) diff --git a/arch/powerpc/platforms/pseries/iommu.c b/arch/powerpc/platforms/pseries/iommu.c index 4e33147825cc..e4198700ed1a 100644 --- a/arch/powerpc/platforms/pseries/iommu.c +++ b/arch/powerpc/platforms/pseries/iommu.c @@ -1066,6 +1066,38 @@ static phys_addr_t ddw_memory_hotplug_max(void) return max_addr; } +/* + * Platforms supporting the DDW option starting with LoPAR level 2.7 implement + * ibm,ddw-extensions, which carries the rtas token for + * ibm,reset-pe-dma-windows. + * That rtas-call can be used to restore the default DMA window for the device. + */ +static void reset_dma_window(struct pci_dev *dev, struct device_node *par_dn) +{ + int ret; + u32 cfg_addr, reset_dma_win; + u64 buid; + struct device_node *dn; + struct pci_dn *pdn; + + ret = ddw_read_ext(par_dn, DDW_EXT_RESET_DMA_WIN, &reset_dma_win); + if (ret) + return; + + dn = pci_device_to_OF_node(dev); + pdn = PCI_DN(dn); + buid = pdn->phb->buid; + cfg_addr = (pdn->busno << 16) | (pdn->devfn << 8); + + ret = rtas_call(reset_dma_win, 3, 1, NULL, cfg_addr, BUID_HI(buid), + BUID_LO(buid)); + if (ret) + dev_info(&dev->dev, + "ibm,reset-pe-dma-windows(%x) %x %x %x returned %d ", + reset_dma_win, cfg_addr, BUID_HI(buid), BUID_LO(buid), + ret); +} + /* * If the PE supports dynamic dma windows, and there is space for a table * that can map all pages in a linear offset, then setup such a table, @@ -1090,6 +1122,7 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) struct property *win64; struct dynamic_dma_window_prop *ddwprop; struct failed_ddw_pdn *fpdn; + bool default_win_removed = false; mutex_lock(&direct_window_init_mutex); @@ -1133,14 +1166,38 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) if (ret != 0) goto out_failed; + /* + * If there is no window available, remove the default DMA window, + * if it's present. This will make all the resources available to the + * new DDW window. + * If anything fails after this, we need to restore it, so also check + * for extensions presence. + */ if (query.windows_available == 0) { - /* - * no additional windows are available for this device. - * We might be able to reallocate the existing window, - * trading in for a larger page size. - */ - dev_dbg(&dev->dev, "no free dynamic windows"); - goto out_failed; + struct property *default_win; + int reset_win_ext; + + default_win = of_find_property(pdn, "ibm,dma-window", NULL); + if (!default_win) + goto out_failed; + + reset_win_ext = ddw_read_ext(pdn, DDW_EXT_RESET_DMA_WIN, NULL); + if (reset_win_ext) + goto out_failed; + + remove_dma_window(pdn, ddw_avail, default_win); + default_win_removed = true; + + /* Query again, to check if the window is available */ + ret = query_ddw(dev, ddw_avail, &query, pdn); + if (ret != 0) + goto out_failed; + + if (query.windows_available == 0) { + /* no windows are available for this device. */ + dev_dbg(&dev->dev, "no free dynamic windows"); + goto out_failed; + } } if (query.page_size & 4) { page_shift = 24; /* 16MB */ @@ -1231,6 +1288,8 @@ static u64 enable_ddw(struct pci_dev *dev, struct device_node *pdn) kfree(win64); out_failed: + if (default_win_removed) + reset_dma_window(dev, pdn); fpdn = kzalloc(sizeof(*fpdn), GFP_KERNEL); if (!fpdn)