From patchwork Tue Mar 3 05:17:45 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bharat Bhushan X-Patchwork-Id: 445549 X-Patchwork-Delegate: scottwood@freescale.com Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 902F7140187 for ; Tue, 3 Mar 2015 16:43:40 +1100 (AEDT) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 687981A03C8 for ; Tue, 3 Mar 2015 16:43:40 +1100 (AEDT) X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Received: from ozlabs.org (ozlabs.org [103.22.144.67]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 559621A0244 for ; Tue, 3 Mar 2015 16:42:22 +1100 (AEDT) Received: by ozlabs.org (Postfix) id 3ABAA140161; Tue, 3 Mar 2015 16:42:22 +1100 (AEDT) Delivered-To: linuxppc-dev@ozlabs.org X-Greylist: delayed 857 seconds by postgrey-1.35 at bilbo; Tue, 03 Mar 2015 16:42:20 AEDT Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0138.outbound.protection.outlook.com [157.56.110.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id EDFD1140146 for ; Tue, 3 Mar 2015 16:42:20 +1100 (AEDT) Received: from BY2PR03CA063.namprd03.prod.outlook.com (10.141.249.36) by BY1PR0301MB1269.namprd03.prod.outlook.com (25.161.205.143) with Microsoft SMTP Server (TLS) id 15.1.99.14; Tue, 3 Mar 2015 05:28:05 +0000 Received: from BN1AFFO11FD039.protection.gbl (2a01:111:f400:7c10::131) by BY2PR03CA063.outlook.office365.com (2a01:111:e400:2c5d::36) with Microsoft SMTP Server (TLS) id 15.1.99.9 via Frontend Transport; Tue, 3 Mar 2015 05:28:04 +0000 Received: from az84smr01.freescale.net (192.88.158.2) by BN1AFFO11FD039.mail.protection.outlook.com (10.58.52.243) with Microsoft SMTP Server (TLS) id 15.1.99.6 via Frontend Transport; Tue, 3 Mar 2015 05:28:00 +0000 Received: from kvm.ap.freescale.net (kvm.ap.freescale.net [10.232.14.24]) by az84smr01.freescale.net (8.14.3/8.14.0) with ESMTP id t235RrVW019197; Mon, 2 Mar 2015 22:27:58 -0700 From: Bharat Bhushan To: Subject: [PATCH 3/4 RFC] fsl/msi: Add MSI bank allocation for kernel owned devices Date: Tue, 3 Mar 2015 10:47:45 +0530 Message-ID: <1425359866-31049-3-git-send-email-Bharat.Bhushan@freescale.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1425359866-31049-1-git-send-email-Bharat.Bhushan@freescale.com> References: <1425359866-31049-1-git-send-email-Bharat.Bhushan@freescale.com> X-EOPAttributedMessage: 0 Received-SPF: Fail (protection.outlook.com: domain of freescale.com does not designate 192.88.158.2 as permitted sender) receiver=protection.outlook.com; client-ip=192.88.158.2; helo=az84smr01.freescale.net; Authentication-Results: spf=fail (sender IP is 192.88.158.2) smtp.mailfrom=Bharat.Bhushan@freescale.com; freescale.mail.onmicrosoft.com; dkim=none (message not signed) header.d=none; X-Forefront-Antispam-Report: CIP:192.88.158.2; CTRY:US; IPV:NLI; EFV:NLI; SFV:NSPM; SFS:(10019020)(6009001)(339900001)(189002)(199003)(2950100001)(77156002)(92566002)(450100001)(86362001)(575784001)(76176999)(77096005)(50986999)(106466001)(19580405001)(6806004)(19580395003)(36756003)(47776003)(105606002)(50466002)(85426001)(46102003)(87936001)(110136001)(48376002)(2371004)(62966003)(104016003)(229853001)(2351001)(50226001); DIR:OUT; SFP:1102; SCL:1; SRVR:BY1PR0301MB1269; H:az84smr01.freescale.net; FPR:; SPF:Fail; MLV:sfv; MX:1; A:1; LANG:en; MIME-Version: 1.0 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BY1PR0301MB1269; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(5005006); SRVR:BY1PR0301MB1269; BCL:0; PCL:0; RULEID:; SRVR:BY1PR0301MB1269; X-Forefront-PRVS: 0504F29D72 X-OriginatorOrg: freescale.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Mar 2015 05:28:00.5563 (UTC) X-MS-Exchange-CrossTenant-Id: 710a03f5-10f6-4d38-9ff4-a80b81da590d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=710a03f5-10f6-4d38-9ff4-a80b81da590d; Ip=[192.88.158.2] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0301MB1269 Cc: Bharat Bhushan , linuxppc-dev@ozlabs.org X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" With this patch a "context" can allocate a MSI bank and use the allocated MSI-bank for the devices in that "context". kernel/host "context" is "NULL", So all devices owned by kernel will share a MSI bank allocated with "context = NULL. This patch is in direction to have separate MSI bank for kernel context and userspace/VM context. We do not want two software context (kernel and VMs) to share a MSI bank for safe/reliable interrupts with full isolation. Follow up patch will add interface to allocate a MSI bank for userspace/VM context. NOTE: This RFC patch allows only one MSI bank to be allocated for kernel context. Which seems to be sufficient to me. But if we see this is limiting some real usecase scanerio then this limitation can be removed One issue which still need to addressed is when to free kernel context allocated MSI bank? Say all MSI capable devices are assigned to VM/userspace then there is no need to have any MSI bank reserved for kernel context. Signed-off-by: Bharat Bhushan --- arch/powerpc/sysdev/fsl_msi.c | 88 ++++++++++++++++++++++++++++++++++++++----- arch/powerpc/sysdev/fsl_msi.h | 4 ++ 2 files changed, 83 insertions(+), 9 deletions(-) diff --git a/arch/powerpc/sysdev/fsl_msi.c b/arch/powerpc/sysdev/fsl_msi.c index 32ba1e3..027aeeb 100644 --- a/arch/powerpc/sysdev/fsl_msi.c +++ b/arch/powerpc/sysdev/fsl_msi.c @@ -142,6 +142,79 @@ static void fsl_teardown_msi_irqs(struct pci_dev *pdev) return; } +/* + * Allocate a MSI Bank for the requested "context". + * NULL "context" means that this request is to allocate + * MSI bank for kernel owned devices. And currently we + * assume that one MSI bank is sufficient for kernel. + */ +static struct fsl_msi *fsl_msi_allocate_msi_bank(void *context) +{ + struct fsl_msi *msi_data; + + /* Kernel context (NULL) can reserve only one msi bank */ + if (!context) { + list_for_each_entry(msi_data, &msi_head, list) { + if ((msi_data->reserved == MSI_RESERVED) && + (msi_data->context == NULL)) + return NULL; + } + } + + list_for_each_entry(msi_data, &msi_head, list) { + if (msi_data->reserved == MSI_FREE) { + msi_data->reserved = MSI_RESERVED; + msi_data->context = context; + return msi_data; + } + } + + return NULL; +} + +/* FIXME: Assumption that host kernel will allocate only one MSI bank */ + __attribute__ ((unused)) static int fsl_msi_free_msi_bank(void *context) +{ + struct fsl_msi *msi_data; + + list_for_each_entry(msi_data, &msi_head, list) { + if ((msi_data->reserved == MSI_RESERVED) && + (msi_data->context == context)) { + msi_data->reserved = MSI_FREE; + msi_data->context = NULL; + return 0; + } + } + return -ENODEV; +} + +/* This API returns the allocated MSI bank of "context" + * to which "pdev" device belongs. + * All kernel owned devices have NULL context. All devices + * in same "context" will share the allocated MSI bank. + * + * Note: If no MSI bank allocated to kernel context then + * we allocate a MSI bank here. + */ +static struct fsl_msi *fsl_msi_get_reserved_msi_bank(struct pci_dev *pdev) +{ + struct fsl_msi *msi_data = NULL; + void *context = NULL; + + list_for_each_entry(msi_data, &msi_head, list) { + if ((msi_data->reserved == MSI_RESERVED) && + (msi_data->context == context)) + return msi_data; + } + + /* If no MSI bank allocated for kernel owned device, allocate one */ + msi_data = fsl_msi_allocate_msi_bank(NULL); + if (msi_data) + return msi_data; + + return NULL; +} + static void fsl_compose_msi_msg(struct pci_dev *pdev, int hwirq, struct msi_msg *msg, struct fsl_msi *fsl_msi_data) @@ -174,7 +247,7 @@ static int fsl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type) struct pci_controller *hose = pci_bus_to_host(pdev->bus); struct device_node *np; phandle phandle = 0; - int rc, hwirq = -ENOMEM; + int rc = -ENODEV, hwirq = -ENOMEM; unsigned int virq; struct msi_desc *entry; struct msi_msg msg; @@ -231,15 +304,12 @@ static int fsl_setup_msi_irqs(struct pci_dev *pdev, int nvec, int type) if (specific_msi_bank) { hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1); } else { - /* - * Loop over all the MSI devices until we find one that has an - * available interrupt. - */ - list_for_each_entry(msi_data, &msi_head, list) { - hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1); - if (hwirq >= 0) - break; + msi_data = fsl_msi_get_reserved_msi_bank(pdev); + if (!msi_data) { + dev_err(&pdev->dev, "No MSI Bank allocated\n"); + goto out_free; } + hwirq = msi_bitmap_alloc_hwirqs(&msi_data->bitmap, 1); } if (hwirq < 0) { diff --git a/arch/powerpc/sysdev/fsl_msi.h b/arch/powerpc/sysdev/fsl_msi.h index 9b0ab84..c69702b 100644 --- a/arch/powerpc/sysdev/fsl_msi.h +++ b/arch/powerpc/sysdev/fsl_msi.h @@ -46,6 +46,10 @@ struct fsl_msi { struct list_head list; /* support multiple MSI banks */ phandle phandle; +#define MSI_FREE 0 +#define MSI_RESERVED 1 + int reserved; + void *context; }; #endif /* _POWERPC_SYSDEV_FSL_MSI_H */