From patchwork Thu Aug 27 21:02:10 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ron Mercer X-Patchwork-Id: 32309 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@bilbo.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from ozlabs.org (ozlabs.org [203.10.76.45]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mx.ozlabs.org", Issuer "CA Cert Signing Authority" (verified OK)) by bilbo.ozlabs.org (Postfix) with ESMTPS id C08FAB7B99 for ; Fri, 28 Aug 2009 07:08:03 +1000 (EST) Received: by ozlabs.org (Postfix) id B379FDDD1B; Fri, 28 Aug 2009 07:08:03 +1000 (EST) Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by ozlabs.org (Postfix) with ESMTP id 47582DDD01 for ; Fri, 28 Aug 2009 07:08:03 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752888AbZH0VHz (ORCPT ); Thu, 27 Aug 2009 17:07:55 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1752814AbZH0VHy (ORCPT ); Thu, 27 Aug 2009 17:07:54 -0400 Received: from avexch1.qlogic.com ([198.70.193.115]:40659 "EHLO avexch1.qlogic.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752781AbZH0VHw (ORCPT ); Thu, 27 Aug 2009 17:07:52 -0400 Received: from linux-ox1b.qlogic.com ([172.17.161.157]) by avexch1.qlogic.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 27 Aug 2009 14:07:31 -0700 Received: by linux-ox1b.qlogic.com (Postfix, from userid 1000) id 0D2EC2C6AD; Thu, 27 Aug 2009 14:02:11 -0700 (PDT) From: Ron Mercer To: davem@davemloft.net Cc: netdev@vger.kernel.org, ron.mercer@qlogic.com Subject: [net-next PATCH 2/3] qlge: Allow running MSIx with fewer vectors. Date: Thu, 27 Aug 2009 14:02:10 -0700 Message-Id: <1251406931-15107-3-git-send-email-ron.mercer@qlogic.com> X-Mailer: git-send-email 1.6.0.2 In-Reply-To: <1251406931-15107-1-git-send-email-ron.mercer@qlogic.com> References: <1251406931-15107-1-git-send-email-ron.mercer@qlogic.com> X-OriginalArrivalTime: 27 Aug 2009 21:07:31.0832 (UTC) FILETIME=[64645F80:01CA275A] Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Currently we downshift to MSI/Legacy if we don't get enough vectors for cpu_count RSS rings plus cpu_count TX completion rings. This patch allows running MSIX with the vector count that the platform provides. Signed-off-by: Ron Mercer --- drivers/net/qlge/qlge_main.c | 85 +++++++++++++++++++----------------------- 1 files changed, 38 insertions(+), 47 deletions(-) diff --git a/drivers/net/qlge/qlge_main.c b/drivers/net/qlge/qlge_main.c index 89ea9c7..0cbda4d 100644 --- a/drivers/net/qlge/qlge_main.c +++ b/drivers/net/qlge/qlge_main.c @@ -2815,17 +2815,20 @@ static void ql_disable_msix(struct ql_adapter *qdev) } } +/* We start by trying to get the number of vectors + * stored in qdev->intr_count. If we don't get that + * many then we reduce the count and try again. + */ static void ql_enable_msix(struct ql_adapter *qdev) { - int i; + int i, err; - qdev->intr_count = 1; /* Get the MSIX vectors. */ if (irq_type == MSIX_IRQ) { /* Try to alloc space for the msix struct, * if it fails then go to MSI/legacy. */ - qdev->msi_x_entry = kcalloc(qdev->rx_ring_count, + qdev->msi_x_entry = kcalloc(qdev->intr_count, sizeof(struct msix_entry), GFP_KERNEL); if (!qdev->msi_x_entry) { @@ -2833,26 +2836,36 @@ static void ql_enable_msix(struct ql_adapter *qdev) goto msi; } - for (i = 0; i < qdev->rx_ring_count; i++) + for (i = 0; i < qdev->intr_count; i++) qdev->msi_x_entry[i].entry = i; - if (!pci_enable_msix - (qdev->pdev, qdev->msi_x_entry, qdev->rx_ring_count)) { - set_bit(QL_MSIX_ENABLED, &qdev->flags); - qdev->intr_count = qdev->rx_ring_count; - QPRINTK(qdev, IFUP, DEBUG, - "MSI-X Enabled, got %d vectors.\n", - qdev->intr_count); - return; - } else { + /* Loop to get our vectors. We start with + * what we want and settle for what we get. + */ + do { + err = pci_enable_msix(qdev->pdev, + qdev->msi_x_entry, qdev->intr_count); + if (err > 0) + qdev->intr_count = err; + } while (err > 0); + + if (err < 0) { kfree(qdev->msi_x_entry); qdev->msi_x_entry = NULL; QPRINTK(qdev, IFUP, WARNING, "MSI-X Enable failed, trying MSI.\n"); + qdev->intr_count = 1; irq_type = MSI_IRQ; + } else if (err == 0) { + set_bit(QL_MSIX_ENABLED, &qdev->flags); + QPRINTK(qdev, IFUP, INFO, + "MSI-X Enabled, got %d vectors.\n", + qdev->intr_count); + return; } } msi: + qdev->intr_count = 1; if (irq_type == MSI_IRQ) { if (!pci_enable_msi(qdev->pdev)) { set_bit(QL_MSI_ENABLED, &qdev->flags); @@ -2876,8 +2889,6 @@ static void ql_resolve_queues_to_irqs(struct ql_adapter *qdev) int i = 0; struct intr_context *intr_context = &qdev->intr_context[0]; - ql_enable_msix(qdev); - if (likely(test_bit(QL_MSIX_ENABLED, &qdev->flags))) { /* Each rx_ring has it's * own intr_context since we have separate @@ -3438,40 +3449,20 @@ static int ql_configure_rings(struct ql_adapter *qdev) int i; struct rx_ring *rx_ring; struct tx_ring *tx_ring; - int cpu_cnt = num_online_cpus(); - - /* - * For each processor present we allocate one - * rx_ring for outbound completions, and one - * rx_ring for inbound completions. Plus there is - * always the one default queue. For the CPU - * counts we end up with the following rx_rings: - * rx_ring count = - * one default queue + - * (CPU count * outbound completion rx_ring) + - * (CPU count * inbound (RSS) completion rx_ring) - * To keep it simple we limit the total number of - * queues to < 32, so we truncate CPU to 8. - * This limitation can be removed when requested. + int cpu_cnt = min(MAX_CPUS, (int)num_online_cpus()); + + /* In a perfect world we have one RSS ring for each CPU + * and each has it's own vector. To do that we ask for + * cpu_cnt vectors. ql_enable_msix() will adjust the + * vector count to what we actually get. We then + * allocate an RSS ring for each. + * Essentially, we are doing min(cpu_count, msix_vector_count). */ - - if (cpu_cnt > MAX_CPUS) - cpu_cnt = MAX_CPUS; - - /* - * rx_ring[0] is always the default queue. - */ - /* Allocate outbound completion ring for each CPU. */ + qdev->intr_count = cpu_cnt; + ql_enable_msix(qdev); + /* Adjust the RSS ring count to the actual vector count. */ + qdev->rss_ring_count = qdev->intr_count; qdev->tx_ring_count = cpu_cnt; - /* Allocate inbound completion (RSS) ring for each CPU. */ - qdev->rss_ring_count = cpu_cnt; - /* - * qdev->rx_ring_count: - * Total number of rx_rings. This includes the one - * default queue, a number of outbound completion - * handler rx_rings, and the number of inbound - * completion handler rx_rings. - */ qdev->rx_ring_count = qdev->tx_ring_count + qdev->rss_ring_count; for (i = 0; i < qdev->tx_ring_count; i++) {