From patchwork Fri Jul 31 20:15:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shannon Nelson X-Patchwork-Id: 1339587 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=pensando.io Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=pensando.io header.i=@pensando.io header.a=rsa-sha256 header.s=google header.b=ZbojopEt; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BJJSM5lsRz9sRK for ; Sat, 1 Aug 2020 06:15:47 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727066AbgGaUPr (ORCPT ); Fri, 31 Jul 2020 16:15:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725767AbgGaUPq (ORCPT ); Fri, 31 Jul 2020 16:15:46 -0400 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E14B7C061574 for ; Fri, 31 Jul 2020 13:15:45 -0700 (PDT) Received: by mail-pl1-x643.google.com with SMTP id b9so17931547plx.6 for ; Fri, 31 Jul 2020 13:15:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pensando.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ji45wpS231u5wkVoxPXZFgb0WawlUJO4xkSAUebuYLA=; b=ZbojopEtoUGyayK7VxO6fOUgReSKzXUAFG2+XY/VVILYezrpcq8khMiaZ0A7HsAg7+ wl0Cvi3nFqk1f8T9y2tl+qLSCzI9wzTZaVlZ6KgZZV1CH41fLVbUievfU3eqJtib7AL2 5dgOOgKMPzg5VQtcQ9tTQle43ppbkgxhENl2l3W601iE8Wt8X+KR4LNjF4WdSVRwdQ7u YVcdrl3hnaQOBRvqjHue55Fc/QaIp0fanHiRvU62eO8dC+Cb4ZptqgspLQonmIrlpNSx 9StzHuCFxm2AqF5PQ8be766GxHSc64n0rV9KBffHoK5JOVzN5ELIhVbh9WPiDlR1R1Th R3Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ji45wpS231u5wkVoxPXZFgb0WawlUJO4xkSAUebuYLA=; b=s1fS2k4iDqvNYFoZMqX6os887O9TdU4U1fFOLxBY9+EY5A3P859nNSCDcd9tXKWsOn aF5gOE6PlRPhdYcr72YAz5/Ko6HAACJ6UU3po4rnrdZkjApaFlWkJbonrN9gcu50I/3e sUZhXQmfVT7NB/4QMv++TFVYGs8g/WAXcAL23kG+V66eAAm0yCChMwdN83jseTZozlOc 9oZmOqaYYOsRItEe//AXVfCDbgArZxWPneqXzrxrocpxqLxCJzGO2IJ/bWFpmKKTGYCe 95oInXv9FqTMWrE+3bTIBoZJWVMIkHJ+EL28lBdCgnTnMVXeXTqI4O/MNJPLy5cJnxMJ cJJg== X-Gm-Message-State: AOAM531u6YF5zTAe+IbThMTXAe0XBLw+gL/KA1sL3zS+b4cedbbnSSLH RoxV0KnmECtPxkEbfujwQKJUfp2ym/Y= X-Google-Smtp-Source: ABdhPJzOkZocrJE4zmPog3mGHXj3vDHlXES+p//OvllFHFMcAVTPVI34ZXwHGpAQWGzPLejmoM4hcw== X-Received: by 2002:a17:902:7787:: with SMTP id o7mr5034307pll.327.1596226544813; Fri, 31 Jul 2020 13:15:44 -0700 (PDT) Received: from driver-dev1.pensando.io ([12.226.153.42]) by smtp.gmail.com with ESMTPSA id h1sm11470513pgn.41.2020.07.31.13.15.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 31 Jul 2020 13:15:44 -0700 (PDT) From: Shannon Nelson To: netdev@vger.kernel.org, davem@davemloft.net Cc: Shannon Nelson Subject: [PATCH v3 net-next 1/3] ionic: use fewer firmware doorbells on rx fill Date: Fri, 31 Jul 2020 13:15:34 -0700 Message-Id: <20200731201536.18246-2-snelson@pensando.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200731201536.18246-1-snelson@pensando.io> References: <20200731201536.18246-1-snelson@pensando.io> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org We really don't need to hit the Rx queue doorbell so many times, we can wait to the end and cause a little less thrash. Signed-off-by: Shannon Nelson --- drivers/net/ethernet/pensando/ionic/ionic_txrx.c | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 85eb8f276a37..e660cd66f9a8 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -331,9 +331,6 @@ static void ionic_rx_page_free(struct ionic_queue *q, struct page *page, __free_page(page); } -#define IONIC_RX_RING_DOORBELL_STRIDE ((1 << 5) - 1) -#define IONIC_RX_RING_HEAD_BUF_SZ 2048 - void ionic_rx_fill(struct ionic_queue *q) { struct net_device *netdev = q->lif->netdev; @@ -345,7 +342,6 @@ void ionic_rx_fill(struct ionic_queue *q) unsigned int remain_len; unsigned int seg_len; unsigned int nfrags; - bool ring_doorbell; unsigned int i, j; unsigned int len; @@ -360,9 +356,7 @@ void ionic_rx_fill(struct ionic_queue *q) page_info = &desc_info->pages[0]; if (page_info->page) { /* recycle the buffer */ - ring_doorbell = ((q->head->index + 1) & - IONIC_RX_RING_DOORBELL_STRIDE) == 0; - ionic_rxq_post(q, ring_doorbell, ionic_rx_clean, NULL); + ionic_rxq_post(q, false, ionic_rx_clean, NULL); continue; } @@ -401,10 +395,11 @@ void ionic_rx_fill(struct ionic_queue *q) page_info++; } - ring_doorbell = ((q->head->index + 1) & - IONIC_RX_RING_DOORBELL_STRIDE) == 0; - ionic_rxq_post(q, ring_doorbell, ionic_rx_clean, NULL); + ionic_rxq_post(q, false, ionic_rx_clean, NULL); } + + ionic_dbell_ring(q->lif->kern_dbpage, q->hw_type, + q->dbval | q->head->index); } static void ionic_rx_fill_cb(void *arg) From patchwork Fri Jul 31 20:15:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shannon Nelson X-Patchwork-Id: 1339589 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=pensando.io Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=pensando.io header.i=@pensando.io header.a=rsa-sha256 header.s=google header.b=turqaxdc; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BJJSS6M5fz9sRK for ; Sat, 1 Aug 2020 06:15:52 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727772AbgGaUPu (ORCPT ); Fri, 31 Jul 2020 16:15:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725767AbgGaUPs (ORCPT ); Fri, 31 Jul 2020 16:15:48 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C1E7C061574 for ; Fri, 31 Jul 2020 13:15:48 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id k13so10117583plk.13 for ; Fri, 31 Jul 2020 13:15:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pensando.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dbrGQk5+OOKy5ircUxKuyjvq+gN8z/VEf1m73BmaeEo=; b=turqaxdcKMxT5f+Km7GEpEU4HEyAs0hDFTp3vcotErrRGk/PqVIH5eZt8HyhXME+U6 UIkh1OkXIya9uVCTP2SVV6gGAwA85xArB1Qi0AaPg/ibkRX0gHNW6zrN4LTGmUsL68FN WdZooHU57k4voyDtcpyiVlc1SvspAgM3cy596Bmj3HmL5pNSK4CB/lvjz1rBy/7Xi8lw Je0bIf7lD3VNiysPgpm7fmUcbW4j0+99ypxaUla+7i8d+nE8ACkjE/Ed3NxWJrmLJQIG nHB45Yr2GgbpLRh3mHYEIgS6Q7EUmFbtHEiVUgsg1g6q1zcignwLgLFFA11FHvUsXM4o jHpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dbrGQk5+OOKy5ircUxKuyjvq+gN8z/VEf1m73BmaeEo=; b=VPHL/rcmmtKSwetXcMs2nvLTmytD5tgzTvLm3UxHQ9gCZYpHEFcp2+rauvE8YowM05 l87KZrztiWRZL9tHtg5j4qG+OTsG+IXh8DbOFiTzEQUJ8QcrMkULr7n+UNFMMx8qkD5Z mW7xt6lo0vlb5jyTYhGVXSPYAEjLxB6gAHsJ1ZVfXAkD1H5TModj7vKNL8OatzvuCqoX jzYbeecFXOmDTZcRv0T+bV5+UiDD9I/fa+961Ah+b00EPDOEwnJb+VEcJAxN7vMv7FNN jPstxLkUANR9mxJ5+ftfDcMzTNnSWZi/WottMJ8vh+IRlz+esYDt9AcW7IHvQCOqNuuk mMww== X-Gm-Message-State: AOAM533RKd6OZ1slnZX5E04BZwilJcwG4L2/jt8pglq3dFd0lpddVGLH PN4c6+5Fdl0s8SIZJfUnT+fNN0mPjvI= X-Google-Smtp-Source: ABdhPJz9XJyBFngHalda97Bah9GINfdqXoTMvds4WpG8BwRao8/EssIlq5QuckJt7wxtgY2js/Awfg== X-Received: by 2002:a17:90a:498b:: with SMTP id d11mr5527014pjh.179.1596226547190; Fri, 31 Jul 2020 13:15:47 -0700 (PDT) Received: from driver-dev1.pensando.io ([12.226.153.42]) by smtp.gmail.com with ESMTPSA id h1sm11470513pgn.41.2020.07.31.13.15.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 31 Jul 2020 13:15:46 -0700 (PDT) From: Shannon Nelson To: netdev@vger.kernel.org, davem@davemloft.net Cc: Shannon Nelson Subject: [PATCH v3 net-next 2/3] ionic: tx separate servicing Date: Fri, 31 Jul 2020 13:15:35 -0700 Message-Id: <20200731201536.18246-3-snelson@pensando.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200731201536.18246-1-snelson@pensando.io> References: <20200731201536.18246-1-snelson@pensando.io> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org We give the tx clean path its own budget and service routine in order to give a little more leeway to be more aggressive, and in preparation for coming changes. We've found this gives us a little better performance in some packet processing scenarios without hurting other scenarios. Signed-off-by: Shannon Nelson --- .../net/ethernet/pensando/ionic/ionic_lif.c | 1 + .../net/ethernet/pensando/ionic/ionic_lif.h | 2 + .../net/ethernet/pensando/ionic/ionic_txrx.c | 106 +++++++++--------- 3 files changed, 53 insertions(+), 56 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 728dd6429d80..b228362363ca 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -2063,6 +2063,7 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index lif->index = index; lif->ntxq_descs = IONIC_DEF_TXRX_DESC; lif->nrxq_descs = IONIC_DEF_TXRX_DESC; + lif->tx_budget = IONIC_TX_BUDGET_DEFAULT; /* Convert the default coalesce value to actual hw resolution */ lif->rx_coalesce_usecs = IONIC_ITR_COAL_USEC_DEFAULT; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index edad17d7aeeb..13f173d83970 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -13,6 +13,7 @@ #define IONIC_MAX_NUM_NAPI_CNTR (NAPI_POLL_WEIGHT + 1) #define IONIC_MAX_NUM_SG_CNTR (IONIC_TX_MAX_SG_ELEMS + 1) #define IONIC_RX_COPYBREAK_DEFAULT 256 +#define IONIC_TX_BUDGET_DEFAULT 256 struct ionic_tx_stats { u64 dma_map_err; @@ -176,6 +177,7 @@ struct ionic_lif { unsigned int ntxq_descs; unsigned int nrxq_descs; u32 rx_copybreak; + u32 tx_budget; unsigned int rx_mode; u64 hw_features; bool mc_overflow; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index e660cd66f9a8..766f4595895c 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -15,6 +15,10 @@ static void ionic_rx_clean(struct ionic_queue *q, struct ionic_cq_info *cq_info, void *cb_arg); +static bool ionic_rx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info); + +static bool ionic_tx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info); + static inline void ionic_txq_post(struct ionic_queue *q, bool ring_dbell, ionic_desc_cb cb_func, void *cb_arg) { @@ -249,29 +253,13 @@ static bool ionic_rx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info) return true; } -static u32 ionic_rx_walk_cq(struct ionic_cq *rxcq, u32 limit) -{ - u32 work_done = 0; - - while (ionic_rx_service(rxcq, rxcq->tail)) { - if (rxcq->tail->last) - rxcq->done_color = !rxcq->done_color; - rxcq->tail = rxcq->tail->next; - DEBUG_STATS_CQE_CNT(rxcq); - - if (++work_done >= limit) - break; - } - - return work_done; -} - void ionic_rx_flush(struct ionic_cq *cq) { struct ionic_dev *idev = &cq->lif->ionic->idev; u32 work_done; - work_done = ionic_rx_walk_cq(cq, cq->num_descs); + work_done = ionic_cq_service(cq, cq->num_descs, + ionic_rx_service, NULL, NULL); if (work_done) ionic_intr_credits(idev->intr_ctrl, cq->bound_intr->index, @@ -439,32 +427,42 @@ int ionic_rx_napi(struct napi_struct *napi, int budget) struct ionic_dev *idev; struct ionic_lif *lif; struct ionic_cq *txcq; + u32 rx_work_done = 0; + u32 tx_work_done = 0; u32 work_done = 0; u32 flags = 0; + bool unmask; lif = rxcq->bound_q->lif; idev = &lif->ionic->idev; txcq = &lif->txqcqs[qi].qcq->cq; - ionic_tx_flush(txcq); - - work_done = ionic_rx_walk_cq(rxcq, budget); + tx_work_done = ionic_cq_service(txcq, lif->tx_budget, + ionic_tx_service, NULL, NULL); - if (work_done) + rx_work_done = ionic_cq_service(rxcq, budget, + ionic_rx_service, NULL, NULL); + if (rx_work_done) ionic_rx_fill_cb(rxcq->bound_q); - if (work_done < budget && napi_complete_done(napi, work_done)) { + unmask = (rx_work_done < budget) && (tx_work_done < lif->tx_budget); + + if (unmask && napi_complete_done(napi, rx_work_done)) { flags |= IONIC_INTR_CRED_UNMASK; DEBUG_STATS_INTR_REARM(rxcq->bound_intr); + work_done = rx_work_done; + } else { + work_done = budget; } if (work_done || flags) { flags |= IONIC_INTR_CRED_RESET_COALESCE; ionic_intr_credits(idev->intr_ctrl, rxcq->bound_intr->index, - work_done, flags); + tx_work_done + rx_work_done, flags); } - DEBUG_STATS_NAPI_POLL(qcq, work_done); + DEBUG_STATS_NAPI_POLL(qcq, rx_work_done); + DEBUG_STATS_NAPI_POLL(qcq, tx_work_done); return work_done; } @@ -552,43 +550,39 @@ static void ionic_tx_clean(struct ionic_queue *q, } } -void ionic_tx_flush(struct ionic_cq *cq) +static bool ionic_tx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info) { - struct ionic_txq_comp *comp = cq->tail->cq_desc; - struct ionic_dev *idev = &cq->lif->ionic->idev; + struct ionic_txq_comp *comp = cq_info->cq_desc; struct ionic_queue *q = cq->bound_q; struct ionic_desc_info *desc_info; - unsigned int work_done = 0; - - /* walk the completed cq entries */ - while (work_done < cq->num_descs && - color_match(comp->color, cq->done_color)) { - - /* clean the related q entries, there could be - * several q entries completed for each cq completion - */ - do { - desc_info = q->tail; - q->tail = desc_info->next; - ionic_tx_clean(q, desc_info, cq->tail, - desc_info->cb_arg); - desc_info->cb = NULL; - desc_info->cb_arg = NULL; - } while (desc_info->index != le16_to_cpu(comp->comp_index)); - - if (cq->tail->last) - cq->done_color = !cq->done_color; - - cq->tail = cq->tail->next; - comp = cq->tail->cq_desc; - DEBUG_STATS_CQE_CNT(cq); - - work_done++; - } + if (!color_match(comp->color, cq->done_color)) + return false; + + /* clean the related q entries, there could be + * several q entries completed for each cq completion + */ + do { + desc_info = q->tail; + q->tail = desc_info->next; + ionic_tx_clean(q, desc_info, cq->tail, desc_info->cb_arg); + desc_info->cb = NULL; + desc_info->cb_arg = NULL; + } while (desc_info->index != le16_to_cpu(comp->comp_index)); + + return true; +} + +void ionic_tx_flush(struct ionic_cq *cq) +{ + struct ionic_dev *idev = &cq->lif->ionic->idev; + u32 work_done; + + work_done = ionic_cq_service(cq, cq->num_descs, + ionic_tx_service, NULL, NULL); if (work_done) ionic_intr_credits(idev->intr_ctrl, cq->bound_intr->index, - work_done, 0); + work_done, IONIC_INTR_CRED_RESET_COALESCE); } void ionic_tx_empty(struct ionic_queue *q) From patchwork Fri Jul 31 20:15:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shannon Nelson X-Patchwork-Id: 1339588 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=pensando.io Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=pensando.io header.i=@pensando.io header.a=rsa-sha256 header.s=google header.b=hHHJ4yOO; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4BJJSQ5XyVz9sRK for ; Sat, 1 Aug 2020 06:15:50 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727823AbgGaUPu (ORCPT ); Fri, 31 Jul 2020 16:15:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727074AbgGaUPt (ORCPT ); Fri, 31 Jul 2020 16:15:49 -0400 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8DFCEC06174A for ; Fri, 31 Jul 2020 13:15:49 -0700 (PDT) Received: by mail-pl1-x644.google.com with SMTP id b9so17931617plx.6 for ; Fri, 31 Jul 2020 13:15:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pensando.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TS9xdhIgyJE3CZ/6Dcgk1WwtLUDZaGlRLhPQx47Z6kw=; b=hHHJ4yOOVPHWiONvyvckOAyp4roC/DJZ9i0xjyPUr/jCDKWHPNkyF29cIEPBErmZMo QlPyGSv02srMQc1/w9FswuFBTfd961i8XIgVZLU4AbqDfgw9ZXu9nHDMoabJgHDOlRbX voADCF3r+5pgODy6VFy4OJDhi7mlOH1GLh2/VufK2zuJPi931wIFJYRbfHZ4Mr4OXyG8 Usb+UhecsdL3DBv2j3CDXwUgwSCztUDP1CRuZ2DbAMHh9AZWMsDwPYsdOZBzOHFV7Vql dzK50deLDXF3ueRQtJEIcn5BLcsjaoko1UJBEe0opTY+aDZnuzV9GEHIafufObpwvbBb n/OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TS9xdhIgyJE3CZ/6Dcgk1WwtLUDZaGlRLhPQx47Z6kw=; b=HX7xcHIwJUSqgS8txhWTVpGbRDiIyqxayjLSoJcuWv6mcCwUMQmvgzO3pgEUmqc32J eWW+IkGD3Z8Ng/5uaR83Hs0rsgAXF6NnAF1Y01qblABibrLTfRvehmnZpl0tCcrZInpl DXfi+aBaoMyfHcslhpiOTSrWOcfbxEARi9ltvv1qHv22Y0ek+ot08Ie5a2lri4jw5wVE 05uAlHjD18h/7J2outTTg80MVtC2BKxEKNzMa4n+dF3fqn1nki/kYMF6xfY84Cltgbbb M4VwgWHBHd4TyxCoCSz2sTLycimY3+rtm2LzJYnF8RwMGfq3/4dWjjvzYn6NkMBbTUh6 K+lQ== X-Gm-Message-State: AOAM5306j7BplToUw9O/DaODAHj7fvzX6BeW7n9DMlGWS/jY5DpzMvtF qXn9iy5ykzVPYZ09xl8iQDnSbuF4/Nk= X-Google-Smtp-Source: ABdhPJzWeywjJDZJkaErECDdIDJNREFbvu/nWBLAd9tyvqKx3YbxQkRYpO9VRqw8sSwq7PZyzWoRGg== X-Received: by 2002:a17:90b:4385:: with SMTP id in5mr5212116pjb.16.1596226548347; Fri, 31 Jul 2020 13:15:48 -0700 (PDT) Received: from driver-dev1.pensando.io ([12.226.153.42]) by smtp.gmail.com with ESMTPSA id h1sm11470513pgn.41.2020.07.31.13.15.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 31 Jul 2020 13:15:47 -0700 (PDT) From: Shannon Nelson To: netdev@vger.kernel.org, davem@davemloft.net Cc: Shannon Nelson Subject: [PATCH v3 net-next 3/3] ionic: separate interrupt for Tx and Rx Date: Fri, 31 Jul 2020 13:15:36 -0700 Message-Id: <20200731201536.18246-4-snelson@pensando.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200731201536.18246-1-snelson@pensando.io> References: <20200731201536.18246-1-snelson@pensando.io> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add the capability to split the Tx queues onto their own interrupts with their own napi contexts. This gives the opportunity for more direct control of Tx interrupt handling, such as CPU affinity and interrupt coalescing, useful for some traffic loads. v2: use ethtool -L, not a vendor specific priv-flag v3: simplify logging, drop unnecessary "no-change" tests Reviewed-by: Jakub Kicinski Signed-off-by: Shannon Nelson --- .../ethernet/pensando/ionic/ionic_ethtool.c | 96 +++++++++++++++---- .../net/ethernet/pensando/ionic/ionic_lif.c | 41 ++++++-- .../net/ethernet/pensando/ionic/ionic_lif.h | 3 + .../net/ethernet/pensando/ionic/ionic_txrx.c | 67 +++++++++++++ .../net/ethernet/pensando/ionic/ionic_txrx.h | 2 + 5 files changed, 182 insertions(+), 27 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c index 095561924bdc..3c57c331729f 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c @@ -403,8 +403,7 @@ static int ionic_get_coalesce(struct net_device *netdev, { struct ionic_lif *lif = netdev_priv(netdev); - /* Tx uses Rx interrupt */ - coalesce->tx_coalesce_usecs = lif->rx_coalesce_usecs; + coalesce->tx_coalesce_usecs = lif->tx_coalesce_usecs; coalesce->rx_coalesce_usecs = lif->rx_coalesce_usecs; return 0; @@ -417,7 +416,8 @@ static int ionic_set_coalesce(struct net_device *netdev, struct ionic_identity *ident; struct ionic_qcq *qcq; unsigned int i; - u32 coal; + u32 rx_coal; + u32 tx_coal; ident = &lif->ionic->ident; if (ident->dev.intr_coal_div == 0) { @@ -426,26 +426,31 @@ static int ionic_set_coalesce(struct net_device *netdev, return -EIO; } - /* Tx uses Rx interrupt, so only change Rx */ - if (coalesce->tx_coalesce_usecs != lif->rx_coalesce_usecs) { + /* Tx normally shares Rx interrupt, so only change Rx */ + if (!test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state) && + coalesce->tx_coalesce_usecs != lif->rx_coalesce_usecs) { netdev_warn(netdev, "only the rx-usecs can be changed\n"); return -EINVAL; } - /* Convert the usec request to a HW useable value. If they asked + /* Convert the usec request to a HW usable value. If they asked * for non-zero and it resolved to zero, bump it up */ - coal = ionic_coal_usec_to_hw(lif->ionic, coalesce->rx_coalesce_usecs); - if (!coal && coalesce->rx_coalesce_usecs) - coal = 1; - - if (coal > IONIC_INTR_CTRL_COAL_MAX) + rx_coal = ionic_coal_usec_to_hw(lif->ionic, coalesce->rx_coalesce_usecs); + if (!rx_coal && coalesce->rx_coalesce_usecs) + rx_coal = 1; + tx_coal = ionic_coal_usec_to_hw(lif->ionic, coalesce->tx_coalesce_usecs); + if (!tx_coal && coalesce->tx_coalesce_usecs) + tx_coal = 1; + + if (rx_coal > IONIC_INTR_CTRL_COAL_MAX || + tx_coal > IONIC_INTR_CTRL_COAL_MAX) return -ERANGE; - /* Save the new value */ + /* Save the new values */ lif->rx_coalesce_usecs = coalesce->rx_coalesce_usecs; - if (coal != lif->rx_coalesce_hw) { - lif->rx_coalesce_hw = coal; + if (rx_coal != lif->rx_coalesce_hw) { + lif->rx_coalesce_hw = rx_coal; if (test_bit(IONIC_LIF_F_UP, lif->state)) { for (i = 0; i < lif->nxqs; i++) { @@ -457,6 +462,23 @@ static int ionic_set_coalesce(struct net_device *netdev, } } + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) + lif->tx_coalesce_usecs = coalesce->tx_coalesce_usecs; + else + lif->tx_coalesce_usecs = coalesce->rx_coalesce_usecs; + if (tx_coal != lif->tx_coalesce_hw) { + lif->tx_coalesce_hw = tx_coal; + + if (test_bit(IONIC_LIF_F_UP, lif->state)) { + for (i = 0; i < lif->nxqs; i++) { + qcq = lif->txqcqs[i].qcq; + ionic_intr_coal_init(lif->ionic->idev.intr_ctrl, + qcq->intr.index, + lif->tx_coalesce_hw); + } + } + } + return 0; } @@ -510,29 +532,63 @@ static void ionic_get_channels(struct net_device *netdev, /* report maximum channels */ ch->max_combined = lif->ionic->ntxqs_per_lif; + ch->max_rx = lif->ionic->ntxqs_per_lif / 2; + ch->max_tx = lif->ionic->ntxqs_per_lif / 2; /* report current channels */ - ch->combined_count = lif->nxqs; + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) { + ch->rx_count = lif->nxqs; + ch->tx_count = lif->nxqs; + } else { + ch->combined_count = lif->nxqs; + } } static void ionic_set_queuecount(struct ionic_lif *lif, void *arg) { struct ethtool_channels *ch = arg; - lif->nxqs = ch->combined_count; + if (ch->combined_count) { + lif->nxqs = ch->combined_count; + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) { + clear_bit(IONIC_LIF_F_SPLIT_INTR, lif->state); + lif->tx_coalesce_usecs = lif->rx_coalesce_usecs; + lif->tx_coalesce_hw = lif->rx_coalesce_hw; + netdev_info(lif->netdev, "Sharing queue interrupts\n"); + } + } else { + lif->nxqs = ch->rx_count; + if (!test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) { + set_bit(IONIC_LIF_F_SPLIT_INTR, lif->state); + netdev_info(lif->netdev, "Splitting queue interrupts\n"); + } + } } static int ionic_set_channels(struct net_device *netdev, struct ethtool_channels *ch) { struct ionic_lif *lif = netdev_priv(netdev); + int new_cnt; - if (!ch->combined_count || ch->other_count || - ch->rx_count || ch->tx_count) + if (ch->rx_count != ch->tx_count) { + netdev_info(netdev, "The rx and tx count must be equal\n"); return -EINVAL; + } - if (ch->combined_count == lif->nxqs) - return 0; + if (ch->combined_count && ch->rx_count) { + netdev_info(netdev, "Use either combined_count or rx/tx_count, not both\n"); + return -EINVAL; + } + + if (ch->combined_count) + new_cnt = ch->combined_count; + else + new_cnt = ch->rx_count; + + if (lif->nxqs != new_cnt) + netdev_info(netdev, "Changing queue count from %d to %d\n", + lif->nxqs, new_cnt); return ionic_reset_queues(lif, ionic_set_queuecount, ch); } diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index b228362363ca..28868d343540 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -616,7 +616,6 @@ static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) .index = cpu_to_le32(q->index), .flags = cpu_to_le16(IONIC_QINIT_F_IRQ | IONIC_QINIT_F_SG), - .intr_index = cpu_to_le16(lif->rxqcqs[q->index].qcq->intr.index), .pid = cpu_to_le16(q->pid), .ring_size = ilog2(q->num_descs), .ring_base = cpu_to_le64(q->base_pa), @@ -624,14 +623,22 @@ static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) .sg_ring_base = cpu_to_le64(q->sg_base_pa), }, }; + unsigned int intr_index; int err; + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) + intr_index = qcq->intr.index; + else + intr_index = lif->rxqcqs[q->index].qcq->intr.index; + ctx.cmd.q_init.intr_index = cpu_to_le16(intr_index); + dev_dbg(dev, "txq_init.pid %d\n", ctx.cmd.q_init.pid); dev_dbg(dev, "txq_init.index %d\n", ctx.cmd.q_init.index); dev_dbg(dev, "txq_init.ring_base 0x%llx\n", ctx.cmd.q_init.ring_base); dev_dbg(dev, "txq_init.ring_size %d\n", ctx.cmd.q_init.ring_size); dev_dbg(dev, "txq_init.flags 0x%x\n", ctx.cmd.q_init.flags); dev_dbg(dev, "txq_init.ver %d\n", ctx.cmd.q_init.ver); + dev_dbg(dev, "txq_init.intr_index %d\n", ctx.cmd.q_init.intr_index); q->tail = q->info; q->head = q->tail; @@ -648,6 +655,10 @@ static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) dev_dbg(dev, "txq->hw_type %d\n", q->hw_type); dev_dbg(dev, "txq->hw_index %d\n", q->hw_index); + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) + netif_napi_add(lif->netdev, &qcq->napi, ionic_tx_napi, + NAPI_POLL_WEIGHT); + qcq->flags |= IONIC_QCQ_F_INITED; return 0; @@ -684,6 +695,7 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) dev_dbg(dev, "rxq_init.ring_size %d\n", ctx.cmd.q_init.ring_size); dev_dbg(dev, "rxq_init.flags 0x%x\n", ctx.cmd.q_init.flags); dev_dbg(dev, "rxq_init.ver %d\n", ctx.cmd.q_init.ver); + dev_dbg(dev, "rxq_init.intr_index %d\n", ctx.cmd.q_init.intr_index); q->tail = q->info; q->head = q->tail; @@ -700,8 +712,12 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) dev_dbg(dev, "rxq->hw_type %d\n", q->hw_type); dev_dbg(dev, "rxq->hw_index %d\n", q->hw_index); - netif_napi_add(lif->netdev, &qcq->napi, ionic_rx_napi, - NAPI_POLL_WEIGHT); + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) + netif_napi_add(lif->netdev, &qcq->napi, ionic_rx_napi, + NAPI_POLL_WEIGHT); + else + netif_napi_add(lif->netdev, &qcq->napi, ionic_txrx_napi, + NAPI_POLL_WEIGHT); qcq->flags |= IONIC_QCQ_F_INITED; @@ -1537,6 +1553,8 @@ static int ionic_txrx_alloc(struct ionic_lif *lif) sg_desc_sz = sizeof(struct ionic_txq_sg_desc); flags = IONIC_QCQ_F_TX_STATS | IONIC_QCQ_F_SG; + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) + flags |= IONIC_QCQ_F_INTR; for (i = 0; i < lif->nxqs; i++) { err = ionic_qcq_alloc(lif, IONIC_QTYPE_TXQ, i, "tx", flags, lif->ntxq_descs, @@ -1547,6 +1565,11 @@ static int ionic_txrx_alloc(struct ionic_lif *lif) if (err) goto err_out; + if (flags & IONIC_QCQ_F_INTR) + ionic_intr_coal_init(lif->ionic->idev.intr_ctrl, + lif->txqcqs[i].qcq->intr.index, + lif->tx_coalesce_hw); + lif->txqcqs[i].qcq->stats = lif->txqcqs[i].stats; ionic_debugfs_add_qcq(lif, lif->txqcqs[i].qcq); } @@ -1562,13 +1585,15 @@ static int ionic_txrx_alloc(struct ionic_lif *lif) if (err) goto err_out; - lif->rxqcqs[i].qcq->stats = lif->rxqcqs[i].stats; - ionic_intr_coal_init(lif->ionic->idev.intr_ctrl, lif->rxqcqs[i].qcq->intr.index, lif->rx_coalesce_hw); - ionic_link_qcq_interrupts(lif->rxqcqs[i].qcq, - lif->txqcqs[i].qcq); + + if (!test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) + ionic_link_qcq_interrupts(lif->rxqcqs[i].qcq, + lif->txqcqs[i].qcq); + + lif->rxqcqs[i].qcq->stats = lif->rxqcqs[i].stats; ionic_debugfs_add_qcq(lif, lif->rxqcqs[i].qcq); } @@ -2069,6 +2094,8 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index lif->rx_coalesce_usecs = IONIC_ITR_COAL_USEC_DEFAULT; lif->rx_coalesce_hw = ionic_coal_usec_to_hw(lif->ionic, lif->rx_coalesce_usecs); + lif->tx_coalesce_usecs = lif->rx_coalesce_usecs; + lif->tx_coalesce_hw = lif->rx_coalesce_hw; snprintf(lif->name, sizeof(lif->name), "lif%u", index); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index 13f173d83970..1ee3b14c8d50 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -137,6 +137,7 @@ enum ionic_lif_state_flags { IONIC_LIF_F_UP, IONIC_LIF_F_LINK_CHECK_REQUESTED, IONIC_LIF_F_FW_RESET, + IONIC_LIF_F_SPLIT_INTR, /* leave this as last */ IONIC_LIF_F_STATE_SIZE @@ -205,6 +206,8 @@ struct ionic_lif { struct dentry *dentry; u32 rx_coalesce_usecs; /* what the user asked for */ u32 rx_coalesce_hw; /* what the hw is using */ + u32 tx_coalesce_usecs; /* what the user asked for */ + u32 tx_coalesce_hw; /* what the hw is using */ struct work_struct tx_timeout_work; }; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 766f4595895c..8107d32c2767 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -419,7 +419,74 @@ void ionic_rx_empty(struct ionic_queue *q) } } +int ionic_tx_napi(struct napi_struct *napi, int budget) +{ + struct ionic_qcq *qcq = napi_to_qcq(napi); + struct ionic_cq *cq = napi_to_cq(napi); + struct ionic_dev *idev; + struct ionic_lif *lif; + u32 work_done = 0; + u32 flags = 0; + + lif = cq->bound_q->lif; + idev = &lif->ionic->idev; + + work_done = ionic_cq_service(cq, budget, + ionic_tx_service, NULL, NULL); + + if (work_done < budget && napi_complete_done(napi, work_done)) { + flags |= IONIC_INTR_CRED_UNMASK; + DEBUG_STATS_INTR_REARM(cq->bound_intr); + } + + if (work_done || flags) { + flags |= IONIC_INTR_CRED_RESET_COALESCE; + ionic_intr_credits(idev->intr_ctrl, + cq->bound_intr->index, + work_done, flags); + } + + DEBUG_STATS_NAPI_POLL(qcq, work_done); + + return work_done; +} + int ionic_rx_napi(struct napi_struct *napi, int budget) +{ + struct ionic_qcq *qcq = napi_to_qcq(napi); + struct ionic_cq *cq = napi_to_cq(napi); + struct ionic_dev *idev; + struct ionic_lif *lif; + u32 work_done = 0; + u32 flags = 0; + + lif = cq->bound_q->lif; + idev = &lif->ionic->idev; + + work_done = ionic_cq_service(cq, budget, + ionic_rx_service, NULL, NULL); + + if (work_done) + ionic_rx_fill(cq->bound_q); + + if (work_done < budget && napi_complete_done(napi, work_done)) { + flags |= IONIC_INTR_CRED_UNMASK; + DEBUG_STATS_INTR_REARM(cq->bound_intr); + } + + if (work_done || flags) { + flags |= IONIC_INTR_CRED_RESET_COALESCE; + ionic_intr_credits(idev->intr_ctrl, + cq->bound_intr->index, + work_done, flags); + } + + DEBUG_STATS_NAPI_POLL(qcq, work_done); + + return work_done; +} + +int ionic_txrx_napi(struct napi_struct *napi, int budget) { struct ionic_qcq *qcq = napi_to_qcq(napi); struct ionic_cq *rxcq = napi_to_cq(napi); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.h b/drivers/net/ethernet/pensando/ionic/ionic_txrx.h index 71973e3c35a6..a5883be0413f 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.h @@ -11,6 +11,8 @@ void ionic_rx_fill(struct ionic_queue *q); void ionic_rx_empty(struct ionic_queue *q); void ionic_tx_empty(struct ionic_queue *q); int ionic_rx_napi(struct napi_struct *napi, int budget); +int ionic_tx_napi(struct napi_struct *napi, int budget); +int ionic_txrx_napi(struct napi_struct *napi, int budget); netdev_tx_t ionic_start_xmit(struct sk_buff *skb, struct net_device *netdev); #endif /* _IONIC_TXRX_H_ */