From patchwork Wed Jul 12 18:50:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Gardner X-Patchwork-Id: 1806948 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=ub1EU4nh; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4R1RfL6Lvcz242X for ; Thu, 13 Jul 2023 04:51:21 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qJevf-0002g5-2O; Wed, 12 Jul 2023 18:51:11 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qJevc-0002fg-Kd for kernel-team@lists.ubuntu.com; Wed, 12 Jul 2023 18:51:08 +0000 Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 986653F189 for ; Wed, 12 Jul 2023 18:51:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1689187867; bh=XXzmykRnaH/KiZCbAN6AN+O+mleAlA0NO1L/uBzzBt0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=ub1EU4nhoMYuIduPXhBpWU/yYiZHdxKzoWP0y67A/mCv9Uf8eyqiIBLGXrvuoXhMu FnF6JlEwhrICHydPuKNJEdlb+6nJFobtCJ6ZEGbWZFVXN/wEJhsI+r6IHh46Lwt9P8 MrGixZK5YKXHofYNMvN7hDhpCJ0wGyzjxucOwYCzjbLTjSgQazPDasPXxH2T/DQG9L 3+Gzpy5h7ObQ1uKVwU8/rRXY2sWfjvIXmD3PAAJEI54IXsFvDrgY9h/Fv70AKQPJFs uk1UHG3k/IA16QMXCD//7bztFBqXVGablGnJxMEbiZMu2BLydUEKmA5VCHQoat2Yjw aTAstJ4z+nDEQ== Received: by mail-pg1-f197.google.com with SMTP id 41be03b00d2f7-53feeb13906so9385186a12.1 for ; Wed, 12 Jul 2023 11:51:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689187865; x=1691779865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XXzmykRnaH/KiZCbAN6AN+O+mleAlA0NO1L/uBzzBt0=; b=keSF7r0JtbWotRDeWCI1DGInLR9vamVZ4cLIsHCIovAHbWCixKq1INkzd7EUdQxFPY U/l9id4k5FgWj6BfMP9D0npvGhSQ8WB+FwbQp3LPv/UX79a75A9LLuBNS3VG05L2MKJS +t6fqUGQmmBsKcmbmtn/8vMR6KKl4LJ32yJvqjVqfKFvjDc9bHgtlku0hh9iUsUWP4El cyebWbM5ZRqpBAyr2UT80ctc5CmMcpuAQmEshUSDPProNZHxYvNPVOpynMiY662JXAdA StiE7LoQV/UZx9AE+QHlNBjuou58LuymwHYOvuHwFToGVway/H9OdkiqHTDIZnBIoZl9 tV3Q== X-Gm-Message-State: ABy/qLbFcCfPncH7IDe+sfNWqOGr13KlsCyi7mU/Y7vtqEJt4sk6od/6 Qomf/A5+V67TYSOS/RvwEgtMrKroLqUuaefCagnI5nKVOZ+PL2yhaXB3hsdQfaRUynFx+layc9d fJG+EPIssa9flr90FOiEl/CqA0PZw3J1T05EgIqaAUOIbCZMvgw== X-Received: by 2002:a05:6a00:3920:b0:682:4edf:b9c7 with SMTP id fh32-20020a056a00392000b006824edfb9c7mr25344789pfb.2.1689187865654; Wed, 12 Jul 2023 11:51:05 -0700 (PDT) X-Google-Smtp-Source: APBJJlHqQUrHK4zIW1KKZsQQpl2YZGS9MHYcSTY+RCcw/usnzZBNLCZBFaSM0wr0eImGyKy3Q3miJQ== X-Received: by 2002:a05:6a00:3920:b0:682:4edf:b9c7 with SMTP id fh32-20020a056a00392000b006824edfb9c7mr25344777pfb.2.1689187865361; Wed, 12 Jul 2023 11:51:05 -0700 (PDT) Received: from smtp.gmail.com (174-045-099-030.res.spectrum.com. [174.45.99.30]) by smtp.gmail.com with ESMTPSA id x41-20020a056a000be900b00682a9325ffcsm3984968pfu.5.2023.07.12.11.51.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Jul 2023 11:51:04 -0700 (PDT) From: Tim Gardner To: kernel-team@lists.ubuntu.com Subject: [PATCH 1/2] UBUNTU: SAUCE: net: mana: Batch ringing RX queue doorbell on receiving packets Date: Wed, 12 Jul 2023 12:50:58 -0600 Message-Id: <20230712185059.723271-2-tim.gardner@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230712185059.723271-1-tim.gardner@canonical.com> References: <20230712185059.723271-1-tim.gardner@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Long Li BugLink: https://bugs.launchpad.net/bugs/2027615 It's inefficient to ring the doorbell page every time a WQE is posted to the received queue. Excessive MMIO writes result in CPU spending more time waiting on LOCK instructions (atomic operations), resulting in poor scaling performance. Move the code for ringing doorbell page to where after we have posted all WQEs to the receive queue during a callback from napi_poll(). With this change, tests showed an improvement from 120G/s to 160G/s on a 200G physical link, with 16 or 32 hardware queues. Tests showed no regression in network latency benchmarks on single connection. Cc: stable@vger.kernel.org Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)") Reviewed-by: Haiyang Zhang Reviewed-by: Dexuan Cui Signed-off-by: Long Li Signed-off-by: Tim Gardner --- drivers/net/ethernet/microsoft/mana/mana_en.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c index 2bf3f999ce97..146b4f99a5d8 100644 --- a/drivers/net/ethernet/microsoft/mana/mana_en.c +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c @@ -1383,8 +1383,8 @@ static void mana_post_pkt_rxq(struct mana_rxq *rxq) recv_buf_oob = &rxq->rx_oobs[curr_index]; - err = mana_gd_post_and_ring(rxq->gdma_rq, &recv_buf_oob->wqe_req, - &recv_buf_oob->wqe_inf); + err = mana_gd_post_work_request(rxq->gdma_rq, &recv_buf_oob->wqe_req, + &recv_buf_oob->wqe_inf); if (WARN_ON_ONCE(err)) return; @@ -1654,6 +1654,12 @@ static void mana_poll_rx_cq(struct mana_cq *cq) mana_process_rx_cqe(rxq, cq, &comp[i]); } + if (comp_read > 0) { + struct gdma_context *gc = rxq->gdma_rq->gdma_dev->gdma_context; + + mana_gd_wq_ring_doorbell(gc, rxq->gdma_rq); + } + if (rxq->xdp_flush) xdp_do_flush(); }