From patchwork Fri Apr 10 00:09:36 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Grover X-Patchwork-Id: 25810 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by ozlabs.org (Postfix) with ESMTP id 6E15CDE1FF for ; Fri, 10 Apr 2009 10:10:11 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S937258AbZDJAKI (ORCPT ); Thu, 9 Apr 2009 20:10:08 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S937386AbZDJAKG (ORCPT ); Thu, 9 Apr 2009 20:10:06 -0400 Received: from rcsinet12.oracle.com ([148.87.113.124]:22797 "EHLO rgminet12.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S936876AbZDJAKA (ORCPT ); Thu, 9 Apr 2009 20:10:00 -0400 Received: from acsinet13.oracle.com (acsinet13.oracle.com [141.146.126.235]) by rgminet12.oracle.com (Switch-3.3.1/Switch-3.3.1) with ESMTP id n3A09hxw019185 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 10 Apr 2009 00:09:44 GMT Received: from acsmt702.oracle.com (acsmt702.oracle.com [141.146.40.80]) by acsinet13.oracle.com (Switch-3.3.1/Switch-3.3.1) with ESMTP id n3A0AKSE022854; Fri, 10 Apr 2009 00:10:21 GMT Received: from localhost.localdomain (/139.185.48.5) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 09 Apr 2009 17:09:52 -0700 From: Andy Grover To: netdev@vger.kernel.org Cc: rds-devel@oss.oracle.com, Steve Wise Subject: [PATCH 01/12] RDS/IW+IB: Set recv ring low water mark to 1/2 full. Date: Thu, 9 Apr 2009 17:09:36 -0700 Message-Id: <1239322187-3764-2-git-send-email-andy.grover@oracle.com> X-Mailer: git-send-email 1.5.6.3 In-Reply-To: <1239322187-3764-1-git-send-email-andy.grover@oracle.com> References: <1239322187-3764-1-git-send-email-andy.grover@oracle.com> X-Source-IP: acsmt702.oracle.com [141.146.40.80] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A090209.49DE8E52.01A0:SCFMA4539814,ss=1,fgs=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Steve Wise Currently the recv ring low water mark is 1/4 the depth. Performance measurements show that this limits iWARP throughput by flow controlling the rds-stress senders. Setting it to 1/2 seems to max the T3 performance. I tried even higher levels but that didn't help and it started to increase the rds thread cpu utilization. Signed-off-by: Steve Wise Signed-off-by: Andy Grover --- net/rds/ib_ring.c | 2 +- net/rds/iw_ring.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/net/rds/ib_ring.c b/net/rds/ib_ring.c index 99a6cca..ff97e8e 100644 --- a/net/rds/ib_ring.c +++ b/net/rds/ib_ring.c @@ -137,7 +137,7 @@ int rds_ib_ring_empty(struct rds_ib_work_ring *ring) int rds_ib_ring_low(struct rds_ib_work_ring *ring) { - return __rds_ib_ring_used(ring) <= (ring->w_nr >> 2); + return __rds_ib_ring_used(ring) <= (ring->w_nr >> 1); } /* diff --git a/net/rds/iw_ring.c b/net/rds/iw_ring.c index d422d4b..da8e3b6 100644 --- a/net/rds/iw_ring.c +++ b/net/rds/iw_ring.c @@ -137,7 +137,7 @@ int rds_iw_ring_empty(struct rds_iw_work_ring *ring) int rds_iw_ring_low(struct rds_iw_work_ring *ring) { - return __rds_iw_ring_used(ring) <= (ring->w_nr >> 2); + return __rds_iw_ring_used(ring) <= (ring->w_nr >> 1); }