From patchwork Mon Mar 30 18:44:46 2009 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Grover X-Patchwork-Id: 25329 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.176.167]) by ozlabs.org (Postfix) with ESMTP id C5ABCDDEE8 for ; Tue, 31 Mar 2009 05:46:10 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758842AbZC3SpZ (ORCPT ); Mon, 30 Mar 2009 14:45:25 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754430AbZC3SpL (ORCPT ); Mon, 30 Mar 2009 14:45:11 -0400 Received: from rcsinet12.oracle.com ([148.87.113.124]:28031 "EHLO rgminet12.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758975AbZC3SpJ (ORCPT ); Mon, 30 Mar 2009 14:45:09 -0400 Received: from rgminet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) by rgminet12.oracle.com (Switch-3.3.1/Switch-3.3.1) with ESMTP id n2UIisbL020912 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Mon, 30 Mar 2009 18:44:55 GMT Received: from acsmt701.oracle.com (acsmt701.oracle.com [141.146.40.71]) by rgminet15.oracle.com (Switch-3.3.1/Switch-3.3.1) with ESMTP id n2UIj6ue011375; Mon, 30 Mar 2009 18:45:08 GMT Received: from localhost.localdomain (/139.185.48.5) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Mon, 30 Mar 2009 18:45:00 +0000 From: Andy Grover To: netdev@vger.kernel.org Cc: rds-devel@oss.oracle.com, Steve Wise Subject: [PATCH 2/9] RDS/IW+IB: Set recv ring low water mark to 1/2 full. Date: Mon, 30 Mar 2009 11:44:46 -0700 Message-Id: <1238438693-29540-3-git-send-email-andy.grover@oracle.com> X-Mailer: git-send-email 1.5.6.3 In-Reply-To: <1238438693-29540-1-git-send-email-andy.grover@oracle.com> References: <1238438693-29540-1-git-send-email-andy.grover@oracle.com> X-Source-IP: acsmt701.oracle.com [141.146.40.71] X-Auth-Type: Internal IP X-CT-RefId: str=0001.0A09020A.49D1132F.00B7:SCFMA4539814,ss=1,fgs=0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Steve Wise Currently the recv ring low water mark is 1/4 the depth. Performance measurements show that this limits iWARP throughput by flow controlling the rds-stress senders. Setting it to 1/2 seems to max the T3 performance. I tried even higher levels but that didn't help and it started to increase the rds thread cpu utilization. Signed-off-by: Steve Wise Signed-off-by: Andy Grover --- net/rds/ib_ring.c | 2 +- net/rds/iw_ring.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/net/rds/ib_ring.c b/net/rds/ib_ring.c index 99a6cca..ff97e8e 100644 --- a/net/rds/ib_ring.c +++ b/net/rds/ib_ring.c @@ -137,7 +137,7 @@ int rds_ib_ring_empty(struct rds_ib_work_ring *ring) int rds_ib_ring_low(struct rds_ib_work_ring *ring) { - return __rds_ib_ring_used(ring) <= (ring->w_nr >> 2); + return __rds_ib_ring_used(ring) <= (ring->w_nr >> 1); } /* diff --git a/net/rds/iw_ring.c b/net/rds/iw_ring.c index d422d4b..da8e3b6 100644 --- a/net/rds/iw_ring.c +++ b/net/rds/iw_ring.c @@ -137,7 +137,7 @@ int rds_iw_ring_empty(struct rds_iw_work_ring *ring) int rds_iw_ring_low(struct rds_iw_work_ring *ring) { - return __rds_iw_ring_used(ring) <= (ring->w_nr >> 2); + return __rds_iw_ring_used(ring) <= (ring->w_nr >> 1); }