From patchwork Fri May 1 03:14:07 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexei Starovoitov X-Patchwork-Id: 466791 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 1C372140761 for ; Fri, 1 May 2015 13:14:20 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753299AbbEADOQ (ORCPT ); Thu, 30 Apr 2015 23:14:16 -0400 Received: from mail-pa0-f42.google.com ([209.85.220.42]:36216 "EHLO mail-pa0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753262AbbEADOP (ORCPT ); Thu, 30 Apr 2015 23:14:15 -0400 Received: by pabsx10 with SMTP id sx10so78850621pab.3 for ; Thu, 30 Apr 2015 20:14:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=m8MB/Flj6Pc2ayfkxpTQPyE4vqEiN35i/VuV/NxmTVE=; b=eo/T/Ocmkhiq3guD/mhrTmmRnRPDCOlhMdWtkarWTxcZ9GATdVM3f+YSIw4Ibae4+G vLkh0I3RqO3IEunRbhHgYwfQhfThi2pQM9BBV6+N/ygR/e3EtIfa3GOh5YfT67pN10Mb TZMpryTTtVlE2cNQ6FwBfkQLAGjECyMt2tm/xhM3q80v8hSCXuJJ1M/z/4FtoQVZnwFg MrW1obd7yKYQELq8QBq+bOyYfUEuSzRKKEmWvwdoWCSyq5LW1Fio16q0dLimvavc+2DB RsfRJl0a7aQOYTUAXqva9/0UtXC8PZ6lSkrPPXj1YFbS7Rttdl4VgEGvT2vcu+38bOZC eo0Q== X-Gm-Message-State: ALoCoQliNCRTtKgeQhP6dOuBOE43eYMwm8ijKAcqy2t3w8DmiQ5afvdhrG/MA/9FEC6kxcuvOhGe X-Received: by 10.68.130.42 with SMTP id ob10mr13799217pbb.151.1430450054656; Thu, 30 Apr 2015 20:14:14 -0700 (PDT) Received: from localhost.localdomain ([12.229.56.227]) by mx.google.com with ESMTPSA id as1sm3471846pbc.39.2015.04.30.20.14.13 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 30 Apr 2015 20:14:14 -0700 (PDT) From: Alexei Starovoitov To: "David S. Miller" Cc: John Fastabend , Jamal Hadi Salim , Daniel Borkmann , netdev@vger.kernel.org Subject: [PATCH net-next] net: sched: run ingress qdisc without locks Date: Thu, 30 Apr 2015 20:14:07 -0700 Message-Id: <1430450047-5671-1-git-send-email-ast@plumgrid.com> X-Mailer: git-send-email 1.7.9.5 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org TC classifiers/actions were converted to RCU by John in the series: http://thread.gmane.org/gmane.linux.network/329739/focus=329739 and many follow on patches. This is the last patch from that series that finally drops ingress spin_lock. Single cpu ingress+u32 performance goes from 22.9 Mpps to 24.5 Mpps. In two cpu case when both cores are receiving traffic on the same device and go into the same ingress+u32 the performance jumps from 4.5 + 4.5 Mpps to 23.5 + 23.5 Mpps Signed-off-by: John Fastabend Signed-off-by: Alexei Starovoitov Signed-off-by: Jamal Hadi Salim Acked-by: Daniel Borkmann --- All the credit goes to John. I merely tested and benchmarked. net/core/dev.c | 2 -- net/sched/sch_ingress.c | 5 +++-- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/net/core/dev.c b/net/core/dev.c index 97a15ae8d07a..862875ec8f2f 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -3538,10 +3538,8 @@ static int ing_filter(struct sk_buff *skb, struct netdev_queue *rxq) q = rcu_dereference(rxq->qdisc); if (q != &noop_qdisc) { - spin_lock(qdisc_lock(q)); if (likely(!test_bit(__QDISC_STATE_DEACTIVATED, &q->state))) result = qdisc_enqueue_root(skb, q); - spin_unlock(qdisc_lock(q)); } return result; diff --git a/net/sched/sch_ingress.c b/net/sched/sch_ingress.c index 4cdbfb85686a..a89cc3278bfb 100644 --- a/net/sched/sch_ingress.c +++ b/net/sched/sch_ingress.c @@ -65,11 +65,11 @@ static int ingress_enqueue(struct sk_buff *skb, struct Qdisc *sch) result = tc_classify(skb, fl, &res); - qdisc_bstats_update(sch, skb); + qdisc_bstats_update_cpu(sch, skb); switch (result) { case TC_ACT_SHOT: result = TC_ACT_SHOT; - qdisc_qstats_drop(sch); + qdisc_qstats_drop_cpu(sch); break; case TC_ACT_STOLEN: case TC_ACT_QUEUED: @@ -91,6 +91,7 @@ static int ingress_enqueue(struct sk_buff *skb, struct Qdisc *sch) static int ingress_init(struct Qdisc *sch, struct nlattr *opt) { net_inc_ingress_queue(); + sch->flags |= TCQ_F_CPUSTATS; return 0; }