From patchwork Thu Apr 10 15:57:06 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rik van Riel X-Patchwork-Id: 338148 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 1228214007D for ; Fri, 11 Apr 2014 01:59:41 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935695AbaDJP67 (ORCPT ); Thu, 10 Apr 2014 11:58:59 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51390 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934732AbaDJP6U (ORCPT ); Thu, 10 Apr 2014 11:58:20 -0400 Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s3AFvjCI031016 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 10 Apr 2014 11:57:46 -0400 Received: from annuminas.surriel.com ([10.10.116.19]) by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s3AFvinE014363; Thu, 10 Apr 2014 11:57:44 -0400 Date: Thu, 10 Apr 2014 11:57:06 -0400 From: Rik van Riel To: linux-kernel@vger.kernel.org Cc: netdev@vger.kernel.org, Jiri Benc , Peter Zijlstra , Thomas Gleixner , David Miller , Frederic Weisbecker Subject: [PATCH] softirq: punt to ksoftirqd if __do_softirq recently looped Message-ID: <20140410115706.662fb5e7@annuminas.surriel.com> Organization: Red Hat, Inc. MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Jiri noticed that netperf throughput had gotten worse in recent years, for smaller message sizes. In the past, ksoftirqd would take around 80% of a CPU, and netserver would take around 100% of another CPU. On current kernels, sometimes all the softirq processing is done in the context of the netperf process, which can result in as much as a 50% performance drop, due to netserver spending all its CPU time "delivering" packets to a socket it rarely empties, and dropping the packets on the floor as a result. This seems silly in an age where even cell phones are multi-core, and we could simply let the ksoftirqd thread handle the softirq load, so the scheduler can migrate the userspace task to another CPU. This patch accomplishes that in a very simplistic way. The code remembers when __do_softirq last looped, and will punt softirq handling to ksoftirqd if another softirq happens in the same jiffie. Netperf results: without patch with patch UDP_STREAM 1472 957.17 / 954.18 957.15 / 951.73 UDP_STREAM 978 936.85 / 930.06 936.84 / 927.63 UDP_STREAM 466 875.98 / 865.62 875.98 / 868.65 UDP_STREAM 210 760.88 / 748.70 760.88 / 748.61 UDP_STREAM 82 554.06 / 329.96 554.06 / 505.95 unstable ^^^^^^ UDP_STREAM 18 158.99 / 108.95 160.73 / 112.68 Signed-off-by: Rik van Riel Cc: Frederic Weisbecker Cc: David Miller Cc: Thomas Gleixner Cc: Peter Zijlstra Tested-by: Jiri Benc Reported-by: Jiri Benc --- kernel/softirq.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/kernel/softirq.c b/kernel/softirq.c index 787b3a0..020be2f 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -56,6 +56,7 @@ EXPORT_SYMBOL(irq_stat); static struct softirq_action softirq_vec[NR_SOFTIRQS] __cacheline_aligned_in_smp; DEFINE_PER_CPU(struct task_struct *, ksoftirqd); +DEFINE_PER_CPU(unsigned long, softirq_looped); char *softirq_to_name[NR_SOFTIRQS] = { "HI", "TIMER", "NET_TX", "NET_RX", "BLOCK", "BLOCK_IOPOLL", @@ -271,6 +272,9 @@ asmlinkage void __do_softirq(void) pending = local_softirq_pending(); if (pending) { + /* Still busy? Remember this for invoke_softirq() below... */ + this_cpu_write(softirq_looped, jiffies); + if (time_before(jiffies, end) && !need_resched() && --max_restart) goto restart; @@ -330,7 +334,11 @@ void irq_enter(void) static inline void invoke_softirq(void) { - if (!force_irqthreads) { + /* + * If force_irqthreads is set, or if we looped in __do_softirq this + * jiffie, punt to ksoftirqd to prevent userland starvation. + */ + if (!force_irqthreads && this_cpu_read(softirq_looped) != jiffies) { /* * We can safely execute softirq on the current stack if * it is the irq stack, because it should be near empty