From patchwork Sun Dec 15 07:57:58 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "fan.du" X-Patchwork-Id: 301317 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id 6383D2C009A for ; Sun, 15 Dec 2013 18:59:00 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754041Ab3LOH65 (ORCPT ); Sun, 15 Dec 2013 02:58:57 -0500 Received: from mail.windriver.com ([147.11.1.11]:57097 "EHLO mail.windriver.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751147Ab3LOH64 (ORCPT ); Sun, 15 Dec 2013 02:58:56 -0500 Received: from ALA-HCA.corp.ad.wrs.com (ala-hca.corp.ad.wrs.com [147.11.189.40]) by mail.windriver.com (8.14.5/8.14.5) with ESMTP id rBF7w5Ux028975 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Sat, 14 Dec 2013 23:58:10 -0800 (PST) Received: from romashell-ThinkPad-T510.corp.ad.wrs.com (128.224.163.146) by ALA-HCA.corp.ad.wrs.com (147.11.189.40) with Microsoft SMTP Server id 14.2.347.0; Sat, 14 Dec 2013 23:58:09 -0800 From: Fan Du To: CC: , , Subject: [PATCHv3 net-next 1/7] {pktgen, xfrm} Correct xfrm state lock usage when transforming Date: Sun, 15 Dec 2013 15:57:58 +0800 Message-ID: <1387094284-2901-2-git-send-email-fan.du@windriver.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1387094284-2901-1-git-send-email-fan.du@windriver.com> References: <1387094284-2901-1-git-send-email-fan.du@windriver.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org xfrm_state lock protects its state, i.e., VALID/DEAD and statistics, not the transforming procedure, as both mode/type output functions are reentrant. Another issue is state lock can be used in BH context when state timer alarmed, after transformation in pktgen, update state statistics acquiring state lock should disabled BH context for a moment. Otherwise LOCKDEP critisize this: [ 62.354339] pktgen: Packet Generator for packet performance testing. Version: 2.74 [ 62.655444] [ 62.655448] ================================= [ 62.655451] [ INFO: inconsistent lock state ] [ 62.655455] 3.13.0-rc2+ #70 Not tainted [ 62.655457] --------------------------------- [ 62.655459] inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-W} usage. [ 62.655463] kpktgend_0/2764 [HC0[0]:SC0[0]:HE1:SE1] takes: [ 62.655466] (&(&x->lock)->rlock){+.?...}, at: [] pktgen_thread_worker+0x1796/0x1860 [pktgen] [ 62.655479] {IN-SOFTIRQ-W} state was registered at: [ 62.655484] [] __lock_acquire+0x62d/0x1d70 [ 62.655492] [] lock_acquire+0x97/0x130 [ 62.655498] [] _raw_spin_lock+0x36/0x70 [ 62.655505] [] xfrm_timer_handler+0x43/0x290 [ 62.655511] [] __tasklet_hrtimer_trampoline+0x17/0x40 [ 62.655519] [] tasklet_hi_action+0xd7/0xf0 [ 62.655523] [] __do_softirq+0xe6/0x2d0 [ 62.655526] [] irq_exit+0x96/0xc0 [ 62.655530] [] smp_apic_timer_interrupt+0x4a/0x60 [ 62.655537] [] apic_timer_interrupt+0x6f/0x80 [ 62.655541] [] arch_cpu_idle+0x26/0x30 [ 62.655547] [] cpu_startup_entry+0x88/0x2b0 [ 62.655552] [] rest_init+0xbc/0xd0 [ 62.655557] [] start_kernel+0x3c4/0x3d1 [ 62.655583] [] x86_64_start_reservations+0x2a/0x2c [ 62.655588] [] x86_64_start_kernel+0xf5/0xfc [ 62.655592] irq event stamp: 77 [ 62.655594] hardirqs last enabled at (77): [] vprintk_emit+0x1b2/0x520 [ 62.655597] hardirqs last disabled at (76): [] vprintk_emit+0x44/0x520 [ 62.655601] softirqs last enabled at (22): [] __do_softirq+0x177/0x2d0 [ 62.655605] softirqs last disabled at (15): [] irq_exit+0x96/0xc0 [ 62.655609] [ 62.655609] other info that might help us debug this: [ 62.655613] Possible unsafe locking scenario: [ 62.655613] [ 62.655616] CPU0 [ 62.655617] ---- [ 62.655618] lock(&(&x->lock)->rlock); [ 62.655622] [ 62.655623] lock(&(&x->lock)->rlock); [ 62.655626] [ 62.655626] *** DEADLOCK *** [ 62.655626] [ 62.655629] no locks held by kpktgend_0/2764. [ 62.655631] [ 62.655631] stack backtrace: [ 62.655636] CPU: 0 PID: 2764 Comm: kpktgend_0 Not tainted 3.13.0-rc2+ #70 [ 62.655638] Hardware name: innotek GmbH VirtualBox, BIOS VirtualBox 12/01/2006 [ 62.655642] ffffffff8216b7b0 ffff88001be43ab8 ffffffff8176af37 0000000000000007 [ 62.655652] ffff88001c8d4fc0 ffff88001be43b18 ffffffff81766d78 0000000000000000 [ 62.655663] ffff880000000001 ffff880000000001 ffffffff8101025f ffff88001be43b18 [ 62.655671] Call Trace: [ 62.655680] [] dump_stack+0x46/0x58 [ 62.655685] [] print_usage_bug+0x1f1/0x202 [ 62.655691] [] ? save_stack_trace+0x2f/0x50 [ 62.655696] [] mark_lock+0x28c/0x2f0 [ 62.655700] [] ? check_usage_forwards+0x150/0x150 [ 62.655704] [] __lock_acquire+0x68a/0x1d70 [ 62.655712] [] ? irq_work_queue+0x69/0xb0 [ 62.655717] [] ? vprintk_emit+0x1b2/0x520 [ 62.655722] [] ? trace_hardirqs_on_caller+0x105/0x1d0 [ 62.655730] [] ? pktgen_thread_worker+0x1796/0x1860 [pktgen] [ 62.655734] [] lock_acquire+0x97/0x130 [ 62.655741] [] ? pktgen_thread_worker+0x1796/0x1860 [pktgen] [ 62.655745] [] _raw_spin_lock+0x36/0x70 [ 62.655752] [] ? pktgen_thread_worker+0x1796/0x1860 [pktgen] [ 62.655758] [] pktgen_thread_worker+0x1796/0x1860 [pktgen] [ 62.655766] [] ? pktgen_thread_worker+0xb19/0x1860 [pktgen] [ 62.655771] [] ? trace_hardirqs_on+0xd/0x10 [ 62.655777] [] ? _raw_spin_unlock_irq+0x30/0x40 [ 62.655785] [] ? e1000_clean+0x9d0/0x9d0 [ 62.655791] [] ? __init_waitqueue_head+0x60/0x60 [ 62.655795] [] ? __init_waitqueue_head+0x60/0x60 [ 62.655800] [] ? mod_cur_headers+0x7f0/0x7f0 [pktgen] [ 62.655806] [] kthread+0xe4/0x100 [ 62.655813] [] ? flush_kthread_worker+0x170/0x170 [ 62.655819] [] ret_from_fork+0x7c/0xb0 [ 62.655824] [] ? flush_kthread_worker+0x170/0x170 Signed-off-by: Fan Du --- net/core/pktgen.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/net/core/pktgen.c b/net/core/pktgen.c index a797fff..b007586 100644 --- a/net/core/pktgen.c +++ b/net/core/pktgen.c @@ -2487,8 +2487,6 @@ static int pktgen_output_ipsec(struct sk_buff *skb, struct pktgen_dev *pkt_dev) if (x->props.mode != XFRM_MODE_TRANSPORT) return 0; - spin_lock(&x->lock); - err = x->outer_mode->output(x, skb); if (err) goto error; @@ -2496,10 +2494,11 @@ static int pktgen_output_ipsec(struct sk_buff *skb, struct pktgen_dev *pkt_dev) if (err) goto error; + spin_lock_bh(&x->lock); x->curlft.bytes += skb->len; x->curlft.packets++; + spin_unlock_bh(&x->lock); error: - spin_unlock(&x->lock); return err; }