From patchwork Thu Dec 19 15:50:32 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Christoph Lameter (Ampere)" X-Patchwork-Id: 303584 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id B1B2C2C0040 for ; Fri, 20 Dec 2013 02:57:05 +1100 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755906Ab3LSP4o (ORCPT ); Thu, 19 Dec 2013 10:56:44 -0500 Received: from qmta12.emeryville.ca.mail.comcast.net ([76.96.27.227]:43381 "EHLO qmta12.emeryville.ca.mail.comcast.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755895Ab3LSP4n (ORCPT ); Thu, 19 Dec 2013 10:56:43 -0500 X-Greylist: delayed 365 seconds by postgrey-1.27 at vger.kernel.org; Thu, 19 Dec 2013 10:56:43 EST Received: from omta06.emeryville.ca.mail.comcast.net ([76.96.30.51]) by qmta12.emeryville.ca.mail.comcast.net with comcast id 3TGA1n00616AWCU01TqesX; Thu, 19 Dec 2013 15:50:38 +0000 Received: from gentwo.org ([98.213.233.247]) by omta06.emeryville.ca.mail.comcast.net with comcast id 3Tqc1n0075Lw0ES8STqdg0; Thu, 19 Dec 2013 15:50:38 +0000 Received: by gentwo.org (Postfix, from userid 1001) id 9E1545B221; Thu, 19 Dec 2013 09:50:32 -0600 (CST) Message-Id: <20131219155032.559429807@linux.com> Date: Thu, 19 Dec 2013 09:50:32 -0600 From: Christoph Lameter To: Tejun Heo Cc: akpm@linuxfoundation.org, rostedt@goodmis.org, linux-kernel@vger.kernel.org, Ingo Molnar , Peter Zijlstra , Thomas Gleixner , netdev@vger.kernel.org, davem@davemloft.net, edumazet@google.com Subject: [PATCH 17/40] net: Replace __this_cpu_inc in route.c with raw_cpu_inc References: <20131219155015.443763038@linux.com> Content-Disposition: inline; filename=preempt_rt_cache_stat DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20121106; t=1387468238; bh=Bv757SBXA+O7s8YWMqHiuju2q4f7PChFOqhynTaCcRk=; h=Received:Received:Received:Message-Id:Date:From:To:Subject: Content-Type; b=MD/RHVNXr++HNCnM8qPYBETwIPPgIqcUIqzErhKKJ45bJ7FOxL92PXhGnewa/Bd9D 5XKIJa3/vDq4nUGv+iJhPbzFb+l5TP75bYVYnvcAnBVo6lI0P9aF47lyUKJ+ZXv2WP 0IWw8eHRePam58Z3JyR+0WPz12ZLSIAUqJU2zhfq8bb6WSNriK6cjURKnNfuWaVG0T eXfAWpU/DcJtLIISf0HN9hFh1FWK4z6UKE0lo7GHlH6WY5eeo7a0yab69V+DEKbZND jGT3iyhzhyWsdNMk18MqJsVoHa3x6EK68TRYeYop7BSh61fLV4rPNjsQx8Sb2i7q3x ExtcPZb9/IqDg== Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org [Patch depends on another patch in this series that introduces raw_cpu_ops] The RT_CACHE_STAT_INC macro triggers the new preemption checks for __this_cpu ops. I do not see any other synchronization that would allow the use of a __this_cpu operation here however in commit dbd2915ce87e811165da0717f8e159276ebb803e Andrew justifies the use of raw_smp_processor_id() here because "we do not care" about races. In the past we agreed that the price of disabling interrupts here to get consistent counters would be too high. These counters may be inaccurate due to race conditions. The use of __this_cpu op improves the situation already from what commit dbd2915ce87e811165da0717f8e159276ebb803e did since the single instruction emitted on x86 does not allow the race to occur anymore. However, non x86 platforms could still experience a race here. Trace: [ 1277.189084] __this_cpu_add operation in preemptible [00000000] code: avahi-daemon/1193 [ 1277.189085] caller is __this_cpu_preempt_check+0x38/0x60 [ 1277.189086] CPU: 1 PID: 1193 Comm: avahi-daemon Tainted: GF 3.12.0-rc4+ #187 [ 1277.189087] Hardware name: FUJITSU CELSIUS W530 Power/D3227-A1, BIOS V4.6.5.4 R1.10.0 for D3227-A1x 09/16/2013 [ 1277.189088] 0000000000000001 ffff8807ef78fa00 ffffffff816d5a57 ffff8807ef78ffd8 [ 1277.189089] ffff8807ef78fa30 ffffffff8137359c ffff8807ef78fba0 ffff88079f822b40 [ 1277.189091] 0000000020000000 ffff8807ee32c800 ffff8807ef78fa70 ffffffff813735f8 [ 1277.189093] Call Trace: [ 1277.189094] [] dump_stack+0x4e/0x82 [ 1277.189096] [] check_preemption_disabled+0xec/0x110 [ 1277.189097] [] __this_cpu_preempt_check+0x38/0x60 [ 1277.189098] [] __ip_route_output_key+0x575/0x8c0 [ 1277.189100] [] ip_route_output_flow+0x27/0x70 [ 1277.189101] [] ? ip_copy_metadata+0x1a0/0x1a0 [ 1277.189102] [] udp_sendmsg+0x825/0xa20 [ 1277.189104] [] ? do_sys_poll+0x449/0x5d0 [ 1277.189105] [] inet_sendmsg+0x85/0xc0 [ 1277.189106] [] sock_sendmsg+0x9c/0xd0 [ 1277.189108] [] ? __this_cpu_preempt_check+0x38/0x60 [ 1277.189109] [] ? move_addr_to_kernel+0x40/0xa0 [ 1277.189111] [] ___sys_sendmsg+0x37c/0x390 [ 1277.189112] [] ? string.isra.3+0x3a/0xd0 [ 1277.189113] [] ? string.isra.3+0x3a/0xd0 [ 1277.189115] [] ? vsnprintf+0x364/0x650 [ 1277.189116] [] ? snprintf+0x39/0x40 [ 1277.189118] [] ? __this_cpu_preempt_check+0x38/0x60 [ 1277.189119] [] __sys_sendmsg+0x49/0x90 [ 1277.189121] [] SyS_sendmsg+0x12/0x20 [ 1277.189122] [] tracesys+0xe1/0xe6 Cc: netdev@vger.kernel.org Cc: davem@davemloft.net Cc: edumazet@google.com Acked-by: Ingo Molnar Signed-off-by: Christoph Lameter --- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Index: linux/net/ipv4/route.c =================================================================== --- linux.orig/net/ipv4/route.c 2013-12-02 16:07:51.964573250 -0600 +++ linux/net/ipv4/route.c 2013-12-02 16:07:51.954573526 -0600 @@ -197,7 +197,7 @@ const __u8 ip_tos2prio[16] = { EXPORT_SYMBOL(ip_tos2prio); static DEFINE_PER_CPU(struct rt_cache_stat, rt_cache_stat); -#define RT_CACHE_STAT_INC(field) __this_cpu_inc(rt_cache_stat.field) +#define RT_CACHE_STAT_INC(field) raw_cpu_inc(rt_cache_stat.field) #ifdef CONFIG_PROC_FS static void *rt_cache_seq_start(struct seq_file *seq, loff_t *pos)