From patchwork Sun Apr 18 15:31:24 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?b?RnLDqWTDqXJpYyBXZWlzYmVja2Vy?= X-Patchwork-Id: 50415 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by ozlabs.org (Postfix) with ESMTP id C034AB7D0C for ; Mon, 19 Apr 2010 01:31:26 +1000 (EST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754039Ab0DRPbZ (ORCPT ); Sun, 18 Apr 2010 11:31:25 -0400 Received: from mail-wy0-f174.google.com ([74.125.82.174]:41818 "EHLO mail-wy0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754020Ab0DRPbY (ORCPT ); Sun, 18 Apr 2010 11:31:24 -0400 Received: by wyb39 with SMTP id 39so2035851wyb.19 for ; Sun, 18 Apr 2010 08:31:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:date:from:to:cc :subject:message-id:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=Fx0wf4OUyZYz3OGYeEQZg2xA324Bs/15hmcsJ7wYCRU=; b=AQdbJonIa3cYmCJE+J9bL88k8sLIAzthpfs9CgJb9OWX2LPtxSJ3wnV8hw0176tmjy d+jk0fLCnxRX8ZF2S2G3o3NNsyY6eqCfjVA7O7ZXYWRz2vATjJ+Bwa9C9mSR5c97CiWh Hj5+gk9Pfsuhau5m2XTxF32oaD2mq65I/jQms= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=E41igiihm/IKORQiMlv8KzsguDKjBEqe5N4wf5HzcsE3q6ASsnP6JloK6TFm4TlIGI gnCHkXlh1HWNFU+ezCijbgL5vbwsAVWMR42gTquEX+c2lxAEXjoMLeRMR533I1DkbH/W BIZM9SrhIan5d36NDxUhPYN34RCTW+sY5S7Qo= Received: by 10.216.86.203 with SMTP id w53mr5493514wee.71.1271604680619; Sun, 18 Apr 2010 08:31:20 -0700 (PDT) Received: from nowhere (ADijon-551-1-104-221.w92-138.abo.wanadoo.fr [92.138.15.221]) by mx.google.com with ESMTPS id z34sm40345181wbv.2.2010.04.18.08.31.19 (version=SSLv3 cipher=RC4-MD5); Sun, 18 Apr 2010 08:31:19 -0700 (PDT) Received: by nowhere (nbSMTP-1.00) for uid 1000 (using TLSv1/SSLv3 with cipher RC4-MD5 (128/128 bits)) fweisbec@gmail.com; Sun, 18 Apr 2010 17:31:25 +0200 (CEST) Date: Sun, 18 Apr 2010 17:31:24 +0200 From: Frederic Weisbecker To: David Miller Cc: rostedt@goodmis.org, sparclinux@vger.kernel.org Subject: Re: [PATCH 7/7] sparc64: Add function graph tracer support. Message-ID: <20100418153121.GA5174@nowhere> References: <20100417172220.GB15037@nowhere> <20100417.142906.24235257.davem@davemloft.net> <20100417213413.GC15037@nowhere> <20100417.143837.24301795.davem@davemloft.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20100417.143837.24301795.davem@davemloft.net> User-Agent: Mutt/1.5.18 (2008-05-17) Sender: sparclinux-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: sparclinux@vger.kernel.org On Sat, Apr 17, 2010 at 02:38:37PM -0700, David Miller wrote: > From: Frederic Weisbecker > Date: Sat, 17 Apr 2010 23:34:15 +0200 > > > I haven't started the watchdog nor perf, I guess NMI don't trigger > > in other cases, right? > > They do, for the NMI watchdog, every few seconds. > > > For now, the only reentrancy I could find was irqs that interrupt > > the tracing path. Which means no good clue there. That said I > > have only logged recursivity on trace entry path, not yet > > on return. > > > > I'm disabling the protections on entry, just to narrow down > > the recursivity place, in case it only happens on return. > > No need to do so much work, when you hit this case simply > disable tracing and dump_stack(). That way you'll see it > clearly. In fact it's quite hard to dump, because most of them I get are irrelevant (irqs that do reentering the tracing path). And after some time, dumps end up crashing. All I could do is narrowing down the source, everything happens well with this patch: That reminds me badly the problems with NMIs, and I saw an NMI path in one of the dumps, so I guess you were right after all as it looks like more an NMIs related problem rather than a recursion. I thought if I won't enable the watchdog I wouldn't have NMIs but I have some in /proc/interrupts --- To unsubscribe from this list: send the line "unsubscribe sparclinux" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c index 9aed1a5..cfcb863 100644 --- a/kernel/trace/trace_functions_graph.c +++ b/kernel/trace/trace_functions_graph.c @@ -287,7 +287,9 @@ void trace_graph_return(struct ftrace_graph_ret *trace) __trace_graph_return(tr, trace, flags, pc); } atomic_dec(&data->disabled); + pause_graph_tracing(); local_irq_restore(flags); + unpause_graph_tracing(); } void set_graph_array(struct trace_array *tr)