From patchwork Thu Oct 3 16:35:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julian Brown X-Patchwork-Id: 1171276 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=gcc.gnu.org (client-ip=209.132.180.131; helo=sourceware.org; envelope-from=gcc-patches-return-510168-incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b="AGlSX5uM"; dkim-atps=neutral Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 46kdtC0tc6z9sVb for ; Fri, 4 Oct 2019 02:36:02 +1000 (AEST) DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:mime-version :content-transfer-encoding:content-type; q=dns; s=default; b=LjD oSoYKWsZJBGcBUElegSSUBC4sBrckOuMi7+eJ2f5Jmkv8WdcPEBOI9s+JIMsv9iF gtKAT6kVew7htwQ61ZsHBj5KFkt8nlNY3pftsfFCHyLfMiPIHaHcEqyKmHOEWMCK UiyIn3W1/ywkHWWTlEjXYvHGtbB07yiQVVNGc95s= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:mime-version :content-transfer-encoding:content-type; s=default; bh=D+guY0KKm f40wkbcdmXCSu+h9sU=; b=AGlSX5uM/8/htLV1YRzCjovQxCRLImc79+xS/QrSo YikiCC2QP2o1ZzSYVVkOvCKRjZFBJqLAfQCFLKmH7FZ8SvvaxbIPbO4WIkVKfChd t0se6/LVNllh/u7WnI1CcnO6oCHEtLNJ67dH/XnT0HRLdu0SA/yPMVMq5iVhtmTU 4o= Received: (qmail 28265 invoked by alias); 3 Oct 2019 16:35:35 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 28209 invoked by uid 89); 3 Oct 2019 16:35:34 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-22.7 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS autolearn=ham version=3.3.1 spammy=oaccparallelc, oacc-parallel.c, UD:oacc-parallel.c X-HELO: esa1.mentor.iphmx.com Received: from esa1.mentor.iphmx.com (HELO esa1.mentor.iphmx.com) (68.232.129.153) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 03 Oct 2019 16:35:32 +0000 IronPort-SDR: w1omNjsQ/9WoNIMMJ4Ju0q2QLMJ7E53LEJ7rRbmSPtLru0vGLqKXdwwFu4VVtB1Nfg+rbz0t3r 80CC1zZP3mF7bt7CbZYBGXoFCcuB9PoPbrh1s91ID3AVU574f/KyFFZmoF61xJwaiwR5X91zN4 X1o+uFfDqfhVs7VuoSnLd47sJ2No2K8es8q6E586X5DlBVh35jOHb6UVCuCCVyVhu/twoLoQih 91EvLR5rJQ/bfgNk1526VHx5UFTCsMGeGN/j5+3PSptith3u2MUUCOEI3qzi0F5ZqJ26Zmbwl5 uJ8= Received: from orw-gwy-02-in.mentorg.com ([192.94.38.167]) by esa1.mentor.iphmx.com with ESMTP; 03 Oct 2019 08:35:31 -0800 IronPort-SDR: tEvEjs3IOskL0XRfNz25QS1rSZyWV86X0gngcPVRXzVbPdq8YBj/eKmoKNBbcFpk2mb16vOxX8 FBOT5JzObi8Q1t2ENaDqkWcyLLij1Buj/y55DlXeu3de5gg5+R+fuxjgNysQInxQ535PoFteYQ uT1Dub1VKhENYqGIL6LuH2/UmvIUd0rIQ6R5o68wEGD3weNTFDT5Gtek3hpUgnXlGOqxSoxGrY +yXU2jzCLftIAoxXuIYhe4fGaKppEVVw6npQJUleHPiN1PdYfiFkKQj30IU9VR8UhmyGPJaJsO 4qY= From: Julian Brown To: CC: Jakub Jelinek , Thomas Schwinge Subject: [PATCH] OpenACC reference count consistency checking Date: Thu, 3 Oct 2019 09:35:05 -0700 Message-ID: <20191003163505.49997-3-julian@codesourcery.com> MIME-Version: 1.0 X-IsSubscribed: yes This patch provides self-checking for consistency of the OpenACC reference-counting implementation in libgomp. Tested alongside (and dependent on) the patch posted adjacent to this one that overhauls said reference-counting implementation. Tested (with RC_CHECKING enabled) with offloading to NVPTX. OK for trunk? Thanks, Julian 2019-10-02 Julian Brown libgomp/ * libgomp.h (RC_CHECKING): New macro, disabled by default, guarding all hunks in this patch. (target_mem_desc): Add refcount_chk, mark fields. (splay_tree_key_s): Add refcount_chk field. (dump_tgt, gomp_rc_check): Add prototypes. * oacc-parallel.c (GOACC_parallel_keyed): Add refcount self-check code. (GOACC_data_start, GOACC_data_end, GOACC_enter_exit_data): Likewise. * target.c (stdio.h): Include. (dump_tgt, rc_check_clear, rc_check_count, rc_check_verify, gomp_rc_check): New functions to consistency-check reference counts. --- libgomp/libgomp.h | 18 ++++ libgomp/oacc-parallel.c | 33 ++++++++ libgomp/target.c | 178 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 229 insertions(+) diff --git a/libgomp/libgomp.h b/libgomp/libgomp.h index 6b7ed7248a1..19553a37c13 100644 --- a/libgomp/libgomp.h +++ b/libgomp/libgomp.h @@ -873,9 +873,17 @@ struct target_var_desc { uintptr_t length; }; +/* Uncomment to enable reference-count consistency checking (for development + use only). */ +//#define RC_CHECKING 1 + struct target_mem_desc { /* Reference count. */ uintptr_t refcount; +#ifdef RC_CHECKING + uintptr_t refcount_chk; + bool mark; +#endif /* All the splay nodes allocated together. */ splay_tree_node array; /* Start of the target region. */ @@ -930,6 +938,10 @@ struct splay_tree_key_s { If set to VREFCOUNT_LINK_KEY (for OpenMP, where this field is not otherwise needed), the union below represents a link key. */ uintptr_t virtual_refcount; +#ifdef RC_CHECKING + /* The recalculated reference count, for verification. */ + uintptr_t refcount_chk; +#endif union { /* Pointer to the original mapping of "omp declare target link" object. Only used for OpenMP. */ @@ -1084,6 +1096,12 @@ extern void gomp_copy_dev2host (struct gomp_device_descr *, struct goacc_asyncqueue *, void *, const void *, size_t); +#ifdef RC_CHECKING +extern void dump_tgt (const char *, struct target_mem_desc *); +extern void gomp_rc_check (struct gomp_device_descr *, + struct target_mem_desc *); +#endif + extern struct target_mem_desc *gomp_map_vars (struct gomp_device_descr *, size_t, void **, void **, size_t *, void *, bool, diff --git a/libgomp/oacc-parallel.c b/libgomp/oacc-parallel.c index 7e72d9c6b24..18feac5f31c 100644 --- a/libgomp/oacc-parallel.c +++ b/libgomp/oacc-parallel.c @@ -339,6 +339,15 @@ GOACC_parallel_keyed (int flags_m, void (*fn) (void *), &api_info); } +#ifdef RC_CHECKING + gomp_mutex_lock (&acc_dev->lock); + assert (tgt); + dump_tgt (__FUNCTION__, tgt); + tgt->prev = thr->mapped_data; + gomp_rc_check (acc_dev, tgt); + gomp_mutex_unlock (&acc_dev->lock); +#endif + devaddrs = gomp_alloca (sizeof (void *) * mapnum); for (i = 0; i < mapnum; i++) if (tgt->list[i].key != NULL) @@ -389,6 +398,12 @@ GOACC_parallel_keyed (int flags_m, void (*fn) (void *), thr->prof_info = NULL; thr->api_info = NULL; } + +#ifdef RC_CHECKING + gomp_mutex_lock (&acc_dev->lock); + gomp_rc_check (acc_dev, thr->mapped_data); + gomp_mutex_unlock (&acc_dev->lock); +#endif } /* Legacy entry point (GCC 5). Only provide host fallback execution. */ @@ -522,6 +537,12 @@ GOACC_data_start (int flags_m, size_t mapnum, thr->prof_info = NULL; thr->api_info = NULL; } + +#ifdef RC_CHECKING + gomp_mutex_lock (&acc_dev->lock); + gomp_rc_check (acc_dev, thr->mapped_data); + gomp_mutex_unlock (&acc_dev->lock); +#endif } void @@ -595,6 +616,12 @@ GOACC_data_end (void) thr->prof_info = NULL; thr->api_info = NULL; } + +#ifdef RC_CHECKING + gomp_mutex_lock (&thr->dev->lock); + gomp_rc_check (thr->dev, thr->mapped_data); + gomp_mutex_unlock (&thr->dev->lock); +#endif } void @@ -832,6 +859,12 @@ GOACC_enter_exit_data (int flags_m, size_t mapnum, thr->prof_info = NULL; thr->api_info = NULL; } + +#ifdef RC_CHECKING + gomp_mutex_lock (&acc_dev->lock); + gomp_rc_check (acc_dev, thr->mapped_data); + gomp_mutex_unlock (&acc_dev->lock); +#endif } static void diff --git a/libgomp/target.c b/libgomp/target.c index b42b4ad2448..29cb7ca8348 100644 --- a/libgomp/target.c +++ b/libgomp/target.c @@ -38,6 +38,9 @@ #include #include #include +#ifdef RC_CHECKING +#include +#endif #ifdef PLUGIN_SUPPORT #include @@ -346,6 +349,181 @@ gomp_free_device_memory (struct gomp_device_descr *devicep, void *devptr) } } +#ifdef RC_CHECKING +void +dump_tgt (const char *where, struct target_mem_desc *tgt) +{ + if (!getenv ("GOMP_DEBUG_TGT")) + return; + + fprintf (stderr, "%s: %s: tgt=%p\n", __FUNCTION__, where, (void*) tgt); + fprintf (stderr, "refcount=%d\n", (int) tgt->refcount); + fprintf (stderr, "tgt_start=%p\n", (void*) tgt->tgt_start); + fprintf (stderr, "tgt_end=%p\n", (void*) tgt->tgt_end); + fprintf (stderr, "to_free=%p\n", tgt->to_free); + fprintf (stderr, "list_count=%d\n", (int) tgt->list_count); + for (int i = 0; i < tgt->list_count; i++) + { + fprintf (stderr, "list item %d:\n", i); + fprintf (stderr, " key: %p\n", (void*) tgt->list[i].key); + if (tgt->list[i].key) + { + fprintf (stderr, " key.host_start=%p\n", + (void*) tgt->list[i].key->host_start); + fprintf (stderr, " key.host_end=%p\n", + (void*) tgt->list[i].key->host_end); + fprintf (stderr, " key.tgt=%p\n", (void*) tgt->list[i].key->tgt); + fprintf (stderr, " key.offset=%d\n", + (int) tgt->list[i].key->tgt_offset); + fprintf (stderr, " key.refcount=%d\n", + (int) tgt->list[i].key->refcount); + if (tgt->list[i].key->virtual_refcount == VREFCOUNT_LINK_KEY) + fprintf (stderr, " key.u.link_key=%p\n", + (void*) tgt->list[i].key->u.link_key); + else + fprintf (stderr, " key.virtual_refcount=%d\n", + (int) tgt->list[i].key->virtual_refcount); + } + } + fprintf (stderr, "\n"); +} + +static void +rc_check_clear (splay_tree_node node) +{ + splay_tree_key k = &node->key; + + k->refcount_chk = 0; + k->tgt->refcount_chk = 0; + k->tgt->mark = false; + + if (node->left) + rc_check_clear (node->left); + if (node->right) + rc_check_clear (node->right); +} + +static void +rc_check_count (splay_tree_node node) +{ + splay_tree_key k = &node->key; + struct target_mem_desc *t; + + /* Add virtual reference counts ("acc enter data", etc.) for this key. */ + k->refcount_chk += k->virtual_refcount; + + t = k->tgt; + t->refcount_chk++; + + if (!t->mark) + { + for (int i = 0; i < t->list_count; i++) + if (t->list[i].key) + t->list[i].key->refcount_chk++; + + t->mark = true; + } + + if (node->left) + rc_check_count (node->left); + if (node->right) + rc_check_count (node->right); +} + +static bool +rc_check_verify (splay_tree_node node, bool noisy, bool errors) +{ + splay_tree_key k = &node->key; + struct target_mem_desc *t; + + if (k->refcount != REFCOUNT_INFINITY) + { + if (noisy) + fprintf (stderr, "key %p (%p..+%d): rc=%d/%d, virt_rc=%d\n", k, + (void *) k->host_start, (int) (k->host_end - k->host_start), + (int) k->refcount, (int) k->refcount_chk, + (int) k->virtual_refcount); + + if (k->refcount != k->refcount_chk) + { + if (noisy) + fprintf (stderr, " -- key refcount mismatch!\n"); + errors = true; + } + + t = k->tgt; + + if (noisy) + fprintf (stderr, "tgt %p: rc=%d/%d\n", t, (int) t->refcount, + (int) t->refcount_chk); + + if (t->refcount != t->refcount_chk) + { + if (noisy) + fprintf (stderr, + " -- target memory descriptor refcount mismatch!\n"); + errors = true; + } + } + + if (node->left) + errors |= rc_check_verify (node->left, noisy, errors); + if (node->right) + errors |= rc_check_verify (node->right, noisy, errors); + + return errors; +} + +/* Call with device locked. */ + +attribute_hidden void +gomp_rc_check (struct gomp_device_descr *devicep, struct target_mem_desc *tgt) +{ + splay_tree sp = &devicep->mem_map; + + bool noisy = getenv ("GOMP_DEBUG_TGT") != 0; + + if (noisy) + fprintf (stderr, "\n*** GOMP_RC_CHECK ***\n\n"); + + if (sp->root) + { + rc_check_clear (sp->root); + + for (struct target_mem_desc *t = tgt; t; t = t->prev) + { + t->refcount_chk = 0; + t->mark = false; + } + + /* Add references for interconnected splay-tree keys. */ + rc_check_count (sp->root); + + /* Add references for the tgt for a currently-executing kernel and/or + any enclosing data directives. */ + for (struct target_mem_desc *t = tgt; t; t = t->prev) + { + t->refcount_chk++; + + if (!t->mark) + { + for (int i = 0; i < t->list_count; i++) + if (t->list[i].key) + t->list[i].key->refcount_chk++; + + t->mark = true; + } + } + + if (rc_check_verify (sp->root, noisy, false)) + { + gomp_mutex_unlock (&devicep->lock); + gomp_fatal ("refcount checking failure"); + } + } +} +#endif + /* Handle the case where gomp_map_lookup, splay_tree_lookup or gomp_map_0len_lookup found oldn for newn. Helper function of gomp_map_vars. */