From patchwork Tue Jun 27 09:16:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tom de Vries X-Patchwork-Id: 781087 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wxgKx6jL4z9s4q for ; Tue, 27 Jun 2017 19:17:09 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=gcc.gnu.org header.i=@gcc.gnu.org header.b="R07gCX23"; dkim-atps=neutral DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:references:subject:message-id:date:mime-version :in-reply-to:content-type; q=dns; s=default; b=JBP2wBM0gAgIDpvwt NQvUMuRZPu0906lEWG/qfVvLGhEVzJKK/MCmQ2qZaQhEo8I2usAM9UR0+B2aT+Z8 OkXJ2vZOAAAnGI8Kimpys9p3FzY1GOzJZtmPhe6EF4aPkjVPHbEGGQ9L0aZOh5WI G9CTTVUyVQdBXCnazXQi5AcyUQ= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:references:subject:message-id:date:mime-version :in-reply-to:content-type; s=default; bh=+INt3/c4yCCA4iVVTOAVsiW cSCw=; b=R07gCX23wV7atQoAUbbTxjZNXH8xlu2rVFJC771N3onFH7ye80ODvDp Zh3OXCM8b8xmnP2+afrYICooZQrfmBWeWuE2m5+HxrGJIB0CGyXEd2YZnJLLkqho hkrPyJ956ij75KdZiC1/8mrXj4RJXwqybklNubUtp4v6OeH4zd3o= Received: (qmail 55295 invoked by alias); 27 Jun 2017 09:17:00 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 55268 invoked by uid 89); 27 Jun 2017 09:16:59 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-24.5 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS, URIBL_RED autolearn=ham version=3.3.2 spammy= X-HELO: relay1.mentorg.com Received: from relay1.mentorg.com (HELO relay1.mentorg.com) (192.94.38.131) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 27 Jun 2017 09:16:57 +0000 Received: from nat-ies.mentorg.com ([192.94.31.2] helo=SVR-IES-MBX-04.mgc.mentorg.com) by relay1.mentorg.com with esmtp id 1dPmcA-0006ts-AC from Tom_deVries@mentor.com ; Tue, 27 Jun 2017 02:16:54 -0700 Received: from [127.0.0.1] (137.202.0.87) by SVR-IES-MBX-04.mgc.mentorg.com (139.181.222.4) with Microsoft SMTP Server (TLS) id 15.0.1263.5; Tue, 27 Jun 2017 10:16:50 +0100 From: Tom de Vries To: Jakub Jelinek CC: GCC Patches , Thomas Schwinge References: <2413b0f6-9cb2-243f-d805-08323a9c9a0a@mentor.com> Subject: [PATCH, 5/4] Handle GOMP_OPENACC_NVPTX_PTXRW in libgomp nvptx plugin Message-ID: <0f449cf7-09fc-0909-a1d3-6e5b78014fe5@mentor.com> Date: Tue, 27 Jun 2017 11:16:46 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.1 MIME-Version: 1.0 In-Reply-To: <2413b0f6-9cb2-243f-d805-08323a9c9a0a@mentor.com> X-ClientProxiedBy: svr-ies-mbx-01.mgc.mentorg.com (139.181.222.1) To SVR-IES-MBX-04.mgc.mentorg.com (139.181.222.4) [ was: Re: [PATCH, 0/4] Handle GOMP_OPENACC_NVPTX_{DISASM,SAVE_TEMPS,JIT} in libgomp nvptx plugin ] On 06/26/2017 01:24 PM, Tom de Vries wrote: > Hi, > > I've written a patch series to facilitate debugging libgomp openacc > testcase failures on the nvptx accelerator. > > > When running an openacc test-case on an nvptx accelerator, the following > happens: > - the plugin obtains the ptx assembly for the acceleration kernels > - it calls the cuda jit to compile and link the ptx into a module > - it loads the module > - it starts an acceleration kernel A typical scenario when developing the compiler is: - run gcc test.c -save-temps - run a.out - edit test.s to fix bug or make code faster or smaller - run gcc test.s - run a.out - edit compiler sources to make the compiler do the same as the .s edit With openacc test-cases, this scenario is currently not available for ptx assembly. Using -save-temps -foffload=-save-temps we can get a .s containing ptx. But to insert the edited .s back into the compilation flow is difficult. This patch facilitates such a scenario in the nvptx plugin. - we define GOMP_OPENACC_NVPTX_PTXRW == 'w', and the plugin writes the ptx assembly into a series of files - we edit one of those files - we define GOMP_OPENACC_NVPTX_PTXRW == 'r', and the plugin reads the ptx assembly back from those files, and uses that instead of the ptx in the executable. I've tested this patch series on top of gomp-4_0-branch, by running an openacc testcase from the command line and going through the write-edit-readscenario with an observable ptx edit. OK for trunk if bootstrap and reg-test on x86_64 with nvidia accelerator succeeds? Thanks, - Tom Handle GOMP_OPENACC_NVPTX_PTXRW in libgomp nvptx plugin 2017-06-27 Tom de Vries * plugin/plugin-nvptx.c (post_process_ptx): New function. (link_ptx): Call post_process_ptx. --- libgomp/plugin/plugin-nvptx.c | 129 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 127 insertions(+), 2 deletions(-) diff --git a/libgomp/plugin/plugin-nvptx.c b/libgomp/plugin/plugin-nvptx.c index db42292..26e453f 100644 --- a/libgomp/plugin/plugin-nvptx.c +++ b/libgomp/plugin/plugin-nvptx.c @@ -1020,6 +1020,128 @@ process_GOMP_OPENACC_NVPTX_JIT (intptr_t *gomp_openacc_nvptx_o, } } +/* If environment variable GOMP_OPENACC_NVPTX_PTXRW=[Ww], write *RES_CODE to + file plugin-nvptx..ptx. If it is [Rr], read *RES_CODE from file + instead. */ + +static void +post_process_ptx (unsigned num, const char **res_code, size_t *res_size) +{ + static int gomp_openacc_nvptx_ptxrw = -1; + + if (gomp_openacc_nvptx_ptxrw == -1) + { + const char *var_name = "GOMP_OPENACC_NVPTX_PTXRW"; + const char *env_var = secure_getenv (var_name); + notify_var (var_name, env_var); + + gomp_openacc_nvptx_ptxrw = 0; + if (env_var == NULL) + ; + else if ((env_var[0] == 'w' || env_var[0] == 'W') + && env_var[1] == '\0') + gomp_openacc_nvptx_ptxrw = 1; + else if ((env_var[0] == 'r' || env_var[0] == 'R') + && env_var[1] == '\0') + gomp_openacc_nvptx_ptxrw = 2; + else + GOMP_PLUGIN_error ("Error parsing %s", var_name); + } + + if (gomp_openacc_nvptx_ptxrw == 0) + return; + + const char *prefix = "plugin-nvptx."; + const char *postfix = ".ptx"; + const int len = (strlen (prefix) + + 10 /* %u. */ + + strlen (postfix) + + 1 /* '\0'. */); + char file_name[len]; + int res = snprintf (file_name, len, "%s%u%s", prefix, + num, postfix); + assert (res < len); /* Assert there's no truncation. */ + + GOMP_PLUGIN_debug (0, "%s %s \n", + (gomp_openacc_nvptx_ptxrw == 1 ? "Writing" : "Reading"), + file_name); + + if (gomp_openacc_nvptx_ptxrw == 1) + { + FILE *ptx_file = fopen (file_name, "w"); + if (ptx_file == NULL) + { + GOMP_PLUGIN_debug (0, "Opening %s failed\n", file_name); + return; + } + + int res = fprintf (ptx_file, "%s", code); + unsigned int write_succeeded = res == size - 1; + if (!write_succeeded) + GOMP_PLUGIN_debug (0, + "Writing %s failed: written %d but expected %zu\n", + file_name, res, size - 1); + + res = fclose (ptx_file); + if (res != 0) + GOMP_PLUGIN_debug (0, "Closing %s failed\n", file_name); + + return; + } + + if (gomp_openacc_nvptx_ptxrw == 2) + { + FILE *ptx_file = fopen (file_name, "r"); + if (ptx_file == NULL) + { + GOMP_PLUGIN_debug (0, "Opening %s failed\n", file_name); + return; + } + + if (fseek (ptx_file, 0L, SEEK_END) != 0) + { + GOMP_PLUGIN_debug (0, "Seeking end of %s failed\n", file_name); + return; + } + + long bufsize = ftell (ptx_file); + if (bufsize == -1) + { + GOMP_PLUGIN_debug (0, "ftell of %s failed\n", file_name); + return; + } + + if (fseek (ptx_file, 0L, SEEK_SET) != 0) + { + GOMP_PLUGIN_debug (0, "Seeking start of %s failed\n", file_name); + return; + } + + char *new_code = GOMP_PLUGIN_malloc (sizeof (char) * (bufsize + 1)); + + size_t new_size = fread (new_code, sizeof (char), bufsize, ptx_file); + if (ferror (ptx_file) != 0) + { + GOMP_PLUGIN_debug (0, "Reading %s failed\n", file_name); + return; + } + + assert (new_size < bufsize + 1); + new_code[new_size++] = '\0'; + + int res = fclose (ptx_file); + if (res != 0) + { + GOMP_PLUGIN_debug (0, "Closing %s failed\n", file_name); + return; + } + + *res_code = new_code; + *res_size = new_size; + return; + } +} + static bool link_ptx (CUmodule *module, const struct targ_ptx_obj *ptx_objs, unsigned num_objs) @@ -1073,11 +1195,14 @@ link_ptx (CUmodule *module, const struct targ_ptx_obj *ptx_objs, for (; num_objs--; ptx_objs++) { + const char *ptx_code = ptx_objs->code; + size_t ptx_size = ptx_objs->size; + post_process_ptx (num_objs, &ptx_code, &ptx_size); + GOMP_PLUGIN_debug (0, "Loading:\n---\n%s\n---\n", ptx_code); /* cuLinkAddData's 'data' argument erroneously omits the const qualifier. */ - GOMP_PLUGIN_debug (0, "Loading:\n---\n%s\n---\n", ptx_objs->code); r = CUDA_CALL_NOCHECK (cuLinkAddData, linkstate, CU_JIT_INPUT_PTX, - (char *) ptx_objs->code, ptx_objs->size, + (char *) ptx_code, ptx_size, 0, 0, 0, 0); if (r != CUDA_SUCCESS) {