From patchwork Thu Nov 29 18:11:39 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Lance Taylor X-Patchwork-Id: 202801 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from sourceware.org (server1.sourceware.org [209.132.180.131]) by ozlabs.org (Postfix) with SMTP id 260902C0087 for ; Fri, 30 Nov 2012 05:12:14 +1100 (EST) Comment: DKIM? See http://www.dkim.org DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=gcc.gnu.org; s=default; x=1354817535; h=Comment: DomainKey-Signature:Received:Received:Received:Received:Received: Received:From:To:Subject:Date:Message-ID:User-Agent:MIME-Version: Content-Type:Mailing-List:Precedence:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:Sender:Delivered-To; bh=Hd8lkak yXjYd1WxLtB9umMLJ8rg=; b=MIZg23IqMsMLkUAQefsbP3o1+3s3X5EE5Jr+BaP I/XoioM4FPnEKauv5lcEwonaI61VeRLg04trnQ6mSPvQyWtJZptak5RdhwDbozzo iINf/38PH4+99uYhAMM5GGDNiR5cqehw9U7a4WNDHPfVOkQOZ+7jjA2Ix3jjUAL/ z9B8= Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=gcc.gnu.org; h=Received:Received:X-SWARE-Spam-Status:X-Spam-Check-By:Received:Received:X-Google-DKIM-Signature:Received:Received:From:To:Subject:Date:Message-ID:User-Agent:MIME-Version:Content-Type:X-Gm-Message-State:X-IsSubscribed:Mailing-List:Precedence:List-Id:List-Unsubscribe:List-Archive:List-Post:List-Help:Sender:Delivered-To; b=O+nxm8KFEs1RDQoxzC0S4UzjZWmzeDFzgPQAN/8nJ/IZaP1+IRS16dHgYQ1IMm 0Zky70B1vqpzhbxoHjh6nDyecjS/RCENL41QQSbsps3fNiID3DwOoojx9aWrwYoU kAyk4T/5MB6L/AUt1VgZToB9dG7LevNZYLOxDX2h1XwtI=; Received: (qmail 29875 invoked by alias); 29 Nov 2012 18:12:03 -0000 Received: (qmail 29851 invoked by uid 22791); 29 Nov 2012 18:12:00 -0000 X-SWARE-Spam-Status: No, hits=-5.4 required=5.0 tests=AWL, BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, KHOP_RCVD_TRUST, RCVD_IN_DNSWL_LOW, RCVD_IN_HOSTKARMA_YE, RP_MATCHES_RCVD, T_TVD_MIME_NO_HEADERS X-Spam-Check-By: sourceware.org Received: from mail-pa0-f47.google.com (HELO mail-pa0-f47.google.com) (209.85.220.47) by sourceware.org (qpsmtpd/0.43rc1) with ESMTP; Thu, 29 Nov 2012 18:11:47 +0000 Received: by mail-pa0-f47.google.com with SMTP id fa10so5194420pad.20 for ; Thu, 29 Nov 2012 10:11:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:to:subject:date:message-id:user-agent:mime-version :content-type:x-gm-message-state; bh=yHpm+KK0IQFHj81xWmf3q/JLegwoHn3gkpeVe0ZzLOc=; b=nQ6lEhVmBXh+4N8RjrVNF3ehYI7Oqy9/60kzfl7Xj0b842rLzmTCSK5Hna2+Qz8plz A+165rQKDUCfaQ5kw1XAfpwckWFqKIY87TtVlrwg3BTQGibSDYmOtlNoYckWoiLPXgxu paFbBG3HKrnVhg/Nw2BEytXmJ/ijwy32CDZKowHexz3AJAjy3+k9qMOofp32s7N/V+iI 7+xNIg+ZjVW708pv3B/hf93h+uEIV6wYhFEHJkldAR3Uhm792NjbaDwSqJEoutz2CuRf Yx/himdlqW0GY1B19m92o0mQNsL6IYNC8kAc8QXqoq2THIrxYeFGcM7CFxQw/Vbx1Rl7 w/8w== Received: by 10.68.234.201 with SMTP id ug9mr70439298pbc.63.1354212707056; Thu, 29 Nov 2012 10:11:47 -0800 (PST) Received: from coign.google.com ([192.168.211.67]) by mx.google.com with ESMTPS id vi9sm1575961pbc.41.2012.11.29.10.11.43 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 29 Nov 2012 10:11:46 -0800 (PST) From: Ian Lance Taylor To: gcc-patches@gcc.gnu.org, gofrontend-dev@googlegroups.com Subject: Go patch committed: Track fields with tag go:"track" Date: Thu, 29 Nov 2012 10:11:39 -0800 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) MIME-Version: 1.0 X-Gm-Message-State: ALoCoQnJrFiupWMYNJ1cBCqTURfo0dROR1v/uxhf+14wYj0ZEW5e6BbN9yU1d7joa22uesUHZVqDW/Ok3Dzqe4lalWKMNC/x46UZjBhNxhXjIvgPLAQXtOdbFsG7LpIrqy9Og2kDpj1IS31UsjyJMkEkGbyky0SHUbpN+ai5wtfMUay/hlOD0NrIhvbpFaCtcL5LtgwG9Ziz X-IsSubscribed: yes Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org This patch implements support for somewhat experimental field tracking. If a struct field has a tag go:"track", the compiler will track references to the field. A new function runtime.Fieldtrack will add the names of all tracked fields to a map. The idea is that linker garbage collection will remove all unreferenced functions, and those permit the program to determine at runtime which fields are actually referenced. The actual implementation is pretty ugly. The compiler inserts function calls for each field reference, passing a string describing the field. The function does nothing. To determine which fields are referenced, the runtime scans the data sections for strings that match the required format. The problem is that we want linker garbage collection to discard unused field references. The middle-end doesn't currently provide any way to associate arbitrary data with a function; it supports putting a switch table in a section associated with a function, but not other data. So I actually store the string in a global variable, and then let the linker garbage collect the function and the global variable. If this experiment turns out to be useful, then at some later point we can enhance the compiler to permit associating arbitrary data with a function. Then the linker garbage collection should do the right thing, discarding the data as well as the function. The data can be put into some other section, which will permit the runtime scan to only look at the relevant data, not the entire data section, and will avoid the remote possibility of a false positive. Bootstrapped and ran Go testsuite on x86_64-unknown-linux-gnu. Committed to mainline. Ian 2012-11-29 Ian Lance Taylor * go-gcc.cc: Include "output.h". (global_variable): Add is_unique_section parameter. (global_variable_set_init): Adjust unique section if necessary. * Make-lang.in (go/go-gcc.o): Add dependency on output.h. Index: gcc/go/go-gcc.cc =================================================================== --- gcc/go/go-gcc.cc (revision 193595) +++ gcc/go/go-gcc.cc (working copy) @@ -28,6 +28,7 @@ #include "tree-iterator.h" #include "gimple.h" #include "toplev.h" +#include "output.h" #include "go-c.h" @@ -267,6 +268,7 @@ class Gcc_backend : public Backend Btype* btype, bool is_external, bool is_hidden, + bool in_unique_section, Location location); void @@ -1277,6 +1279,7 @@ Gcc_backend::global_variable(const std:: Btype* btype, bool is_external, bool is_hidden, + bool in_unique_section, Location location) { tree type_tree = btype->get_tree(); @@ -1308,6 +1311,9 @@ Gcc_backend::global_variable(const std:: } TREE_USED(decl) = 1; + if (in_unique_section) + resolve_unique_section (decl, 0, 1); + go_preserve_from_gc(decl); return new Bvariable(decl); @@ -1326,6 +1332,16 @@ Gcc_backend::global_variable_set_init(Bv if (var_decl == error_mark_node) return; DECL_INITIAL(var_decl) = expr_tree; + + // If this variable goes in a unique section, it may need to go into + // a different one now that DECL_INITIAL is set. + if (DECL_HAS_IMPLICIT_SECTION_NAME_P (var_decl)) + { + DECL_SECTION_NAME (var_decl) = NULL_TREE; + resolve_unique_section (var_decl, + compute_reloc_for_constant (expr_tree), + 1); + } } // Make a local variable. Index: gcc/go/Make-lang.in =================================================================== --- gcc/go/Make-lang.in (revision 193595) +++ gcc/go/Make-lang.in (working copy) @@ -1,6 +1,6 @@ # Make-lang.in -- Top level -*- makefile -*- fragment for gcc Go frontend. -# Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc. +# Copyright (C) 2009, 2010, 2011, 2012 Free Software Foundation, Inc. # This file is part of GCC. @@ -254,7 +254,7 @@ go/go-lang.o: go/go-lang.c $(CONFIG_H) $ GOINCLUDES = -I $(srcdir)/go -I $(srcdir)/go/gofrontend go/go-gcc.o: go/go-gcc.cc $(GO_SYSTEM_H) $(TREE_H) tree-iterator.h \ - $(GIMPLE_H) toplev.h $(GO_C_H) $(GO_GOGO_H) \ + $(GIMPLE_H) toplev.h output.h $(GO_C_H) $(GO_GOGO_H) \ go/gofrontend/backend.h $(CXX) -c $(GOINCLUDES) $(ALL_CPPFLAGS) $(ALL_CXXFLAGS) $< $(OUTPUT_OPTION) Index: gcc/go/gofrontend/gogo.cc =================================================================== --- gcc/go/gofrontend/gogo.cc (revision 193874) +++ gcc/go/gofrontend/gogo.cc (working copy) @@ -3075,7 +3075,8 @@ Function::Function(Function_type* type, closure_var_(NULL), block_(block), location_(location), labels_(), local_type_count_(0), fndecl_(NULL), defer_stack_(NULL), results_are_named_(false), nointerface_(false), calls_recover_(false), - is_recover_thunk_(false), has_recover_thunk_(false) + is_recover_thunk_(false), has_recover_thunk_(false), + in_unique_section_(false) { } @@ -3896,7 +3897,7 @@ Variable::Variable(Type* type, Expressio seen_(false), init_is_lowered_(false), type_from_init_tuple_(false), type_from_range_index_(false), type_from_range_value_(false), type_from_chan_element_(false), is_type_switch_var_(false), - determined_type_(false) + determined_type_(false), in_unique_section_(false) { go_assert(type != NULL || init != NULL); go_assert(!is_parameter || init == NULL); @@ -4315,6 +4316,7 @@ Variable::get_backend_variable(Gogo* gog btype, package != NULL, Gogo::is_hidden_name(name), + this->in_unique_section_, this->location_); else if (function == NULL) { Index: gcc/go/gofrontend/runtime.def =================================================================== --- gcc/go/gofrontend/runtime.def (revision 193595) +++ gcc/go/gofrontend/runtime.def (working copy) @@ -354,6 +354,10 @@ DEF_GO_RUNTIME(PRINT_SPACE, "__go_print_ DEF_GO_RUNTIME(PRINT_NL, "__go_print_nl", P0(), R0()) +// Used for field tracking for data analysis. +DEF_GO_RUNTIME(FIELDTRACK, "__go_fieldtrack", P1(POINTER), R0()) + + // Remove helper macros. #undef ABFT6 #undef ABFT2 Index: gcc/go/gofrontend/gogo.h =================================================================== --- gcc/go/gofrontend/gogo.h (revision 193874) +++ gcc/go/gofrontend/gogo.h (working copy) @@ -1032,6 +1032,11 @@ class Function set_has_recover_thunk() { this->has_recover_thunk_ = true; } + // Mark the function as going into a unique section. + void + set_in_unique_section() + { this->in_unique_section_ = true; } + // Swap with another function. Used only for the thunk which calls // recover. void @@ -1139,6 +1144,9 @@ class Function bool is_recover_thunk_; // True if this function already has a recover thunk. bool has_recover_thunk_; + // True if this function should be put in a unique section. This is + // turned on for field tracking. + bool in_unique_section_ : 1; }; // A snapshot of the current binding state. @@ -1414,6 +1422,14 @@ class Variable set_is_type_switch_var() { this->is_type_switch_var_ = true; } + // Mark the variable as going into a unique section. + void + set_in_unique_section() + { + go_assert(this->is_global_); + this->in_unique_section_ = true; + } + // Traverse the initializer expression. int traverse_expression(Traverse*, unsigned int traverse_mask); @@ -1504,6 +1520,9 @@ class Variable bool is_type_switch_var_ : 1; // True if we have determined types. bool determined_type_ : 1; + // True if this variable should be put in a unique section. This is + // used for field tracking. + bool in_unique_section_ : 1; }; // A variable which is really the name for a function return value, or Index: gcc/go/gofrontend/expressions.h =================================================================== --- gcc/go/gofrontend/expressions.h (revision 193595) +++ gcc/go/gofrontend/expressions.h (working copy) @@ -1842,7 +1842,7 @@ class Field_reference_expression : publi Field_reference_expression(Expression* expr, unsigned int field_index, Location location) : Expression(EXPRESSION_FIELD_REFERENCE, location), - expr_(expr), field_index_(field_index) + expr_(expr), field_index_(field_index), called_fieldtrack_(false) { } // Return the struct expression. @@ -1868,6 +1868,9 @@ class Field_reference_expression : publi do_traverse(Traverse* traverse) { return Expression::traverse(&this->expr_, traverse); } + Expression* + do_lower(Gogo*, Named_object*, Statement_inserter*, int); + Type* do_type(); @@ -1906,6 +1909,8 @@ class Field_reference_expression : publi Expression* expr_; // The zero-based index of the field we are retrieving. unsigned int field_index_; + // Whether we have already emitted a fieldtrack call. + bool called_fieldtrack_; }; // A reference to a field of an interface. Index: gcc/go/gofrontend/gogo-tree.cc =================================================================== --- gcc/go/gofrontend/gogo-tree.cc (revision 193596) +++ gcc/go/gofrontend/gogo-tree.cc (working copy) @@ -1316,6 +1316,9 @@ Function::get_or_make_decl(Gogo* gogo, N DECL_ATTRIBUTES(decl) = tree_cons(attr, NULL_TREE, NULL_TREE); } + if (this->in_unique_section_) + resolve_unique_section (decl, 0, 1); + go_preserve_from_gc(decl); if (this->closure_var_ != NULL) Index: gcc/go/gofrontend/backend.h =================================================================== --- gcc/go/gofrontend/backend.h (revision 193595) +++ gcc/go/gofrontend/backend.h (working copy) @@ -326,8 +326,11 @@ class Backend // option. NAME is the name of the variable. BTYPE is the type of // the variable. IS_EXTERNAL is true if the variable is defined in // some other package. IS_HIDDEN is true if the variable is not - // exported (name begins with a lower case letter). LOCATION is - // where the variable was defined. + // exported (name begins with a lower case letter). + // IN_UNIQUE_SECTION is true if the variable should be put into a + // unique section if possible; this is intended to permit the linker + // to garbage collect the variable if it is not referenced. + // LOCATION is where the variable was defined. virtual Bvariable* global_variable(const std::string& package_name, const std::string& pkgpath, @@ -335,6 +338,7 @@ class Backend Btype* btype, bool is_external, bool is_hidden, + bool in_unique_section, Location location) = 0; // A global variable will 1) be initialized to zero, or 2) be Index: gcc/go/gofrontend/expressions.cc =================================================================== --- gcc/go/gofrontend/expressions.cc (revision 193596) +++ gcc/go/gofrontend/expressions.cc (working copy) @@ -10580,6 +10580,102 @@ Expression::make_map_index(Expression* m // Class Field_reference_expression. +// Lower a field reference expression. There is nothing to lower, but +// this is where we generate the tracking information for fields with +// the magic go:"track" tag. + +Expression* +Field_reference_expression::do_lower(Gogo* gogo, Named_object* function, + Statement_inserter* inserter, int) +{ + Struct_type* struct_type = this->expr_->type()->struct_type(); + if (struct_type == NULL) + { + // Error will be reported elsewhere. + return this; + } + const Struct_field* field = struct_type->field(this->field_index_); + if (field == NULL) + return this; + if (!field->has_tag()) + return this; + if (field->tag().find("go:\"track\"") == std::string::npos) + return this; + + // We have found a reference to a tracked field. Build a call to + // the runtime function __go_fieldtrack with a string that describes + // the field. FIXME: We should only call this once per referenced + // field per function, not once for each reference to the field. + + if (this->called_fieldtrack_) + return this; + this->called_fieldtrack_ = true; + + Location loc = this->location(); + + std::string s = "fieldtrack \""; + Named_type* nt = this->expr_->type()->named_type(); + if (nt == NULL || nt->named_object()->package() == NULL) + s.append(gogo->pkgpath()); + else + s.append(nt->named_object()->package()->pkgpath()); + s.push_back('.'); + if (nt != NULL) + s.append(nt->name()); + s.push_back('.'); + s.append(field->field_name()); + s.push_back('"'); + + // We can't use a string here, because internally a string holds a + // pointer to the actual bytes; when the linker garbage collects the + // string, it won't garbage collect the bytes. So we use a + // [...]byte. + + mpz_t val; + mpz_init_set_ui(val, s.length()); + Expression* length_expr = Expression::make_integer(&val, NULL, loc); + mpz_clear(val); + + Type* byte_type = gogo->lookup_global("byte")->type_value(); + Type* array_type = Type::make_array_type(byte_type, length_expr); + + Expression_list* bytes = new Expression_list(); + for (std::string::const_iterator p = s.begin(); p != s.end(); p++) + { + mpz_init_set_ui(val, *p); + Expression* byte = Expression::make_integer(&val, NULL, loc); + mpz_clear(val); + bytes->push_back(byte); + } + + Expression* e = Expression::make_composite_literal(array_type, 0, false, + bytes, loc); + + Variable* var = new Variable(array_type, e, true, false, false, loc); + + static int count; + char buf[50]; + snprintf(buf, sizeof buf, "fieldtrack.%d", count); + ++count; + + Named_object* no = gogo->add_variable(buf, var); + e = Expression::make_var_reference(no, loc); + e = Expression::make_unary(OPERATOR_AND, e, loc); + + Expression* call = Runtime::make_call(Runtime::FIELDTRACK, loc, 1, e); + inserter->insert(Statement::make_statement(call, false)); + + // Put this function, and the global variable we just created, into + // unique sections. This will permit the linker to garbage collect + // them if they are not referenced. The effect is that the only + // strings, indicating field references, that will wind up in the + // executable will be those for functions that are actually needed. + function->func_value()->set_in_unique_section(); + var->set_in_unique_section(); + + return this; +} + // Return the type of a field reference. Type* Index: libgo/runtime/go-fieldtrack.c =================================================================== --- libgo/runtime/go-fieldtrack.c (revision 0) +++ libgo/runtime/go-fieldtrack.c (revision 0) @@ -0,0 +1,101 @@ +/* go-fieldtrack.c -- structure field data analysis. + + Copyright 2012 The Go Authors. All rights reserved. + Use of this source code is governed by a BSD-style + license that can be found in the LICENSE file. */ + +#include "runtime.h" +#include "go-type.h" +#include "map.h" + +/* The compiler will track fields that have the tag go:"track". Any + function that refers to such a field will call this function with a + string + fieldtrack "package.type.field" + + This function does not actually do anything. Instead, we gather + the field tracking information by looking for strings of that form + in the read-only data section. This is, of course, a horrible + hack, but it's good enough for now. We can improve it, e.g., by a + linker plugin, if this turns out to be useful. */ + +void +__go_fieldtrack (byte *p __attribute__ ((unused))) +{ +} + +/* A runtime function to add all the tracked fields to a + map[string]bool. */ + +extern const char _etext[] __attribute__ ((weak)); +extern const char __etext[] __attribute__ ((weak)); +extern const char __data_start[] __attribute__ ((weak)); +extern const char _edata[] __attribute__ ((weak)); +extern const char __edata[] __attribute__ ((weak)); +extern const char __bss_start[] __attribute__ ((weak)); + +void runtime_Fieldtrack (struct __go_map *) __asm__ ("runtime.Fieldtrack"); + +void +runtime_Fieldtrack (struct __go_map *m) +{ + const char *p; + const char *pend; + const char *prefix; + size_t prefix_len; + + p = __data_start; + if (p == NULL) + p = __etext; + if (p == NULL) + p = _etext; + if (p == NULL) + return; + + pend = __edata; + if (pend == NULL) + pend = _edata; + if (pend == NULL) + pend = __bss_start; + if (pend == NULL) + return; + + prefix = "fieldtrack "; + prefix_len = __builtin_strlen (prefix); + + while (p < pend) + { + const char *q1; + const char *q2; + + q1 = __builtin_memchr (p + prefix_len, '"', pend - (p + prefix_len)); + if (q1 == NULL) + break; + + if (__builtin_memcmp (q1 - prefix_len, prefix, prefix_len) != 0) + { + p = q1 + 1; + continue; + } + + q1++; + q2 = __builtin_memchr (q1, '"', pend - q1); + if (q2 == NULL) + break; + + if (__builtin_memchr (q1, '\0', q2 - q1) == NULL) + { + String s; + void *v; + _Bool *pb; + + s.str = (const byte *) q1; + s.len = q2 - q1; + v = __go_map_index (m, &s, 1); + pb = (_Bool *) v; + *pb = 1; + } + + p = q2; + } +} Index: libgo/go/runtime/debug.go =================================================================== --- libgo/go/runtime/debug.go (revision 193688) +++ libgo/go/runtime/debug.go (working copy) @@ -168,3 +168,10 @@ func BlockProfile(p []BlockProfileRecord // If all is true, Stack formats stack traces of all other goroutines // into buf after the trace for the current goroutine. func Stack(buf []byte, all bool) int + +// Get field tracking information. Only fields with a tag go:"track" +// are tracked. This function will add every such field that is +// referenced to the map. The keys in the map will be +// PkgPath.Name.FieldName. The value will be true for each field +// added. +func Fieldtrack(map[string]bool) Index: libgo/Makefile.am =================================================================== --- libgo/Makefile.am (revision 193688) +++ libgo/Makefile.am (working copy) @@ -450,6 +450,7 @@ runtime_files = \ runtime/go-deferred-recover.c \ runtime/go-eface-compare.c \ runtime/go-eface-val-compare.c \ + runtime/go-fieldtrack.c \ runtime/go-getgoroot.c \ runtime/go-int-array-to-string.c \ runtime/go-int-to-string.c \