From patchwork Tue Jul 9 15:28:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 1958520 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gcc.gnu.org (client-ip=8.43.85.97; helo=server2.sourceware.org; envelope-from=gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org; receiver=patchwork.ozlabs.org) Received: from server2.sourceware.org (server2.sourceware.org [8.43.85.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WJPzN0lnPz1ySg for ; Wed, 10 Jul 2024 01:29:04 +1000 (AEST) Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 633EB3841FD8 for ; Tue, 9 Jul 2024 15:29:02 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by sourceware.org (Postfix) with ESMTP id 2E78138432E2 for ; Tue, 9 Jul 2024 15:28:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 2E78138432E2 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=arm.com ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 2E78138432E2 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720538914; cv=none; b=c+o0feu4D+88e0Nh0n2iXmdm23r/N+g53u8Oh/oeRBjNF+12iqSvxMhzsqHLSLUGm9I26tQpXzbbasa55Zru2BOD+qwYK5Q3EW/zK0YSIMshLkxxtXYQQSwF38jUPka2x5zqTQ8puXOtoRQTadceo5I4Qz8/RIFb7Boq0pAd+WA= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1720538914; c=relaxed/simple; bh=omVLBr/I1PkRw+gkm/dr4hRtGIz3U9FHBP3Q12EntF0=; h=From:To:Subject:Date:Message-ID:MIME-Version; b=mSe6MYcgFZ8ZCqeZaC3AI8K6ENAcLaqSHyBnxtrv7KXMyjFL9l1e1dh4f2mvzJ25S/AugBlpu94z+i095oZk8OW2rf9lJtn9p7vw/3oU4a2C5qr4XcrJCubOGPi/NdkNjmmowII/OjnhyEnp/jCsmVKEhwAedcrpzqBbXbb94PM= ARC-Authentication-Results: i=1; server2.sourceware.org Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E92E2153B for ; Tue, 9 Jul 2024 08:28:53 -0700 (PDT) Received: from localhost (e121540-lin.manchester.arm.com [10.32.110.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 01BAE3F766 for ; Tue, 9 Jul 2024 08:28:27 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@arm.com Subject: [PATCH 2/2] rtl-ssa: Fix prev_any_insn [PR115785] Date: Tue, 09 Jul 2024 16:28:26 +0100 Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 X-Spam-Status: No, score=-19.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_DMARC_NONE, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces~incoming=patchwork.ozlabs.org@gcc.gnu.org Bit of a brown paper bag issue, but: due to the representation of the insn chain, insn_info::prev_any_insn would sometimes skip over instructions. This led to an invalid update in the PR when adding and removing instructions. I think one of the reasons I failed to spot this when checking the code is that m_prev_insn_or_last_debug_insn is misnamed: it's the previous instruction *of the same type* or the last debug instruction in a group. The patch therefore renames it to m_prev_sametype_or_last_debug_insn (with the term prev_sametype already being used in some accessors). The reason this didn't show up earlier is that (a) prev_any_insn is rarely used directly, (b) no instructions were lost from the def-use chains, and (c) only consecutive debug instructions were skipped when walking the insn chain. The chaining scheme makes prev_any_insn more complicated than next_any_insn, prev_nondebug_insn and next_nondebug_insn, but the object code produced is still relatively simple. Bootstrapped & regression-tested on aarch64-linux-gnu and x86_64-linux-gnu. OK to install? Richard gcc/ PR rtl-optimization/115785 * rtl-ssa/insns.h (insn_info::prev_insn_or_last_debug_insn) (insn_info::next_nondebug_or_debug_insn): Remove typedefs. (insn_info::m_prev_insn_or_last_debug_insn): Rename to... (insn_info::m_prev_sametype_or_last_debug_insn): ...this. * rtl-ssa/internals.inl (insn_info::insn_info): Update after above renaming. (insn_info::copy_prev_from): Likewise. (insn_info::set_prev_sametype_insn): Likewise. (insn_info::set_last_debug_insn): Likewise. (insn_info::clear_insn_links): Likewise. (insn_info::has_insn_links): Likewise. * rtl-ssa/member-fns.inl (insn_info::prev_nondebug_insn): Likewise. (insn_info::prev_any_insn): Fix moves from non-debug to debug insns. gcc/testsuite/ PR rtl-optimization/115785 * g++.dg/torture/pr115785.C: New test. --- gcc/rtl-ssa/insns.h | 54 +- gcc/rtl-ssa/internals.inl | 13 +- gcc/rtl-ssa/member-fns.inl | 25 +- gcc/testsuite/g++.dg/torture/pr115785.C | 696 ++++++++++++++++++++++++ 4 files changed, 747 insertions(+), 41 deletions(-) create mode 100644 gcc/testsuite/g++.dg/torture/pr115785.C diff --git a/gcc/rtl-ssa/insns.h b/gcc/rtl-ssa/insns.h index 80eae5eaa1e..1304b18e085 100644 --- a/gcc/rtl-ssa/insns.h +++ b/gcc/rtl-ssa/insns.h @@ -339,32 +339,6 @@ private: }; using order_splay_tree = default_rootless_splay_tree; - // prev_insn_or_last_debug_insn represents a choice between two things: - // - // (1) A pointer to the previous instruction in the list that has the - // same is_debug_insn () value, or null if no such instruction exists. - // - // (2) A pointer to the end of a sublist of debug instructions. - // - // (2) is used if this instruction is a debug instruction and the - // previous instruction is not. (1) is used otherwise. - // - // next_nondebug_or_debug_insn points to the next instruction but also - // records whether that next instruction is a debug instruction or a - // nondebug instruction. - // - // Thus the list is chained as follows: - // - // ----> ----> ----> ----> ----> - // NONDEBUG NONDEBUG DEBUG DEBUG DEBUG NONDEBUG ... - // <---- ^ +-- <---- <---- ^ +-- - // | | | | - // | +------------------------+ | - // | | - // +-----------------------------------+ - using prev_insn_or_last_debug_insn = pointer_mux; - using next_nondebug_or_debug_insn = pointer_mux; - insn_info (bb_info *bb, rtx_insn *rtl, int cost_or_uid); static void print_uid (pretty_printer *, int); @@ -395,9 +369,33 @@ private: void clear_insn_links (); bool has_insn_links (); + // m_prev_sametye_or_last_debug_insn represents a choice between two things: + // + // (1) A pointer to the previous instruction in the list that has the + // same is_debug_insn () value, or null if no such instruction exists. + // + // (2) A pointer to the end of a sublist of debug instructions. + // + // (2) is used if this instruction is a debug instruction and the + // previous instruction is not. (1) is used otherwise. + // + // m_next_nondebug_or_debug_insn points to the next instruction but also + // records whether that next instruction is a debug instruction or a + // nondebug instruction. + // + // Thus the list is chained as follows: + // + // ----> ----> ----> ----> ----> + // NONDEBUG NONDEBUG DEBUG DEBUG DEBUG NONDEBUG ... + // <---- ^ +-- <---- <---- ^ +-- + // | | | | + // | +------------------------+ | + // | | + // +-----------------------------------+ + pointer_mux m_prev_sametype_or_last_debug_insn; + pointer_mux m_next_nondebug_or_debug_insn; + // The values returned by the accessors above. - prev_insn_or_last_debug_insn m_prev_insn_or_last_debug_insn; - next_nondebug_or_debug_insn m_next_nondebug_or_debug_insn; bb_info *m_bb; rtx_insn *m_rtl; diff --git a/gcc/rtl-ssa/internals.inl b/gcc/rtl-ssa/internals.inl index 73581a02bad..c736877479e 100644 --- a/gcc/rtl-ssa/internals.inl +++ b/gcc/rtl-ssa/internals.inl @@ -403,7 +403,7 @@ inline insn_call_clobbers_note::insn_call_clobbers_note (unsigned int abi_id, // If the instruction is real, COST_OR_UID is the value of cost (), // otherwise it is the value of uid (). inline insn_info::insn_info (bb_info *bb, rtx_insn *rtl, int cost_or_uid) - : m_prev_insn_or_last_debug_insn (nullptr), + : m_prev_sametype_or_last_debug_insn (nullptr), m_next_nondebug_or_debug_insn (nullptr), m_bb (bb), m_rtl (rtl), @@ -486,7 +486,8 @@ insn_info::get_known_order_node () const inline void insn_info::copy_prev_from (insn_info *other) { - m_prev_insn_or_last_debug_insn = other->m_prev_insn_or_last_debug_insn; + m_prev_sametype_or_last_debug_insn + = other->m_prev_sametype_or_last_debug_insn; } // Copy the overloaded next link from OTHER. @@ -504,7 +505,7 @@ insn_info::copy_next_from (insn_info *other) inline void insn_info::set_prev_sametype_insn (insn_info *prev) { - m_prev_insn_or_last_debug_insn.set_first (prev); + m_prev_sametype_or_last_debug_insn.set_first (prev); } // Only valid for debug instructions. Record that this instruction starts @@ -512,7 +513,7 @@ insn_info::set_prev_sametype_insn (insn_info *prev) inline void insn_info::set_last_debug_insn (insn_info *last) { - m_prev_insn_or_last_debug_insn.set_second (last); + m_prev_sametype_or_last_debug_insn.set_second (last); } // Record that the next instruction of any kind is NEXT. @@ -529,7 +530,7 @@ insn_info::set_next_any_insn (insn_info *next) inline void insn_info::clear_insn_links () { - m_prev_insn_or_last_debug_insn = nullptr; + m_prev_sametype_or_last_debug_insn = nullptr; m_next_nondebug_or_debug_insn = nullptr; m_point = 0; } @@ -539,7 +540,7 @@ insn_info::clear_insn_links () inline bool insn_info::has_insn_links () { - return (m_prev_insn_or_last_debug_insn + return (m_prev_sametype_or_last_debug_insn || m_next_nondebug_or_debug_insn || m_point); } diff --git a/gcc/rtl-ssa/member-fns.inl b/gcc/rtl-ssa/member-fns.inl index 833907b62c9..d39184fb8cd 100644 --- a/gcc/rtl-ssa/member-fns.inl +++ b/gcc/rtl-ssa/member-fns.inl @@ -554,7 +554,7 @@ inline insn_info * insn_info::prev_nondebug_insn () const { gcc_checking_assert (!is_debug_insn ()); - return m_prev_insn_or_last_debug_insn.known_first (); + return m_prev_sametype_or_last_debug_insn.known_first (); } inline insn_info * @@ -570,12 +570,23 @@ insn_info::next_nondebug_insn () const inline insn_info * insn_info::prev_any_insn () const { - const insn_info *from = this; - if (insn_info *last_debug = m_prev_insn_or_last_debug_insn.second_or_null ()) + if (auto *last_debug = m_prev_sametype_or_last_debug_insn.second_or_null ()) // This instruction is the first in a subsequence of debug instructions. - // Move to the following nondebug instruction. - from = last_debug->m_next_nondebug_or_debug_insn.known_first (); - return from->m_prev_insn_or_last_debug_insn.known_first (); + // Move to the following nondebug instruction and get the previous one + // from there. + return (last_debug->m_next_nondebug_or_debug_insn.known_first () + ->m_prev_sametype_or_last_debug_insn.known_first ()); + auto *prev = m_prev_sametype_or_last_debug_insn.known_first (); + if (prev) + { + auto *next = prev->next_any_insn (); + if (next != this) + // This instruction is a non-debug instruction and there are some + // debug instructions between it and PREV. NEXT is the first of + // the debug instructions; get the last. + return next->m_prev_sametype_or_last_debug_insn.known_second (); + } + return prev; } inline insn_info * @@ -663,7 +674,7 @@ insn_info::find_note () const inline insn_info * insn_info::last_debug_insn () const { - return m_prev_insn_or_last_debug_insn.known_second (); + return m_prev_sametype_or_last_debug_insn.known_second (); } inline insn_range_info::insn_range_info (insn_info *first, insn_info *last) diff --git a/gcc/testsuite/g++.dg/torture/pr115785.C b/gcc/testsuite/g++.dg/torture/pr115785.C new file mode 100644 index 00000000000..498dfc9cb28 --- /dev/null +++ b/gcc/testsuite/g++.dg/torture/pr115785.C @@ -0,0 +1,696 @@ +// { dg-do compile { target { x86_64-*-* i?86-*-* } } } +// { dg-require-effective-target c++11 } +// { dg-additional-options "-fno-vect-cost-model -fno-strict-overflow -fno-delete-null-pointer-checks -fwrapv -fsigned-char -w -mf16c -mavx2 -mfma -mlzcnt -mbmi -mbmi2" } + +void *operator new(__SIZE_TYPE__, void *); +struct a { + a(int, int); +}; +typedef float b __attribute__((__vector_size__(16))); +b ad, d, f; +b e(b h, b m) { return __builtin_ia32_maxps(h, m); } +typedef int g __attribute__((__vector_size__(16))); +typedef long long aa __attribute__((__vector_size__(16))); +aa af(b h) { return aa(h); } +aa ae; +struct ah { + template ah(ag, int); +}; +template aj ai(aj h, aj m, aj p) { return an(ak(h, p), m); } +struct q { + b j; + q(b h) : j(h) {} + operator b() { return j; } +}; +struct t { + aa j; + t(aa h) : j(h) {} + operator aa() { return j; } + t(int) : j() {} + t(b h) : j(af(h)) {} +}; +template struct M { + b j; + M(b h) : j(h) {} + operator b() { return j; } + M(int); +}; +template struct ap { + using aq = M; + using ar = M; + using as = t; + using at = q; +}; +template using aq = typename ap::aq; +template using ar = typename ap::ar; +template using as = typename ap::as; +template using at = typename ap::at; +typedef at<4> au; +ar<4> operator&(ar<4> h, ar<4>) { + b av, aw = h; + av = __builtin_ia32_andps(aw, d); + return av; +} +int ax, ay = 0, az, ba, bb, bc; +bool any(ar<4> h) { + __attribute__((__vector_size__(4 * sizeof(float)))) float bd = h; + ax = __builtin_ia32_movmskps(bd); + return ax; +} +ar<4> operator<(as<4> h, as<4>) { + aa be, bf = h, ac; + be = bf < ac; + return b(be); +} +as<4> ak(as<4> h, as<4>) { + aa bg, bh = h; + bg = (aa)__builtin_ia32_pminsd128((g)bh, (g)ae); + return bg; +} +as<4> an(as<4> h, as<4> m) { + aa bi, bj = h, bk = m; + bi = (aa)__builtin_ia32_pmaxsd128((g)bj, (g)bk); + return bi; +} +au operator-(au h, au) { + b bl, bm = h; + bl = bm - ad; + return bl; +} +au operator*(au h, au m) { + b bn, bo = h, k = m; + bn = bo * k; + return bn; + __attribute__(()) float bp; + ; + ; +} +struct br { + b bs; + br() {} + br(b h) : bs(h) {} + br(const br &h) { bs = h.bs; } + operator au() { return bs; } +}; +br ak(br &h, br &m) { + b bq; + { + b ab = h.bs; + b ac = m.bs; + bq = __builtin_ia32_minps(ab, ac); + } + return bq; +} +br an(br h, br m) { + { f = __builtin_ia32_maxps(h.bs, m.bs); } + return f; +} +struct bt { + b bs; + unsigned n; + operator br() const { return bs; } +}; +template struct bu { + br al, am; + bu() {} + bu(br h, br m) : al(h), am(m) {} + void ca(bu &h) { + al = ak(al, h.al); + am = an(am, h.am); + } + void ca(br h) { + br bv; + al = ak(al, h); + bv = e(am.bs, h.bs); + am = bv; + } + br bw() { + b bx; + br by, cf; + bx = al.bs + am.bs; + by = bx; + br cg = by; + cf = cg; + return cf; + } + br cb() { + br cc = al.bs + am.bs; + return cc; + } +}; +typedef bu
cd; +template using ce = ah; +struct ci { + cd bounds() { return cd(al, am); } + bt al; + bt am; +} o; +template struct cm { + void ch(ci &h) { + cd bounds = h.bounds(); + cj.ca(bounds); + br ck = bounds.cb(); + cl.ca(ck); + } + bz cj; + cd cl; +}; +template struct cn : cm { + void co(ci &h, int m) { + cm::ch(h); + cp += m; + } + int cp; +}; +struct cq { + struct cu { + cu(cq *); + }; +}; +template +__attribute__((always_inline)) void ct(cn> &h, cn> &m, cr p, + const cs cv) { + ci *l = &o, *r; + while (1) { + while (__builtin_expect(l <= r && p(*l), true)) { + cv(h, *l); + ++l; + } + while (__builtin_expect(l <= r && p(*r), true)) { + cv(m, *r); + --r; + } + if (r < l) + break; + cv(h, *r); + cv(m, *l); + } +} +struct cw { + as<4> bin(br h) { + q s = h, cx = s - cy, cz = cx * da; + as<4> i(cz); + return ai(i, as<4>(0), as<4>(1)); + } + au cy, da; +}; +struct db { + cw dc; +}; +struct dd : cm { + dd(); + dd(cm h) : cm(h) {} +} de; +template struct df { + typedef db dg; + typedef int u; + df(void(), ci *, cm); + u dh; + void di(u, dd, dd, dd) { dj(); } + dg &v; + dd w; + dd x; + a dj() { + cn y, z; + int dk; + as<4> dl(0); + aq<4> dm(dk); + ct( + y, z, + [&](ci &dn) { + br __trans_tmp_1293 = dn.bounds().bw(), + __trans_tmp_1292 = __trans_tmp_1293, + __trans_tmp_1291 = __trans_tmp_1292, + __trans_tmp_1290 = __trans_tmp_1291, + __trans_tmp_1289 = __trans_tmp_1290, + __trans_tmp_1288 = __trans_tmp_1289, + __trans_tmp_1287 = __trans_tmp_1288, + __trans_tmp_1286 = __trans_tmp_1287, + __trans_tmp_1285 = __trans_tmp_1286, + __trans_tmp_1284 = __trans_tmp_1285, + __trans_tmp_1283 = __trans_tmp_1284, + __trans_tmp_1282 = __trans_tmp_1283, + __trans_tmp_1281 = __trans_tmp_1282, + __trans_tmp_1280 = __trans_tmp_1281, + __trans_tmp_1279 = __trans_tmp_1280, + __trans_tmp_1278 = __trans_tmp_1279, + __trans_tmp_1277 = __trans_tmp_1278, + __trans_tmp_1276 = __trans_tmp_1277, + __trans_tmp_1275 = __trans_tmp_1276, + __trans_tmp_1274 = __trans_tmp_1275, + __trans_tmp_1273 = __trans_tmp_1274, + __trans_tmp_1272 = __trans_tmp_1273, + __trans_tmp_1271 = __trans_tmp_1272, + __trans_tmp_1270 = __trans_tmp_1271, + __trans_tmp_1269 = __trans_tmp_1270, + __trans_tmp_1268 = __trans_tmp_1269, + __trans_tmp_1267 = __trans_tmp_1268, + __trans_tmp_1266 = __trans_tmp_1267, + __trans_tmp_1265 = __trans_tmp_1266, + __trans_tmp_1264 = __trans_tmp_1265, + __trans_tmp_1263 = __trans_tmp_1264, + __trans_tmp_1262 = __trans_tmp_1263, + __trans_tmp_1261 = __trans_tmp_1262, + __trans_tmp_1260 = __trans_tmp_1261, + __trans_tmp_1259 = __trans_tmp_1260, + __trans_tmp_1258 = __trans_tmp_1259, + __trans_tmp_1257 = __trans_tmp_1258, + __trans_tmp_1256 = __trans_tmp_1257, + __trans_tmp_1255 = __trans_tmp_1256, + __trans_tmp_1254 = __trans_tmp_1255, + __trans_tmp_1253 = __trans_tmp_1254, + __trans_tmp_1252 = __trans_tmp_1253, + __trans_tmp_1251 = __trans_tmp_1252, + __trans_tmp_1250 = __trans_tmp_1251, + __trans_tmp_1249 = __trans_tmp_1250, + __trans_tmp_1248 = __trans_tmp_1249, + __trans_tmp_1247 = __trans_tmp_1248, + __trans_tmp_1246 = __trans_tmp_1247, + __trans_tmp_1245 = __trans_tmp_1246, + __trans_tmp_1244 = __trans_tmp_1245, + __trans_tmp_1243 = __trans_tmp_1244, + __trans_tmp_1242 = __trans_tmp_1243, + __trans_tmp_1241 = __trans_tmp_1242, + __trans_tmp_1240 = __trans_tmp_1241, + __trans_tmp_1239 = __trans_tmp_1240, + __trans_tmp_1238 = __trans_tmp_1239, + __trans_tmp_1237 = __trans_tmp_1238, + __trans_tmp_1236 = __trans_tmp_1237, + __trans_tmp_1235 = __trans_tmp_1236, + __trans_tmp_1234 = __trans_tmp_1235, + __trans_tmp_1233 = __trans_tmp_1234, + __trans_tmp_1232 = __trans_tmp_1233, + __trans_tmp_1231 = __trans_tmp_1232, + __trans_tmp_1230 = __trans_tmp_1231, + __trans_tmp_1229 = __trans_tmp_1230, + __trans_tmp_1228 = __trans_tmp_1229, + __trans_tmp_1227 = __trans_tmp_1228, + __trans_tmp_1226 = __trans_tmp_1227, + __trans_tmp_1225 = __trans_tmp_1226, + __trans_tmp_1224 = __trans_tmp_1225, + __trans_tmp_1223 = __trans_tmp_1224, + __trans_tmp_1222 = __trans_tmp_1223, + __trans_tmp_1221 = __trans_tmp_1222, + __trans_tmp_1220 = __trans_tmp_1221, + __trans_tmp_1219 = __trans_tmp_1220, + __trans_tmp_1218 = __trans_tmp_1219, + __trans_tmp_1217 = __trans_tmp_1218, + __trans_tmp_1216 = __trans_tmp_1217, + __trans_tmp_1215 = __trans_tmp_1216, + __trans_tmp_1214 = __trans_tmp_1215, + __trans_tmp_1213 = __trans_tmp_1214, + __trans_tmp_1212 = __trans_tmp_1213, + __trans_tmp_1211 = __trans_tmp_1212, + __trans_tmp_1210 = __trans_tmp_1211, + __trans_tmp_1209 = __trans_tmp_1210, + __trans_tmp_1208 = __trans_tmp_1209, + __trans_tmp_1207 = __trans_tmp_1208, + __trans_tmp_1206 = __trans_tmp_1207, + __trans_tmp_1205 = __trans_tmp_1206, + __trans_tmp_1204 = __trans_tmp_1205, + __trans_tmp_1203 = __trans_tmp_1204, + __trans_tmp_1202 = __trans_tmp_1203, + __trans_tmp_1201 = __trans_tmp_1202, + __trans_tmp_1200 = __trans_tmp_1201, + __trans_tmp_1199 = __trans_tmp_1200, + __trans_tmp_1198 = __trans_tmp_1199, + __trans_tmp_1197 = __trans_tmp_1198, + __trans_tmp_1196 = __trans_tmp_1197, + __trans_tmp_1195 = __trans_tmp_1196, + __trans_tmp_1194 = __trans_tmp_1195, + __trans_tmp_1193 = __trans_tmp_1194, + __trans_tmp_1192 = __trans_tmp_1193, + __trans_tmp_1191 = __trans_tmp_1192, + __trans_tmp_1190 = __trans_tmp_1191, + __trans_tmp_1189 = __trans_tmp_1190, + __trans_tmp_1188 = __trans_tmp_1189, + __trans_tmp_1187 = __trans_tmp_1188, + __trans_tmp_1186 = __trans_tmp_1187, + __trans_tmp_1185 = __trans_tmp_1186, + __trans_tmp_1184 = __trans_tmp_1185, + __trans_tmp_1183 = __trans_tmp_1184, + __trans_tmp_1182 = __trans_tmp_1183, + __trans_tmp_1181 = __trans_tmp_1182, + __trans_tmp_1180 = __trans_tmp_1181, + __trans_tmp_1179 = __trans_tmp_1180, + __trans_tmp_1178 = __trans_tmp_1179, + __trans_tmp_1177 = __trans_tmp_1178, + __trans_tmp_1176 = __trans_tmp_1177, + __trans_tmp_1175 = __trans_tmp_1176, + __trans_tmp_1174 = __trans_tmp_1175, + __trans_tmp_1173 = __trans_tmp_1174, + __trans_tmp_1172 = __trans_tmp_1173, + __trans_tmp_1171 = __trans_tmp_1172, + __trans_tmp_1170 = __trans_tmp_1171, + __trans_tmp_1169 = __trans_tmp_1170, + __trans_tmp_1168 = __trans_tmp_1169, + __trans_tmp_1167 = __trans_tmp_1168, + __trans_tmp_1166 = __trans_tmp_1167, + __trans_tmp_1165 = __trans_tmp_1166, + __trans_tmp_1164 = __trans_tmp_1165, + __trans_tmp_1163 = __trans_tmp_1164, + __trans_tmp_1162 = __trans_tmp_1163, + __trans_tmp_1161 = __trans_tmp_1162, + __trans_tmp_1160 = __trans_tmp_1161, + __trans_tmp_1159 = __trans_tmp_1160, + __trans_tmp_1158 = __trans_tmp_1159, + __trans_tmp_1157 = __trans_tmp_1158, + __trans_tmp_1156 = __trans_tmp_1157, + __trans_tmp_1155 = __trans_tmp_1156, + __trans_tmp_1154 = __trans_tmp_1155, + __trans_tmp_1153 = __trans_tmp_1154, + __trans_tmp_1152 = __trans_tmp_1153, + __trans_tmp_1151 = __trans_tmp_1152, + __trans_tmp_1150 = __trans_tmp_1151, + __trans_tmp_1149 = __trans_tmp_1150, + __trans_tmp_1148 = __trans_tmp_1149, + __trans_tmp_1147 = __trans_tmp_1148, + __trans_tmp_1146 = __trans_tmp_1147, + __trans_tmp_1145 = __trans_tmp_1146, + __trans_tmp_1144 = __trans_tmp_1145, + __trans_tmp_1143 = __trans_tmp_1144, + __trans_tmp_1142 = __trans_tmp_1143, + __trans_tmp_1141 = __trans_tmp_1142, + __trans_tmp_1140 = __trans_tmp_1141, + __trans_tmp_1139 = __trans_tmp_1140, + __trans_tmp_1138 = __trans_tmp_1139, + __trans_tmp_1137 = __trans_tmp_1138, + __trans_tmp_1136 = __trans_tmp_1137, + __trans_tmp_1135 = __trans_tmp_1136, + __trans_tmp_1134 = __trans_tmp_1135, + __trans_tmp_1133 = __trans_tmp_1134, + __trans_tmp_1132 = __trans_tmp_1133, + __trans_tmp_1131 = __trans_tmp_1132, + __trans_tmp_1130 = __trans_tmp_1131, + __trans_tmp_1129 = __trans_tmp_1130, + __trans_tmp_1128 = __trans_tmp_1129, + __trans_tmp_1127 = __trans_tmp_1128, + __trans_tmp_1126 = __trans_tmp_1127, + __trans_tmp_1125 = __trans_tmp_1126, + __trans_tmp_1124 = __trans_tmp_1125, + __trans_tmp_1123 = __trans_tmp_1124, + __trans_tmp_1122 = __trans_tmp_1123, + __trans_tmp_1121 = __trans_tmp_1122, + __trans_tmp_1120 = __trans_tmp_1121, + __trans_tmp_1119 = __trans_tmp_1120, + __trans_tmp_1118 = __trans_tmp_1119, + __trans_tmp_1117 = __trans_tmp_1118, + __trans_tmp_1116 = __trans_tmp_1117, + __trans_tmp_1115 = __trans_tmp_1116, + __trans_tmp_1114 = __trans_tmp_1115, + __trans_tmp_1113 = __trans_tmp_1114, + __trans_tmp_1112 = __trans_tmp_1113, + __trans_tmp_1111 = __trans_tmp_1112, + __trans_tmp_1110 = __trans_tmp_1111, + __trans_tmp_1109 = __trans_tmp_1110, + __trans_tmp_1108 = __trans_tmp_1109, + __trans_tmp_1107 = __trans_tmp_1108, + __trans_tmp_1106 = __trans_tmp_1107, + __trans_tmp_1105 = __trans_tmp_1106, + __trans_tmp_1104 = __trans_tmp_1105, + __trans_tmp_1103 = __trans_tmp_1104, + __trans_tmp_1102 = __trans_tmp_1103, + __trans_tmp_1101 = __trans_tmp_1102, + __trans_tmp_1100 = __trans_tmp_1101, + __trans_tmp_1099 = __trans_tmp_1100, + __trans_tmp_1098 = __trans_tmp_1099, + __trans_tmp_1097 = __trans_tmp_1098, + __trans_tmp_1096 = __trans_tmp_1097, + __trans_tmp_1095 = __trans_tmp_1096, + __trans_tmp_1094 = __trans_tmp_1095, + __trans_tmp_1093 = __trans_tmp_1094, + __trans_tmp_1092 = __trans_tmp_1093, + __trans_tmp_1091 = __trans_tmp_1092, + __trans_tmp_1090 = __trans_tmp_1091, + __trans_tmp_1089 = __trans_tmp_1090, + __trans_tmp_1088 = __trans_tmp_1089, + __trans_tmp_1087 = __trans_tmp_1088, + __trans_tmp_1086 = __trans_tmp_1087, + __trans_tmp_1085 = __trans_tmp_1086, + __trans_tmp_1084 = __trans_tmp_1085, + __trans_tmp_1083 = __trans_tmp_1084, + __trans_tmp_1082 = __trans_tmp_1083, + __trans_tmp_1081 = __trans_tmp_1082, + __trans_tmp_1080 = __trans_tmp_1081, + __trans_tmp_1079 = __trans_tmp_1080, + __trans_tmp_1078 = __trans_tmp_1079, + __trans_tmp_1077 = __trans_tmp_1078, + __trans_tmp_1076 = __trans_tmp_1077, + __trans_tmp_1075 = __trans_tmp_1076, + __trans_tmp_1074 = __trans_tmp_1075, + __trans_tmp_1073 = __trans_tmp_1074, + __trans_tmp_1072 = __trans_tmp_1073, + __trans_tmp_1071 = __trans_tmp_1072, + __trans_tmp_1070 = __trans_tmp_1071, + __trans_tmp_1069 = __trans_tmp_1070, + __trans_tmp_1068 = __trans_tmp_1069, + __trans_tmp_1067 = __trans_tmp_1068, + __trans_tmp_1066 = __trans_tmp_1067, + __trans_tmp_1065 = __trans_tmp_1066, + __trans_tmp_1064 = __trans_tmp_1065, + __trans_tmp_1063 = __trans_tmp_1064, + __trans_tmp_1062 = __trans_tmp_1063, + __trans_tmp_1061 = __trans_tmp_1062, + __trans_tmp_1060 = __trans_tmp_1061, + __trans_tmp_1059 = __trans_tmp_1060, + __trans_tmp_1058 = __trans_tmp_1059, + __trans_tmp_1057 = __trans_tmp_1058, + __trans_tmp_1056 = __trans_tmp_1057, + __trans_tmp_1055 = __trans_tmp_1056, + __trans_tmp_1054 = __trans_tmp_1055, + __trans_tmp_1053 = __trans_tmp_1054, + __trans_tmp_1052 = __trans_tmp_1053, + __trans_tmp_1051 = __trans_tmp_1052, + __trans_tmp_1050 = __trans_tmp_1051, + __trans_tmp_1049 = __trans_tmp_1050, + __trans_tmp_1048 = __trans_tmp_1049, + __trans_tmp_1047 = __trans_tmp_1048, + __trans_tmp_1046 = __trans_tmp_1047, + __trans_tmp_1045 = __trans_tmp_1046, + __trans_tmp_1044 = __trans_tmp_1045, + __trans_tmp_1043 = __trans_tmp_1044, + __trans_tmp_1042 = __trans_tmp_1043, + __trans_tmp_1041 = __trans_tmp_1042, + __trans_tmp_1040 = __trans_tmp_1041, + __trans_tmp_1039 = __trans_tmp_1040, + __trans_tmp_1038 = __trans_tmp_1039, + __trans_tmp_1037 = __trans_tmp_1038, + __trans_tmp_1036 = __trans_tmp_1037, + __trans_tmp_1035 = __trans_tmp_1036, + __trans_tmp_1034 = __trans_tmp_1035, + __trans_tmp_1033 = __trans_tmp_1034, + __trans_tmp_1032 = __trans_tmp_1033, + __trans_tmp_1031 = __trans_tmp_1032, + __trans_tmp_1030 = __trans_tmp_1031, + __trans_tmp_1029 = __trans_tmp_1030, + __trans_tmp_1028 = __trans_tmp_1029, + __trans_tmp_1027 = __trans_tmp_1028, + __trans_tmp_1026 = __trans_tmp_1027, + __trans_tmp_1025 = __trans_tmp_1026, + __trans_tmp_1024 = __trans_tmp_1025, + __trans_tmp_1023 = __trans_tmp_1024, + __trans_tmp_1022 = __trans_tmp_1023, + __trans_tmp_1021 = __trans_tmp_1022, + __trans_tmp_1020 = __trans_tmp_1021, + __trans_tmp_1019 = __trans_tmp_1020, + __trans_tmp_1018 = __trans_tmp_1019, + __trans_tmp_1017 = __trans_tmp_1018, + __trans_tmp_1016 = __trans_tmp_1017, + __trans_tmp_1015 = __trans_tmp_1016, + __trans_tmp_1014 = __trans_tmp_1015, + __trans_tmp_1013 = __trans_tmp_1014, + __trans_tmp_1012 = __trans_tmp_1013, + __trans_tmp_1011 = __trans_tmp_1012, + __trans_tmp_1010 = __trans_tmp_1011, + __trans_tmp_1009 = __trans_tmp_1010, + __trans_tmp_1008 = __trans_tmp_1009, + __trans_tmp_1007 = __trans_tmp_1008, + __trans_tmp_1006 = __trans_tmp_1007, + __trans_tmp_1005 = __trans_tmp_1006, + __trans_tmp_1004 = __trans_tmp_1005, + __trans_tmp_1003 = __trans_tmp_1004, + __trans_tmp_1002 = __trans_tmp_1003, + __trans_tmp_1001 = __trans_tmp_1002, + __trans_tmp_1000 = __trans_tmp_1001, + __trans_tmp_999 = __trans_tmp_1000, + __trans_tmp_998 = __trans_tmp_999, + __trans_tmp_997 = __trans_tmp_998, + __trans_tmp_996 = __trans_tmp_997, + __trans_tmp_995 = __trans_tmp_996, + __trans_tmp_994 = __trans_tmp_995, + __trans_tmp_993 = __trans_tmp_994, + __trans_tmp_992 = __trans_tmp_993, + __trans_tmp_991 = __trans_tmp_992, + __trans_tmp_990 = __trans_tmp_991, + __trans_tmp_989 = __trans_tmp_990, + __trans_tmp_988 = __trans_tmp_989, + __trans_tmp_987 = __trans_tmp_988, + __trans_tmp_986 = __trans_tmp_987, + __trans_tmp_985 = __trans_tmp_986, + __trans_tmp_984 = __trans_tmp_985, + __trans_tmp_983 = __trans_tmp_984, + __trans_tmp_982 = __trans_tmp_983, + __trans_tmp_981 = __trans_tmp_982, + __trans_tmp_980 = __trans_tmp_981, + __trans_tmp_979 = __trans_tmp_980, + __trans_tmp_978 = __trans_tmp_979, + __trans_tmp_977 = __trans_tmp_978, + __trans_tmp_976 = __trans_tmp_977, + __trans_tmp_975 = __trans_tmp_976, + __trans_tmp_974 = __trans_tmp_975, + __trans_tmp_973 = __trans_tmp_974, + __trans_tmp_972 = __trans_tmp_973, + __trans_tmp_971 = __trans_tmp_972, + __trans_tmp_970 = __trans_tmp_971, + __trans_tmp_969 = __trans_tmp_970, + __trans_tmp_968 = __trans_tmp_969, + __trans_tmp_967 = __trans_tmp_968, + __trans_tmp_966 = __trans_tmp_967, + __trans_tmp_965 = __trans_tmp_966, + __trans_tmp_964 = __trans_tmp_965, + __trans_tmp_963 = __trans_tmp_964, + __trans_tmp_962 = __trans_tmp_963, + __trans_tmp_961 = __trans_tmp_962, + __trans_tmp_960 = __trans_tmp_961, + __trans_tmp_959 = __trans_tmp_960, + __trans_tmp_958 = __trans_tmp_959, + __trans_tmp_957 = __trans_tmp_958, + __trans_tmp_956 = __trans_tmp_957, + __trans_tmp_955 = __trans_tmp_956, + __trans_tmp_954 = __trans_tmp_955, + __trans_tmp_953 = __trans_tmp_954, + __trans_tmp_952 = __trans_tmp_953, + __trans_tmp_951 = __trans_tmp_952, + __trans_tmp_950 = __trans_tmp_951, + __trans_tmp_949 = __trans_tmp_950, + __trans_tmp_948 = __trans_tmp_949, + __trans_tmp_947 = __trans_tmp_948, + __trans_tmp_946 = __trans_tmp_947, + __trans_tmp_945 = __trans_tmp_946, + __trans_tmp_944 = __trans_tmp_945, + __trans_tmp_943 = __trans_tmp_944, + __trans_tmp_942 = __trans_tmp_943, + __trans_tmp_941 = __trans_tmp_942, + __trans_tmp_940 = __trans_tmp_941, + __trans_tmp_939 = __trans_tmp_940, + __trans_tmp_938 = __trans_tmp_939, + __trans_tmp_937 = __trans_tmp_938, + __trans_tmp_936 = __trans_tmp_937, + __trans_tmp_935 = __trans_tmp_936, + __trans_tmp_934 = __trans_tmp_935, + __trans_tmp_933 = __trans_tmp_934, + __trans_tmp_932 = __trans_tmp_933, + __trans_tmp_931 = __trans_tmp_932, + __trans_tmp_930 = __trans_tmp_931, + __trans_tmp_929 = __trans_tmp_930, + __trans_tmp_928 = __trans_tmp_929, + __trans_tmp_927 = __trans_tmp_928, + __trans_tmp_926 = __trans_tmp_927, + __trans_tmp_925 = __trans_tmp_926, + __trans_tmp_924 = __trans_tmp_925, + __trans_tmp_923 = __trans_tmp_924, + __trans_tmp_922 = __trans_tmp_923, + __trans_tmp_921 = __trans_tmp_922, + __trans_tmp_920 = __trans_tmp_921, + __trans_tmp_919 = __trans_tmp_920, + __trans_tmp_918 = __trans_tmp_919, + __trans_tmp_917 = __trans_tmp_918, + __trans_tmp_916 = __trans_tmp_917, + __trans_tmp_915 = __trans_tmp_916, + __trans_tmp_914 = __trans_tmp_915, + __trans_tmp_913 = __trans_tmp_914, + __trans_tmp_912 = __trans_tmp_913, + __trans_tmp_911 = __trans_tmp_912, + __trans_tmp_910 = __trans_tmp_911, + __trans_tmp_909 = __trans_tmp_910, + __trans_tmp_908 = __trans_tmp_909, + __trans_tmp_907 = __trans_tmp_908, + __trans_tmp_906 = __trans_tmp_907, + __trans_tmp_905 = __trans_tmp_906, + __trans_tmp_904 = __trans_tmp_905, + __trans_tmp_903 = __trans_tmp_904, + __trans_tmp_902 = __trans_tmp_903, + __trans_tmp_901 = __trans_tmp_902, + __trans_tmp_900 = __trans_tmp_901, + __trans_tmp_899 = __trans_tmp_900, + __trans_tmp_898 = __trans_tmp_899, + __trans_tmp_897 = __trans_tmp_898, + __trans_tmp_896 = __trans_tmp_897, + __trans_tmp_895 = __trans_tmp_896, + __trans_tmp_894 = __trans_tmp_895, + __trans_tmp_893 = __trans_tmp_894, + __trans_tmp_892 = __trans_tmp_893, + __trans_tmp_891 = __trans_tmp_892, + __trans_tmp_890 = __trans_tmp_891, + __trans_tmp_889 = __trans_tmp_890, + __trans_tmp_888 = __trans_tmp_889, + __trans_tmp_887 = __trans_tmp_888, + __trans_tmp_886 = __trans_tmp_887, + __trans_tmp_885 = __trans_tmp_886, + __trans_tmp_884 = __trans_tmp_885, + __trans_tmp_883 = __trans_tmp_884, + __trans_tmp_882 = __trans_tmp_883, + __trans_tmp_881 = __trans_tmp_882, + __trans_tmp_880 = __trans_tmp_881, + __trans_tmp_879 = __trans_tmp_880, + __trans_tmp_878 = __trans_tmp_879, + __trans_tmp_877 = __trans_tmp_878, + __trans_tmp_876 = __trans_tmp_877, + __trans_tmp_875 = __trans_tmp_876, + __trans_tmp_874 = __trans_tmp_875, + __trans_tmp_873 = __trans_tmp_874, + __trans_tmp_872 = __trans_tmp_873, + __trans_tmp_871 = __trans_tmp_872, + __trans_tmp_870 = __trans_tmp_871, + __trans_tmp_869 = __trans_tmp_870, + __trans_tmp_868 = __trans_tmp_869, + __trans_tmp_867 = __trans_tmp_868, + __trans_tmp_866 = __trans_tmp_867, + __trans_tmp_865 = __trans_tmp_866, + __trans_tmp_864 = __trans_tmp_865, + __trans_tmp_863 = __trans_tmp_864, c = __trans_tmp_863; + bool __trans_tmp_861 = any(v.dc.bin(c) < dl & dm); + return __trans_tmp_861; + }, + [](cn &pinfo, ci &dn) { pinfo.co(dn, dn.al.n); }); + int left_weight = y.cp; + new (&w) dd(y); + new (&x) dd(z); + a(left_weight, z.cp); + } +}; +struct G {}; +struct J { + J(int, df<16>, cq::cu, int, int, cq *, int, int, int, int); + void recurse(G h, cq::cu, bool) { heuristic.di(ay, de, de, de); } + df<6> heuristic; +}; +struct I { + template + static void build(Heuristic h, cq::cu m, cq *p) { + J builder(0, h, m, bc, bb, p, int(), int(), ba, az); + G record; + builder.recurse(record, nullptr, true); + } +}; +struct BVHBuilderBinnedFastSpatialSAH { + template + static void build(cq::cu h, int, int, cq *cv, SplitPrimitiveFunc p5, int, + cn p7, int) { + ci de; + typedef df<16> Heuristic; + Heuristic heuristic(p5, &de, p7); + I::build(heuristic, h, cv); + } +} BVHNBuilderFastSpatialSAH_scene; +struct BVHNBuilderFastSpatialSAH { + cq bvh; + ce prims0; + int settings; + BVHNBuilderFastSpatialSAH() : prims0(BVHNBuilderFastSpatialSAH_scene, 0) { + cn pinfo; + void splitter(); + BVHBuilderBinnedFastSpatialSAH::build(&bvh, int(), int(), &bvh, splitter, 0, + pinfo, settings); + } +} BVH4Triangle4SceneBuilderFastSpatialSAH_bvh;