From patchwork Fri May 15 06:56:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Rogers X-Patchwork-Id: 1290900 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=L1Pcz6gn; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49NfNg1LD8z9sTC for ; Fri, 15 May 2020 16:57:23 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726661AbgEOG4f (ORCPT ); Fri, 15 May 2020 02:56:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48336 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726313AbgEOG4e (ORCPT ); Fri, 15 May 2020 02:56:34 -0400 Received: from mail-qk1-x74a.google.com (mail-qk1-x74a.google.com [IPv6:2607:f8b0:4864:20::74a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B0E9C05BD0A for ; Thu, 14 May 2020 23:56:33 -0700 (PDT) Received: by mail-qk1-x74a.google.com with SMTP id r124so1273831qkf.1 for ; Thu, 14 May 2020 23:56:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=tvl7gLFsS/dAedcEqfrKtncclZsMityBUyr8g2OugcQ=; b=L1Pcz6gnqUzZkJrzhBUQp0QUfmzqMeD11Syg0lPwtZra+G6eYVcIwSdxLHbea30lMH VPEiAhP4to/UFfvAX0LAMJ1tyxOpktCcAdayUEcYBXl6F8VgXGGU/TEUYJ3y15A/3R5E YU6LMqnfuUcMQBUHHdnob6YvUNLUlY7ezab93T8g6Q4fi9fcQCkBISB+hhwPcYVCeE8R 1CUJySRpZreY8cMtNlU8Cm6Tfuw8qoyN8n86PLRd4NRTktgnuecp7WTaCr2RQI8IQ5td cCkgS+2KoQy94ltW9Gy0mZAHyMKpiXB+4bjqQRJZt7KHphQ7+zuJxxd+VRgQ0t2PZ1MJ xouw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tvl7gLFsS/dAedcEqfrKtncclZsMityBUyr8g2OugcQ=; b=FWSPE3AOa+a5aj4GDUoVcyjzsAptLIjXgOOSnlYIeJp/URhgrYr0yeRccDW7wmY8xA UVyKvyJrr4fisT+65ZX8uxD1YDVGWxkD48JBkfAaZUIfgayhtJ+KDELkuyWFu1c0lHwU KY30AiodtLSokbcB82qVJJEOcFMTOwuBGftgz6H99jY62gxFCHT5HB5YFkKYX7MkxW9P MPLCifzEFPNawJ/zZ2Z4w7OXLpdIkrBpm/M9OTBI1/J5H90jwXcBBfolYkWW6d6JAczX w5EuZ3EJLJjQdpMVd9N1JvbLfrfOh90B0XsAhjqWia2WhIrCQM0S8hwjNc1C7o7IBctI +Vnw== X-Gm-Message-State: AOAM530nSUNAIOW3FXemnHHdEZ8gMOZN+H2H2OF8hIZiq+i25Hr1YD3+ xp0jfJejVueABI6kPOMmUztEOY/U3sL4 X-Google-Smtp-Source: ABdhPJxDHoNPBj+Xy4XlruWwU/uTMdd15h3I5dMGic4z9Trd8Ynf0UfZutAZW27seEr+Fu0dO/wbEvgsqVcx X-Received: by 2002:a0c:ac41:: with SMTP id m1mr2102104qvb.71.1589525792231; Thu, 14 May 2020 23:56:32 -0700 (PDT) Date: Thu, 14 May 2020 23:56:17 -0700 In-Reply-To: <20200515065624.21658-1-irogers@google.com> Message-Id: <20200515065624.21658-2-irogers@google.com> Mime-Version: 1.0 References: <20200515065624.21658-1-irogers@google.com> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be-goog Subject: [PATCH 1/8] libbpf: Fix memory leak and possible double-free in hashmap__clear From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , Kajol Jain , Andi Kleen , John Garry , Jin Yao , Kan Liang , Cong Wang , Kim Phillips , Adrian Hunter , Leo Yan , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Stephane Eranian , Alston Tang , Ian Rogers Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org From: Andrii Nakryiko Fix memory leak in hashmap_clear() not freeing hashmap_entry structs for each of the remaining entries. Also NULL-out bucket list to prevent possible double-free between hashmap__clear() and hashmap__free(). Running test_progs-asan flavor clearly showed this problem. Reported-by: Alston Tang Signed-off-by: Andrii Nakryiko Signed-off-by: Alexei Starovoitov Link: https://lore.kernel.org/bpf/20200429012111.277390-5-andriin@fb.com Signed-off-by: Ian Rogers --- tools/lib/bpf/hashmap.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tools/lib/bpf/hashmap.c b/tools/lib/bpf/hashmap.c index 54c30c802070..cffb96202e0d 100644 --- a/tools/lib/bpf/hashmap.c +++ b/tools/lib/bpf/hashmap.c @@ -59,7 +59,14 @@ struct hashmap *hashmap__new(hashmap_hash_fn hash_fn, void hashmap__clear(struct hashmap *map) { + struct hashmap_entry *cur, *tmp; + int bkt; + + hashmap__for_each_entry_safe(map, cur, tmp, bkt) { + free(cur); + } free(map->buckets); + map->buckets = NULL; map->cap = map->cap_bits = map->sz = 0; } From patchwork Fri May 15 06:56:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Rogers X-Patchwork-Id: 1290897 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=l3OKb9/P; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49NfNd3mrmz9sT8 for ; Fri, 15 May 2020 16:57:21 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727876AbgEOG5U (ORCPT ); Fri, 15 May 2020 02:57:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48350 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726659AbgEOG4f (ORCPT ); Fri, 15 May 2020 02:56:35 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DF4FC05BD0C for ; Thu, 14 May 2020 23:56:35 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id x26so1613135qvd.20 for ; Thu, 14 May 2020 23:56:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=6zJdeczCxXIR05ROPjfL5M3I5d1ozekeF2WaxmhUPXo=; b=l3OKb9/Pu0/xrbfvi4lE2TdFagJPZ+BcPCVUsNj1x3y0yxGzAHd4rZFb5smd6fBjNj E5sWXxqdr81LqYGmCrXKMXD26Kg4w64KQ4PrQb0AMH475KRgrmYJgqfinUUapGN7Kz0+ JomMgqVpcsgelaft2fGQcqHV65twgmBNwO6YywGIvz5fFPrQKjW4r2P9MSpe+MLL9z0V RQOYhDIkOX8dkiQyqL8tEyMl+XCpKwT8/zrxBEQJE6gVIki9Z/7z27Q3MngZfJG4k6yh uyC+MInwMKFZubfIYfNfwhotQRMCL9SLOigBr4GX5+3V5evNYARi8ESBmeBfvf2faHMy 31Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=6zJdeczCxXIR05ROPjfL5M3I5d1ozekeF2WaxmhUPXo=; b=pPaVeLrOklktWpSIIoN0cH4hCl91w9D/AGlCGBvuIXgbSRaaKVquy1ZB31GiyWFVy2 Qzhw3AltNpQcOcgWcz9BITF6p0FDrTLRhV9bymazkBx1D62LvkaxN0Ut2oRODd5msnJG kmuv/9rdZRqYBHm5deqxJPjYBRivt+0Qiv7Kx3/ZDalxz/8QEkmkcZdBVhHkZpHZ38y9 cB8wkMuA/WBJhbx1Dn5quL3bkAgGKIFMyMtZBiTFoIC33By3uCPV/bxM7A88GQk7HGTf DBZy7rfKKvmqx71ZozTym7UNWUMCbOeKONEW1PYzLkxOyXxhI4QKi9y3eJCZ/Q8VYyWf mAEQ== X-Gm-Message-State: AOAM531RKOrcp0w002CntBNyB37fZPXZmtn8zNXBBZWTNNTn6AX9/SSq aa3LSnbS6Xabsls9o+D/1vN62l/J7iXi X-Google-Smtp-Source: ABdhPJxgP3QPfqhrDHlYbZvoBePrEE1LeamWczkyl3CHCs2p0Alr5VTy1rx2TlK6084IIR4Gb7TzZvSscmhG X-Received: by 2002:ad4:5584:: with SMTP id e4mr2000014qvx.191.1589525794216; Thu, 14 May 2020 23:56:34 -0700 (PDT) Date: Thu, 14 May 2020 23:56:18 -0700 In-Reply-To: <20200515065624.21658-1-irogers@google.com> Message-Id: <20200515065624.21658-3-irogers@google.com> Mime-Version: 1.0 References: <20200515065624.21658-1-irogers@google.com> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be-goog Subject: [PATCH 2/8] libbpf hashmap: Remove unused #include From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , Kajol Jain , Andi Kleen , John Garry , Jin Yao , Kan Liang , Cong Wang , Kim Phillips , Adrian Hunter , Leo Yan , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Stephane Eranian , Ian Rogers Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Remove #include of libbpf_internal.h that is unused. Discussed in this thread: https://lore.kernel.org/lkml/CAEf4BzZRmiEds_8R8g4vaAeWvJzPb4xYLnpF0X2VNY8oTzkphQ@mail.gmail.com/ Signed-off-by: Ian Rogers --- tools/lib/bpf/hashmap.h | 1 - 1 file changed, 1 deletion(-) diff --git a/tools/lib/bpf/hashmap.h b/tools/lib/bpf/hashmap.h index bae8879cdf58..e823b35e7371 100644 --- a/tools/lib/bpf/hashmap.h +++ b/tools/lib/bpf/hashmap.h @@ -15,7 +15,6 @@ #else #include #endif -#include "libbpf_internal.h" static inline size_t hash_bits(size_t h, int bits) { From patchwork Fri May 15 06:56:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ian Rogers X-Patchwork-Id: 1290884 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=EZrZEXW7; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49NfMt1VFcz9sTc for ; Fri, 15 May 2020 16:56:41 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726716AbgEOG4i (ORCPT ); Fri, 15 May 2020 02:56:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48360 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726696AbgEOG4h (ORCPT ); Fri, 15 May 2020 02:56:37 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 001DFC05BD09 for ; Thu, 14 May 2020 23:56:36 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id z1so1616308qvd.23 for ; Thu, 14 May 2020 23:56:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=BlBgjK5xziAM5vwiVDuTtOoE4PbK5TxWadJH4JBlKYM=; b=EZrZEXW7Viyk80Bux2NOOgwyk1D6bHXWVb/zlJ7c5MJdmc/2B55H2QFyMiG/6Jh3oe 5RwV0XnzkKjqLyJ4L1WTK03fYm0P4+W+zxkf5AbSvcWZdZoSbjUzSBroTZx8DEmMszcy 2V/caUMHX3fCDFKQWYvT+Q3i4j+VCljdEk41+w+qTQCMTbvXUAx43MobqI3DrMcNWeu4 bCszGbUeYdunqPsjaHsNJQXJwg39Lzr4M2imLnC5Tza+41rvDHVzTqB4rlMjoI11c9Uz 6zkfdg1u6D5ocppBaDpCnunArW32oHIUfartVuiUILVUWdgbSjz4M+TTCSI0AcY0XXG1 fd2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=BlBgjK5xziAM5vwiVDuTtOoE4PbK5TxWadJH4JBlKYM=; b=uTubugOa0WvTsaKzAFPDgWSMjKlk4u28yWKNpoaCuCPFfMTwDcx/a4CC8IjUBQJN0R QwejCMQcEygolLyngVjmrqo/ISoUekbuSngYqZovBJZArSsTVp8G8ROJE5ZR7HmxUW2V hNBEYpW0HdSIHjG6lrv543+Op1dDNjjvF6IwQ0qgk0ABM7JQUxnaCby00lg3Igs8yqC6 rwENqrOTLyGvA5dg63pey7CS+2kOGme3hE9WsB2+F+Oqr0uN6VsFhqYQbRg47l7H/51x 9Lq02GTumS5YBijs9/5pD4cGkE0rNagW+adojl/QuqZEyC6yqP5OXf1yRk7lv9tUSHL0 tW2w== X-Gm-Message-State: AOAM533bzfBAwQaUy82BRRpG+3K9qIMezhPTfXebyjumW1Lh/pYFpfki hxzGRBFXVWIhnYbjKwSCclDPtOYSUCdf X-Google-Smtp-Source: ABdhPJz8XtegKSh8yOMx1iWS4vUX0ml1o219kpFcONA39sezAoXcY+JdpGmDcGCC+GD5zV3AYR4TQQRMVetq X-Received: by 2002:a0c:8262:: with SMTP id h89mr1968365qva.173.1589525796081; Thu, 14 May 2020 23:56:36 -0700 (PDT) Date: Thu, 14 May 2020 23:56:19 -0700 In-Reply-To: <20200515065624.21658-1-irogers@google.com> Message-Id: <20200515065624.21658-4-irogers@google.com> Mime-Version: 1.0 References: <20200515065624.21658-1-irogers@google.com> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be-goog Subject: [PATCH 3/8] libbpf hashmap: Fix signedness warnings From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , Kajol Jain , Andi Kleen , John Garry , Jin Yao , Kan Liang , Cong Wang , Kim Phillips , Adrian Hunter , Leo Yan , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Stephane Eranian , Ian Rogers Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Fixes the following warnings: hashmap.c: In function ‘hashmap__clear’: hashmap.h:150:20: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare] 150 | for (bkt = 0; bkt < map->cap; bkt++) \ hashmap.c: In function ‘hashmap_grow’: hashmap.h:150:20: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare] 150 | for (bkt = 0; bkt < map->cap; bkt++) \ Signed-off-by: Ian Rogers --- tools/lib/bpf/hashmap.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/hashmap.c b/tools/lib/bpf/hashmap.c index cffb96202e0d..a405dad068f5 100644 --- a/tools/lib/bpf/hashmap.c +++ b/tools/lib/bpf/hashmap.c @@ -60,7 +60,7 @@ struct hashmap *hashmap__new(hashmap_hash_fn hash_fn, void hashmap__clear(struct hashmap *map) { struct hashmap_entry *cur, *tmp; - int bkt; + size_t bkt; hashmap__for_each_entry_safe(map, cur, tmp, bkt) { free(cur); @@ -100,8 +100,7 @@ static int hashmap_grow(struct hashmap *map) struct hashmap_entry **new_buckets; struct hashmap_entry *cur, *tmp; size_t new_cap_bits, new_cap; - size_t h; - int bkt; + size_t h, bkt; new_cap_bits = map->cap_bits + 1; if (new_cap_bits < HASHMAP_MIN_CAP_BITS) From patchwork Fri May 15 06:56:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Rogers X-Patchwork-Id: 1290890 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=KLdZkd7h; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49NfNH35dBz9sTr for ; Fri, 15 May 2020 16:57:03 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726795AbgEOG4l (ORCPT ); Fri, 15 May 2020 02:56:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726723AbgEOG4j (ORCPT ); Fri, 15 May 2020 02:56:39 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10D48C05BD09 for ; Thu, 14 May 2020 23:56:39 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 132so1539279ybc.22 for ; Thu, 14 May 2020 23:56:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=rDN5X6ee0p3DPWcQJnEO6bz3ZGrQ0VwuHJwyeJb8UjA=; b=KLdZkd7htipcJURCMR14vvzpQP9A0y+OVxW6HSLzZHzFBZJHTKwz9IVBBxRF4GvUEm FSMZk7uKRPn2mlBTMmDdAt4esd/0s/s2wgWy5aAXz7LuQtGXRB+iIyTXNziR2p53QKE3 T5PtllNVjRZJj4jCsHWttYebRTmUjfvvmRoDZtP0cJ7NI9MB3S+RPaCwNn8KeFIkYLI8 0cJto9KU6HcZEH+wKsu4HJgbFDpQ5jMfV2/lsGTVbKVcr1J9sClqOkTc/McfHP0SuS/F XhDi9KXz9PRd5yRyXKq2SgvSHeLXQ916hTUQ41QXQgXuFNTFyzU/fLuAraZvRzw69afs DX7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=rDN5X6ee0p3DPWcQJnEO6bz3ZGrQ0VwuHJwyeJb8UjA=; b=KXReQLftmSvN2gw4DQGFuvEFS6Hlrb97jfHhkNL+CFSZUCxhGvp6eTumV8MsTYPqPX WpISN5iHCvzt1SMApe6JLkQRV9EfhpdBpuCX2/P+HakYi/zdLhr3fun+sTfdStEfL5T0 soV7n5O5Y6zZvMoDGxvjH/FpguqRRkWBxlWlBu7FiaSx59piqVbOy7YZ0txj1ASvWwsL dgzCXoY4uZ4SqJlgt2WPEq1tUhrUMuvDy00gS3+/jGsemynFYpPFgfF46o537ALYshaj y2SW+MAipWfTE6uYjTZbIsvxHarVjhzIm6C8cDvbvO9SPsRujj6hiYUDf01Gaff+cexX VLog== X-Gm-Message-State: AOAM5305jCPPoPxc9pRob5HBUuSGyDUMezuem2zLDhq1wDWnTdLhT67n wJm2dOovFgX36/ChUFZkyztTism5vWu0 X-Google-Smtp-Source: ABdhPJyAKrEhG9hP88436bqQfTqIsJLRSZBZRWxiUsuHq6Q11uRmf18lQUT127u8JrDy04si5qc0Uj/kFk74 X-Received: by 2002:a25:a184:: with SMTP id a4mr3068042ybi.255.1589525797993; Thu, 14 May 2020 23:56:37 -0700 (PDT) Date: Thu, 14 May 2020 23:56:20 -0700 In-Reply-To: <20200515065624.21658-1-irogers@google.com> Message-Id: <20200515065624.21658-5-irogers@google.com> Mime-Version: 1.0 References: <20200515065624.21658-1-irogers@google.com> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be-goog Subject: [PATCH 4/8] libbpf hashmap: Localize static hashmap__* symbols From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , Kajol Jain , Andi Kleen , John Garry , Jin Yao , Kan Liang , Cong Wang , Kim Phillips , Adrian Hunter , Leo Yan , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Stephane Eranian , Ian Rogers Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Localize the hashmap__* symbols in libbpf.a. To allow for a version in libapi. Before: $ nm libbpf.a ... 000000000002088a t hashmap_add_entry 000000000001712a t hashmap__append 0000000000020aa3 T hashmap__capacity 000000000002099c T hashmap__clear 00000000000208b3 t hashmap_del_entry 0000000000020fc1 T hashmap__delete 0000000000020f29 T hashmap__find 0000000000020c6c t hashmap_find_entry 0000000000020a61 T hashmap__free 0000000000020b08 t hashmap_grow 00000000000208dd T hashmap__init 0000000000020d35 T hashmap__insert 0000000000020ab5 t hashmap_needs_to_grow 0000000000020947 T hashmap__new 0000000000000775 t hashmap__set 00000000000212f8 t hashmap__set 0000000000020a91 T hashmap__size ... After: $ nm libbpf.a ... 000000000002088a t hashmap_add_entry 000000000001712a t hashmap__append 0000000000020aa3 t hashmap__capacity 000000000002099c t hashmap__clear 00000000000208b3 t hashmap_del_entry 0000000000020fc1 t hashmap__delete 0000000000020f29 t hashmap__find 0000000000020c6c t hashmap_find_entry 0000000000020a61 t hashmap__free 0000000000020b08 t hashmap_grow 00000000000208dd t hashmap__init 0000000000020d35 t hashmap__insert 0000000000020ab5 t hashmap_needs_to_grow 0000000000020947 t hashmap__new 0000000000000775 t hashmap__set 00000000000212f8 t hashmap__set 0000000000020a91 t hashmap__size ... Signed-off-by: Ian Rogers --- tools/lib/bpf/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/lib/bpf/Makefile b/tools/lib/bpf/Makefile index aee7f1a83c77..4a1cdbceb04e 100644 --- a/tools/lib/bpf/Makefile +++ b/tools/lib/bpf/Makefile @@ -20,6 +20,7 @@ srctree := $(patsubst %/,%,$(dir $(srctree))) endif INSTALL = install +OBJCOPY ?= objcopy # Use DESTDIR for installing into a different root directory. # This is useful for building a package. The program will be @@ -181,6 +182,7 @@ $(BPF_IN_SHARED): force elfdep zdep bpfdep $(BPF_HELPER_DEFS) $(BPF_IN_STATIC): force elfdep zdep bpfdep $(BPF_HELPER_DEFS) $(Q)$(MAKE) $(build)=libbpf OUTPUT=$(STATIC_OBJDIR) + $(Q)$(OBJCOPY) -w -L hashmap__\* $@ $(BPF_HELPER_DEFS): $(srctree)/tools/include/uapi/linux/bpf.h $(QUIET_GEN)$(srctree)/scripts/bpf_helpers_doc.py --header \ From patchwork Fri May 15 06:56:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Rogers X-Patchwork-Id: 1290887 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=AMis5iCt; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49NfND3Y66z9sTr for ; Fri, 15 May 2020 16:57:00 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726946AbgEOG4p (ORCPT ); Fri, 15 May 2020 02:56:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726726AbgEOG4l (ORCPT ); Fri, 15 May 2020 02:56:41 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 094C4C061A0C for ; Thu, 14 May 2020 23:56:41 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id l4so1279863qke.2 for ; Thu, 14 May 2020 23:56:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=x6LP+ciDY4RCiG/s/KExKhc3Bihf0x1pjrWNO79ic+U=; b=AMis5iCt3aVs8bAeWYA+2hJYwYYIsw5wAGUlAwd2FHdkAscpOiLrQfWkMexWQL/FdV BeWoRWMEPl6v1c2BTCw/RoT5XIBkDYSs0sEwM9/+1rTKfKO09mOpvUkC3UQZUhmxo3ka 1IPNYOOisIo9AmJ73Q37Dqf8a1pOdNK8oAcQtOoDl8wUxh+eBU7QA5u81c1nesj8OWSS MOJrT9Q3/nQn71Z4F+AK+DBE9ZWiKQoLfcoA5I694kmDD3LIKFxkYqaFRjJJbGHzRKre RSuyplCXFjqhonTk+7fYX4L71qAAb+lFyLpVgZLeAPnXMsgqT8sXMhb5r1K36DQ35xqs HopA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=x6LP+ciDY4RCiG/s/KExKhc3Bihf0x1pjrWNO79ic+U=; b=PBUhkEaIaL+KBRreAfdtusIsh+4unHbatwcDb/Ow3XHcdMbIcZ+om8j91AR4DZ5WNz ghFYILzVVobvS0XM5+cptr8e2C1OxVF4qvG8NNwSKHxbMuctW7W46FLrcnIAXMRTUazq IJzFZR7cbwXRs+82WbnfTzxzWtf4rTyeF2YB9vL/D2sSJMV5lCyHKfEWet5JI2AFVyzO NIW1Gf0KcjggpIaTlzP75mu+tjQyBmtsuScPgdZ9c88ARwo2B12QyrRyp+511U7bzi1r r03Fy+aS3Yg4frOJq9GPsm3+5jrSEV3BJr7rdlQZ2ZlfSiDDNJO/ZZnqQVBwzsbYDIYd nE+Q== X-Gm-Message-State: AOAM531T/00EgisGFK6HrExWyOM6Nw9fv66MMMsR9PffcUF1dEAGCjw9 SqFSO97ewvNVXQJFw0RMVGMr4qDPpH2R X-Google-Smtp-Source: ABdhPJxBahYwCzOGyVWRvHYjPNoLRl7jhRzgs3VwaG3zrGcgYhV+hC4dygILP9fAvSLAatp3bCsz7luEzTE2 X-Received: by 2002:a0c:b45c:: with SMTP id e28mr2137447qvf.74.1589525800107; Thu, 14 May 2020 23:56:40 -0700 (PDT) Date: Thu, 14 May 2020 23:56:21 -0700 In-Reply-To: <20200515065624.21658-1-irogers@google.com> Message-Id: <20200515065624.21658-6-irogers@google.com> Mime-Version: 1.0 References: <20200515065624.21658-1-irogers@google.com> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be-goog Subject: [PATCH 5/8] tools lib/api: Copy libbpf hashmap to libapi From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , Kajol Jain , Andi Kleen , John Garry , Jin Yao , Kan Liang , Cong Wang , Kim Phillips , Adrian Hunter , Leo Yan , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Stephane Eranian , Ian Rogers Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Allow use of hashmap in more than just libbpf, where it isn't an exported symbol. Place in libapi as that is a required part of tools/perf whereas libbpf is currently optional. Modify perf's check-headers.sh script to check that the files are kept in sync, in the same way kernel headers are checked. This will warn if they are out of sync at the start of a perf build. Signed-off-by: Ian Rogers --- tools/lib/api/Build | 1 + tools/lib/api/hashmap.c | 238 ++++++++++++++++++++++++++++++++++++ tools/lib/api/hashmap.h | 177 +++++++++++++++++++++++++++ tools/perf/check-headers.sh | 4 + 4 files changed, 420 insertions(+) create mode 100644 tools/lib/api/hashmap.c create mode 100644 tools/lib/api/hashmap.h diff --git a/tools/lib/api/Build b/tools/lib/api/Build index 6e2373db5598..2c8787a88080 100644 --- a/tools/lib/api/Build +++ b/tools/lib/api/Build @@ -2,6 +2,7 @@ libapi-y += fd/ libapi-y += fs/ libapi-y += cpu.o libapi-y += debug.o +libapi-y += hashmap.o libapi-y += str_error_r.o $(OUTPUT)str_error_r.o: ../str_error_r.c FORCE diff --git a/tools/lib/api/hashmap.c b/tools/lib/api/hashmap.c new file mode 100644 index 000000000000..a405dad068f5 --- /dev/null +++ b/tools/lib/api/hashmap.c @@ -0,0 +1,238 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) + +/* + * Generic non-thread safe hash map implementation. + * + * Copyright (c) 2019 Facebook + */ +#include +#include +#include +#include +#include +#include "hashmap.h" + +/* make sure libbpf doesn't use kernel-only integer typedefs */ +#pragma GCC poison u8 u16 u32 u64 s8 s16 s32 s64 + +/* start with 4 buckets */ +#define HASHMAP_MIN_CAP_BITS 2 + +static void hashmap_add_entry(struct hashmap_entry **pprev, + struct hashmap_entry *entry) +{ + entry->next = *pprev; + *pprev = entry; +} + +static void hashmap_del_entry(struct hashmap_entry **pprev, + struct hashmap_entry *entry) +{ + *pprev = entry->next; + entry->next = NULL; +} + +void hashmap__init(struct hashmap *map, hashmap_hash_fn hash_fn, + hashmap_equal_fn equal_fn, void *ctx) +{ + map->hash_fn = hash_fn; + map->equal_fn = equal_fn; + map->ctx = ctx; + + map->buckets = NULL; + map->cap = 0; + map->cap_bits = 0; + map->sz = 0; +} + +struct hashmap *hashmap__new(hashmap_hash_fn hash_fn, + hashmap_equal_fn equal_fn, + void *ctx) +{ + struct hashmap *map = malloc(sizeof(struct hashmap)); + + if (!map) + return ERR_PTR(-ENOMEM); + hashmap__init(map, hash_fn, equal_fn, ctx); + return map; +} + +void hashmap__clear(struct hashmap *map) +{ + struct hashmap_entry *cur, *tmp; + size_t bkt; + + hashmap__for_each_entry_safe(map, cur, tmp, bkt) { + free(cur); + } + free(map->buckets); + map->buckets = NULL; + map->cap = map->cap_bits = map->sz = 0; +} + +void hashmap__free(struct hashmap *map) +{ + if (!map) + return; + + hashmap__clear(map); + free(map); +} + +size_t hashmap__size(const struct hashmap *map) +{ + return map->sz; +} + +size_t hashmap__capacity(const struct hashmap *map) +{ + return map->cap; +} + +static bool hashmap_needs_to_grow(struct hashmap *map) +{ + /* grow if empty or more than 75% filled */ + return (map->cap == 0) || ((map->sz + 1) * 4 / 3 > map->cap); +} + +static int hashmap_grow(struct hashmap *map) +{ + struct hashmap_entry **new_buckets; + struct hashmap_entry *cur, *tmp; + size_t new_cap_bits, new_cap; + size_t h, bkt; + + new_cap_bits = map->cap_bits + 1; + if (new_cap_bits < HASHMAP_MIN_CAP_BITS) + new_cap_bits = HASHMAP_MIN_CAP_BITS; + + new_cap = 1UL << new_cap_bits; + new_buckets = calloc(new_cap, sizeof(new_buckets[0])); + if (!new_buckets) + return -ENOMEM; + + hashmap__for_each_entry_safe(map, cur, tmp, bkt) { + h = hash_bits(map->hash_fn(cur->key, map->ctx), new_cap_bits); + hashmap_add_entry(&new_buckets[h], cur); + } + + map->cap = new_cap; + map->cap_bits = new_cap_bits; + free(map->buckets); + map->buckets = new_buckets; + + return 0; +} + +static bool hashmap_find_entry(const struct hashmap *map, + const void *key, size_t hash, + struct hashmap_entry ***pprev, + struct hashmap_entry **entry) +{ + struct hashmap_entry *cur, **prev_ptr; + + if (!map->buckets) + return false; + + for (prev_ptr = &map->buckets[hash], cur = *prev_ptr; + cur; + prev_ptr = &cur->next, cur = cur->next) { + if (map->equal_fn(cur->key, key, map->ctx)) { + if (pprev) + *pprev = prev_ptr; + *entry = cur; + return true; + } + } + + return false; +} + +int hashmap__insert(struct hashmap *map, const void *key, void *value, + enum hashmap_insert_strategy strategy, + const void **old_key, void **old_value) +{ + struct hashmap_entry *entry; + size_t h; + int err; + + if (old_key) + *old_key = NULL; + if (old_value) + *old_value = NULL; + + h = hash_bits(map->hash_fn(key, map->ctx), map->cap_bits); + if (strategy != HASHMAP_APPEND && + hashmap_find_entry(map, key, h, NULL, &entry)) { + if (old_key) + *old_key = entry->key; + if (old_value) + *old_value = entry->value; + + if (strategy == HASHMAP_SET || strategy == HASHMAP_UPDATE) { + entry->key = key; + entry->value = value; + return 0; + } else if (strategy == HASHMAP_ADD) { + return -EEXIST; + } + } + + if (strategy == HASHMAP_UPDATE) + return -ENOENT; + + if (hashmap_needs_to_grow(map)) { + err = hashmap_grow(map); + if (err) + return err; + h = hash_bits(map->hash_fn(key, map->ctx), map->cap_bits); + } + + entry = malloc(sizeof(struct hashmap_entry)); + if (!entry) + return -ENOMEM; + + entry->key = key; + entry->value = value; + hashmap_add_entry(&map->buckets[h], entry); + map->sz++; + + return 0; +} + +bool hashmap__find(const struct hashmap *map, const void *key, void **value) +{ + struct hashmap_entry *entry; + size_t h; + + h = hash_bits(map->hash_fn(key, map->ctx), map->cap_bits); + if (!hashmap_find_entry(map, key, h, NULL, &entry)) + return false; + + if (value) + *value = entry->value; + return true; +} + +bool hashmap__delete(struct hashmap *map, const void *key, + const void **old_key, void **old_value) +{ + struct hashmap_entry **pprev, *entry; + size_t h; + + h = hash_bits(map->hash_fn(key, map->ctx), map->cap_bits); + if (!hashmap_find_entry(map, key, h, &pprev, &entry)) + return false; + + if (old_key) + *old_key = entry->key; + if (old_value) + *old_value = entry->value; + + hashmap_del_entry(pprev, entry); + free(entry); + map->sz--; + + return true; +} + diff --git a/tools/lib/api/hashmap.h b/tools/lib/api/hashmap.h new file mode 100644 index 000000000000..e823b35e7371 --- /dev/null +++ b/tools/lib/api/hashmap.h @@ -0,0 +1,177 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ + +/* + * Generic non-thread safe hash map implementation. + * + * Copyright (c) 2019 Facebook + */ +#ifndef __LIBBPF_HASHMAP_H +#define __LIBBPF_HASHMAP_H + +#include +#include +#ifdef __GLIBC__ +#include +#else +#include +#endif + +static inline size_t hash_bits(size_t h, int bits) +{ + /* shuffle bits and return requested number of upper bits */ + return (h * 11400714819323198485llu) >> (__WORDSIZE - bits); +} + +typedef size_t (*hashmap_hash_fn)(const void *key, void *ctx); +typedef bool (*hashmap_equal_fn)(const void *key1, const void *key2, void *ctx); + +struct hashmap_entry { + const void *key; + void *value; + struct hashmap_entry *next; +}; + +struct hashmap { + hashmap_hash_fn hash_fn; + hashmap_equal_fn equal_fn; + void *ctx; + + struct hashmap_entry **buckets; + size_t cap; + size_t cap_bits; + size_t sz; +}; + +#define HASHMAP_INIT(hash_fn, equal_fn, ctx) { \ + .hash_fn = (hash_fn), \ + .equal_fn = (equal_fn), \ + .ctx = (ctx), \ + .buckets = NULL, \ + .cap = 0, \ + .cap_bits = 0, \ + .sz = 0, \ +} + +void hashmap__init(struct hashmap *map, hashmap_hash_fn hash_fn, + hashmap_equal_fn equal_fn, void *ctx); +struct hashmap *hashmap__new(hashmap_hash_fn hash_fn, + hashmap_equal_fn equal_fn, + void *ctx); +void hashmap__clear(struct hashmap *map); +void hashmap__free(struct hashmap *map); + +size_t hashmap__size(const struct hashmap *map); +size_t hashmap__capacity(const struct hashmap *map); + +/* + * Hashmap insertion strategy: + * - HASHMAP_ADD - only add key/value if key doesn't exist yet; + * - HASHMAP_SET - add key/value pair if key doesn't exist yet; otherwise, + * update value; + * - HASHMAP_UPDATE - update value, if key already exists; otherwise, do + * nothing and return -ENOENT; + * - HASHMAP_APPEND - always add key/value pair, even if key already exists. + * This turns hashmap into a multimap by allowing multiple values to be + * associated with the same key. Most useful read API for such hashmap is + * hashmap__for_each_key_entry() iteration. If hashmap__find() is still + * used, it will return last inserted key/value entry (first in a bucket + * chain). + */ +enum hashmap_insert_strategy { + HASHMAP_ADD, + HASHMAP_SET, + HASHMAP_UPDATE, + HASHMAP_APPEND, +}; + +/* + * hashmap__insert() adds key/value entry w/ various semantics, depending on + * provided strategy value. If a given key/value pair replaced already + * existing key/value pair, both old key and old value will be returned + * through old_key and old_value to allow calling code do proper memory + * management. + */ +int hashmap__insert(struct hashmap *map, const void *key, void *value, + enum hashmap_insert_strategy strategy, + const void **old_key, void **old_value); + +static inline int hashmap__add(struct hashmap *map, + const void *key, void *value) +{ + return hashmap__insert(map, key, value, HASHMAP_ADD, NULL, NULL); +} + +static inline int hashmap__set(struct hashmap *map, + const void *key, void *value, + const void **old_key, void **old_value) +{ + return hashmap__insert(map, key, value, HASHMAP_SET, + old_key, old_value); +} + +static inline int hashmap__update(struct hashmap *map, + const void *key, void *value, + const void **old_key, void **old_value) +{ + return hashmap__insert(map, key, value, HASHMAP_UPDATE, + old_key, old_value); +} + +static inline int hashmap__append(struct hashmap *map, + const void *key, void *value) +{ + return hashmap__insert(map, key, value, HASHMAP_APPEND, NULL, NULL); +} + +bool hashmap__delete(struct hashmap *map, const void *key, + const void **old_key, void **old_value); + +bool hashmap__find(const struct hashmap *map, const void *key, void **value); + +/* + * hashmap__for_each_entry - iterate over all entries in hashmap + * @map: hashmap to iterate + * @cur: struct hashmap_entry * used as a loop cursor + * @bkt: integer used as a bucket loop cursor + */ +#define hashmap__for_each_entry(map, cur, bkt) \ + for (bkt = 0; bkt < map->cap; bkt++) \ + for (cur = map->buckets[bkt]; cur; cur = cur->next) + +/* + * hashmap__for_each_entry_safe - iterate over all entries in hashmap, safe + * against removals + * @map: hashmap to iterate + * @cur: struct hashmap_entry * used as a loop cursor + * @tmp: struct hashmap_entry * used as a temporary next cursor storage + * @bkt: integer used as a bucket loop cursor + */ +#define hashmap__for_each_entry_safe(map, cur, tmp, bkt) \ + for (bkt = 0; bkt < map->cap; bkt++) \ + for (cur = map->buckets[bkt]; \ + cur && ({tmp = cur->next; true; }); \ + cur = tmp) + +/* + * hashmap__for_each_key_entry - iterate over entries associated with given key + * @map: hashmap to iterate + * @cur: struct hashmap_entry * used as a loop cursor + * @key: key to iterate entries for + */ +#define hashmap__for_each_key_entry(map, cur, _key) \ + for (cur = ({ size_t bkt = hash_bits(map->hash_fn((_key), map->ctx),\ + map->cap_bits); \ + map->buckets ? map->buckets[bkt] : NULL; }); \ + cur; \ + cur = cur->next) \ + if (map->equal_fn(cur->key, (_key), map->ctx)) + +#define hashmap__for_each_key_entry_safe(map, cur, tmp, _key) \ + for (cur = ({ size_t bkt = hash_bits(map->hash_fn((_key), map->ctx),\ + map->cap_bits); \ + cur = map->buckets ? map->buckets[bkt] : NULL; }); \ + cur && ({ tmp = cur->next; true; }); \ + cur = tmp) \ + if (map->equal_fn(cur->key, (_key), map->ctx)) + +#endif /* __LIBBPF_HASHMAP_H */ diff --git a/tools/perf/check-headers.sh b/tools/perf/check-headers.sh index cf147db4e5ca..87fcffa33a4d 100755 --- a/tools/perf/check-headers.sh +++ b/tools/perf/check-headers.sh @@ -128,4 +128,8 @@ check arch/x86/lib/insn.c '-I "^#include [\"<]\(../include/\)*asm/in # diff non-symmetric files check_2 tools/perf/arch/x86/entry/syscalls/syscall_64.tbl arch/x86/entry/syscalls/syscall_64.tbl +# check duplicated library files +check_2 tools/lib/api/hashmap.h tools/lib/bpf/hashmap.h +check_2 tools/lib/api/hashmap.c tools/lib/bpf/hashmap.c + cd tools/perf From patchwork Fri May 15 06:56:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Rogers X-Patchwork-Id: 1290892 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: incoming-bpf@patchwork.ozlabs.org Delivered-To: patchwork-incoming-bpf@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=bpf-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=j0EVcEkU; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49NfNH70g2z9sT8 for ; Fri, 15 May 2020 16:57:03 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726722AbgEOG5D (ORCPT ); Fri, 15 May 2020 02:57:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726855AbgEOG4n (ORCPT ); Fri, 15 May 2020 02:56:43 -0400 Received: from mail-qk1-x749.google.com (mail-qk1-x749.google.com [IPv6:2607:f8b0:4864:20::749]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 013D7C05BD09 for ; Thu, 14 May 2020 23:56:43 -0700 (PDT) Received: by mail-qk1-x749.google.com with SMTP id f200so1243966qke.14 for ; Thu, 14 May 2020 23:56:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=HRcsgixZKIGHXE809MZwKkXe7/yWNFRQw3Pqfn6iPQ4=; b=j0EVcEkUBLjPQgdo3WSyM3qJ84Z+fbTZaJe/5D+JTuRP4nQM87i/fzxocDWW33TKP0 KWVRqqAGrHS/zckUbOO9aOYW+AaTF0WycfWPgIQKnZu7I0pMHT3WB9nv/r7FCKd43QtM /lbxGNGZCFS8wXUCTC6hR9ecRXJvX65b67ftpeDTLYb9RtQCN0azwYHUXxOAIWjXlDqT EWrj9dO3op7IaLqNKerRK5f3alhUP6wi/TiEgXjMbeRw+ipFRXzYJrZhgZOEjbz0BlQy YcgLhsTOSNZ3P0u0hFLaYgsgbRoQF/ejZU30j5+INhQHqe4p+07dP/KWEpDVvRk5ABj4 UyYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=HRcsgixZKIGHXE809MZwKkXe7/yWNFRQw3Pqfn6iPQ4=; b=A7teXPR1hCwkWgurMCcvZMJz/NrSbv68cdEOInfUsCvcLSHGZDvxPm21kKhBg2TpFS 2t1WaRanPrZjg2xLmJfv2XTFSorM5uNT5jtZv0kZj2J4FXKucaqoA45bl3VnSpNAymIc R2bIRck7oKCXTFJ0H+cijwEyT0Z38OZFlDRfNXBJq3QvACxW385a9Q7vaVGx2vtSVqno MEQCPOSZ1hT6GFTbPGNFIxcmvxZB6/cezkgV65ZlrZfbFxrN7b13BlaSYCtLqVZEtJdI xVINv3hKqCTQd+x/rfKQXkA0bV+SO32EJjuDei7gWv1WXg388BN/J6nIrDQpWrIK+5C8 F4Iw== X-Gm-Message-State: AOAM531XzcCz1tIsWPQfu9ZRBK01BL1DnYM0M8EZ6HiG0M47yYgn7cMQ 6/PAlA9Gk8SEMzz6ZCCyzC8mkmNq0lVh X-Google-Smtp-Source: ABdhPJwSFLjEFtRA2Y0gUBr1G22pIPriwKT5ufbbnoybsN93DW5fiomOe6xfzfWE6tcnEJeXnwfNiLMlj5N8 X-Received: by 2002:ad4:5903:: with SMTP id ez3mr2102437qvb.105.1589525802128; Thu, 14 May 2020 23:56:42 -0700 (PDT) Date: Thu, 14 May 2020 23:56:22 -0700 In-Reply-To: <20200515065624.21658-1-irogers@google.com> Message-Id: <20200515065624.21658-7-irogers@google.com> Mime-Version: 1.0 References: <20200515065624.21658-1-irogers@google.com> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be-goog Subject: [PATCH 6/8] perf test: Provide a subtest callback to ask for the reason for skipping a subtest From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , Kajol Jain , Andi Kleen , John Garry , Jin Yao , Kan Liang , Cong Wang , Kim Phillips , Adrian Hunter , Leo Yan , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Stephane Eranian , Ian Rogers , Paul Clarke , Arnaldo Carvalho de Melo Sender: bpf-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Now subtests can inform why a test was skipped. The upcoming patch improvint PMU event metric testing will use it. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Andi Kleen Cc: Jin Yao Cc: Jiri Olsa Cc: John Garry Cc: Kajol Jain Cc: Kan Liang Cc: Leo Yan Cc: Mark Rutland Cc: Namhyung Kim Cc: Paul Clarke Cc: Peter Zijlstra Cc: Stephane Eranian Link: http://lore.kernel.org/lkml/20200513212933.41273-1-irogers@google.com [ split from a larger patch ] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/tests/builtin-test.c | 11 +++++++++-- tools/perf/tests/tests.h | 1 + 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c index 3471ec52ea11..baee735e6aa5 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -429,8 +429,15 @@ static int test_and_print(struct test *t, bool force_skip, int subtest) case TEST_OK: pr_info(" Ok\n"); break; - case TEST_SKIP: - color_fprintf(stderr, PERF_COLOR_YELLOW, " Skip\n"); + case TEST_SKIP: { + const char *skip_reason = NULL; + if (t->subtest.skip_reason) + skip_reason = t->subtest.skip_reason(subtest); + if (skip_reason) + color_fprintf(stderr, PERF_COLOR_YELLOW, " Skip (%s)\n", skip_reason); + else + color_fprintf(stderr, PERF_COLOR_YELLOW, " Skip\n"); + } break; case TEST_FAIL: default: diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h index d6d4ac34eeb7..88e45aeab94f 100644 --- a/tools/perf/tests/tests.h +++ b/tools/perf/tests/tests.h @@ -34,6 +34,7 @@ struct test { bool skip_if_fail; int (*get_nr)(void); const char *(*get_desc)(int subtest); + const char *(*skip_reason)(int subtest); } subtest; bool (*is_supported)(void); void *priv; From patchwork Fri May 15 06:56:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Rogers X-Patchwork-Id: 1290885 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=l6axO6bp; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49NfN13nfQz9sTc for ; Fri, 15 May 2020 16:56:49 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727097AbgEOG4s (ORCPT ); Fri, 15 May 2020 02:56:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1727001AbgEOG4q (ORCPT ); Fri, 15 May 2020 02:56:46 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADAD2C05BD0B for ; Thu, 14 May 2020 23:56:44 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id h129so1584822ybc.3 for ; Thu, 14 May 2020 23:56:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Qx5DR7q9yKe+bR/yiFue61eKIkA72C71MJNDdOxOzy4=; b=l6axO6bpPJ7he34PyFvu8nbo+5/gL2/wskwqJcsi+3p/gzezErAmQXMVXTfenMNQZ1 k6q2a7FcJhnSKpOm4kAvgrxirH4c7PSVQVPUd7NEcdO9v/0wxLdGI6uij7Db1BKjRFZJ rwu3eUq0dJCEkUxzIZ/ooYoAVRtJ2XXa09toHvdbgcG/2hwPrdwyWGdR3YarILVfRqTr QkBzTiEtTXoXKMOuHJ6lNiry06XO8jpE6PqO7zd/HltHVEjWX+93N7Fk8/+vTaSewTt3 nn4OwQiPIhFmjuZQq4JdUWSF1l8Cp5fTOdItSdZIv31MpSfy2BBV/nH6HFZfF3rW5XQE ehwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Qx5DR7q9yKe+bR/yiFue61eKIkA72C71MJNDdOxOzy4=; b=U4T5V+TUZe2IvQTFrJWu8FXlWKMHj9SrrmgybvJ5gidUi0Y7n88eOhv70j1nWx5eO4 hyjT6461QPzn6zx+eCnDbZv2BPED9o1a8c/7eXl8D1I/UM4/iSGX9fZGtLmCzWb5sCtY oamWjb0UB/9rVUdMRoOCE1WjvIIKgeJa9ALZj1H2hAIXBK5583nk+nrLqHrVaJLdrYhh zN/kLgIrpUkrUHjSBf9xhE4lZuYDfcSsdie8N8REy5lEZkwYj+n+5v7PqJUJd9UurGot yTfxWQiRHDbgHHcJJpOGA1ooQUiq7G924Gs+gopNHwFZZRsL3uUAuLeoQkA2uPpJO/LR O1Yw== X-Gm-Message-State: AOAM531NrMZ0KLgFXFMe8Q1RyVO8Mmg8FJ2xq7A7yT9/I7pH8bhxgDNf BWz66vaAINT0ITY/1fyMuSxp3G1Hk0ZS X-Google-Smtp-Source: ABdhPJz9N+s3X6xjDoBQW0ErfYd0gLJbKsfNrHL07fBzBUoDJYr0IlyawWKrulzcmnBOi338kjc5lHSHvShy X-Received: by 2002:a25:810e:: with SMTP id o14mr3112716ybk.453.1589525803900; Thu, 14 May 2020 23:56:43 -0700 (PDT) Date: Thu, 14 May 2020 23:56:23 -0700 In-Reply-To: <20200515065624.21658-1-irogers@google.com> Message-Id: <20200515065624.21658-8-irogers@google.com> Mime-Version: 1.0 References: <20200515065624.21658-1-irogers@google.com> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be-goog Subject: [PATCH 7/8] perf test: Improve pmu event metric testing From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , Kajol Jain , Andi Kleen , John Garry , Jin Yao , Kan Liang , Cong Wang , Kim Phillips , Adrian Hunter , Leo Yan , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Stephane Eranian , Ian Rogers , Paul Clarke , Arnaldo Carvalho de Melo Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Break pmu-events test into 2 and add a test to verify that all pmu metric expressions simply parse. Try to parse all metric ids/events, skip/warn if metrics for the current architecture fail to parse. To support warning for a skip, and an ability for a subtest to describe why it skips. Tested on power9, skylakex, haswell, broadwell, westmere, sandybridge and ivybridge. May skip/warn on other architectures if metrics are invalid. In particular s390 is untested, but its expressions are trivial. The untested architectures with expressions are power8, cascadelakex, tremontx, skylake, jaketown, ivytown and variants of haswell and broadwell. v3. addresses review comments from John Garry , Jiri Olsa and Arnaldo Carvalho de Melo . v2. changes the commit message as event parsing errors no longer cause the test to fail. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Andi Kleen Cc: Jin Yao Cc: Jiri Olsa Cc: John Garry Cc: Kajol Jain Cc: Kan Liang Cc: Leo Yan Cc: Mark Rutland Cc: Namhyung Kim Cc: Paul Clarke Cc: Peter Zijlstra Cc: Stephane Eranian Link: http://lore.kernel.org/lkml/20200513212933.41273-1-irogers@google.com [ split from a larger patch ] Signed-off-by: Arnaldo Carvalho de Melo --- tools/perf/tests/builtin-test.c | 7 ++ tools/perf/tests/pmu-events.c | 168 ++++++++++++++++++++++++++++++-- tools/perf/tests/tests.h | 3 + 3 files changed, 172 insertions(+), 6 deletions(-) diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c index baee735e6aa5..9553f8061772 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -75,6 +75,13 @@ static struct test generic_tests[] = { { .desc = "PMU events", .func = test__pmu_events, + .subtest = { + .skip_if_fail = false, + .get_nr = test__pmu_events_subtest_get_nr, + .get_desc = test__pmu_events_subtest_get_desc, + .skip_reason = test__pmu_events_subtest_skip_reason, + }, + }, { .desc = "DSO data read", diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c index d64261da8bf7..e21f0addcfbb 100644 --- a/tools/perf/tests/pmu-events.c +++ b/tools/perf/tests/pmu-events.c @@ -8,6 +8,9 @@ #include #include "debug.h" #include "../pmu-events/pmu-events.h" +#include "util/evlist.h" +#include "util/expr.h" +#include "util/parse-events.h" struct perf_pmu_test_event { struct pmu_event event; @@ -144,7 +147,7 @@ static struct pmu_events_map *__test_pmu_get_events_map(void) } /* Verify generated events from pmu-events.c is as expected */ -static int __test_pmu_event_table(void) +static int test_pmu_event_table(void) { struct pmu_events_map *map = __test_pmu_get_events_map(); struct pmu_event *table; @@ -347,14 +350,11 @@ static int __test__pmu_event_aliases(char *pmu_name, int *count) return res; } -int test__pmu_events(struct test *test __maybe_unused, - int subtest __maybe_unused) + +static int test_aliases(void) { struct perf_pmu *pmu = NULL; - if (__test_pmu_event_table()) - return -1; - while ((pmu = perf_pmu__scan(pmu)) != NULL) { int count = 0; @@ -377,3 +377,159 @@ int test__pmu_events(struct test *test __maybe_unused, return 0; } + +static bool is_number(const char *str) +{ + char *end_ptr; + + strtod(str, &end_ptr); + return end_ptr != str; +} + +static int check_parse_id(const char *id, bool same_cpu, struct pmu_event *pe) +{ + struct parse_events_error error; + struct evlist *evlist; + int ret; + + /* Numbers are always valid. */ + if (is_number(id)) + return 0; + + evlist = evlist__new(); + memset(&error, 0, sizeof(error)); + ret = parse_events(evlist, id, &error); + if (ret && same_cpu) { + pr_warning("Parse event failed metric '%s' id '%s' expr '%s'\n", + pe->metric_name, id, pe->metric_expr); + pr_warning("Error string '%s' help '%s'\n", error.str, + error.help); + } else if (ret) { + pr_debug3("Parse event failed, but for an event that may not be supported by this CPU.\nid '%s' metric '%s' expr '%s'\n", + id, pe->metric_name, pe->metric_expr); + ret = 0; + } + evlist__delete(evlist); + free(error.str); + free(error.help); + free(error.first_str); + free(error.first_help); + return ret; +} + +static void expr_failure(const char *msg, + const struct pmu_events_map *map, + const struct pmu_event *pe) +{ + pr_debug("%s for map %s %s %s\n", + msg, map->cpuid, map->version, map->type); + pr_debug("On metric %s\n", pe->metric_name); + pr_debug("On expression %s\n", pe->metric_expr); +} + +static int test_parsing(void) +{ + struct pmu_events_map *cpus_map = perf_pmu__find_map(NULL); + struct pmu_events_map *map; + struct pmu_event *pe; + int i, j, k; + const char **ids; + int idnum; + int ret = 0; + struct expr_parse_ctx ctx; + double result; + + i = 0; + for (;;) { + map = &pmu_events_map[i++]; + if (!map->table) + break; + j = 0; + for (;;) { + pe = &map->table[j++]; + if (!pe->name && !pe->metric_group && !pe->metric_name) + break; + if (!pe->metric_expr) + continue; + if (expr__find_other(pe->metric_expr, NULL, + &ids, &idnum, 0) < 0) { + expr_failure("Parse other failed", map, pe); + ret++; + continue; + } + expr__ctx_init(&ctx); + + /* + * Add all ids with a made up value. The value may + * trigger divide by zero when subtracted and so try to + * make them unique. + */ + for (k = 0; k < idnum; k++) + expr__add_id(&ctx, ids[k], k + 1); + + for (k = 0; k < idnum; k++) { + if (check_parse_id(ids[k], map == cpus_map, pe)) + ret++; + } + + if (expr__parse(&result, &ctx, pe->metric_expr, 0)) { + expr_failure("Parse failed", map, pe); + ret++; + } + for (k = 0; k < idnum; k++) + zfree(&ids[k]); + free(ids); + } + } + /* TODO: fail when not ok */ + return ret == 0 ? TEST_OK : TEST_SKIP; +} + +static const struct { + int (*func)(void); + const char *desc; +} pmu_events_testcase_table[] = { + { + .func = test_pmu_event_table, + .desc = "PMU event table sanity", + }, + { + .func = test_aliases, + .desc = "PMU event map aliases", + }, + { + .func = test_parsing, + .desc = "Parsing of PMU event table metrics", + }, +}; + +const char *test__pmu_events_subtest_get_desc(int subtest) +{ + if (subtest < 0 || + subtest >= (int)ARRAY_SIZE(pmu_events_testcase_table)) + return NULL; + return pmu_events_testcase_table[subtest].desc; +} + +const char *test__pmu_events_subtest_skip_reason(int subtest) +{ + if (subtest < 0 || + subtest >= (int)ARRAY_SIZE(pmu_events_testcase_table)) + return NULL; + if (pmu_events_testcase_table[subtest].func != test_parsing) + return NULL; + return "some metrics failed"; +} + +int test__pmu_events_subtest_get_nr(void) +{ + return (int)ARRAY_SIZE(pmu_events_testcase_table); +} + +int test__pmu_events(struct test *test __maybe_unused, int subtest) +{ + if (subtest < 0 || + subtest >= (int)ARRAY_SIZE(pmu_events_testcase_table)) + return TEST_FAIL; + return pmu_events_testcase_table[subtest].func(); +} diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h index 88e45aeab94f..6c6c4b6a4796 100644 --- a/tools/perf/tests/tests.h +++ b/tools/perf/tests/tests.h @@ -51,6 +51,9 @@ int test__perf_evsel__tp_sched_test(struct test *test, int subtest); int test__syscall_openat_tp_fields(struct test *test, int subtest); int test__pmu(struct test *test, int subtest); int test__pmu_events(struct test *test, int subtest); +const char *test__pmu_events_subtest_get_desc(int subtest); +const char *test__pmu_events_subtest_skip_reason(int subtest); +int test__pmu_events_subtest_get_nr(void); int test__attr(struct test *test, int subtest); int test__dso_data(struct test *test, int subtest); int test__dso_data_cache(struct test *test, int subtest); From patchwork Fri May 15 06:56:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Rogers X-Patchwork-Id: 1290886 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=GJojiVvC; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 49NfN40Mh0z9sTc for ; Fri, 15 May 2020 16:56:52 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727785AbgEOG4v (ORCPT ); Fri, 15 May 2020 02:56:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48414 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1727111AbgEOG4s (ORCPT ); Fri, 15 May 2020 02:56:48 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AA112C061A0C for ; Thu, 14 May 2020 23:56:46 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id x10so1579093ybx.8 for ; Thu, 14 May 2020 23:56:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ASXgkl9jjy97Vpt8ThFDh/vzQo4yJ10C2p5zMR9IFP0=; b=GJojiVvC4Xwev66Pkan0IFOjzyWd7pAaohzST4ZVmrciflpEo+Rp3NyJGGM3HLFHez uuYr1TnzHLKZ52btL6CJeN4UhuZuQ3fnwo/sXVqeCpZZ4dUERs26gQtFTjpywqpdKJdb QqS9InjR/ywXcMRzZkCTGjihOvlLE7HYFRH7B3mRaRS8ARcJKZFnQxtrs591yftC9vkr 4uiO24lwP6sNcvx6AScuOQW9Z1ckX/07KKcsp0pYzq25yTQYYh83vQciN3b4rYSjU8kS RnTMqMsoD+kyEJwXPzHGXBYAAMdqbvLyI+rMTW3RtcNEJT9pcBMyeCkE8UWoW+DsPHbl T6UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ASXgkl9jjy97Vpt8ThFDh/vzQo4yJ10C2p5zMR9IFP0=; b=g/Y+j7NVo7DiST6cjdM4e7fBF6BQ+VlUgcYZkQjcYxny5/rsVkVYPw2BktdyajPoRZ FXzTR1XVH6hxFw9hqUPWbK2jgw+e2mGcpZO/1MJRcrBSyyDVNMuQMAi9uAj/MYy+qgm9 wlnT2yw30zwHfH2/DrbUxq+rE9QfVktai5l1vtJSVgfPy1g12P6FaobYg8r2FOQBioM6 lnb93xnTGDl+Oc9GaB/76sWQ+qnlUSKn9+fOkXY4TLUcAzaxhKgUF6CiySD1hpdT48s6 wKak3aX8nM8xwEJrNU+r7Xz+aQdkK7HxpgPgQ/2QgvJm/WuR3aQrayt4wN5ehKdV4eM2 fUhg== X-Gm-Message-State: AOAM5319+ECtZ06A/FdGxyIxP0t3R1oYgLSky9YQNWpGuCIalK3P3zYb 3b83ILkoYJUeXhc1/ObqOjNW8buVe8Sy X-Google-Smtp-Source: ABdhPJzOmEg9tICLafCKUYry6ScxTMNQ/Vqn/NS2thX2FUAiW2pEaeSQP9fPqk66WjITsXsBIQP1TzllKVaT X-Received: by 2002:a25:7903:: with SMTP id u3mr3366128ybc.251.1589525805856; Thu, 14 May 2020 23:56:45 -0700 (PDT) Date: Thu, 14 May 2020 23:56:24 -0700 In-Reply-To: <20200515065624.21658-1-irogers@google.com> Message-Id: <20200515065624.21658-9-irogers@google.com> Mime-Version: 1.0 References: <20200515065624.21658-1-irogers@google.com> X-Mailer: git-send-email 2.26.2.761.g0e0b3e54be-goog Subject: [PATCH 8/8] perf expr: Migrate expr ids table to a hashmap From: Ian Rogers To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Andrii Nakryiko , John Fastabend , KP Singh , Kajol Jain , Andi Kleen , John Garry , Jin Yao , Kan Liang , Cong Wang , Kim Phillips , Adrian Hunter , Leo Yan , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Cc: Stephane Eranian , Ian Rogers Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Use a hashmap between a char* string and a double* value. While bpf's hashmap entries are size_t in size, we can't guarantee sizeof(size_t) >= sizeof(double). Avoid a memory allocation when gathering ids by making 0.0 a special value encoded as NULL. Original map suggestion by Andi Kleen: https://lore.kernel.org/lkml/20200224210308.GQ160988@tassilo.jf.intel.com/ and seconded by Jiri Olsa: https://lore.kernel.org/lkml/20200423112915.GH1136647@krava/ Signed-off-by: Ian Rogers --- tools/perf/tests/expr.c | 40 ++++++----- tools/perf/tests/pmu-events.c | 25 +++---- tools/perf/util/expr.c | 129 +++++++++++++++++++--------------- tools/perf/util/expr.h | 22 +++--- tools/perf/util/expr.y | 22 +----- tools/perf/util/metricgroup.c | 87 +++++++++++------------ tools/perf/util/stat-shadow.c | 49 ++++++++----- 7 files changed, 193 insertions(+), 181 deletions(-) diff --git a/tools/perf/tests/expr.c b/tools/perf/tests/expr.c index 3f742612776a..5e606fd5a2c6 100644 --- a/tools/perf/tests/expr.c +++ b/tools/perf/tests/expr.c @@ -19,11 +19,9 @@ static int test(struct expr_parse_ctx *ctx, const char *e, double val2) int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) { const char *p; - const char **other; - double val; - int i, ret; + double val, *val_ptr; + int ret; struct expr_parse_ctx ctx; - int num_other; expr__ctx_init(&ctx); expr__add_id(&ctx, "FOO", 1); @@ -52,25 +50,29 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused) ret = expr__parse(&val, &ctx, p, 1); TEST_ASSERT_VAL("missing operand", ret == -1); + hashmap__clear(&ctx.ids); TEST_ASSERT_VAL("find other", - expr__find_other("FOO + BAR + BAZ + BOZO", "FOO", &other, &num_other, 1) == 0); - TEST_ASSERT_VAL("find other", num_other == 3); - TEST_ASSERT_VAL("find other", !strcmp(other[0], "BAR")); - TEST_ASSERT_VAL("find other", !strcmp(other[1], "BAZ")); - TEST_ASSERT_VAL("find other", !strcmp(other[2], "BOZO")); - TEST_ASSERT_VAL("find other", other[3] == NULL); + expr__find_other("FOO + BAR + BAZ + BOZO", "FOO", + &ctx, 1) == 0); + TEST_ASSERT_VAL("find other", hashmap__size(&ctx.ids) == 3); + TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "BAR", + (void **)&val_ptr)); + TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "BAZ", + (void **)&val_ptr)); + TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "BOZO", + (void **)&val_ptr)); + hashmap__clear(&ctx.ids); TEST_ASSERT_VAL("find other", - expr__find_other("EVENT1\\,param\\=?@ + EVENT2\\,param\\=?@", NULL, - &other, &num_other, 3) == 0); - TEST_ASSERT_VAL("find other", num_other == 2); - TEST_ASSERT_VAL("find other", !strcmp(other[0], "EVENT1,param=3/")); - TEST_ASSERT_VAL("find other", !strcmp(other[1], "EVENT2,param=3/")); - TEST_ASSERT_VAL("find other", other[2] == NULL); + expr__find_other("EVENT1\\,param\\=?@ + EVENT2\\,param\\=?@", + NULL, &ctx, 3) == 0); + TEST_ASSERT_VAL("find other", hashmap__size(&ctx.ids) == 2); + TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "EVENT1,param=3/", + (void **)&val_ptr)); + TEST_ASSERT_VAL("find other", hashmap__find(&ctx.ids, "EVENT2,param=3/", + (void **)&val_ptr)); - for (i = 0; i < num_other; i++) - zfree(&other[i]); - free((void *)other); + expr__ctx_clear(&ctx); return 0; } diff --git a/tools/perf/tests/pmu-events.c b/tools/perf/tests/pmu-events.c index e21f0addcfbb..3de59564deb0 100644 --- a/tools/perf/tests/pmu-events.c +++ b/tools/perf/tests/pmu-events.c @@ -433,8 +433,6 @@ static int test_parsing(void) struct pmu_events_map *map; struct pmu_event *pe; int i, j, k; - const char **ids; - int idnum; int ret = 0; struct expr_parse_ctx ctx; double result; @@ -446,29 +444,34 @@ static int test_parsing(void) break; j = 0; for (;;) { + struct hashmap_entry *cur; + size_t bkt; + pe = &map->table[j++]; if (!pe->name && !pe->metric_group && !pe->metric_name) break; if (!pe->metric_expr) continue; - if (expr__find_other(pe->metric_expr, NULL, - &ids, &idnum, 0) < 0) { + expr__ctx_init(&ctx); + if (expr__find_other(pe->metric_expr, NULL, &ctx, 0) + < 0) { expr_failure("Parse other failed", map, pe); ret++; continue; } - expr__ctx_init(&ctx); /* * Add all ids with a made up value. The value may * trigger divide by zero when subtracted and so try to * make them unique. */ - for (k = 0; k < idnum; k++) - expr__add_id(&ctx, ids[k], k + 1); + k = 1; + hashmap__for_each_entry((&ctx.ids), cur, bkt) + expr__add_id(&ctx, strdup(cur->key), k++); - for (k = 0; k < idnum; k++) { - if (check_parse_id(ids[k], map == cpus_map, pe)) + hashmap__for_each_entry((&ctx.ids), cur, bkt) { + if (check_parse_id(cur->key, map == cpus_map, + pe)) ret++; } @@ -476,9 +479,7 @@ static int test_parsing(void) expr_failure("Parse failed", map, pe); ret++; } - for (k = 0; k < idnum; k++) - zfree(&ids[k]); - free(ids); + expr__ctx_clear(&ctx); } } /* TODO: fail when not ok */ diff --git a/tools/perf/util/expr.c b/tools/perf/util/expr.c index 8b4ce704a68d..f64ab91c432b 100644 --- a/tools/perf/util/expr.c +++ b/tools/perf/util/expr.c @@ -4,25 +4,76 @@ #include "expr.h" #include "expr-bison.h" #include "expr-flex.h" +#include #ifdef PARSER_DEBUG extern int expr_debug; #endif +static size_t key_hash(const void *key, void *ctx __maybe_unused) +{ + const char *str = (const char *)key; + size_t hash = 0; + + while (*str != '\0') { + hash *= 31; + hash += *str; + str++; + } + return hash; +} + +static bool key_equal(const void *key1, const void *key2, + void *ctx __maybe_unused) +{ + return !strcmp((const char *)key1, (const char *)key2); +} + /* Caller must make sure id is allocated */ -void expr__add_id(struct expr_parse_ctx *ctx, const char *name, double val) +int expr__add_id(struct expr_parse_ctx *ctx, const char *name, double val) { - int idx; + double *val_ptr = NULL, *old_val = NULL; + char *old_key = NULL; + int ret; + + if (val != 0.0) { + val_ptr = malloc(sizeof(double)); + if (!val_ptr) + return -ENOMEM; + *val_ptr = val; + } + ret = hashmap__set(&ctx->ids, name, val_ptr, + (const void **)&old_key, (void **)&old_val); + free(old_key); + free(old_val); + return ret; +} + +int expr__get_id(struct expr_parse_ctx *ctx, const char *id, double *val_ptr) +{ + double *data; - assert(ctx->num_ids < MAX_PARSE_ID); - idx = ctx->num_ids++; - ctx->ids[idx].name = name; - ctx->ids[idx].val = val; + if (!hashmap__find(&ctx->ids, id, (void **)&data)) + return -1; + *val_ptr = (data == NULL) ? 0.0 : *data; + return 0; } void expr__ctx_init(struct expr_parse_ctx *ctx) { - ctx->num_ids = 0; + hashmap__init(&ctx->ids, key_hash, key_equal, NULL); +} + +void expr__ctx_clear(struct expr_parse_ctx *ctx) +{ + struct hashmap_entry *cur; + size_t bkt; + + hashmap__for_each_entry((&ctx->ids), cur, bkt) { + free((char *)cur->key); + free(cur->value); + } + hashmap__clear(&ctx->ids); } static int @@ -56,61 +107,25 @@ __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr, return ret; } -int expr__parse(double *final_val, struct expr_parse_ctx *ctx, const char *expr, int runtime) +int expr__parse(double *final_val, struct expr_parse_ctx *ctx, + const char *expr, int runtime) { return __expr__parse(final_val, ctx, expr, EXPR_PARSE, runtime) ? -1 : 0; } -static bool -already_seen(const char *val, const char *one, const char **other, - int num_other) -{ - int i; - - if (one && !strcasecmp(one, val)) - return true; - for (i = 0; i < num_other; i++) - if (!strcasecmp(other[i], val)) - return true; - return false; -} - -int expr__find_other(const char *expr, const char *one, const char ***other, - int *num_other, int runtime) +int expr__find_other(const char *expr, const char *one, + struct expr_parse_ctx *ctx, int runtime) { - int err, i = 0, j = 0; - struct expr_parse_ctx ctx; - - expr__ctx_init(&ctx); - err = __expr__parse(NULL, &ctx, expr, EXPR_OTHER, runtime); - if (err) - return -1; - - *other = malloc((ctx.num_ids + 1) * sizeof(char *)); - if (!*other) - return -ENOMEM; - - for (i = 0, j = 0; i < ctx.num_ids; i++) { - const char *str = ctx.ids[i].name; - - if (already_seen(str, one, *other, j)) - continue; - - str = strdup(str); - if (!str) - goto out; - (*other)[j++] = str; - } - (*other)[j] = NULL; - -out: - if (i != ctx.num_ids) { - while (--j) - free((char *) (*other)[i]); - free(*other); - err = -1; + double *old_val = NULL; + char *old_key = NULL; + int ret = __expr__parse(NULL, ctx, expr, EXPR_OTHER, runtime); + + if (one) { + hashmap__delete(&ctx->ids, one, + (const void **)&old_key, (void **)&old_val); + free(old_key); + free(old_val); } - *num_other = j; - return err; + return ret; } diff --git a/tools/perf/util/expr.h b/tools/perf/util/expr.h index 40fc452b0f2b..03c961e85e65 100644 --- a/tools/perf/util/expr.h +++ b/tools/perf/util/expr.h @@ -2,17 +2,10 @@ #ifndef PARSE_CTX_H #define PARSE_CTX_H 1 -#define EXPR_MAX_OTHER 64 -#define MAX_PARSE_ID EXPR_MAX_OTHER - -struct expr_parse_id { - const char *name; - double val; -}; +#include struct expr_parse_ctx { - int num_ids; - struct expr_parse_id ids[MAX_PARSE_ID]; + struct hashmap ids; }; struct expr_scanner_ctx { @@ -21,9 +14,12 @@ struct expr_scanner_ctx { }; void expr__ctx_init(struct expr_parse_ctx *ctx); -void expr__add_id(struct expr_parse_ctx *ctx, const char *id, double val); -int expr__parse(double *final_val, struct expr_parse_ctx *ctx, const char *expr, int runtime); -int expr__find_other(const char *expr, const char *one, const char ***other, - int *num_other, int runtime); +void expr__ctx_clear(struct expr_parse_ctx *ctx); +int expr__add_id(struct expr_parse_ctx *ctx, const char *id, double val); +int expr__get_id(struct expr_parse_ctx *ctx, const char *id, double *val_ptr); +int expr__parse(double *final_val, struct expr_parse_ctx *ctx, + const char *expr, int runtime); +int expr__find_other(const char *expr, const char *one, + struct expr_parse_ctx *ids, int runtime); #endif diff --git a/tools/perf/util/expr.y b/tools/perf/util/expr.y index 21e82a1e11a2..ec5a48bf5f34 100644 --- a/tools/perf/util/expr.y +++ b/tools/perf/util/expr.y @@ -46,19 +46,6 @@ static void expr_error(double *final_val __maybe_unused, pr_debug("%s\n", s); } -static int lookup_id(struct expr_parse_ctx *ctx, char *id, double *val) -{ - int i; - - for (i = 0; i < ctx->num_ids; i++) { - if (!strcasecmp(ctx->ids[i].name, id)) { - *val = ctx->ids[i].val; - return 0; - } - } - return -1; -} - %} %% @@ -72,12 +59,7 @@ all_other: all_other other other: ID { - if (ctx->num_ids + 1 >= EXPR_MAX_OTHER) { - pr_err("failed: way too many variables"); - YYABORT; - } - - ctx->ids[ctx->num_ids++].name = $1; + expr__add_id(ctx, $1, 0.0); } | MIN | MAX | IF | ELSE | SMT_ON | NUMBER | '|' | '^' | '&' | '-' | '+' | '*' | '/' | '%' | '(' | ')' | ',' @@ -92,7 +74,7 @@ if_expr: ; expr: NUMBER - | ID { if (lookup_id(ctx, $1, &$$) < 0) { + | ID { if (expr__get_id(ctx, $1, &$$)) { pr_debug("%s not found\n", $1); YYABORT; } diff --git a/tools/perf/util/metricgroup.c b/tools/perf/util/metricgroup.c index b071df373f8b..37be5a368d6e 100644 --- a/tools/perf/util/metricgroup.c +++ b/tools/perf/util/metricgroup.c @@ -85,8 +85,7 @@ static void metricgroup__rblist_init(struct rblist *metric_events) struct egroup { struct list_head nd; - int idnum; - const char **ids; + struct expr_parse_ctx pctx; const char *metric_name; const char *metric_expr; const char *metric_unit; @@ -94,19 +93,21 @@ struct egroup { }; static struct evsel *find_evsel_group(struct evlist *perf_evlist, - const char **ids, - int idnum, + struct expr_parse_ctx *pctx, struct evsel **metric_events, bool *evlist_used) { struct evsel *ev; - int i = 0, j = 0; bool leader_found; + const size_t idnum = hashmap__size(&pctx->ids); + size_t i = 0; + int j = 0; + double *val_ptr; evlist__for_each_entry (perf_evlist, ev) { if (evlist_used[j++]) continue; - if (!strcmp(ev->name, ids[i])) { + if (hashmap__find(&pctx->ids, ev->name, (void **)&val_ptr)) { if (!metric_events[i]) metric_events[i] = ev; i++; @@ -118,7 +119,8 @@ static struct evsel *find_evsel_group(struct evlist *perf_evlist, memset(metric_events, 0, sizeof(struct evsel *) * idnum); - if (!strcmp(ev->name, ids[i])) { + if (hashmap__find(&pctx->ids, ev->name, + (void **)&val_ptr)) { if (!metric_events[i]) metric_events[i] = ev; i++; @@ -175,19 +177,20 @@ static int metricgroup__setup_events(struct list_head *groups, list_for_each_entry (eg, groups, nd) { struct evsel **metric_events; - metric_events = calloc(sizeof(void *), eg->idnum + 1); + metric_events = calloc(sizeof(void *), + hashmap__size(&eg->pctx.ids) + 1); if (!metric_events) { ret = -ENOMEM; break; } - evsel = find_evsel_group(perf_evlist, eg->ids, eg->idnum, - metric_events, evlist_used); + evsel = find_evsel_group(perf_evlist, &eg->pctx, metric_events, + evlist_used); if (!evsel) { pr_debug("Cannot resolve %s: %s\n", eg->metric_name, eg->metric_expr); continue; } - for (i = 0; i < eg->idnum; i++) + for (i = 0; metric_events[i]; i++) metric_events[i]->collect_stat = true; me = metricgroup__lookup(metric_events_list, evsel, true); if (!me) { @@ -415,20 +418,20 @@ void metricgroup__print(bool metrics, bool metricgroups, char *filter, } static void metricgroup__add_metric_weak_group(struct strbuf *events, - const char **ids, - int idnum) + struct expr_parse_ctx *ctx) { + struct hashmap_entry *cur; + size_t bkt, i = 0; bool no_group = false; - int i; - for (i = 0; i < idnum; i++) { - pr_debug("found event %s\n", ids[i]); + hashmap__for_each_entry((&ctx->ids), cur, bkt) { + pr_debug("found event %s\n", (const char *)cur->key); /* * Duration time maps to a software event and can make * groups not count. Always use it outside a * group. */ - if (!strcmp(ids[i], "duration_time")) { + if (!strcmp(cur->key, "duration_time")) { if (i > 0) strbuf_addf(events, "}:W,"); strbuf_addf(events, "duration_time"); @@ -437,21 +440,22 @@ static void metricgroup__add_metric_weak_group(struct strbuf *events, } strbuf_addf(events, "%s%s", i == 0 || no_group ? "{" : ",", - ids[i]); + (const char *)cur->key); no_group = false; + i++; } if (!no_group) strbuf_addf(events, "}:W"); } static void metricgroup__add_metric_non_group(struct strbuf *events, - const char **ids, - int idnum) + struct expr_parse_ctx *ctx) { - int i; + struct hashmap_entry *cur; + size_t bkt; - for (i = 0; i < idnum; i++) - strbuf_addf(events, ",%s", ids[i]); + hashmap__for_each_entry((&ctx->ids), cur, bkt) + strbuf_addf(events, ",%s", (const char *)cur->key); } static void metricgroup___watchdog_constraint_hint(const char *name, bool foot) @@ -495,32 +499,32 @@ int __weak arch_get_runtimeparam(void) static int __metricgroup__add_metric(struct strbuf *events, struct list_head *group_list, struct pmu_event *pe, int runtime) { - - const char **ids; - int idnum; struct egroup *eg; - if (expr__find_other(pe->metric_expr, NULL, &ids, &idnum, runtime) < 0) - return -EINVAL; - - if (events->len > 0) - strbuf_addf(events, ","); - - if (metricgroup__has_constraint(pe)) - metricgroup__add_metric_non_group(events, ids, idnum); - else - metricgroup__add_metric_weak_group(events, ids, idnum); - eg = malloc(sizeof(*eg)); if (!eg) return -ENOMEM; - eg->ids = ids; - eg->idnum = idnum; + expr__ctx_init(&eg->pctx); eg->metric_name = pe->metric_name; eg->metric_expr = pe->metric_expr; eg->metric_unit = pe->unit; eg->runtime = runtime; + + if (expr__find_other(pe->metric_expr, NULL, &eg->pctx, runtime) < 0) { + expr__ctx_clear(&eg->pctx); + free(eg); + return -EINVAL; + } + + if (events->len > 0) + strbuf_addf(events, ","); + + if (metricgroup__has_constraint(pe)) + metricgroup__add_metric_non_group(events, &eg->pctx); + else + metricgroup__add_metric_weak_group(events, &eg->pctx); + list_add_tail(&eg->nd, group_list); return 0; @@ -603,12 +607,9 @@ static int metricgroup__add_metric_list(const char *list, struct strbuf *events, static void metricgroup__free_egroups(struct list_head *group_list) { struct egroup *eg, *egtmp; - int i; list_for_each_entry_safe (eg, egtmp, group_list, nd) { - for (i = 0; i < eg->idnum; i++) - zfree(&eg->ids[i]); - zfree(&eg->ids); + expr__ctx_clear(&eg->pctx); list_del_init(&eg->nd); free(eg); } diff --git a/tools/perf/util/stat-shadow.c b/tools/perf/util/stat-shadow.c index 9bd7a8d2a858..c44dc814b377 100644 --- a/tools/perf/util/stat-shadow.c +++ b/tools/perf/util/stat-shadow.c @@ -323,35 +323,46 @@ void perf_stat__collect_metric_expr(struct evlist *evsel_list) { struct evsel *counter, *leader, **metric_events, *oc; bool found; - const char **metric_names; + struct expr_parse_ctx ctx; + struct hashmap_entry *cur; + size_t bkt; int i; - int num_metric_names; + expr__ctx_init(&ctx); evlist__for_each_entry(evsel_list, counter) { bool invalid = false; leader = counter->leader; if (!counter->metric_expr) continue; + + expr__ctx_clear(&ctx); metric_events = counter->metric_events; if (!metric_events) { - if (expr__find_other(counter->metric_expr, counter->name, - &metric_names, &num_metric_names, 1) < 0) + if (expr__find_other(counter->metric_expr, + counter->name, + &ctx, 1) < 0) continue; metric_events = calloc(sizeof(struct evsel *), - num_metric_names + 1); - if (!metric_events) + hashmap__size(&ctx.ids) + 1); + if (!metric_events) { + expr__ctx_clear(&ctx); return; + } counter->metric_events = metric_events; } - for (i = 0; i < num_metric_names; i++) { + i = 0; + hashmap__for_each_entry((&ctx.ids), cur, bkt) { + const char *metric_name = (const char *)cur->key; + found = false; if (leader) { /* Search in group */ for_each_group_member (oc, leader) { - if (!strcasecmp(oc->name, metric_names[i]) && + if (!strcasecmp(oc->name, + metric_name) && !oc->collect_stat) { found = true; break; @@ -360,7 +371,8 @@ void perf_stat__collect_metric_expr(struct evlist *evsel_list) } if (!found) { /* Search ignoring groups */ - oc = perf_stat__find_event(evsel_list, metric_names[i]); + oc = perf_stat__find_event(evsel_list, + metric_name); } if (!oc) { /* Deduping one is good enough to handle duplicated PMUs. */ @@ -373,27 +385,28 @@ void perf_stat__collect_metric_expr(struct evlist *evsel_list) * of events. So we ask the user instead to add the missing * events. */ - if (!printed || strcasecmp(printed, metric_names[i])) { + if (!printed || + strcasecmp(printed, metric_name)) { fprintf(stderr, "Add %s event to groups to get metric expression for %s\n", - metric_names[i], + metric_name, counter->name); - printed = strdup(metric_names[i]); + printed = strdup(metric_name); } invalid = true; continue; } - metric_events[i] = oc; + metric_events[i++] = oc; oc->collect_stat = true; } metric_events[i] = NULL; - free(metric_names); if (invalid) { free(metric_events); counter->metric_events = NULL; counter->metric_expr = NULL; } } + expr__ctx_clear(&ctx); } static double runtime_stat_avg(struct runtime_stat *st, @@ -738,7 +751,10 @@ static void generic_metric(struct perf_stat_config *config, expr__ctx_init(&pctx); /* Must be first id entry */ - expr__add_id(&pctx, name, avg); + n = strdup(name); + if (!n) + return; + expr__add_id(&pctx, n, avg); for (i = 0; metric_events[i]; i++) { struct saved_value *v; struct stats *stats; @@ -814,8 +830,7 @@ static void generic_metric(struct perf_stat_config *config, (metric_name ? metric_name : name) : "", 0); } - for (i = 1; i < pctx.num_ids; i++) - zfree(&pctx.ids[i].name); + expr__ctx_clear(&pctx); } void perf_stat__print_shadow_stats(struct perf_stat_config *config,