From patchwork Tue Aug 27 00:42:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Ruffell X-Patchwork-Id: 1977068 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=185.125.189.65; helo=lists.ubuntu.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from lists.ubuntu.com (lists.ubuntu.com [185.125.189.65]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Wt80D4BBTz1yXd for ; Tue, 27 Aug 2024 10:42:51 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=lists.ubuntu.com) by lists.ubuntu.com with esmtp (Exim 4.86_2) (envelope-from ) id 1sikIA-0002Ny-6f; Tue, 27 Aug 2024 00:42:38 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by lists.ubuntu.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1sikI8-0002Nr-Sv for kernel-team@lists.ubuntu.com; Tue, 27 Aug 2024 00:42:36 +0000 Received: from mail-ot1-f72.google.com (mail-ot1-f72.google.com [209.85.210.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id B35963F285 for ; Tue, 27 Aug 2024 00:42:36 +0000 (UTC) Received: by mail-ot1-f72.google.com with SMTP id 46e09a7af769-709435310b7so6269938a34.2 for ; Mon, 26 Aug 2024 17:42:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724719355; x=1725324155; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=e6a1/b+gOtPVmqFAHOxMmgQfyz3migF64ISF1FxSQu0=; b=NeBuDQeO7Q8NBMFJkBk4uQHuKFwYV+dPIGSYCytIlpjbL2n/UT1eKcnafzi2laIQsQ ZLNzTANWn/I/Vh45H/EDN6L2277NMIcFKJ5nVJw/XkhrOvvAi/KsQ8uTPqF0kxIKVTaj z1GsG283YfjcqHRlz4VOwXCbDebTza8EBKBTpon8Jf23pxvA881IazzQzWBp2xROGYy9 Qh/dFz14F3qBeCmrVWWgqF7XM1unskU//dBr44Cvjgzm4onXE8mO+EegxvOe5ZUdgc22 DbnWbsgoRYz0GARvCsrCYZb845QuEuyiV0wM0XpznE72inxeOgHXAiICIYTLDxm/PqFD 0s7w== X-Gm-Message-State: AOJu0YwXIhABBIHvvQeu4PM7YpVknd66NZXbEvkx37qjgP94AA0On1lm 9kkF1XnhWikKgKg114rpJ3tateCXD0ZkPK24du0B8cMSAWU1W1wKeTiuv8ncmun/EL+KWH48UjK xJCsyPHwwKPd+ccLfZRs+b+Ekdd2shaXOy0dFYqYOJwKjDPglk5oEczYX6WN8AfuTQI8qTA55dv Rr/nQPnLJ+rA== X-Received: by 2002:a05:6830:3905:b0:709:2fa3:133a with SMTP id 46e09a7af769-70e0ebfa24cmr14611937a34.18.1724719355586; Mon, 26 Aug 2024 17:42:35 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG5dC5JWfmVLguSwCOcb5aYvyYYyCCPzIrHAsxt7IFEcFRaJNKq0dmiMX5uZ+JCDntYiR4MNg== X-Received: by 2002:a05:6830:3905:b0:709:2fa3:133a with SMTP id 46e09a7af769-70e0ebfa24cmr14611931a34.18.1724719355175; Mon, 26 Aug 2024 17:42:35 -0700 (PDT) Received: from ThinkPad-X1.. (125-239-191-70-fibre.sparkbb.co.nz. [125.239.191.70]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71434340419sm7540177b3a.205.2024.08.26.17.42.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 26 Aug 2024 17:42:34 -0700 (PDT) From: Matthew Ruffell To: kernel-team@lists.ubuntu.com Subject: [SRU][Jammy][PATCH 0/1] turbostat fails with too many open files on large systems Date: Tue, 27 Aug 2024 12:42:05 +1200 Message-ID: <20240827004228.16253-1-matthew.ruffell@canonical.com> X-Mailer: git-send-email 2.45.2 MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: https://bugs.launchpad.net/bugs/2069961 [Impact] On large systems, e.g. with 512 cpus or more, turbostat fails to run due to exceeding the rlimit for number of files. 512 cpus requires 1028 file descriptors, but the current limit is 999. $ lscpu ... CPU(s): 512 On-line CPU(s) list: 0-511 ... $ sudo turbostat ... turbostat: /sys/devices/system/cpu/cpu477/cpuidle/state0/usage: open failed: Too many open files There is no workaround, apart from maybe using powerstat instead. [Fix] The fix is to increase the rlimit to increase the amount of file descriptors that turbostat can open to 2^15, which should be plenty for some time to come. commit 3ac1d14d0583a2de75d49a5234d767e2590384dd Author: Wyes Karny Date: Tue Oct 3 05:07:51 2023 +0000 Subject: tools/power turbostat: Increase the limit for fd opened Link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3ac1d14d0583a2de75d49a5234d767e2590384dd This landed in 6.9-rc4, and requires a backport for minor context adjustment in the first hunk for jammy. Noble got fixed already through upstream stable. [Testcase] Deploy a bare metal system with 512 or more cpus. Install linux-tools: $ sudo apt install linux-tools-$(uname -r) Run turbostat: $ sudo turbostat ... turbostat: /sys/devices/system/cpu/cpu477/cpuidle/state0/usage: open failed: Too many open files There are test kernels available in the following ppa: https://launchpad.net/~mruffell/+archive/ubuntu/sf388491-test If you install them, you should be able to see normal turbostat output for all cpus installed in the system. [Where problems can occur] We are simply increasing the rlimit for file descriptors that turbostat can open. This should have no impact on any existing systems. If a regression should occur, then turbostat functionality might not work. Users could use powerstat instead as a workaround while things are fixed. Wyes Karny (1): tools/power turbostat: Increase the limit for fd opened tools/power/x86/turbostat/turbostat.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) Acked-by: Stefan Bader Acked-by: Manuel Diewald