| Message ID | 20200721124300.65615-1-psampat@linux.ibm.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Selftest for cpuidle latency measurement | expand |
On 21/07/2020 14:42, Pratik Rajesh Sampat wrote: > v2: https://lkml.org/lkml/2020/7/17/369 > Changelog v2-->v3 > Based on comments from Gautham R. Shenoy adding the following in the > selftest, > 1. Grepping modules to determine if already loaded > 2. Wrapper to enable/disable states > 3. Preventing any operation/test on offlined CPUs > --- > > The patch series introduces a mechanism to measure wakeup latency for > IPI and timer based interrupts > The motivation behind this series is to find significant deviations > behind advertised latency and resisdency values Why do you want to measure for the timer and the IPI ? Whatever the source of the wakeup, the exit latency remains the same, no ? Is all this kernel-ish code really needed ? What about using a highres periodic timer and make it expires every eg. 50ms x 2400, so it is 120 secondes and measure the deviation. Repeat the operation for each idle states. And in order to make it as much accurate as possible, set the program affinity on a CPU and isolate this one by preventing other processes to be scheduled on and migrate the interrupts on the other CPUs. That will be all userspace code, no?
Hello Daniel, On 21/07/20 8:27 pm, Daniel Lezcano wrote: > On 21/07/2020 14:42, Pratik Rajesh Sampat wrote: >> v2: https://lkml.org/lkml/2020/7/17/369 >> Changelog v2-->v3 >> Based on comments from Gautham R. Shenoy adding the following in the >> selftest, >> 1. Grepping modules to determine if already loaded >> 2. Wrapper to enable/disable states >> 3. Preventing any operation/test on offlined CPUs >> --- >> >> The patch series introduces a mechanism to measure wakeup latency for >> IPI and timer based interrupts >> The motivation behind this series is to find significant deviations >> behind advertised latency and resisdency values > Why do you want to measure for the timer and the IPI ? Whatever the > source of the wakeup, the exit latency remains the same, no ? > > Is all this kernel-ish code really needed ? > > What about using a highres periodic timer and make it expires every eg. > 50ms x 2400, so it is 120 secondes and measure the deviation. Repeat the > operation for each idle states. > > And in order to make it as much accurate as possible, set the program > affinity on a CPU and isolate this one by preventing other processes to > be scheduled on and migrate the interrupts on the other CPUs. > > That will be all userspace code, no? > > The kernel module may not needed now that you mention it. IPI latencies could be measured using pipes and threads using pthread_attr_setaffinity_np to control the experiment, as you suggested. This should internally fire a smp_call_function_single. The original idea was to essentially measure it as closely as possible in the kernel without involving the kernel-->userspace overhead. However, the user-space approach may not be too much of a problem as we are collecting a baseline and the delta of the latency is what we would be concerned about anyways! With respect to measuring both timers and IPI latencies: In principle yes, the exit latency should remain the same but if there is a deviation in reality we may want to measure it. I'll implement this experiment in the userspace and get back with the numbers to confirm. Thanks for your comments! Best, Pratik > >
v2: https://lkml.org/lkml/2020/7/17/369 Changelog v2-->v3 Based on comments from Gautham R. Shenoy adding the following in the selftest, 1. Grepping modules to determine if already loaded 2. Wrapper to enable/disable states 3. Preventing any operation/test on offlined CPUs --- The patch series introduces a mechanism to measure wakeup latency for IPI and timer based interrupts The motivation behind this series is to find significant deviations behind advertised latency and resisdency values To achieve this, we introduce a kernel module and expose its control knobs through the debugfs interface that the selftests can engage with. The kernel module provides the following interfaces within /sys/kernel/debug/latency_test/ for, 1. IPI test: ipi_cpu_dest # Destination CPU for the IPI ipi_cpu_src # Origin of the IPI ipi_latency_ns # Measured latency time in ns 2. Timeout test: timeout_cpu_src # CPU on which the timer to be queued timeout_expected_ns # Timer duration timeout_diff_ns # Difference of actual duration vs expected timer To include the module, check option and include as module kernel hacking -> Cpuidle latency selftests The selftest inserts the module, disables all the idle states and enables them one by one testing the following: 1. Keeping source CPU constant, iterates through all the CPUS measuring IPI latency for baseline (CPU is busy with "cat /dev/random > /dev/null" workload) and the when the CPU is allowed to be at rest 2. Iterating through all the CPUs, sending expected timer durations to be equivalent to the residency of the the deepest idle state enabled and extracting the difference in time between the time of wakeup and the expected timer duration Usage ----- Can be used in conjuction to the rest of the selftests. Default Output location in: tools/testing/cpuidle/cpuidle.log To run this test specifically: $ make -C tools/testing/selftests TARGETS="cpuidle" run_tests There are a few optinal arguments too that the script can take [-h <help>] [-m <location of the module>] [-o <location of the output>] Sample output snippet --------------------- --IPI Latency Test--- --Baseline IPI Latency measurement: CPU Busy-- SRC_CPU DEST_CPU IPI_Latency(ns) ... 0 8 1996 0 9 2125 0 10 1264 0 11 1788 0 12 2045 Baseline Average IPI latency(ns): 1843 ---Enabling state: 5--- SRC_CPU DEST_CPU IPI_Latency(ns) 0 8 621719 0 9 624752 0 10 622218 0 11 623968 0 12 621303 Expected IPI latency(ns): 100000 Observed Average IPI latency(ns): 622792 --Timeout Latency Test-- --Baseline Timeout Latency measurement: CPU Busy-- Wakeup_src Baseline_delay(ns) ... 8 2249 9 2226 10 2211 11 2183 12 2263 Baseline Average timeout diff(ns): 2226 ---Enabling state: 5--- 8 10749 9 10911 10 10912 11 12100 12 73276 Expected timeout(ns): 10000200 Observed Average timeout diff(ns): 23589 Pratik Rajesh Sampat (2): cpuidle: Trace IPI based and timer based wakeup latency from idle states selftest/cpuidle: Add support for cpuidle latency measurement drivers/cpuidle/Makefile | 1 + drivers/cpuidle/test-cpuidle_latency.c | 150 ++++++++++ lib/Kconfig.debug | 10 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/cpuidle/Makefile | 6 + tools/testing/selftests/cpuidle/cpuidle.sh | 310 +++++++++++++++++++++ tools/testing/selftests/cpuidle/settings | 1 + 7 files changed, 479 insertions(+) create mode 100644 drivers/cpuidle/test-cpuidle_latency.c create mode 100644 tools/testing/selftests/cpuidle/Makefile create mode 100755 tools/testing/selftests/cpuidle/cpuidle.sh create mode 100644 tools/testing/selftests/cpuidle/settings