Message ID | 20190610063229.32560-1-anju@linux.vnet.ibm.com (mailing list archive) |
---|---|
State | Accepted |
Commit | 9c9f8fb71feed144973a70455e0a4ee3da57ed2a |
Headers | show |
Series | [v2] powerpc/perf: Use cpumask_last() to determine the designated cpu for nest/core units. | expand |
Context | Check | Description |
---|---|---|
snowpatch_ozlabs/apply_patch | success | Successfully applied on branch next (a3bf9fbdad600b1e4335dd90979f8d6072e4f602) |
snowpatch_ozlabs/build-ppc64le | success | Build succeeded |
snowpatch_ozlabs/build-ppc64be | success | Build succeeded |
snowpatch_ozlabs/build-ppc64e | success | Build succeeded |
snowpatch_ozlabs/build-pmac32 | success | Build succeeded |
snowpatch_ozlabs/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 26 lines checked |
On Mon, 2019-06-10 at 12:02 +0530, Anju T Sudhakar wrote: > Nest and core imc(In-memory Collection counters) assigns a particular > cpu as the designated target for counter data collection. > During system boot, the first online cpu in a chip gets assigned as > the designated cpu for that chip(for nest-imc) and the first online cpu > in a core gets assigned as the designated cpu for that core(for core-imc). > > If the designated cpu goes offline, the next online cpu from the same > chip(for nest-imc)/core(for core-imc) is assigned as the next target, > and the event context is migrated to the target cpu. > Currently, cpumask_any_but() function is used to find the target cpu. > Though this function is expected to return a `random` cpu, this always > returns the next online cpu. > > If all cpus in a chip/core is offlined in a sequential manner, starting > from the first cpu, the event migration has to happen for all the cpus > which goes offline. Since the migration process involves a grace period, > the total time taken to offline all the cpus will be significantly high. > > Example: > In a system which has 2 sockets, with > NUMA node0 CPU(s): 0-87 > NUMA node8 CPU(s): 88-175 > > Time taken to offline cpu 88-175: > real 2m56.099s > user 0m0.191s > sys 0m0.000s > > Use cpumask_last() to choose the target cpu, when the designated cpu > goes online, so the migration will happen only when the last_cpu in the > mask goes offline. This way the time taken to offline all cpus in a > chip/core can be reduced. > > With the patch, > > Time taken to offline cpu 88-175: > real 0m12.207s > user 0m0.171s > sys 0m0.000s > > > Offlining all cpus in reverse order is also taken care because, > cpumask_any_but() is used to find the designated cpu if the last cpu in > the mask goes offline. Since cpumask_any_but() always return the first > cpu in the mask, that becomes the designated cpu and migration will happen > only when the first_cpu in the mask goes offline. > > Example: > With the patch, > > Time taken to offline cpu from 175-88: > real 0m9.330s > user 0m0.110s > sys 0m0.000s Seems like a very interesting work. Out of curiosity, have you used 'chcpu -d' to create your benchmark? > > Signed-off-by: Anju T Sudhakar <anju@linux.vnet.ibm.com> > Reviewed-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com> > --- > > Changes from v1: > Modified the commit log with more info. > --- > > arch/powerpc/perf/imc-pmu.c | 14 ++++++++++++-- > 1 file changed, 12 insertions(+), 2 deletions(-) > > diff --git a/arch/powerpc/perf/imc-pmu.c b/arch/powerpc/perf/imc-pmu.c > index 31fa753..fbfd6e7 100644 > --- a/arch/powerpc/perf/imc-pmu.c > +++ b/arch/powerpc/perf/imc-pmu.c > @@ -366,7 +366,14 @@ static int ppc_nest_imc_cpu_offline(unsigned int cpu) > */ > nid = cpu_to_node(cpu); > l_cpumask = cpumask_of_node(nid); > - target = cpumask_any_but(l_cpumask, cpu); > + target = cpumask_last(l_cpumask); > + > + /* > + * If this(target) is the last cpu in the cpumask for this chip, > + * check for any possible online cpu in the chip. > + */ > + if (unlikely(target == cpu)) > + target = cpumask_any_but(l_cpumask, cpu); > > /* > * Update the cpumask with the target cpu and > @@ -671,7 +678,10 @@ static int ppc_core_imc_cpu_offline(unsigned int cpu) > return 0; > > /* Find any online cpu in that core except the current "cpu" */ > - ncpu = cpumask_any_but(cpu_sibling_mask(cpu), cpu); > + ncpu = cpumask_last(cpu_sibling_mask(cpu)); > + > + if (unlikely(ncpu == cpu)) > + ncpu = cpumask_any_but(cpu_sibling_mask(cpu), cpu); > > if (ncpu >= 0 && ncpu < nr_cpu_ids) { > cpumask_set_cpu(ncpu, &core_imc_cpumask);
Hi Leonardo, On 6/11/19 12:17 AM, Leonardo Bras wrote: > On Mon, 2019-06-10 at 12:02 +0530, Anju T Sudhakar wrote: >> Nest and core imc(In-memory Collection counters) assigns a particular >> cpu as the designated target for counter data collection. >> During system boot, the first online cpu in a chip gets assigned as >> the designated cpu for that chip(for nest-imc) and the first online cpu >> in a core gets assigned as the designated cpu for that core(for core-imc). >> >> If the designated cpu goes offline, the next online cpu from the same >> chip(for nest-imc)/core(for core-imc) is assigned as the next target, >> and the event context is migrated to the target cpu. >> Currently, cpumask_any_but() function is used to find the target cpu. >> Though this function is expected to return a `random` cpu, this always >> returns the next online cpu. >> >> If all cpus in a chip/core is offlined in a sequential manner, starting >> from the first cpu, the event migration has to happen for all the cpus >> which goes offline. Since the migration process involves a grace period, >> the total time taken to offline all the cpus will be significantly high. > Seems like a very interesting work. > Out of curiosity, have you used 'chcpu -d' to create your benchmark? Here I did not use chcpu to disable the cpu. I used a script which will offline cpus 88-175 by echoing `0` to /sys/devices/system/cpu/cpu*/online. Regards, Anju
diff --git a/arch/powerpc/perf/imc-pmu.c b/arch/powerpc/perf/imc-pmu.c index 31fa753..fbfd6e7 100644 --- a/arch/powerpc/perf/imc-pmu.c +++ b/arch/powerpc/perf/imc-pmu.c @@ -366,7 +366,14 @@ static int ppc_nest_imc_cpu_offline(unsigned int cpu) */ nid = cpu_to_node(cpu); l_cpumask = cpumask_of_node(nid); - target = cpumask_any_but(l_cpumask, cpu); + target = cpumask_last(l_cpumask); + + /* + * If this(target) is the last cpu in the cpumask for this chip, + * check for any possible online cpu in the chip. + */ + if (unlikely(target == cpu)) + target = cpumask_any_but(l_cpumask, cpu); /* * Update the cpumask with the target cpu and @@ -671,7 +678,10 @@ static int ppc_core_imc_cpu_offline(unsigned int cpu) return 0; /* Find any online cpu in that core except the current "cpu" */ - ncpu = cpumask_any_but(cpu_sibling_mask(cpu), cpu); + ncpu = cpumask_last(cpu_sibling_mask(cpu)); + + if (unlikely(ncpu == cpu)) + ncpu = cpumask_any_but(cpu_sibling_mask(cpu), cpu); if (ncpu >= 0 && ncpu < nr_cpu_ids) { cpumask_set_cpu(ncpu, &core_imc_cpumask);