| Message ID | 20181016074504.2042-1-abrodkin@synopsys.com |
|---|---|
| State | New |
| Headers | show |
| Series | clocksource/drivers/arc_timer: Utilize generic sched_clock | expand |
On 10/16/2018 12:45 AM, Alexey Brodkin wrote: > It turned out we used to use default implementation of sched_clock() > from kernel/sched/clock.c which was as precise as 1/HZ, i.e. > by default we had 10 msec granularity of time measurement. > > Now given ARC built-in timers are clocked with the same frequency as > CPU cores we may get much higher precision of time tracking. Can you do LMBench runs with and w/o and see if there's any other changes. I'm hoping lat_ctx will be more consistent. > diff --git a/drivers/clocksource/Kconfig b/drivers/clocksource/Kconfig > index dec0dd88ec15..3268dad4effe 100644 > --- a/drivers/clocksource/Kconfig > +++ b/drivers/clocksource/Kconfig > @@ -290,6 +290,7 @@ config CLKSRC_MPS2 > > config ARC_TIMERS > bool "Support for 32-bit TIMERn counters in ARC Cores" if COMPILE_TEST > + depends on GENERIC_SCHED_CLOCK It needs to select, not depends on > @@ -88,6 +89,11 @@ static u64 arc_read_gfrc(struct clocksource *cs) > return (((u64)h) << 32) | l; > } > > +static u64 arc_gfrc_clock_read(void) Needs to be notrace like other such routines. > > + > static struct clocksource arc_counter_timer1 = { > .name = "ARC Timer1", > .rating = 300, > @@ -209,6 +229,8 @@ static int __init arc_cs_setup_timer1(struct device_node *node) > write_aux_reg(ARC_REG_TIMER1_CNT, 0); > write_aux_reg(ARC_REG_TIMER1_CTRL, TIMER_CTRL_NH); > > + sched_clock_register(arc_timer1_clock_read, 64, arc_timer_freq); TIMER1 is 32 bits wide. -Vineet
Hi Vineet, > -----Original Message----- > From: Vineet Gupta > Sent: Tuesday, October 16, 2018 7:03 PM > To: Alexey Brodkin <abrodkin@synopsys.com>; linux-kernel@vger.kernel.org > Cc: linux-snps-arc@lists.infradead.org; Daniel Lezcano <daniel.lezcano@linaro.org>; Thomas Gleixner <tglx@linutronix.de> > Subject: Re: [PATCH] clocksource/drivers/arc_timer: Utilize generic sched_clock > > On 10/16/2018 12:45 AM, Alexey Brodkin wrote: > > It turned out we used to use default implementation of sched_clock() > > from kernel/sched/clock.c which was as precise as 1/HZ, i.e. > > by default we had 10 msec granularity of time measurement. > > > > Now given ARC built-in timers are clocked with the same frequency as > > CPU cores we may get much higher precision of time tracking. > > Can you do LMBench runs with and w/o and see if there's any other changes. I'm > hoping lat_ctx will be more consistent. Sure, I have this, see below: ======================================================== L M B E N C H 3 . 0 S U M M A R Y ------------------------------------ (Alpha software, do not distribute) Basic system parameters -------------------------------------------------------------------------------------- Host OS Description Mhz tlb cache mem scal pages line par load bytes ----------------- ------------- --------------------------------------- ---- ----- ----- ------ ---- hz-2018.10.16.log Linux 4.18.14 hz-2018.10.16.log 998 8 128 1.7500 1 sched_clock-2018. Linux 4.18.14 sched_clock-2018.10.16.log 997 8 1.7500 1 Processor, Processes - times in microseconds - smaller is better ------------------------------------------------------------------------------ Host OS Mhz null null open slct sig sig fork exec sh call I/O stat clos TCP inst hndl proc proc proc --------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- hz-2018.1 Linux 4.18.14 998 0.38 0.59 2.76 5.02 22.4 0.54 2.23 319. 1780 3347 sched_clo Linux 4.18.14 997 0.38 0.61 2.66 5.01 22.3 0.54 2.25 325. 1775 3315 Basic integer operations - times in nanoseconds - smaller is better ------------------------------------------------------------------- Host OS intgr intgr intgr intgr intgr bit add mul div mod --------- ------------- ------ ------ ------ ------ ------ hz-2018.1 Linux 4.18.14 1.0100 1.1500 13.0 7.9900 sched_clo Linux 4.18.14 1.0100 1.1500 13.0 7.9900 Basic uint64 operations - times in nanoseconds - smaller is better ------------------------------------------------------------------ Host OS int64 int64 int64 int64 int64 bit add mul div mod --------- ------------- ------ ------ ------ ------ ------ hz-2018.1 Linux 4.18.14 11. 10.2 126.2 96.5 sched_clo Linux 4.18.14 11. 10.2 126.2 96.6 Basic float operations - times in nanoseconds - smaller is better ----------------------------------------------------------------- Host OS float float float float add mul div bogo --------- ------------- ------ ------ ------ ------ hz-2018.1 Linux 4.18.14 184.5 181.5 375.0 935.3 sched_clo Linux 4.18.14 185.1 185.7 375.0 882.9 Basic double operations - times in nanoseconds - smaller is better ------------------------------------------------------------------ Host OS double double double double add mul div bogo --------- ------------- ------ ------ ------ ------ hz-2018.1 Linux 4.18.14 253.9 285.6 1490.0 2264.3 sched_clo Linux 4.18.14 255.0 283.6 1515.0 2284.3 Context switching - times in microseconds - smaller is better ------------------------------------------------------------------------- Host OS 2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw ctxsw --------- ------------- ------ ------ ------ ------ ------ ------- ------- hz-2018.1 Linux 4.18.14 3.4400 6.7700 0.7300 9.6300 68.1 15.3 115.0 sched_clo Linux 4.18.14 3.5100 7.1300 5.7900 7.9200 71.8 17.6 124.5 *Local* Communication latencies in microseconds - smaller is better --------------------------------------------------------------------- Host OS 2p/0K Pipe AF UDP RPC/ TCP RPC/ TCP ctxsw UNIX UDP TCP conn --------- ------------- ----- ----- ---- ----- ----- ----- ----- ---- hz-2018.1 Linux 4.18.14 3.440 14.6 16.0 31.5 48.0 94. sched_clo Linux 4.18.14 3.510 15.5 13.4 31.5 49.4 158. *Remote* Communication latencies in microseconds - smaller is better --------------------------------------------------------------------- Host OS UDP RPC/ TCP RPC/ TCP UDP TCP conn --------- ------------- ----- ----- ----- ----- ---- hz-2018.1 Linux 4.18.14 sched_clo Linux 4.18.14 File & VM system latencies in microseconds - smaller is better ------------------------------------------------------------------------------- Host OS 0K File 10K File Mmap Prot Page 100fd Create Delete Create Delete Latency Fault Fault selct --------- ------------- ------ ------ ------ ------ ------- ----- ------- ----- hz-2018.1 Linux 4.18.14 11.0 7.8438 54.3 12.8 221.0 0.475 1.16420 8.607 sched_clo Linux 4.18.14 10.8 7.7240 54.1 12.9 223.0 0.505 1.16280 8.475 *Local* Communication bandwidths in MB/s - bigger is better ----------------------------------------------------------------------------- Host OS Pipe AF TCP File Mmap Bcopy Bcopy Mem Mem UNIX reread reread (libc) (hand) read write --------- ------------- ---- ---- ---- ------ ------ ------ ------ ---- ----- hz-2018.1 Linux 4.18.14 224. 456. 111. 248.0 404.8 384.6 232.4 405. 454.4 sched_clo Linux 4.18.14 220. 458. 145. 241.6 404.6 384.3 232.3 404. 453.9 Memory latencies in nanoseconds - smaller is better (WARNING - may not be correct, check graphs) ------------------------------------------------------------------------------ Host OS Mhz L1 $ L2 $ Main mem Rand mem Guesses --------- ------------- --- ---- ---- -------- -------- ------- hz-2018.1 Linux 4.18.14 998 3.0820 24.3 210.5 371.0 sched_clo Linux 4.18.14 997 3.0820 24.3 210.6 371.2 ======================================================== > > diff --git a/drivers/clocksource/Kconfig b/drivers/clocksource/Kconfig > > index dec0dd88ec15..3268dad4effe 100644 > > --- a/drivers/clocksource/Kconfig > > +++ b/drivers/clocksource/Kconfig > > @@ -290,6 +290,7 @@ config CLKSRC_MPS2 > > > > config ARC_TIMERS > > bool "Support for 32-bit TIMERn counters in ARC Cores" if COMPILE_TEST > > + depends on GENERIC_SCHED_CLOCK > > It needs to select, not depends on Well from what I may see git grepping: 1) Architcture selects GENERIC_SCHED_CLOCK 2) Clocksource driver depends on GENERIC_SCHED_CLOCK > > @@ -88,6 +89,11 @@ static u64 arc_read_gfrc(struct clocksource *cs) > > return (((u64)h) << 32) | l; > > } > > > > +static u64 arc_gfrc_clock_read(void) > > Needs to be notrace like other such routines. Ok will add this. > > > > + > > static struct clocksource arc_counter_timer1 = { > > .name = "ARC Timer1", > > .rating = 300, > > @@ -209,6 +229,8 @@ static int __init arc_cs_setup_timer1(struct device_node *node) > > write_aux_reg(ARC_REG_TIMER1_CNT, 0); > > write_aux_reg(ARC_REG_TIMER1_CTRL, TIMER_CTRL_NH); > > > > + sched_clock_register(arc_timer1_clock_read, 64, arc_timer_freq); > > TIMER1 is 32 bits wide. Yep , blind copy-pasting is not good, will change it. -Alexey
diff --git a/arch/arc/Kconfig b/arch/arc/Kconfig index 5151d81476a1..714f769389a4 100644 --- a/arch/arc/Kconfig +++ b/arch/arc/Kconfig @@ -9,6 +9,7 @@ config ARC def_bool y select ARC_TIMERS + select GENERIC_SCHED_CLOCK select ARCH_HAS_SYNC_DMA_FOR_CPU select ARCH_HAS_SYNC_DMA_FOR_DEVICE select ARCH_HAS_SG_CHAIN diff --git a/drivers/clocksource/Kconfig b/drivers/clocksource/Kconfig index dec0dd88ec15..3268dad4effe 100644 --- a/drivers/clocksource/Kconfig +++ b/drivers/clocksource/Kconfig @@ -290,6 +290,7 @@ config CLKSRC_MPS2 config ARC_TIMERS bool "Support for 32-bit TIMERn counters in ARC Cores" if COMPILE_TEST + depends on GENERIC_SCHED_CLOCK select TIMER_OF help These are legacy 32-bit TIMER0 and TIMER1 counters found on all ARC cores diff --git a/drivers/clocksource/arc_timer.c b/drivers/clocksource/arc_timer.c index 20da9b1d7f7d..6d0ebf64a1c5 100644 --- a/drivers/clocksource/arc_timer.c +++ b/drivers/clocksource/arc_timer.c @@ -23,6 +23,7 @@ #include <linux/cpu.h> #include <linux/of.h> #include <linux/of_irq.h> +#include <linux/sched_clock.h> #include <soc/arc/timers.h> #include <soc/arc/mcip.h> @@ -88,6 +89,11 @@ static u64 arc_read_gfrc(struct clocksource *cs) return (((u64)h) << 32) | l; } +static u64 arc_gfrc_clock_read(void) +{ + return arc_read_gfrc(NULL); +} + static struct clocksource arc_counter_gfrc = { .name = "ARConnect GFRC", .rating = 400, @@ -111,6 +117,8 @@ static int __init arc_cs_setup_gfrc(struct device_node *node) if (ret) return ret; + sched_clock_register(arc_gfrc_clock_read, 64, arc_timer_freq); + return clocksource_register_hz(&arc_counter_gfrc, arc_timer_freq); } TIMER_OF_DECLARE(arc_gfrc, "snps,archs-timer-gfrc", arc_cs_setup_gfrc); @@ -139,6 +147,11 @@ static u64 arc_read_rtc(struct clocksource *cs) return (((u64)h) << 32) | l; } +static u64 arc_rtc_clock_read(void) +{ + return arc_read_rtc(NULL); +} + static struct clocksource arc_counter_rtc = { .name = "ARCv2 RTC", .rating = 350, @@ -170,6 +183,8 @@ static int __init arc_cs_setup_rtc(struct device_node *node) write_aux_reg(AUX_RTC_CTRL, 1); + sched_clock_register(arc_rtc_clock_read, 64, arc_timer_freq); + return clocksource_register_hz(&arc_counter_rtc, arc_timer_freq); } TIMER_OF_DECLARE(arc_rtc, "snps,archs-timer-rtc", arc_cs_setup_rtc); @@ -185,6 +200,11 @@ static u64 arc_read_timer1(struct clocksource *cs) return (u64) read_aux_reg(ARC_REG_TIMER1_CNT); } +static u64 arc_timer1_clock_read(void) +{ + return arc_read_timer1(NULL); +} + static struct clocksource arc_counter_timer1 = { .name = "ARC Timer1", .rating = 300, @@ -209,6 +229,8 @@ static int __init arc_cs_setup_timer1(struct device_node *node) write_aux_reg(ARC_REG_TIMER1_CNT, 0); write_aux_reg(ARC_REG_TIMER1_CTRL, TIMER_CTRL_NH); + sched_clock_register(arc_timer1_clock_read, 64, arc_timer_freq); + return clocksource_register_hz(&arc_counter_timer1, arc_timer_freq); }
It turned out we used to use default implementation of sched_clock() from kernel/sched/clock.c which was as precise as 1/HZ, i.e. by default we had 10 msec granularity of time measurement. Now given ARC built-in timers are clocked with the same frequency as CPU cores we may get much higher precision of time tracking. Thus we switch to generic sched_clock which really reads ARC hardware counters. This is especially helpful for measuring short events. That's what we used to have: ------------------------------>8------------------------ $ perf stat /bin/sh -c /root/lmbench-master/bin/arc/hello > /dev/null Performance counter stats for '/bin/sh -c /root/lmbench-master/bin/arc/hello': 10.000000 task-clock (msec) # 2.832 CPUs utilized 1 context-switches # 0.100 K/sec 1 cpu-migrations # 0.100 K/sec 63 page-faults # 0.006 M/sec 3049480 cycles # 0.305 GHz 1091259 instructions # 0.36 insn per cycle 256828 branches # 25.683 M/sec 27026 branch-misses # 10.52% of all branches 0.003530687 seconds time elapsed 0.000000000 seconds user 0.010000000 seconds sys ------------------------------>8------------------------ And now we'll see: ------------------------------>8------------------------ $ perf stat /bin/sh -c /root/lmbench-master/bin/arc/hello > /dev/null Performance counter stats for '/bin/sh -c /root/lmbench-master/bin/arc/hello': 3.004322 task-clock (msec) # 0.865 CPUs utilized 1 context-switches # 0.333 K/sec 1 cpu-migrations # 0.333 K/sec 63 page-faults # 0.021 M/sec 2986734 cycles # 0.994 GHz 1087466 instructions # 0.36 insn per cycle 255209 branches # 84.947 M/sec 26002 branch-misses # 10.19% of all branches 0.003474829 seconds time elapsed 0.003519000 seconds user 0.000000000 seconds sys ------------------------------>8------------------------ Note how much more meaningful is the second output - time spent for execution pretty much matches number of cycles spent (we're runnign @ 1GHz here). Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Cc: Daniel Lezcano <daniel.lezcano@linaro.org> Cc: Vineet Gupta <vgupta@synopsys.com> Cc: Thomas Gleixner <tglx@linutronix.de> --- arch/arc/Kconfig | 1 + drivers/clocksource/Kconfig | 1 + drivers/clocksource/arc_timer.c | 22 ++++++++++++++++++++++ 3 files changed, 24 insertions(+)