[Ksummit-discuss] [TECH TOPIC] benchmarking and performance trends

[Fengguang Wu]

On Wed, Jul 15, 2015 at 09:35:51PM -0400, Len Brown wrote:
> > It would be fun to use turbostat or a rack power meter to
> > measure/compare power usage between two kernels in a given benchmark.  I
> > think the power meters we do have are not going to be fine grained
> > enough to give valid results, but if turbostat is consistent enough we
> > could try it.
> 
> The RAPL power meters exported by turbostat can correlate surprisingly
> well with highly accurate external power meters.  But even if perfect,
> RAPL doesn't know about
> the hardware outside of the processor package (except Xeon DRAM), so
> the absolute
> numbers will not match an AC power meter.  But differences are visible
> and consistent.
> The accuracy and the quality of correlation with actual electricals
> varies a lot with
> the type of processor.  In general, Xeon is the best, followed by
> desktop/mobile core,
> and Atom's RAPL power meters have been the least accurate of those
> shipped, so far.
> 
> Yes, 0-day is using this output today to identify regressions, without
> any external power meters.  But they are also adding external power meters.

Yeah we collect turbostat stats in every benchmark it runs and the
machines that support RAPL. It has been effective in catching power
regressions.

There are also 4 external power meters to measure whole-machine power
consumption, however that number is limited comparing to the machines
that support RAPL.

Thanks,
Fengguang