[cgl_discussion] Enhancements to mpstat utility
Eric.Chacron at alcatel.fr
Eric.Chacron at alcatel.fr
Mon Dec 1 05:31:28 PST 2003
David,
>Another item for release 3, from a customer who is moving multi-threaded
APPs to Linux.
>
>They reported using mpstat to help tune the locking in the original
environment, but were
>
>dismayed that there is no Linux equivalent or other tools for doing the
same.
>
>
>
>After looking at the open source mpstat that is available for Red Hat
Linux, it just displays
>
>the minimal level of processor utilization statistics
(processor/%user/%system/%idle/intr).
>
>Other versions for Unix report additional information:
>
>- Minor/Major faults
>
This is already reported by mpstat 0.1.1 ( i have used your mpstat URL ) .
What does it mean ?
>- Interprocessor cross calls
>
>- Context switches
This is already reported my mpstat and by vmstat too.
>
>- System calls
>
>- Process migrations from this processor
>
>- Kernel lock statistics (Mutex stalls)
>
>- Wait time
>
>
I think this is a first set of measurements we'd like to have.
But these are very general ( for instance intr is the sum of every
interrupt events)
and it could be good to have also more detailed
event summary like per event en per processor interrupts
( as you have in /proc/interrupt on a UP machine).
>
>This customer said that the inability to use these tools on Linux was
hindering their ability
>
>to tune their application effectively for Linux.
>
>
>
>My guess is that the current Linux mpstat is exposing everything that is
available from the
>
>kernel, and that to do better will require additional statistics gathering
in the kernel as well
>
>as the hooks in the mpstat utility to display them. This may be hard to
sell to LKML.
>
>
I think that using LTT model for instance ( a hook based tool ) could be
enough to
add features in mpstat or other tool .
>
>While having all of this under on hood is likely best for the customer,
i.e. in mpstat, it would
>
>seem reasonable if the functionality were offered through several tools; I
was thinking of the
>
>lockmeter framework and utilities for the kernel locking statistics.
Having a complete set of
>
>tools for determining bottlenecks would be very useful. Documentation that
describes how
>
>to use the utilities, and how to identify and debug real problems is also
needed.
>
Thanks for the link to lockmeter . I have glanced and i think it's a quite
good tool
for tracing spin lock contention but with a quite significant probe effect
too.
Yes we nned some tool for having knowledge of key mechnisms like the
SMP lock contentions, and others: scheduling , ...
>
>
>For customers who are used to using these tools for tuning on other
Unixes, keeping as
>
>much of the presentation of the stats similar would help, but may be
difficult or impossible
>
>for Linux.
Why do you think it could be impossible ?
Eric
"Howell, David P" <david.p.howell at intel.com>@lists.osdl.org on 10/20/2003
07:19:12 PM
Sent by: cgl_discussion-bounces at lists.osdl.org
To: "Hu, Boris" <boris.hu at intel.com>, "cgl_discussion"
<cgl_discussion at lists.osdl.org>
cc: "Smith, Stanley W" <stanley.w.smith at intel.com>, "Wenjen, Wendell"
<wendell.wenjen at intel.com>
Subject: RE: [cgl_discussion] Enhancements to mpstat utility
This was one of the tools that I thought could be used to make up the full
requirement.
In conjunction with lockmeter we'd have enough there for users to begin
doing the tuning
that the white papers call out using mpstat.
There are some practical considerations, like the performance impact of
having a lockmeter
enabled kernel to deal with. Any of these performance counters will have a
cost.
Thanks,
Dave Howell
-----Original Message-----
From: Hu, Boris
Sent: Saturday, October 18, 2003 5:41 AM
To: Howell, David P; cgl_discussion
Cc: Smith, Stanley W; Wenjen, Wendell
Subject: RE: [cgl_discussion] Enhancements to mpstat utility
In the linux, vmstat could provide some of the lacking information, such as
minor faults, context switches, system calls.
boris
There are my thoughts, not my employer's.
-----Original Message-----
From: cgl_discussion-bounces at lists.osdl.org
[mailto:cgl_discussion-bounces at lists.osdl.org] On Behalf Of Howell, David P
Sent: Friday, October 10, 20033:34 AM
To: cgl_discussion
Cc: Smith, Stanley W; Wenjen, Wendell
Subject: [cgl_discussion] Enhancements to mpstat utility
Another item for release 3, from a customer who is moving multi-threaded
APPs to Linux.
They reported using mpstat to help tune the locking in the original
environment, but were
dismayed that there is no Linux equivalent or other tools for doing the
same.
After looking at the open source mpstat that is available for Red Hat
Linux, it just displays
the minimal level of processor utilization statistics
(processor/%user/%system/%idle/intr).
Other versions for Unix report additional information:
- Minor/Major faults
- Interprocessor cross calls
- Context switches
- System calls
- Process migrations from this processor
- Kernel lock statistics (Mutex stalls)
- Wait time
This customer said that the inability to use these tools on Linux was
hindering their ability
to tune their application effectively for Linux.
My guess is that the current Linux mpstat is exposing everything that is
available from the
kernel, and that to do better will require additional statistics gathering
in the kernel as well
as the hooks in the mpstat utility to display them. This may be hard to
sell to LKML.
While having all of this under on hood is likely best for the customer,
i.e. in mpstat, it would
seem reasonable if the functionality were offered through several tools; I
was thinking of the
lockmeter framework and utilities for the kernel locking statistics. Having
a complete set of
tools for determining bottlenecks would be very useful. Documentation that
describes how
to use the utilities, and how to identify and debug real problems is also
needed.
For customers who are used to using these tools for tuning on other Unixes,
keeping as
much of the presentation of the stats similar would help, but may be
difficult or impossible
for Linux.
Reference projects:
http://freshmeat.net/projects/mpstat/?topic_id=862 - Utility
version used on Linux (?)
LKML, for kernel changes to enable additional statistics
Documentation:
- Using the mpstat load displayer (SCO)
- http://docs.sun.com/db/doc/806-0625/6j9vfilr4?q=mpstat&a=view
-
http://sunsite.uakom.sk/sunworldonline/swol-08-1998/swol-08-perf.html
These are my opinions and not official opinions of Intel Corp.
David Howell
Intel Corporation
Telco Server Development
Server Products Division
Voice: (803) 216-2359 Fax: (803) 216-2178
Intel Corporation
Columbia Design Center, CBA-1
100 Center Point Circle, Suite 210
Columbia, SC 29210
david.p.howell at intel.com
_______________________________________________
cgl_discussion mailing list
cgl_discussion at lists.osdl.org
http://lists.osdl.org/mailman/listinfo/cgl_discussion
More information about the cgl_discussion
mailing list