[cgl_discussion] [Fwd: [ANNOUNCE] Native POSIX Thread Library0.1]

george anzinger george at mvista.com
Fri Sep 20 16:48:42 PDT 2002


"Howell, David P" wrote:
> 
> Actually, we have that in the CLONE_THREAD functionality that is in the
> 2.4 kernels now, i.e. proper signal delivery. This should not be an issue.

I think what is there now is a kluge that sends the signals
to a signal handler thread which is supposed to figure out
where it should really go.  The means the signal gets
delivered twice :(.  The kernel is really quite different in
this area in 2.5.
> 
> I would contend that binary compatibility with what exists will be the
> biggest issue. Think of all of those existing APPs to qualify ...
> 
> Dave Howell
> 
> ** This is my opinion, I do not speak for Intel Corporation **
> 
> -----Original Message-----
> From: george anzinger [mailto:george at mvista.com]
> Sent: Friday, September 20, 2002 6:00 PM
> To: Brian Stevens
> Cc: Craig Thomas; Khalid Aziz; cgl discussion
> Subject: Re: [cgl_discussion] [Fwd: [ANNOUNCE] Native POSIX Thread
> Library 0.1]
> 
> Brian Stevens wrote:
> >
> > It is reasonable to expect it will be backported to 2.4.x soon.
> 
> On first blush that my seem to be true, but me thinks they
> use far too many 2.5 new features to make a back port
> reasonable.
> 
> After all, the big threads issue was the signal system in
> the kernel.  It has been rewritten for 2.5 to make it POSIX
> compliant.
> 
> -g
> >
> > Brian
> >
> > Craig Thomas wrote:
> >
> > >It says here that the build environment (to build the threaded
> > >library) must be 2.5, so I am assuming that it is implied that the
> > >library will be released on 2.5.  This looks like something to consider,
> > >but will there be a need to have this back ported to 2.4.18?
> > >
> > >On Fri, 2002-09-20 at 09:25, Khalid Aziz wrote:
> > >
> > >>Here is another implementation of POSIX threads. Might be interesting
> > >>for POSIX threads project to look at. People have already started asking
> > >>for comparison with NGPT on lkml.
> > >>
> > >>--
> > >>Khalid
> > >>
> > >>Ulrich Drepper wrote:
> > >>
> > >>>-----BEGIN PGP SIGNED MESSAGE-----
> > >>>Hash: SHA1
> > >>>
> > >>>We are pleased to announce the first publically available source
> > >>>release of a new POSIX thread library for Linux.  As part of the
> > >>>continuous effort to improve Linux's capabilities as a client, server,
> > >>>and computing platform Red Hat sponsored the development of this
> > >>>completely new implementation of a POSIX thread library, called Native
> > >>>POSIX Thread Library, NPTL.
> > >>>
> > >>>Unless major flaws in the design are found this code is intended to
> > >>>become the standard POSIX thread library on Linux system and it will
> > >>>be included in the GNU C library distribution.
> > >>>
> > >>>The work visible here is the result of close collaboration of kernel
> > >>>and runtime developers.  The collaboration proceeded by developing the
> > >>>kernel changes while writing the appropriate parts of the thread
> > >>>library.  Whenever something couldn't be implemented optimally some
> > >>>interface was changed to eliminate the issue.  The result is this
> > >>>thread library which is, unlike previous attempts, a very thin layer
> > >>>on top of the kernel.  This helps to achieve a maximum of performance
> > >>>for a minimal price.
> > >>>
> > >>>A white paper (still in its draft stage, though) describing the design
> > >>>is available at
> > >>>
> > >>>   http://people.redhat.com/drepper/nptl-design.pdf
> > >>>
> > >>>It provides a larger number of details on the design and insight into
> > >>>the design process.  At this point we want to repeat only a few
> > >>>important points:
> > >>>
> > >>>- - the new library is based on an 1-on-1 model.  Earlier design
> > >>>   documents stated that an M-on-N implementation was necessary to
> > >>>   support a scalable thread library.  This was especially true for
> > >>>   the IA-32 and x86-64 platforms since the ABI with respect to threads
> > >>>   forces the use of segment registers and the only way to use those
> > >>>   registers was with the Local Descriptor Table (LDT) data structure
> > >>>   of the processor.
> > >>>
> > >>>   The kernel limitations the earlier designs were based on have been
> > >>>   eliminated as part of this project, opening the road to a 1-on-1
> > >>>   implementation which has many advantages such as
> > >>>
> > >>>   + less complex implementation;
> > >>>   + avoidance of two-level scheduling, enabling the kernel to make all
> > >>>     scheduling decisions;
> > >>>   + direct interaction between kernel and user-level code (e.g., when
> > >>>     delivering signals);
> > >>>   + and more and more.
> > >>>
> > >>>   It is not generally accepted that a 1-on-1 model is superior but our
> > >>>   tests showed the viability of this approach and by comparing it with
> > >>>   the overhead added by existing M-on-N implementations we became
> > >>>   convinced that 1-on-1 is the right approach.
> > >>>
> > >>>   Initial confirmations were test runs with huge numbers of threads.
> > >>>   Even on IA-32 with its limited address space and memory handling
> > >>>   running 100,000 concurrent threads was no problem at all, creating
> > >>>   and destroying the threads did not take more than two seconds.  This
> > >>>   all was made possible by the kernel work performed as part of this
> > >>>   project.
> > >>>
> > >>>   The only limiting factors on the number of threads today are
> > >>>   resource availability (RAM and processor resources) and architecture
> > >>>   limitations.  Since every thread needs at least a stack and data
> > >>>   structures describing the thread the number is capped.  On 64-bit
> > >>>   machines the architecture does not add any limitations anymore (at
> > >>>   least for the moment) and with enough resources the number of
> > >>>   threads can be grown arbitrarily.
> > >>>
> > >>>   This does not mean that using hundreds of thousands of threads is a
> > >>>   desirable design for the majority of applications.  At least not
> > >>>   unless the number of processors matches the number of threads.  But
> > >>>   it is important to note that the design on the library does not have
> > >>>   a fixed limit.
> > >>>
> > >>>   The kernel work to optimize for a high thread count is still
> > >>>   ongoing.  Some places in which the kernel iterates over process and
> > >>>   threads remain and other places need to be cleaned up.  But it has
> > >>>   already been shown that given sufficient resources and a reasonable
> > >>>   architecture an order of magnitude more threads can be created than
> > >>>   in our tests on IA-32.
> > >>>
> > >>>- - The futex system call is used extensively in all synchronization
> > >>>   primitives and other places which need some kind of
> > >>>   synchronization.  The futex mechanism is generic enough to support
> > >>>   the standard POSIX synchronization mechanisms with very little
> > >>>   effort.
> > >>>
> > >>>   The fact that this is possible is also essential for the selection
> > >>>   of the 1-on-1 model since only with the kernel seeing all the
> > >>>   waiters and knowing that they are blocked for synchronization
> > >>>   purposes will allow the scheduler to make decisions as good as a
> > >>>   thread library would be able to in an M-on-N model implementation.
> > >>>
> > >>>   Futexes also allow the implementation of inter-process
> > >>>   synchronization primitives, a sorely missed feature in the old
> > >>>   LinuxThreads implementation (Hi jbj!).
> > >>>
> > >>>- - Substantial effort went into making the thread creation and
> > >>>   destruction as fast as possible.  Extensions to the clone(2) system
> > >>>   call were introduced to eliminate the need for a helper thread in
> > >>>   either creation or destruction.  The exit process in the kernel was
> > >>>   optimized (previously not a high priority).  The library itself
> > >>>   optimizes the memory allocation so that in many cases the creation
> > >>>   of a new thread can be achieved with one single system call.
> > >>>
> > >>>   On an old IA-32 dual 450MHz PII Xeon system 100,000 threads can be
> > >>>   created and destroyed in 2.3 secs (with up to 50 threads running at
> > >>>   any one time).
> > >>>
> > >>>- - Programs indirectly linked against the thread library had problems
> > >>>   with the old implementation because of the way symbols are looked
> > >>>   up. This should not be a problem anymore.
> > >>>
> > >>>The thread library is designed to be binary compatible with the old
> > >>>LinuxThreads implementation.  This compatibility obviously has some
> > >>>limitations.  In places where the LinuxThreads implementation diverged
> > >>>from the POSIX standard incompatibilities exist.  Users of the old
> > >>>library have been warned from day one that this day will come and code
> > >>>which added work-arounds for the POSIX non-compliance better be
> > >>>prepared to remove that code.  The visible changes of the library
> > >>>include:
> > >>>
> > >>>- - The signal handling changes from per-thread signal handling to the
> > >>>   POSIX process signal handling.  This change will require changes in
> > >>>   programs which exploit the non-conformance of the old
> implementation.
> > >>>
> > >>>   One consequence of this is that SIGSTOP works on the process.  Job
> > >>>control
> > >>>   in the shell and stopping the whole process in a debugger work now.
> > >>>
> > >>>- - getpid() now returns the same value in all threads
> > >>>
> > >>>- - the exec functions are implemented correctly: the exec'ed process
> gets
> > >>>   the PID of the process.  The parent of the multi-threaded
> application
> > >>>   is only notified when the exec'ed process terminates.
> > >>>
> > >>>- - thread handlers registered with pthread_atfork are not anymore run
> > >>>   if vfork is used.  This isn't required by the standard (which does
> > >>>   not define vfork) and all which is allowed in the child is calling
> > >>>   exit() or an exec function.  A user of vfork better knows what s/he
> > >>>   does.
> > >>>
> > >>>- - libpthread should now be much more resistant to linking problems:
> even
> > >>>   if the application doesn't list libpthread as a direct dependency
> > >>>   functions which are extended by libpthread should work correctly.
> > >>>
> > >>>- - no manager thread
> > >>>
> > >>>- - inter-process mutex, read-write lock, conditional variable, and
> barrier
> > >>>   implementations are available
> > >>>
> > >>>- - the pthread_kill_other_threads_np function is not available.  It
> was
> > >>>   needed to work around the broken signal handling.  If somebody shows
> > >>>   some existing code which makes legitimate use of this function we
> > >>>   might add it back.
> > >>>
> > >>>- - requires a kernel with the threading capabilities of Linux 2.5.36.
> > >>>
> > >>>The sources for the new library are for the time being available at
> > >>>
> > >>>   ftp://people.redhat.com/drepper/nptl/
> > >>>
> > >>>The current sources contain support only for IA-32 but this will
> > >>>change very quickly.  The thread library is built as part of glibc so
> > >>>the complete set of glibc sources is available as well.  The current
> > >>>snapshot for glibc 2.3 (or glibc 2.3 when released) is necessary.  You
> > >>>can find it at
> > >>>
> > >>>   ftp://sources.redhat.com/pub/glibc/snapshots
> > >>>
> > >>>Final releases will be available on ftp.gnu.org and its mirrors.
> > >>>
> > >>>Building glibc with the new thread library is demanding on the
> > >>>compilation environment.
> > >>>
> > >>>- - The 2.5.36 kernel or above must be installed and used.  To compile
> > >>>   glibc it is necessary to create the symbolic link
> > >>>
> > >>>      /lib/modules/$(uname -r)/build
> > >>>
> > >>>   to point to the build directory.
> > >>>
> > >>>- - The general compiler requirement for glibc is at least gcc 3.2.
> For
> > >>>   the new thread code it is even necessary to have working support for
> > >>>   the __thread keyword.
> > >>>
> > >>>   Similarly, binutils with functioning TLS support are needed.
> > >>>
> > >>>   The (Null) beta release of the upcoming Red Hat Linux product is
> > >>>   known to have the necessary tools available after updating from the
> > >>>   latest binaries on the FTP site.  This is no ploy to force everybody
> > >>>   to use Red Hat Linux, it's just the only environment known to date
> > >>>   which works.  If alternatives are known they can be announced on the
> > >>>   mailing list.
> > >>>
> > >>>- - To configure glibc it is necessary to run in the build directory
> > >>>   (which always should be separate from the source directory):
> > >>>
> > >>>    /path/to/glibc/configure --prefix=/usr
> --enable-add-ons=linuxthreads2 \
> > >>>       --enable-kernel=current --with-tls
> > >>>
> > >>>   The --enable-kernel parameter requires that the 2.5.36+ kernel is
> > >>>   running.  It is not strictly necessary but helps to avoid mistakes.
> > >>>   It might also be a good idea to add --disable-profile, just to speed
> > >>>   up the compilation.
> > >>>
> > >>>   When configured as above the library must not be installed since it
> > >>>   would overwrite the system's library.  If you want to install the
> > >>>   resulting library choose a different --prefix parameter value.
> > >>>   Otherwise the new code can be used without installation.  Running
> > >>>   existing binaries is possible with
> > >>>
> > >>>    elf/ld.so --library-path .:linuxthreads2:dlfcn:math <binary>
> <args>...
> > >>>
> > >>>   Alternatively the binary could be build to find the dynamic linker
> > >>>   and DSO by itself.  This is a much easier way to debug the code
> > >>>   since gdb can start the binary.  Compiling is a bit more complicated
> > >>>   in this case:
> > >>>
> > >>>    gcc -nostdlib -nostartfiles -o <OUTPUT> csu/crt1.o csu/crti.o \
> > >>>      $(gcc --print-file-name=crtbegin.o) <INPUTS> \
> > >>>      -Wl,-rpath,$PWD,-dynamic-linker,$PWD/ld-linux.so.2 \
> > >>>      linuxthreads2/libpthread.so.0 ./libc.so.6 ./libc_nonshared.a \
> > >>>      elf/ld-linux.so.2 $(gcc --print-file-name=crtend.o) csu/crtn.o
> > >>>
> > >>>   This command assumes that it is run in the build directory.  Correct
> > >>>   the paths if necessary.  The compilation will use the system's
> > >>>   headers which is a good test but might lead to strange effects if
> > >>>   there are compatibility bugs left.
> > >>>
> > >>>Once all these prerequisites are met compiling glibc should be easy.
> > >>>But there are some tests which will flunk.  For good reasons we aren't
> > >>>officially releasing the code yet.  The bugs are either in the TLS
> > >>>code which is not enabled in the standard glibc build, or obviously in
> > >>>the thread library itself.  To run the tests for the thread library
> > >>>run
> > >>>
> > >>>   make subdirs=linuxthreads2 check
> > >>>
> > >>>One word on the name 'linuxthreads2' of the directory.  This is only a
> > >>>convenience thing so that the glibc configure scripts don't complain
> > >>>about missing thread support.  It will we changed to reflect the real
> > >>>name of the library ASAP.
> > >>>
> > >>>What can you expect?
> > >>>
> > >>>This is a very early version of the code so the obvious answer is:
> > >>>some problems.  The test suite for the new thread code should pass but
> > >>>beside that and some performance measurement tool we haven't run much
> > >>>code.  Ideally we would get people to write many more of these small
> > >>>test programs which are included in the sources.  Compiling big
> > >>>programs would mean not being able to locate problems easy.  But I
> > >>>certainly won't object to people running and debugging bigger
> > >>>applications.  Please report successes and failures to the mailing
> > >>>list.
> > >>>
> > >>>People who are interested in contributing must be aware that for any
> > >>>non-trivial change we need an assignment of the code to the FSF.  The
> > >>>process is unfortunately necessary in today's world.
> > >>>
> > >>>People who are contaminated by having worked on proprietary thread
> > >>>library implementation should not participate in discussions on the
> > >>>mailing list unless they willfully disclose the information.  Every
> > >>>bit of information is publically available from the mailing list
> > >>>archive.
> > >>>
> > >>>Which brings us to the final point: the mailing list for *all*
> > >>>discussions related to this thread library implementation is
> > >>>
> > >>>   phil-list at redhat.com
> > >>>
> > >>>Go to
> > >>>
> > >>>   https://listman.redhat.com/mailman/listinfo/phil-list
> > >>>
> > >>>to subscribe, unsubscribe, or review the archive.
> > >>>
> > >>>- --
> > >>>- ---------------.                          ,-.   1325 Chesapeake
> Terrace
> > >>>Ulrich Drepper  \    ,-------------------'   \  Sunnyvale, CA 94089 USA
> > >>>Red Hat          `--' drepper at redhat.com   `------------------------
> > >>>-----BEGIN PGP SIGNATURE-----
> > >>>Version: GnuPG v1.0.7 (GNU/Linux)
> > >>>Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> > >>>
> > >>>iD8DBQE9im7E2ijCOnn/RHQRApe9AKCN20A8A5ITi3DUq+3IRZ0gsSVHTQCeKqEu
> > >>>fA5OFtNuzYqltxSMoL8Ambw=
> > >>>=4pb4
> > >>>-----END PGP SIGNATURE-----
> > >>>
> > >>>-
> > >>>To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> in
> > >>>the body of a message to majordomo at vger.kernel.org
> > >>>More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > >>>Please read the FAQ at  http://www.tux.org/lkml/
> > >>>
> > >>====================================================================
> > >>Khalid Aziz                                   Linux Systems Division
> > >>(970)898-9214                                        Hewlett-Packard
> > >>khalid at fc.hp.com                                    Fort Collins, CO
> > >>
> > >>"The Linux kernel is subject to relentless development"
> > >>                              - Alessandro Rubini
> > >>_______________________________________________
> > >>cgl_discussion mailing list
> > >>cgl_discussion at lists.osdl.org
> > >>http://lists.osdl.org/mailman/listinfo/cgl_discussion
> > >>
> >
> > _______________________________________________
> > cgl_discussion mailing list
> > cgl_discussion at lists.osdl.org
> > http://lists.osdl.org/mailman/listinfo/cgl_discussion
> 
> --
> George Anzinger   george at mvista.com
> High-res-timers:
> http://sourceforge.net/projects/high-res-timers/
> Preemption patch:
> http://www.kernel.org/pub/linux/kernel/people/rml
> _______________________________________________
> cgl_discussion mailing list
> cgl_discussion at lists.osdl.org
> http://lists.osdl.org/mailman/listinfo/cgl_discussion

-- 
George Anzinger   george at mvista.com
High-res-timers: 
http://sourceforge.net/projects/high-res-timers/
Preemption patch:
http://www.kernel.org/pub/linux/kernel/people/rml



More information about the cgl_discussion mailing list