[lsb-discuss] What do we do for complex tests that don't have standards rationale?

Carlos O'Donell carlos at redhat.com
Wed Sep 11 10:22:47 UTC 2013


Dear LSB,

Red Hat has been working hard to ensure that glibc-related
LSB issues are resolved upstream, waived, or the appropriate
course of action suggested in an LSB bug.

As an upstream glibc, fedora, and rhel glibc maintainer 
I am involved in the low-level efforts to make sure that any
LSB failures as they relate to the core GNU C library have
some kind of resolution e.g. waiver, fix, etc.

Lately we have come up against a complex problem. There is
one remaining test that fails because of a precondition
failure in the model-driven test.

The failing test is filed here:
https://lsbbugs.linuxfoundation.org/show_bug.cgi?id=3788

The single largest problem with this failure is that the
model-driven framework isn't clearly coupled in any way with
a standards requirements, making it difficult for a reviewer
to easily determine what part of the standard is violated.

In the abstract the test should be waived, it's 
not correct to check for implementation-defined behaviour
(like the setting of IEXTEN) unless you have a documented
behaviour for the implementation, which you don't.

Practically speaking though the test is allowed to expect
IEXTEN to be set for local terminals on Linux and that the
behaviour shall be similar across all targets (the generic
tty layer doesn't change).

The `{app.read.10}' precondition failure for read_tty_spec, 
has no documented standards-conforming reason for the test 
to expect this result or value. The test code in 
src/model/io/term/term_model.sec explains what IEXTEN is, 
but fails to explain why it's expected to be on. The code
can be called from several other functions which also
don't document what state they are in and if it's valid
for them to expect IEXTEN on Linux.

On Linux IEXTEN is always set for local mode in terminals
created by the OS. The terminal is always ready to process
extended characters locally.

POSIX says:
~~~
The initial local control value after open() is 
implementation-defined.
~~~

The GNU C library acting as the intermediary between Linux
and userspace for terminals is mostly implementing POSIX.

To support that glibc/linux are working correctly I have 
created several tests outside of the olver framework
and they all work exactly as expected with regards to
IEXTEN on x86-64, s390x, and ppc64.

I'm including a tcgetattr IEXTEN test that shows ppc64
behaves as expected by the standard (via glibc):

./test-tcgetattr 
PASS: openpty passed and returned master = 3, slave = 4, name = /dev/pts/1
PASS: IEXTEN set for local mode.
PASS: IEXTEN set for local mode.
PASS: IEXTEN unset for local mode.
PASS: IEXTEN unset for local mode after cfmakeraw emulation.

It just isn't reasonable that for these model-driven tests,
after each failure, the reviewer has to re-work their way through
the entire test proving each step leads to the next step which
leads to the next all within what is expected. Such reasoning
should have been built into the model, either as comments or
formal specification and it's not there.

Debugging io_term_rw_canon_scenario has been very very
difficult, and even now I don't know which part of the
model sets the initial erroneous c_lflag value, but it's
not glibc, and it's not the kernel (I can show that via
strace and gdb).

Has anyone debugged one of these model-driven tests?

Comments?

Cheers,
Carlos.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build.sh
Type: application/x-shellscript
Size: 81 bytes
Desc: not available
URL: <http://lists.linuxfoundation.org/pipermail/lsb-discuss/attachments/20130911/71357d5d/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test-tcgetattr.c
Type: text/x-csrc
Size: 3023 bytes
Desc: not available
URL: <http://lists.linuxfoundation.org/pipermail/lsb-discuss/attachments/20130911/71357d5d/attachment-0001.bin>


More information about the lsb-discuss mailing list