[Fuego] [PATCH 07/12] skip cases because 'netstat -s' not support

Bird, Tim Tim.Bird at sony.com
Thu Oct 1 22:35:19 UTC 2020



> -----Original Message-----
> From: daniel.sangorrin at toshiba.co.jp > 
> Hi Tim, Qiu
> 
> I am guessing this work is being done for the old 2018 LTP tarball. However, I think that LTP has changed quite a few things since then (I
> need to catchup on that). I think it is a good time to update the test to use the latest LTP and the new features that were added, what do
> you think?

I'm OK with this.  We could do this:

diff --git a/tests/Functional.LTP/spec.json b/tests/Functional.LTP/spec.json
index 7c409ff..388097c 100644
--- a/tests/Functional.LTP/spec.json
+++ b/tests/Functional.LTP/spec.json
@@ -75,6 +75,12 @@
             "tests": "cve",
             "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
             "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}
+        },
+        "latest": {
+            "gitrepo": "https://github.com/linux-test-project/ltp.git",
+            "tests": "syscalls",
+            "extra_success_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"},
+            "extra_fail_links": {"xlsx": "results.xlsx", "skiplist": "skiplist.txt"}
         }
-    }
+   }
 }

But that would only do one test from the latest (and it wouldn't be tied to a release of LTP).

Having been lurking on the LTP list for some time now, I have a couple of opinions:
 1. LTP top of tree moves and has bugs during development, so I would definitely want to
   specify a specific release commit, or use a snapshot release tarball
    - my preference is a snapshot tarball.  In general, I like avoiding network dependencies on
       external projects at test runtime, if possible.
 2. LTP has gotten a lot better over time, from what I can see.  I was initially hesitant to move forward
    to a new version, but I think that hesitation was misfounded.  I think Fuego is missing out on
    test improvements by not moving forward as LTP is improved.  It will cause churn in the testcase
    list produced between different versions of LTP, but I think that's a lesser problem than I expected.
    No one appears to be comparing LTP results to each other anyway.

So - sounds good to me!

Note that LTP just had a release this week, and I put a few new patches into Fuego related to that
release.  Now might be a good time to add a more current test option for LTP, and make it default.

> > -----Original Message-----
> > From: Fuego <fuego-bounces at lists.linuxfoundation.org> On Behalf Of Qiu, Tingting
> > Sent: Friday, August 7, 2020 10:42 AM
> > To: Bird, Tim <Tim.Bird at sony.com>
> > Cc: fuego at lists.linuxfoundation.org
> > Subject: Re: [Fuego] [PATCH 07/12] skip cases because 'netstat -s' not support
> >
> > Tim
> >
> > Thanks for your suggestion.
> > Using a board-specific LTP skiplist is a good idear.
> >
> > Thanks,
> >   -- QiuTingting
> >
> > -----Original Message-----
> > From: Bird, Tim [mailto:Tim.Bird at sony.com]
> > Sent: Friday, August 7, 2020 7:28 AM
> > To: Qiu, Tingting/仇 婷婷 <qiutt at cn.fujitsu.com>
> > Cc: fuego at lists.linuxfoundation.org
> > Subject: RE: [Fuego] [PATCH 07/12] skip cases because 'netstat -s' not support
> >
> >
> >
> > > -----Original Message-----
> > > From: Qiu, Tingting <qiutt at cn.fujitsu.com>
> > >
> > > -----Original Message-----
> > > From: Bird, Tim [mailto:Tim.Bird at sony.com]
> > > Sent: Thursday, July 9, 2020 6:20 AM
> > > To: Qiu, Tingting/仇 婷婷 <qiutt at cn.fujitsu.com>;
> > > fuego at lists.linuxfoundation.org
> > > Subject: RE: [Fuego] [PATCH 07/12] skip cases because 'netstat -s' not
> > > support
> > >
> > > > -----Original Message-----
> > > > From:  qiutt
> > > >
> > > > for aarch64(m3ulcb) skip cases in net.tcp_cmds
> > > >
> > > > Signed-off-by: qiutt <qiutt at cn.fujitsu.com>
> > > > ---
> > > >  tests/Functional.LTP/fuego_test.sh | 4 ++++
> > > >  1 file changed, 4 insertions(+)
> > > >
> > > > diff --git a/tests/Functional.LTP/fuego_test.sh
> > > > b/tests/Functional.LTP/fuego_test.sh
> > > > index ab4165d..2254474 100755
> > > > --- a/tests/Functional.LTP/fuego_test.sh
> > > > +++ b/tests/Functional.LTP/fuego_test.sh
> > > > @@ -352,6 +352,10 @@ function test_pre_check {
> > > >      if [ "$ARCHITECTURE" != "i386" ]; then
> > > >          skip_tests "modify_ldt01 modify_ldt02 modify_ldt03" # Only work on i386
> > > >      fi
> > > > +
> > > > +    if [ "$ARCHITECTURE" == "aarch64" ]; then
> > > > +        skip_tests "netstat" # for net.tcp_cmds, 'netstat -s' not support
> > > > +    fi
> > >
> > > This isn't right.  It may be true that netstat -s is not supported on m3ulcb, but it is supported just fine on other aarch64 platforms.
> > >
> > > What is the issue with supporting 'netstat -s' on m3ulcb?
> > > ->
> > > The patch was made for m3ulcb, and m3ulcb used the AGL system.
> > > The "netstat" command used was provided by "busybox"(*1).
> > > The error of "netstat -s" refer to (*2).
> > > Check the code(aarch64-agl-linux/busybox/1.29.3-r0/busybox-1.29.3/networking/netstat.c), the contents refer to (*3).
> > > According to the code, you can know that the netstat provided by
> > > busybox only supports the parameter "-laentuwxrWp", and does not support the parameter "-s".
> > >
> > > (*1)m3ulcb:~# which netstat
> > > /bin/netstat
> > > m3ulcb:~# ls -al /bin/netstat
> > > lrwxrwxrwx 1 root root 19 Feb 25  2020 /bin/netstat ->
> > > /bin/busybox.nosuid
> > >
> > > (*2)m3ulcb:~# netstat -s
> > > netstat: invalid option -- 's'
> > > BusyBox v1.29.3 (2020-01-16 16:59:41 UTC) multi-call binary.
> > > Usage: netstat [-ral] [-tuwx] [-en]
> > >
> > > (*3)
> > > //usage:#define netstat_trivial_usage
> > > //usage:       "[-"IF_ROUTE("r")"al] [-tuwx] [-en"IF_FEATURE_NETSTAT_WIDE("W")IF_FEATURE_NETSTAT_PRG("p")"]"
> > > //usage:#define netstat_full_usage "\n\n"
> > > //usage:       "Display networking information\n"
> > > //usage:        IF_ROUTE(
> > > //usage:     "\n        -r      Routing table"
> > > //usage:        )
> > > //usage:     "\n        -a      All sockets"
> > > //usage:     "\n        -l      Listening sockets"
> > > //usage:     "\n                Else: connected sockets"
> > > //usage:     "\n        -t      TCP sockets"
> > > //usage:     "\n        -u      UDP sockets"
> > > //usage:     "\n        -w      Raw sockets"
> > > //usage:     "\n        -x      Unix sockets"
> > > //usage:     "\n                Else: all socket types"
> > > //usage:     "\n        -e      Other/more information"
> > > //usage:     "\n        -n      Don't resolve names"
> > > //usage:        IF_FEATURE_NETSTAT_WIDE(
> > > //usage:     "\n        -W      Wide display"
> > > //usage:        )
> > > //usage:        IF_FEATURE_NETSTAT_PRG(
> > > //usage:     "\n        -p      Show PID/program name for sockets"
> > > //usage:        )
> > >
> > > #define NETSTAT_OPTS "laentuwx" \
> > >         IF_ROUTE(               "r") \
> > >         IF_FEATURE_NETSTAT_WIDE("W") \
> > >         IF_FEATURE_NETSTAT_PRG( "p")
> > > ...
> > >
> >
> > I see.  I was able to reproduce a problem running the netstat01 sub-test of 'net.tcp_cmds'
> > test in LTP, on a machine that only had busybox netstat.
> >
> > This should not be dependent on the machine architecture, which is unrelated, but rather on whether the machine supports 'netstat -s'.
> > But that's actually one of the functions of the netstat01 test in LTP (to test 'netstat -s' support).  In this case, the test is reporting
> > correctly that the m3ulcb board doesn't support that option.  So the test is correct.
> > Other Fuego users of the test may wish to see that problem reported.  It is not appropriate to mask out this correct bug report by
> > adding to a skiplist that applies to everyone.
> >
> > I'm not sure exactly what you are trying to accomplish (maybe hiding this bug so that it doesn't show up for this board, because your
> > organization has decided to ignore it?).  However, if you want to skip the test, then I would recommend adding this to a board-specific
> > LTP skiplist.
> >
> > In your m3ulcb.board file, you should add:
> > FUNCTIONAL_LTP_BOARD_SKIPLIST="netstat01"
> >
> > Let me know if this does not solve your problem.
> >  -- Tim
> >
> >
> >
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego


More information about the Fuego mailing list