[Netem] About reordering with netem: and how to get jitter WITHOUT reordering...

Jonathan P. Crone jonathan.crone at magorcorp.com
Wed Apr 6 13:01:41 PDT 2011


Brian Lu's recent question about using Netem to clean up reordering and
Stephen's response netem being designed to make reordering worse
for testing purposes  has prompted me to ask a question
I have been meaning to ask for quite some time. 

Specifically: 
The netem page on the linuxfoundation website
 SEEMS to have an error in its description of using a pfifo
of how  to get jitter without reordering. 
Testing on 3 different kernels,  I get a varying behaviour
(mostly errors)  using the example from the linuxfoundation website. 

Details follow: 

I work in a telecollaboration development company and use netem
for creation of  various impaired networks for video streaming testing.

One area in which netem does not meet my needs is the introduction of jitter. 

The specific issue is reported as a virtue on the authoritative 
netem page at the linux foundation with the following suggestion:
(which is technically a very pathological example, in my experience
 only a cellular data  type network would have THIS MUCH jitter...  10ms  plus/minus 100ms!!! ) 
quoting the website: 
Starting with version 1.1 (in 2.6.15), netem will reorder packets if the delay value has lots of jitter.

If you don't want this behaviour then replace the internal queue discipline tfifo with a pure packet fifo pfifo. The following example has lots of jitter, but the packets will stay in order.

 # tc qdisc add dev eth0 root handle 1: netem delay 10ms 100ms
 # tc qdisc add dev eth0 parent 1:1 pfifo limit 1000

The problem is:  I DO NOT want 'reorder'   i just want the jitter.
(IE:  I want  the interval between packet 100 and packet 101 to be 100ms, 
the interval between packet 101 and 102 to be 92 ms,   between
packet 102 and 103  to be 105 ms    etc.) 
(IF I want packet  102 to arrive before packet 101,  I want to explicitly
 control that reorder behaviour independently of jitter. ) 
(a bonded T1 might typically have out of order packets,
 but a very stable delay distribution ) 

So an example of how I would do this: 

This should put a static 100ms delay, with a random jitter  Plus/Minus 10ms . 

tc qdisc  add dev eth0 root handle 1: netem delay 100ms  10ms  

This causes jitter AND reordering... 
The netem page says,  add a pfifo to eliminate the reorder: 

Here's the problem: 
On my 9.10   ubuntu boxes  running the 2.6.31-14-generic  kernel:
if I use the example of enabling the pfifo as a child of the netem parent, 
I get an error message: 
RTNETLINK answers: Invalid argument

When I attempt to enable the pfifo... 

If I try on my ubuntu 10.04 boxes running the 2.6.32-28-generic kernel,
and I try to enable the pfifo,  I get error message:
RTNETLINK answers: Operation not supported

If I use an ubuntu  8.04  box  running a  2.6.24-27  kernel, 
I don't get an RTNETLINK error,  
I get jitter,  and I don't get the reordering 
(My desired behaviour )
Problem is:   the 8.04 boxes are my "application" boxes... 

So.... clearly its  tied to kernel behaviour....  but I apologize
for being at a bit of a loss as to why the 'authoritative' netem
reference  would provide a solution which doesn't seem to work 
on some of the kernels currently out there. 

(for my lab's needs:   the 9.10  boxes are the "default"  netem boxes,
  and the 8.04  and 10.04 boxes are running the application endpoints. ) 
Having the jitter work on the 8.04 based systems is not optimal,
 as the 9.10 boxes are the 'stable'   network emulator boxes,  representing
the pipe   that the 8.04 and 10.04 talk over. 


Suggestions, thoughts, recommendations would be appreciated. 



Jonathan Crone P.Eng
Verification Engineering
Magor Communications
jonathan.crone at magorcorp.com


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.linux-foundation.org/pipermail/netem/attachments/20110406/eaa70cd5/attachment.htm 


More information about the Netem mailing list