User Tools

Site Tools


networking:netem

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
networking:netem [2018/09/14 06:30]
mlouielu
networking:netem [2023/03/08 18:50] (current)
shemminger
Line 1: Line 1:
 ======netem====== ======netem======
  
-**netem** provides [[http://​en.wikipedia.com/​wiki/​Network_emulation|Network Emulation]] functionality for testing protocols by emulating the properties of wide area networks. The current version emulates variable delay, loss, duplication and re-ordering. +**netem** provides [[https://​en.wikipedia.com/​wiki/​Network_emulation|Network Emulation]] functionality for testing protocols by emulating the properties of wide area networks. ​
- +
-If you run a current 2.6 distribution,​ ([[http://​en.wikipedia.com/​wiki/​Fedora_Core|Fedora]],​ [[http://​en.wikipedia.com/​wiki/​Open_Suse|OpenSuse]],​ [[http://​en.wikipedia.com/​wiki/​Gentoo_Linux|Gentoo]],​ [[http://​en.wikipedia.com/​wiki/​Debian|Debian]],​ [[http://​en.wikipedia.com/​wiki/​Mandriva|Mandriva]],​ [[http://​en.wikipedia.com/​wiki/​Ubuntu_%28Linux_distribution%29|Ubuntu]]),​ then netem is already enabled in the kernel and a current version of [[:​networking:​iproute2|iproute2]] is included. The netem kernel component is enabled under: +
-<​code>​ Networking --> +
-   ​Networking Options --> +
-     QoS and/or fair queuing --> +
-        Network emulator</​code>​ +
- +
-Netem is controlled by the command line tool '​tc'​ which is part of the [[:​networking:​iproute2|iproute2]] package of tools. The tc command uses shared libraries and data files in the /usr/lib/tc directory. +
-=====Contents===== +
- +
-  * [[#​Examples|1 Examples]] +
-    * [[https://​www.linuxfoundation.org/#​Emulating_wide_area_network_delays|1.1 Emulating wide area network delays]] +
-    * [[https://​www.linuxfoundation.org/#​Delay_distribution|1.2 Delay distribution]] +
-    * [[https://​www.linuxfoundation.org/#​Packet_loss|1.3 Packet loss]] +
-      * [[https://​www.linuxfoundation.org/#​Caveats|1.3.1 Caveats]] +
-    * [[https://​www.linuxfoundation.org/#​Packet_duplication|1.4 Packet duplication]] +
-    * [[https://​www.linuxfoundation.org/#​Packet_corruption|1.5 Packet corruption]] +
-    * [[https://​www.linuxfoundation.org/#​Packet_re-ordering|1.6 Packet re-ordering]] +
-      * [[https://​www.linuxfoundation.org/#​Caveats_2|1.6.1 Caveats]] +
-    * [[https://​www.linuxfoundation.org/#​Rate_control|1.7 Rate control]] +
-    * [[https://​www.linuxfoundation.org/#​Non_FIFO_queuing|1.8 Non FIFO queuing]] +
-    * [[https://​www.linuxfoundation.org/#​Delaying_only_some_traffic|1.9 Delaying only some traffic]] +
-  * [[https://​www.linuxfoundation.org/#​FAQ|2 FAQ]] +
-    * [[https://​www.linuxfoundation.org/#​How_come_first_ping_takes_longer.3F|2.1 How come first ping takes longer?]] +
-    * [[https://​www.linuxfoundation.org/#​How_come_TCP_is_so_slow_over_netem.3F|2.2 How come TCP is so slow over netem?]] +
-    * [[https://​www.linuxfoundation.org/#​How_can_I_use_netem_on_incoming_traffic.3F|2.3 How can I use netem on incoming traffic?​]] +
-    * [[https://​www.linuxfoundation.org/#​How_to_reorder_packets_based_on_jitter.3F|2.4 How to reorder packets based on jitter?]] +
-    * [[https://​www.linuxfoundation.org/#​How_does_the_value_of_HZ_impact_Netem.3F|2.5 How does the value of HZ impact Netem?]] +
-  * [[https://​www.linuxfoundation.org/#​Links|3 Links]] +
-  * [[https://​www.linuxfoundation.org/#​Contact_Info|4 Contact Info]] +
- +
-=====Examples===== +
-====Emulating wide area network delays==== +
- +
-  +
- +
-This is the simplest example, it just adds a fixed amount of delay to all packets going out of the local Ethernet. +
-<​code>​ # tc qdisc add dev eth0 root netem delay 100ms</​code>​ +
- +
-Now a simple ping test to host on the local network should show an increase of 100 milliseconds. The delay is limited by the clock resolution of the kernel (HZ). On most 2.4 systems, the system clock runs at 100hz which allows delays in increments of 10ms. On 2.6, the value is a configuration parameter from 1000 to 100 hz. +
- +
-Later examples just change parameters without reloading the qdisc +
- +
-Real wide area networks show variability so it is possible to add random variation. +
-<​code>​ # tc qdisc change dev eth0 root netem delay 100ms 10ms</​code>​ +
- +
-This causes the added delay to be 100ms ± 10ms. Network delay variation isn't purely random, so to emulate that there is a [[http://​en.wikipedia.com/​wiki/​correlation|correlation]] value as well. +
-<​code>​ # tc qdisc change dev eth0 root netem delay 100ms 10ms 25%</​code>​ +
- +
-This causes the added delay to be 100ms ± 10ms with the next random element depending 25% on the last one. This isn't true statistical [[http://​en.wikipedia.com/​wiki/​correlation|correlation]],​ but an approximation. +
-====Delay distribution==== +
- +
-  +
- +
-Typically, the delay in a network is not uniform. It is more common to use a something like a [[http://​en.wikipedia.com/​wiki/​Normal_Distribution|normal distribution]] to describe the variation in delay. The netem discipline can take a table to specify a non-uniform distribution. +
-<​code>​ # tc qdisc change dev eth0 root netem delay 100ms 20ms distribution normal</​code>​ +
- +
-The actual tables (normal, pareto, paretonormal) are generated as part of the [[:​networking:​iproute2|iproute2]] compilation and placed in /​usr/​lib/​tc;​ so it is possible with some effort to make your own distribution based on experimental data. +
-====Packet loss==== +
- +
-  +
- +
-Random packet loss is specified in the '​tc'​ command in percent. The smallest possible non-zero value is: +
- +
-2<​sup>​32</​sup>​ = 0.0000000232% +
-<​code>​ # tc qdisc change dev eth0 root netem loss 0.1%</​code>​ +
- +
-This causes 1/10th of a percent (i.e 1 out of 1000) packets to be randomly dropped. +
- +
-An optional correlation may also be added. This causes the random number generator to be less random and can be used to emulate packet burst losses. +
-<​code>​ # tc qdisc change dev eth0 root netem loss 0.3% 25%</​code>​ +
- +
-This will cause 0.3% of packets to be lost, and each successive probability depends by a quarter on the last one. +
- +
-Prob<​sub>​n</​sub>​ = .25 * Prob<​sub>​n-1</​sub>​ + .75 * Random +
- +
-  +
-===Caveats=== +
- +
-  +
- +
-  * When loss is used locally (not on a bridge or router), the loss is reported to the upper level protocols. This may cause TCP to resend and behave as if there was no loss. When testing protocol reponse to loss it is best to use a netem on a [[https://​www.linuxfoundation.org/​node/​add/​wiki?​gids[]=5066| bridge ]] or [[https://​www.linuxfoundation.org/​node/​add/​wiki?​gids[]=5066| router ]] +
-====Packet duplication==== +
- +
-  +
- +
-Packet duplication is specified the same way as packet loss. +
-<​code>​ # tc qdisc change dev eth0 root netem duplicate 1%</​code>​ +
-====Packet corruption==== +
- +
-  +
- +
-Random noise can be emulated (in 2.6.16 or later) with the corrupt option. This introduces a single bit error at a random offset in the packet. +
-<​code>​ # tc qdisc change dev eth0 root netem corrupt 0.1%</​code>​ +
-====Packet re-ordering==== +
- +
-There are two different ways to specify reordering. The first method **gap** uses a fixed sequence and reorders every //Nth// packet. A simple usage of this is: +
-<​code>​ # tc qdisc change dev eth0 root netem gap 5 delay 10ms</​code>​ +
- +
-This causes every 5th (10th, 15th, ...) packet to go to be sent immediately and every other packet to be delayed by 10ms. This is predictable and useful for base protocol testing like reassembly. +
- +
-The second form **reorder** of re-ordering is more like real life. It causes a certain percentage of the packets to get mis-ordered. +
-<​code>​ # tc qdisc change dev eth0 root netem delay 10ms reorder 25% 50%</​code>​ +
- +
-In this example, 25% of packets (with a correlation of 50%) will get sent immediately,​ others will be delayed by 10ms. +
- +
-Newer versions of netem will also re-order packets if the random delay values are out of order. The following will cause some reordering:​ +
-<​code>​ # tc qdisc change dev eth0 root netem delay 100ms 75ms</​code>​ +
- +
-If the first packet gets a random delay of 100ms (100ms base - 0ms jitter) and the second packet is sent 1ms later and gets a delay of 50ms (100ms base - 50ms jitter); the second packet will be sent first. This is because the queue discipline //tfifo// inside netem, keeps packets in order by time to send. +
-===Caveats=== +
- +
-  +
- +
-  * Mixing forms of reordering may lead to unexpected results +
-  * Any method of reordering to work, some delay is necessary. +
-  * If the delay is less than the inter-packet arrival time then no reordering will be seen. +
-====Rate control==== +
- +
-  +
- +
-There is no rate control built-in to the netem discipline, instead use one of the other disciplines that does do rate control. In this example, we use [[http://​en.wikipedia.com/​wiki/​Token_bucket|Token Bucket]] Filter (TBF) to limit output. +
-<​code>​ # tc qdisc add dev eth0 root handle 1:0 netem delay 100ms +
- # tc qdisc add dev eth0 parent 1:1 handle 10: tbf rate 256kbit buffer 1600 limit 3000 +
- # tc -s qdisc ls dev eth0 +
- qdisc netem 1: limit 1000 delay 100.0ms +
-  Sent 0 bytes 0 pkts (dropped 0, overlimits 0 ) +
- qdisc tbf 10: rate 256Kbit burst 1599b lat 26.6ms +
-  Sent 0 bytes 0 pkts (dropped 0, overlimits 0 )</​code>​ +
- +
-Check on the options for buffer and limit as you might find you need bigger defaults than these (they are in bytes) +
- +
-For more explanation about how to use classful queuing disciplines see: [[http://​lartc.org/​howto/​lartc.qdisc.classful.html|Linux Advanced Routing HOWTO - classes]] +
-====Non FIFO queuing==== +
- +
-  +
- +
-Just like the previous example, any of the other queuing disciplines (GRED, CBQ, etc) can be used. +
-====Delaying only some traffic==== +
- +
-  +
- +
-Here is a simple example that only controls traffic to one IP address. +
-<​code>​ # tc qdisc add dev eth0 root handle 1: prio +
- # tc qdisc add dev eth0 parent 1:3 handle 30: \ +
-tbf rate 20kbit buffer 1600 limit  3000 +
- # tc qdisc add dev eth0 parent 30:1 handle 31: \ +
-netem  delay 200ms 10ms distribution normal +
- # tc filter add dev eth0 protocol ip parent 1:0 prio 3 u32 \ +
-     match ip dst 65.172.181.4/​32 flowid 1:​3</​code>​ +
- +
-The commands makes a simple priority queueing discipline, then a TBF is added to do rate control, then attaches a basic netem. Finally, a filter classifies all packets going to 65.172.181.4 as being priority 3. For more info on traffic classification see [[http://​lartc.org/​howto/​lartc.qdisc.filters.html|LARTC -- filters]] +
-=====FAQ===== +
- +
-  +
- +
-  +
-====How come first ping takes longer?​==== +
- +
-The first ICMP packet in a ping requires an ARP request/​response as well. +
-====How come TCP is so slow over netem?​==== +
- +
-When you run [[http://​en.wikipedia.com/​wiki/​Transmission_Control_Protocol|TCP]] over large [[http://​en.wikipedia.com/​wiki/​Bandwidth-delay_product| Bandwidth Delay Product]] links, you need to do some [[http://​en.wikipedia.com/​wiki/​TCP_Tuning|TCP tuning]] to increase the maximum possible buffer space. +
-====How can I use netem on incoming traffic?​==== +
- +
-You need to use the Intermediate Functional Block pseudo-device [[:​networking:​ifb| IFB ]]. This network device allows attaching queuing discplines to incoming packets. +
-<​code>​ # modprobe ifb +
- # ip link set dev ifb0 up +
- # tc qdisc add dev eth0 ingress +
- # tc filter add dev eth0 parent ffff: \  +
-   ​protocol ip u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb0 +
- # tc qdisc add dev ifb0 root netem delay 750ms</​code>​ +
- +
-Another way is to use another machine as an Ethernet [[https://​www.linuxfoundation.org/​node/​add/​wiki?​gids[]=5066| bridge ]], and apply netem to both Ethernet devices. +
-====How to reorder packets based on jitter?​==== +
- +
-Starting with version 1.1 (in 2.6.15), netem will reorder packets if the delay value has lots of jitter. +
- +
-If you don't want this behaviour then replace the internal queue discipline //tfifo// with a pure packet fifo //pfifo//. The following example has lots of jitter, but the packets will stay in order. +
-<​code>​ # tc qdisc add dev eth0 root handle 1: netem delay 10ms 100ms +
- # tc qdisc add dev eth0 parent 1:1 pfifo limit 1000</​code>​ +
-====How does the value of HZ impact Netem?​==== +
- +
-In the 2.6 line of kernels, HZ is a configurable parameter that takes values of either 100, 250, or 1000. Because it affects the granularity with which Netem is able to delay packets, it is most beneficial to set HZ to 1000, which will allow for delays in increments of 1ms. See [[http://​lists.osdl.org/​pipermail/​netem/​2006-March/​000343.html|this mailing list post]] for a more detailed discussion of the impact of HZ. +
- +
-In kernel versions, 2.6.22 or later, netem will use high resolution timers, if they are enabled. This allows for finer granularity (sub-jiffie) resolution. +
-=====Links===== +
- +
-  +
- +
-  * Linux Conf Au [[http://​developer.osdl.org/​shemminger/​LCA2005_netem.pdf|presentation]] and [[http://​developer.osdl.org/​shemminger/​LCA2005_paper.pdf|paper]]. +
-  * [[http://​info.iet.unipi.it/​~luigi/​ip_dummynet/​|dummynet an network emulator in FreeBSD]] +
-  * [[http://​snad.ncsl.nist.gov/​itg/​nistnet/​|NISTnet]] +
-=====Contact Info===== +
- +
-  +
- +
-Since netem is part of the core Linux subsystem, all bug reports and patches should be sent to [[mailto:​netdev@vger.kernel.org|Linux Network Developers]] mailing list. +
- +
-The netem@osdl.org mailing list is available for user discussions. Mail from non-subscribers is moderated to prevent spam. To subscribe or unsubscribe use the [[http://​lists.osdl.org/​mailman/​listinfo/​netem|Netem mailing list interface]].+
  
 +For more information see [[https://​man7.org/​linux/​man-pages/​man8/​tc-netem.8.html|netem man page]].
networking/netem.1536906611.txt.gz · Last modified: 2018/09/14 06:30 by mlouielu