For example you have 2 links to ISP, one is 64Kbit and another is metered (you pay for traffic) much higher speed.
Latency of links must be very similar, otherwise in this example, packets from one session can come in different order to final destination!
tc qdisc del dev eth0.5 root tc qdisc add dev eth0.5 handle 1: root prio tc filter add dev eth0.5 parent 1: protocol ip prio 10 u32 \ match ip src 195.69.208.252/32 flowid 1:16 \ action police rate 64kbit burst 64k conform-exceed pipe/continue \ action mirred egress redirect dev eth0.6
This example will send 64kbit/s only over eth0.5, and remaining over eth0.6
In given example, you have 3 links to upstream ISP, they are all located in one ethernet segment, and on one network card. It will become kind of difficult to shape them conventional way. Let's say you have multiple incoming interfaces also, eth0 (customers1) and eth1 (customers2). Also you have connection to 3 ISP, ISP1 - you use only to peer with kernel.org (let's say 204.152.191.0/24), ISP2 must be used as highly reliable, for customers who is paying a lot, and coming over eth0. ISP3 used for cheap customers, who is coming from eth1.
Shaper:
tc qdisc del dev eth2 root tc qdisc add dev eth2 root handle 1: htb default 2000 tc class add dev eth2 parent 1: classid 1:1 htb rate 100Mbit ceil 100Mbit quantum 1600 tc class add dev eth2 parent 1:1 classid 1:100 htb rate 4000Kbit ceil 4000Kbit quantum 1600 # ISP N1, kernel org peering tc qdisc add dev eth2 parent 1:100 handle 100: bfifo limit 100000 #Buffering 512Kb, or 1 second of data tc filter add dev eth2 parent 1:0 protocol ip prio 100 route to 10 classid 1:100 #Realm 10 tc class add dev eth2 parent 1:1 classid 1:200 htb rate 4000Kbit ceil 4000Kbit quantum 1600 # ISP N2 tc qdisc add dev eth2 parent 1:200 handle 200: bfifo limit 512000 #Buffering 512Kb, or 1 second of data tc filter add dev eth2 parent 1:0 protocol ip prio 100 route to 11 classid 1:200 tc class add dev eth2 parent 1:1 classid 1:300 htb rate 3000Kbit ceil 3000Kbit quantum 1600 # ISP N3 tc qdisc add dev eth2 parent 1:300 handle 300: bfifo limit 384000 #Also 1 second of data tc filter add dev eth2 parent 1:0 protocol ip prio 100 route to 12 classid 1:300
Routing
#Reset old tables ip rule del dev eth0 table 200 ip rule del dev eth1 table 201 ip rule add dev eth0 table 200 ip rule add dev eth1 table 201 #Table 200 - customers1 ip route add table 200 204.152.191.0/24 via 10.95.95.1 realm 10 #Local Peering to kernel.org ip route add table 200 0.0.0.0/0 via 10.95.95.2 realm 11 #Default, for customers1, ISP N2 #Table 201 - customers2 ip route add table 201 204.152.191.0/24 via 10.95.95.1 realm 10 #Local Peering to kernel.org ip route add table 201 0.0.0.0/0 via 10.95.95.3 realm 12 #Default, for customers2, ISP N3