[j-nsp] load balancing between multiple BGP links

Discussion:

Joe Shen

2005-09-14 11:27:50 UTC

Hi,

How could load on multiple BGP peer links be balanced
automatically?

The situation we are facing:

---------------------------|
| Service provider |
| |
--R1--------------------R2---
|\ |
| \ E-BGP |
| \ ----------------\ |
| \|
----R3--------------------R4-|
| Our Network |
| (OSPF) |
-----------------------------

The three links between our network and Service
provider network are all 1Gbps. Now, we noticed that
load on link R1-R3, R1-R4 is about 50% (in/out), but
load on R2-R4 are about 90% ( in/out).

How could we balance those load on the three links
automatically? or must we tune the route mannually?

Is there any technical guide on this?

thanks in advance.

Joe

___________________________________________________________
????G???????????????????
http://cn.mail.yahoo.com

Guy Davies

2005-09-14 11:50:33 UTC

Permalink

Hi Joe,

It can't with the physical layout you described. BGP selects one next-hop as the best (and only one). The only way to load share is to have two (or more) parallel links between just two routers. You then create a static route on each end via each physical next-hop to the loopback on the far end router. Then, you setup an ebgp multihop session between the loopbacks on the two routers. That then uses IGP load sharing to balance traffic across the two parallel links.

Other than that, you're stuck with crude methods of dividing up the destination prefixes to approximate load sharing. You could set localpref on routers based on AS path length or some other totally arbitrary parameter to ensure that you're getting something approximating 50/50. But that's pretty horrible, pretty crude and rarely gives stable results.

Rgds,

Guy

-----Original Message-----
From: juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net] On Behalf Of Joe Shen
Sent: 14 September 2005 11:28
To: juniper-nsp at puck.nether.net
Cc: cisco-nsp at puck.nether.net
Subject: [j-nsp] load balancing between multiple BGP links
Hi,
How could load on multiple BGP peer links be balanced
automatically?
---------------------------|
| Service provider |
| |
--R1--------------------R2---
|\ |
| \ E-BGP |
| \ ----------------\ |
| \|
----R3--------------------R4-|
| Our Network |
| (OSPF) |
-----------------------------
The three links between our network and Service
provider network are all 1Gbps. Now, we noticed that
load on link R1-R3, R1-R4 is about 50% (in/out), but
load on R2-R4 are about 90% ( in/out).
How could we balance those load on the three links
automatically? or must we tune the route mannually?
Is there any technical guide on this?
thanks in advance.
Joe
___________________________________________________________
????G???????????????????
http://cn.mail.yahoo.com
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

This e-mail is private and may be confidential and is for the intended recipient only. If misdirected, please notify us by telephone and confirm that it has been deleted from your system and any copies destroyed. If you are not the intended recipient you are strictly prohibited from using, printing, copying, distributing or disseminating this e-mail or any information contained in it. We use reasonable endeavours to virus scan all e-mails leaving the Company but no warranty is given that this e-mail and any attachments are virus free. You should undertake your own virus checking. The right to monitor e-mail communications through our network is reserved by us.

Alok

2005-09-14 14:11:42 UTC

Permalink

----- Original Message -----
From: "Guy Davies" <Guy.Davies at telindus.co.uk>
To: "Joe Shen" <sj_hznm at yahoo.com.cn>; <juniper-nsp at puck.nether.net>
Cc: <cisco-nsp at puck.nether.net>
Sent: Wednesday, September 14, 2005 4:20 PM
Subject: RE: [j-nsp] load balancing between multiple BGP links

Post by Guy Davies
Hi Joe,
It can't with the physical layout you described. BGP selects one next-hop

as the best (and only one). The only way to load share is to have two (or
more) parallel links between just two routers. You then create a static
route on each end via each physical next-hop to the loopback on the far end
router. Then, you setup an ebgp multihop session between the loopbacks on
the two routers. That then uses IGP load sharing to balance traffic across
the two parallel links.
Anyone knows what to expect in this scenario?

R1===========R2

EBGP multihop between them over 2 links
link 1 =10Mbps
link 2 = 20 Mbps

total traffic = 25Mbps

Is there a way to ensure that link 1 doesnt get choked?

-thanks
Alok

Alok

2005-09-15 18:53:47 UTC

Permalink

Thanks folks for all your replies.

wonder who decided that the next commercially deployed link capacity after a
T1 should be a T3 and the next commercially deployed link after a E1 should
be an E3

:-)
----- Original Message -----
From: "Phillip Vandry" <vandry at TZoNE.ORG>
To: "Alok" <alokdube at hotpop.com>
Cc: "Guy Davies" <Guy.Davies at telindus.co.uk>; "Joe Shen"
<sj_hznm at yahoo.com.cn>; <juniper-nsp at puck.nether.net>;
<cisco-nsp at puck.nether.net>
Sent: Thursday, September 15, 2005 11:15 PM
Subject: Re: [c-nsp] Re: [j-nsp] load balancing between multiple BGP links

Post by Alok
R1===========R2
EBGP multihop between them over 2 links
link 1 =10Mbps
link 2 = 20 Mbps
total traffic = 25Mbps
Is there a way to ensure that link 1 doesnt get choked?

I have not tried this, but if you can somehow treat link 2 as two links
and install two routes, then you can do equal cost load balancing
over link 1, link 2, and link 2 again. Options for doing that are
secondary IP addresses on each side of link 2 and a second static
route pointing to the secondary address on the other side (if using
static routes) or run link 2 as a trunk and put 2 vlans on it
(if using OSPF instead of static routes).
This solution is only practical when the greatest common divisor of
the link sizes is large.
-Phil

Rafal Szarecki (WA/EPO)

2005-09-14 13:23:26 UTC

Permalink

Hi,

Guy is not 100% right. Moder BGP implementation allows for hack to ignore last criteria in path selection algorythm.
This is "BGP Multi-path". This is supported on both JUNOS and IOS.

If you run this command on R4, then trafic leaving your AS by R4 will be distributed over 2 links. Additionaly you has to configure
alghorytm used to load-balancing - per destination-perfix or per flow.
Of course traffic leaving your AS by R3 will use only R3-R1 link.

So You can't automaticaly guarantee that all 3 links will be used equaly. You can simply go to situatin when R4-R1 and R4-R2 will be
loaded equaly, byt R3-R1 can be loaded on different level.

To play with this, You has to paly with BYP attributes per prefix/prefix set:
Local Preference for play with output traffic.
MED and AS path for input traffic.

Note.1 Because SP is autonomous system their policy and BGP setup has to be tunned also. There is no way to guarantee
expected load balancing behavior of SP routers by configuring Customer routers. (e.g. is R3, R4 will use MED,
then SP can overwrite them by use of Local Preference)

So for output traffic, using BGP multi-path you can reach:
R3-->R1 50%
R4-->R1 70%
R4-->R2 70%

And then play with policy/roadmat on R3 to set-up Local Preference >100 to some set of prefixes (e.g. some AS) and annouce this to R4.
In effect R4 will send packet addresed to this prefixes to R3 to use R3->R1 link.

Rafal Jan Szarecki JNCIE #136
Senior Consultant - Datacom Networks
Ericsson Poland EPO/S/D
Office: +48 22 6916635
ECN: 837 6635
Mobile: +48 602418971
Skype: callto://Rafal_Szarecki <callto://Rafal_Szarecki/>

-----Original Message-----
From: juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net]On Behalf Of Guy Davies
Sent: Wednesday, September 14, 2005 12:50 PM
To: Joe Shen; juniper-nsp at puck.nether.net
Cc: cisco-nsp at puck.nether.net
Subject: RE: [j-nsp] load balancing between multiple BGP links
Hi Joe,
It can't with the physical layout you described. BGP selects
one next-hop as the best (and only one). The only way to
load share is to have two (or more) parallel links between
just two routers. You then create a static route on each end
via each physical next-hop to the loopback on the far end
router. Then, you setup an ebgp multihop session between the
loopbacks on the two routers. That then uses IGP load
sharing to balance traffic across the two parallel links.
Other than that, you're stuck with crude methods of dividing
up the destination prefixes to approximate load sharing. You
could set localpref on routers based on AS path length or
some other totally arbitrary parameter to ensure that you're
getting something approximating 50/50. But that's pretty
horrible, pretty crude and rarely gives stable results.
Rgds,
Guy

This e-mail is private and may be confidential and is for the
intended recipient only. If misdirected, please notify us by
telephone and confirm that it has been deleted from your
system and any copies destroyed. If you are not the intended
recipient you are strictly prohibited from using, printing,
copying, distributing or disseminating this e-mail or any
information contained in it. We use reasonable endeavours to
virus scan all e-mails leaving the Company but no warranty is
given that this e-mail and any attachments are virus free.
You should undertake your own virus checking. The right to
monitor e-mail communications through our network is reserved by us.
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

Robert Raszuk

2005-09-14 13:54:11 UTC

Permalink

First Joe has not even indicated if in his OSPF net he runs IBGP or not.
If not then the local pref tips may not turn to be very useful :)

As to the loadbalancing from R4 point of view outbound sure EBGP
multipath has been supported for years - no need for parallel links like
Guy says.

One advice I could add (since cisco-nsp mail alias has been cc-ed :)) is
to use src-dst based hashing (cisco default) rather then just
destination based one.

Using the latter in the case when you have very popular server being
accessed by many src hosts results in all of them traversing the same
link R4-R2 or R4-R1 :(((

Post by Rafal Szarecki (WA/EPO)

Post by Joe Shen
The three links between our network and Service
provider network are all 1Gbps. Now, we noticed that
load on link R1-R3, R1-R4 is about 50% (in/out), but
load on R2-R4 are about 90% ( in/out).

Cheers,
R.

Post by Rafal Szarecki (WA/EPO)
Hi,
Guy is not 100% right. Moder BGP implementation allows for hack to ignore last criteria in path selection algorythm.
This is "BGP Multi-path". This is supported on both JUNOS and IOS.
If you run this command on R4, then trafic leaving your AS by R4 will be distributed over 2 links. Additionaly you has to configure
alghorytm used to load-balancing - per destination-perfix or per flow.
Of course traffic leaving your AS by R3 will use only R3-R1 link.
So You can't automaticaly guarantee that all 3 links will be used equaly. You can simply go to situatin when R4-R1 and R4-R2 will be
loaded equaly, byt R3-R1 can be loaded on different level.
Local Preference for play with output traffic.
MED and AS path for input traffic.
Note.1 Because SP is autonomous system their policy and BGP setup has to be tunned also. There is no way to guarantee
expected load balancing behavior of SP routers by configuring Customer routers. (e.g. is R3, R4 will use MED,
then SP can overwrite them by use of Local Preference)
R3-->R1 50%
R4-->R1 70%
R4-->R2 70%
And then play with policy/roadmat on R3 to set-up Local Preference >100 to some set of prefixes (e.g. some AS) and annouce this to R4.
In effect R4 will send packet addresed to this prefixes to R3 to use R3->R1 link.
Rafal Jan Szarecki JNCIE #136
Senior Consultant - Datacom Networks
Ericsson Poland EPO/S/D
Office: +48 22 6916635
ECN: 837 6635
Mobile: +48 602418971
Skype: callto://Rafal_Szarecki <callto://Rafal_Szarecki/>

Post by Joe Shen
-----Original Message-----
From: juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net] On Behalf Of Joe Shen
Sent: 14 September 2005 11:28
To: juniper-nsp at puck.nether.net
Cc: cisco-nsp at puck.nether.net
Subject: [j-nsp] load balancing between multiple BGP links
Hi,
How could load on multiple BGP peer links be balanced
automatically?
---------------------------|
| Service provider |
| |
--R1--------------------R2---
|\ |
| \ E-BGP |
| \ ----------------\ |
| \|
----R3--------------------R4-|
| Our Network |
| (OSPF) |
-----------------------------
The three links between our network and Service
provider network are all 1Gbps. Now, we noticed that
load on link R1-R3, R1-R4 is about 50% (in/out), but
load on R2-R4 are about 90% ( in/out).
How could we balance those load on the three links
automatically? or must we tune the route mannually?
Is there any technical guide on this?
thanks in advance.
Joe
___________________________________________________________
????G???????????????????
http://cn.mail.yahoo.com
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

This e-mail is private and may be confidential and is for the
intended recipient only. If misdirected, please notify us by
telephone and confirm that it has been deleted from your
system and any copies destroyed. If you are not the intended
recipient you are strictly prohibited from using, printing,
copying, distributing or disseminating this e-mail or any
information contained in it. We use reasonable endeavours to
virus scan all e-mails leaving the Company but no warranty is
given that this e-mail and any attachments are virus free.
You should undertake your own virus checking. The right to
monitor e-mail communications through our network is reserved by us.
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

Sabri Berisha

2005-09-14 14:03:04 UTC

Permalink

On Wed, Sep 14, 2005 at 05:53:54AM -0700, Robert Raszuk wrote:

Hi,

Post by Robert Raszuk
One advice I could add (since cisco-nsp mail alias has been cc-ed :)) is
to use src-dst based hashing (cisco default) rather then just
destination based one.

The IP2 implements microflow based loadbalancing; it's not just
destination based.

Thanks,

--
Sabri Berisha,
Juniper Certified - JNCIA #747 | Cisco Certified - CCNA
email: sabri at cluecentral.net | cell: +31 6 19890416
http://www.cluecentral.net/ | http://www.virt-ix.net/

Rafal Szarecki (WA/EPO)

2005-09-14 14:08:52 UTC

Permalink

Yes, if you configure. in forwarding option load balancing policy to per-packet.

By default (factory config) JUNIPER do PER DESTINATION PREFIX. load ballancing.

Rafa? Jan Szarecki JNCIE #136

Skype: callto://Rafal_Szarecki <callto://Rafal_Szarecki/>

-----Original Message-----
From: juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net]On Behalf Of Sabri Berisha
Sent: Wednesday, September 14, 2005 3:03 PM
To: juniper-nsp at puck.nether.net
Subject: Re: [j-nsp] load balancing between multiple BGP links
Hi,

Post by Robert Raszuk
One advice I could add (since cisco-nsp mail alias has been

cc-ed :)) is

Post by Robert Raszuk
to use src-dst based hashing (cisco default) rather then just
destination based one.

The IP2 implements microflow based loadbalancing; it's not just
destination based.
Thanks,
--
Sabri Berisha,
Juniper Certified - JNCIA #747 | Cisco Certified - CCNA
email: sabri at cluecentral.net | cell: +31 6 19890416
http://www.cluecentral.net/ | http://www.virt-ix.net/
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

Alok

2005-09-14 14:42:38 UTC

Permalink

It isnt juniper in my case though but I just remebered the per packet thing
when I ran into this with another vendor

however, even link bandwidth via BGP , which i perhaps remember has been
there since 5.6 doesnt consider the "immediate util" at the time of
transmitting the packet....

Perhaps "per packet" with BGP link bandwidth would be good infact i think..
.
or perhaps for flow based, flow setup based on "actual util at the time of
setup".... though nothing still seems to beat per packet :-) with link
bandwidth i guess...

----- Original Message -----
From: "Rafal Szarecki (WA/EPO)" <rafal.szarecki at ericsson.com>
To: "Alok" <alokdube at hotpop.com>
Sent: Wednesday, September 14, 2005 7:01 PM
Subject: RE: [j-nsp] load balancing between multiple BGP links

Alok,

Post by Alok
EBGP multihop between them over 2 links
link 1 =10Mbps
link 2 = 20 Mbps
total traffic = 25Mbps

Perpetum mobile? Or statistics inacuracy.

Post by Alok
Is there a way to ensure that link 1 doesnt get choked?

- Bgp multipath is simpler to run.
- Is it Juniper? If the pleas check forwarding option to identyfy
loadbalancing hash alghoritm.
http://www.juniper.net/techpubs/software/junos/junos73/swconfig73-policy/html/policy-actions-config13.html#1021911

Rafa? Szarecki JNCIE

skype me <callto://Rafal_Szarecki/>

sin

2005-09-14 18:13:41 UTC

Permalink

Post by Alok
It isnt juniper in my case though but I just remebered the per packet thing
when I ran into this with another vendor
however, even link bandwidth via BGP , which i perhaps remember has been
there since 5.6 doesnt consider the "immediate util" at the time of
transmitting the packet....
Perhaps "per packet" with BGP link bandwidth would be good infact i think..
.
or perhaps for flow based, flow setup based on "actual util at the time of
setup".... though nothing still seems to beat per packet :-) with link
bandwidth i guess...

i know that with cisco you can setup an acl that can match even/odd ip
addresses and then use that acl in a route map to distribute the traffic
across two links to another bgp speaker (that in it's turn can do the
same with the link back to you). this might be an option for you.

Alok

2005-09-14 19:02:19 UTC

Permalink

except for the need of 2 sessions,

per packet makes sense..somehow...

a. how is a link between any 2 associations different from any end to end
associations... (given all protocols implement some flow control on
top)........
b. on a per node bases it still boils down to treating a bundle between 2
nodes as a single FA

----- Original Message -----
From: "sin" <sin at pvs.ro>
To: "Alok" <alokdube at hotpop.com>
Cc: <juniper-nsp at puck.nether.net>; <cisco-nsp at puck.nether.net>
Sent: Wednesday, September 14, 2005 10:43 PM
Subject: Re: [c-nsp] Re: [j-nsp] load balancing between multiple BGP links

Post by sin

Chris Morrow

2005-09-14 21:13:08 UTC

Permalink

Post by sin

uhm, I'm positive another person mentioned this before, but your IGP does
the loadbalancing, so get some form of IGP to tell you that the 2 links
are available for the same destination next-hop and be done with it.
Policy routing is just so not a solution...

Example, your bgp neighbor is over 2 links, static route the /32 across
both links and reset the next-hop to neighbor IP for all inbound routes
(this should be the default, but with multihop ebgp the neighbor can send
you an alternate next-hop so you should reset it for your own protection).

When traffic is sent to the next-hop it'll get automagically loadshared
across both links... in a 'per flow' way, so it's not 50/50 more like
45/55 to 70/30.

Rendo

2005-09-15 00:03:28 UTC

Permalink

Hi all,

May I change the topic for little bit? I hope so :)
Basically, my problem is still very related to bgp load balancing, but, i
need to load balancing data traffic over mpls over multiple e1 links. I use
rsvp for signalling.

Router A --------4E1---------------Router B

the best result that i can get is, the traffic is load balanced by
destination prefix in the routing table as seen below,
this is some of "show route table VRF-A" result
10.2.178.68/30 *[BGP/170] 13:54:28, localpref 100, from 10.2.178.253
AS path: I
via e1-0/0/2.0, label-switched-path wpi_to_tbs_3
via e1-0/0/3.0, label-switched-path wpi_to_tbs_4
via e1-0/0/0.0, label-switched-path wpi_to_tbs

via e1-0/0/1.0, label-switched-path wpi_to_tbs_2

10.2.178.247/32 *[BGP/170] 13:54:28, localpref 100, from 10.2.178.253
AS path: I

via e1-0/0/2.0, label-switched-path wpi_to_tbs_3

via e1-0/0/3.0, label-switched-path wpi_to_tbs_4
via e1-0/0/0.0, label-switched-path wpi_to_tbs
via e1-0/0/1.0, label-switched-path wpi_to_tbs_2
172.17.128.0/19 *[BGP/170] 13:54:28, MED 0, localpref 100, from
10.2.178.253
AS path: I
via e1-0/0/2.0, label-switched-path wpi_to_tbs_3
via e1-0/0/3.0, label-switched-path wpi_to_tbs_4
via e1-0/0/0.0, label-switched-path wpi_to_tbs

via e1-0/0/1.0, label-switched-path wpi_to_tbs_2

and this is some of "show route forwarding-table table VRF-A" result
10.2.178.68/30 user 0 indr 788 8
ulst 937 1
Push 210576 e1-0/0/2.0
Push 210576 e1-0/0/3.0
Push 210576 e1-0/0/0.0
Push 210576 e1-0/0/1.0
10.2.178.247/32 user 0 indr 788 8
ulst 937 1
Push 210576 e1-0/0/2.0
Push 210576 e1-0/0/3.0
Push 210576 e1-0/0/0.0
Push 210576 e1-0/0/1.0
172.17.128.0/19 user 0 indr 788 8
ulst 937 1
Push 210576 e1-0/0/2.0
Push 210576 e1-0/0/3.0
Push 210576 e1-0/0/0.0
Push 210576 e1-0/0/1.0

My current configuration are:
- i create 4 LSP with strict next-hop option for each e1 link
- i already put per packet load balancing policy in forwarding-options
- I have ospf for IGP between two routers

I'm not happy with this, because all the server is located in the same
subnet and the client also aggregated in another same subnet too, so i can
say that most traffic still use same link.

Any idea how can i load balance this?

thanks.

-rendo-

----- Original Message -----
From: "Chris Morrow" <morrowc at ops-netman.net>
To: "sin" <sin at pvs.ro>
Cc: <juniper-nsp at puck.nether.net>; <cisco-nsp at puck.nether.net>
Sent: Thursday, September 15, 2005 3:13 AM
Subject: Re: [c-nsp] Re: [j-nsp] load balancing between multiple BGP links

Post by sin

uhm, I'm positive another person mentioned this before, but your IGP does
the loadbalancing, so get some form of IGP to tell you that the 2 links
are available for the same destination next-hop and be done with it.
Policy routing is just so not a solution...
Example, your bgp neighbor is over 2 links, static route the /32 across
both links and reset the next-hop to neighbor IP for all inbound routes
(this should be the default, but with multihop ebgp the neighbor can send
you an alternate next-hop so you should reset it for your own protection).
When traffic is sent to the next-hop it'll get automagically loadshared
across both links... in a 'per flow' way, so it's not 50/50 more like
45/55 to 70/30.
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

Piotr Marecki

2005-09-15 07:10:34 UTC

Permalink

Post by Rendo
I'm not happy with this, because all the server is located in the same
subnet and the client also aggregated in another same subnet too, so i can
say that most traffic still use same link.
Any idea how can i load balance this?
thanks.
-rendo-

Try to add layer4 information ( port numbers ) to set of inputs for IP2
hashing function . It should help , since
at least clients shall use different tcp/udp ports numbers from ephemeral
range

bubu at lab-m5-2# show forwarding-options
hash-key {
family inet {
layer-3;
layer-4;
}
}

regards

Piotr Marecki

Harry Reynolds

2005-09-14 14:49:28 UTC

Permalink

The link bandwidth community, as of 5.6, should balance accordingly:

http://www.juniper.net/techpubs/software/junos/junos56/rn-sw-56/html/rn-
sw-563.html#1130062

BGP multipath link-bandwidth attribute--When per-packet load-balancing
is enabled, you can specify bandwidth values resulting in unequally
distributed traffic when you set up a community policy for BGP multipath
path selection. By default, when you use BGP multipath, the router
distributes traffic equally among the several paths it calculates. To
configure an unequal distribution, include the bandwidth attribute,
specifying the local autonomous system and link bandwidth in bytes per
second, at the [edit policy-options community name] hierarchy level. The
show route detail and show route forwarding-table commands display the
new output. [Policy, Command Reference]

HTHs

Post by Guy Davies
Hi Joe,
It can't with the physical layout you described. BGP

selects one next-hop
as the best (and only one). The only way to load share is to
have two (or
more) parallel links between just two routers. You then
create a static
route on each end via each physical next-hop to the loopback
on the far end
router. Then, you setup an ebgp multihop session between the
loopbacks on
the two routers. That then uses IGP load sharing to balance
traffic across
the two parallel links.
Anyone knows what to expect in this scenario?
R1===========R2
EBGP multihop between them over 2 links
link 1 =10Mbps
link 2 = 20 Mbps
total traffic = 25Mbps
Is there a way to ensure that link 1 doesnt get choked?
-thanks
Alok
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

Alok

2005-09-14 16:49:41 UTC

Permalink

thanks folks,

Any one knows if the "link bandwidth" can be set to 2 different values for 1
BGP session?

In other words how is it different from simply tracking the link util on
each of the 2 ECMP routes and sending the packet/setup the flow based on the
instant value

It would work for non BGP ECMP scenarios too...right? (2 links of unequal
costs but to the same peer and simply schedule cells based on the util
value)

-thanks
Alok

----- Original Message -----
From: "Harry Reynolds" <harry at juniper.net>
To: "Alok" <alokdube at hotpop.com>; "Guy Davies" <Guy.Davies at telindus.co.uk>;
"Joe Shen" <sj_hznm at yahoo.com.cn>; <juniper-nsp at puck.nether.net>
Cc: <cisco-nsp at puck.nether.net>
Sent: Wednesday, September 14, 2005 7:19 PM
Subject: RE: [j-nsp] load balancing between multiple BGP links

The link bandwidth community, as of 5.6, should balance accordingly:

http://www.juniper.net/techpubs/software/junos/junos56/rn-sw-56/html/rn-
sw-563.html#1130062

BGP multipath link-bandwidth attribute--When per-packet load-balancing
is enabled, you can specify bandwidth values resulting in unequally
distributed traffic when you set up a community policy for BGP multipath
path selection. By default, when you use BGP multipath, the router
distributes traffic equally among the several paths it calculates. To
configure an unequal distribution, include the bandwidth attribute,
specifying the local autonomous system and link bandwidth in bytes per
second, at the [edit policy-options community name] hierarchy level. The
show route detail and show route forwarding-table commands display the
new output. [Policy, Command Reference]

HTHs

Post by Guy Davies
Hi Joe,
It can't with the physical layout you described. BGP

Guy Davies

2005-09-14 16:24:50 UTC

Permalink

Of course :-( Long week (already!). Need to think more before blurting
out an answer ;-)

Rgds,

Guy

-----Original Message-----
From: Harry Reynolds [mailto:harry at juniper.net]
Sent: 14 September 2005 14:49
To: Alok; Guy Davies; Joe Shen; juniper-nsp at puck.nether.net
Cc: cisco-nsp at puck.nether.net
Subject: RE: [j-nsp] load balancing between multiple BGP links
http://www.juniper.net/techpubs/software/junos/junos56/rn-sw-56

/html/rn-

sw-563.html#1130062
BGP multipath link-bandwidth attribute--When per-packet load-balancing
is enabled, you can specify bandwidth values resulting in unequally
distributed traffic when you set up a community policy for BGP
multipath
path selection. By default, when you use BGP multipath, the router
distributes traffic equally among the several paths it calculates. To
configure an unequal distribution, include the bandwidth attribute,
specifying the local autonomous system and link bandwidth in bytes per
second, at the [edit policy-options community name] hierarchy
level. The
show route detail and show route forwarding-table commands display the
new output. [Policy, Command Reference]
HTHs

Post by Guy Davies
Hi Joe,
It can't with the physical layout you described. BGP

Harry Reynolds

2005-09-15 02:05:20 UTC

Permalink

I think this is per-session, per-directly attached link. You will need
to EBGP sessions with each advertising the BW associated with its
directly connected EBGP peering link.

Regards

-----Original Message-----
From: juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net] On Behalf Of Alok
Sent: Wednesday, September 14, 2005 8:45 AM
To: juniper-nsp at puck.nether.net
Cc: cisco-nsp at puck.nether.net
Subject: Re: [j-nsp] load balancing between multiple BGP links
thanks folks,
Any one knows if the "link bandwidth" can be set to 2
different values for 1 BGP session?
In other words how is it different from simply tracking the
link util on each of the 2 ECMP routes and sending the
packet/setup the flow based on the instant value
It would work for non BGP ECMP scenarios too...right? (2
links of unequal costs but to the same peer and simply
schedule cells based on the util
value)
-thanks
Alok
----- Original Message -----
From: "Harry Reynolds" <harry at juniper.net>
To: "Alok" <alokdube at hotpop.com>; "Guy Davies"
<Guy.Davies at telindus.co.uk>; "Joe Shen"
<sj_hznm at yahoo.com.cn>; <juniper-nsp at puck.nether.net>
Cc: <cisco-nsp at puck.nether.net>
Sent: Wednesday, September 14, 2005 7:19 PM
Subject: RE: [j-nsp] load balancing between multiple BGP links
http://www.juniper.net/techpubs/software/junos/junos56/rn-sw-5

6/html/rn-

sw-563.html#1130062
BGP multipath link-bandwidth attribute--When per-packet load-balancing
is enabled, you can specify bandwidth values resulting in unequally
distributed traffic when you set up a community policy for
BGP multipath
path selection. By default, when you use BGP multipath, the router
distributes traffic equally among the several paths it calculates. To
configure an unequal distribution, include the bandwidth attribute,
specifying the local autonomous system and link bandwidth in bytes per
second, at the [edit policy-options community name] hierarchy
level. The
show route detail and show route forwarding-table commands display the
new output. [Policy, Command Reference]
HTHs

Post by Guy Davies
Hi Joe,
It can't with the physical layout you described. BGP

_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

Harry Reynolds

2005-09-15 19:20:07 UTC

Permalink

It was a gentleman named "economics"

-----Original Message-----
From: juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net] On Behalf Of Alok
Sent: Thursday, September 15, 2005 10:52 AM
To: Phillip Vandry
Cc: juniper-nsp at puck.nether.net; Guy Davies
Subject: Re: [c-nsp] Re: [j-nsp] load balancing between
multiple BGP links
Thanks folks for all your replies.
wonder who decided that the next commercially deployed link
capacity after a
T1 should be a T3 and the next commercially deployed link
after a E1 should be an E3
:-)
----- Original Message -----
From: "Phillip Vandry" <vandry at TZoNE.ORG>
To: "Alok" <alokdube at hotpop.com>
Cc: "Guy Davies" <Guy.Davies at telindus.co.uk>; "Joe Shen"
<sj_hznm at yahoo.com.cn>; <juniper-nsp at puck.nether.net>;
<cisco-nsp at puck.nether.net>
Sent: Thursday, September 15, 2005 11:15 PM
Subject: Re: [c-nsp] Re: [j-nsp] load balancing between
multiple BGP links

Post by Alok
R1===========R2
EBGP multihop between them over 2 links
link 1 =10Mbps
link 2 = 20 Mbps
total traffic = 25Mbps
Is there a way to ensure that link 1 doesnt get choked?

I have not tried this, but if you can somehow treat link 2

as two links

and install two routes, then you can do equal cost load balancing
over link 1, link 2, and link 2 again. Options for doing that are
secondary IP addresses on each side of link 2 and a second static
route pointing to the secondary address on the other side (if using
static routes) or run link 2 as a trunk and put 2 vlans on it
(if using OSPF instead of static routes).
This solution is only practical when the greatest common divisor of
the link sizes is large.
-Phil

_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

Alok

2005-09-15 19:58:44 UTC

Permalink

yep...makes perfect sense from his view point :-)

----- Original Message -----
From: "Harry Reynolds" <harry at juniper.net>
To: "Alok" <alokdube at hotpop.com>; "Phillip Vandry" <vandry at TZoNE.ORG>
Cc: <juniper-nsp at puck.nether.net>; "Guy Davies" <Guy.Davies at telindus.co.uk>
Sent: Thursday, September 15, 2005 11:49 PM
Subject: RE: [c-nsp] Re: [j-nsp] load balancing between multiple BGP links

It was a gentleman named "economics"

William Phang

2005-09-16 06:51:39 UTC

Permalink

Hi Rendo,

In this case, you may need Filter-based forwarding. Suppose that from server
A to Client is using LSP A (means link E1-1), server B A to Client is using
LSP B (means link E1-2), and so on. On the other side, from Client to server
A using LSP A, and so on.

I have tested using this way and it is working.

Regards,

William

-----Original Message-----
From: juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net] On Behalf Of Rendo
Sent: Thursday, September 15, 2005 6:03 AM
To: juniper-nsp at puck.nether.net
Subject: LSP load balancing (was Re: [c-nsp] Re: [j-nsp] load
balancingbetween multiple BGP links)

Hi all,

May I change the topic for little bit? I hope so :)
Basically, my problem is still very related to bgp load balancing, but, i
need to load balancing data traffic over mpls over multiple e1 links. I use
rsvp for signalling.

Router A --------4E1---------------Router B

the best result that i can get is, the traffic is load balanced by
destination prefix in the routing table as seen below,
this is some of "show route table VRF-A" result
10.2.178.68/30 *[BGP/170] 13:54:28, localpref 100, from 10.2.178.253
AS path: I
via e1-0/0/2.0, label-switched-path wpi_to_tbs_3
via e1-0/0/3.0, label-switched-path wpi_to_tbs_4
via e1-0/0/0.0, label-switched-path wpi_to_tbs

via e1-0/0/1.0, label-switched-path wpi_to_tbs_2

10.2.178.247/32 *[BGP/170] 13:54:28, localpref 100, from 10.2.178.253
AS path: I

via e1-0/0/2.0, label-switched-path wpi_to_tbs_3

via e1-0/0/1.0, label-switched-path wpi_to_tbs_2

Post by sin

Post by Alok
It isnt juniper in my case though but I just remebered the per packet

thing

Post by sin

Post by Alok
when I ran into this with another vendor
however, even link bandwidth via BGP , which i perhaps remember has

been

Post by sin

Post by Alok
there since 5.6 doesnt consider the "immediate util" at the time of
transmitting the packet....
Perhaps "per packet" with BGP link bandwidth would be good infact i

think..

Post by sin

Post by Alok
.
or perhaps for flow based, flow setup based on "actual util at the time

Post by sin

Post by Alok
setup".... though nothing still seems to beat per packet :-) with link
bandwidth i guess...

uhm, I'm positive another person mentioned this before, but your IGP does
the loadbalancing, so get some form of IGP to tell you that the 2 links
are available for the same destination next-hop and be done with it.
Policy routing is just so not a solution...
Example, your bgp neighbor is over 2 links, static route the /32 across
both links and reset the next-hop to neighbor IP for all inbound routes
(this should be the default, but with multihop ebgp the neighbor can send
you an alternate next-hop so you should reset it for your own protection).
When traffic is sent to the next-hop it'll get automagically loadshared
across both links... in a 'per flow' way, so it's not 50/50 more like
45/55 to 70/30.
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

Rendo

2005-09-17 13:26:47 UTC

Permalink

Thanks William,

Currently I have used Piotr and Rafal suggestion to use layer-4 ip
information as the hash-key. The current limitation is the single session
traffic still can not exceed 2Mbps (e1 link capacity). But I think it is
enough for me now.
If you have any other idea, it would be great.

-rendo-

----- Original Message -----
From: "William Phang" <phangjk at hotpop.com>
To: "'Rendo'" <r3nd0 at yahoo.com>; <juniper-nsp at puck.nether.net>
Sent: Friday, September 16, 2005 12:51 PM
Subject: RE: LSP load balancing (was Re: [c-nsp] Re: [j-nsp] load
balancingbetween multiple BGP links)

Hi Rendo,

In this case, you may need Filter-based forwarding. Suppose that from server
A to Client is using LSP A (means link E1-1), server B A to Client is using
LSP B (means link E1-2), and so on. On the other side, from Client to server
A using LSP A, and so on.

I have tested using this way and it is working.

Regards,

William

-----Original Message-----
From: juniper-nsp-bounces at puck.nether.net
[mailto:juniper-nsp-bounces at puck.nether.net] On Behalf Of Rendo
Sent: Thursday, September 15, 2005 6:03 AM
To: juniper-nsp at puck.nether.net
Subject: LSP load balancing (was Re: [c-nsp] Re: [j-nsp] load
balancingbetween multiple BGP links)

Hi all,

May I change the topic for little bit? I hope so :)
Basically, my problem is still very related to bgp load balancing, but, i
need to load balancing data traffic over mpls over multiple e1 links. I use
rsvp for signalling.

Router A --------4E1---------------Router B

the best result that i can get is, the traffic is load balanced by
destination prefix in the routing table as seen below,
this is some of "show route table VRF-A" result
10.2.178.68/30 *[BGP/170] 13:54:28, localpref 100, from 10.2.178.253
AS path: I
via e1-0/0/2.0, label-switched-path wpi_to_tbs_3
via e1-0/0/3.0, label-switched-path wpi_to_tbs_4
via e1-0/0/0.0, label-switched-path wpi_to_tbs

via e1-0/0/1.0, label-switched-path wpi_to_tbs_2

10.2.178.247/32 *[BGP/170] 13:54:28, localpref 100, from 10.2.178.253
AS path: I

via e1-0/0/2.0, label-switched-path wpi_to_tbs_3

via e1-0/0/1.0, label-switched-path wpi_to_tbs_2

Post by sin

Post by Alok
It isnt juniper in my case though but I just remebered the per packet

thing

Post by sin

Post by Alok
when I ran into this with another vendor
however, even link bandwidth via BGP , which i perhaps remember has

been

Post by sin

Post by Alok
there since 5.6 doesnt consider the "immediate util" at the time of
transmitting the packet....
Perhaps "per packet" with BGP link bandwidth would be good infact i

think..

Post by sin

Post by Alok
.
or perhaps for flow based, flow setup based on "actual util at the time

Post by sin

Post by Alok
setup".... though nothing still seems to beat per packet :-) with link
bandwidth i guess...

uhm, I'm positive another person mentioned this before, but your IGP does
the loadbalancing, so get some form of IGP to tell you that the 2 links
are available for the same destination next-hop and be done with it.
Policy routing is just so not a solution...
Example, your bgp neighbor is over 2 links, static route the /32 across
both links and reset the next-hop to neighbor IP for all inbound routes
(this should be the default, but with multihop ebgp the neighbor can send
you an alternate next-hop so you should reset it for your own protection).
When traffic is sent to the next-hop it'll get automagically loadshared
across both links... in a 'per flow' way, so it's not 50/50 more like
45/55 to 70/30.
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
http://puck.nether.net/mailman/listinfo/juniper-nsp