[j-nsp] 6PE and BGP signaled lsps

[Tobias Heister]

Dear All,

I am trying to wrap my head around this for some days now and it seems I am stuck. Sorry for the lengthy post, bit I tried to give as much information as possible.
I have 6 routers setup like this, currently all are logical-systems on a MX80-48T running 10.4R9 in our lab.

/--- 9 --- 8 --- \
10 | | 5
\--- 3 --- 4 --- /

all routers in the same AS, the IGP is ISIS with a single level2, all loopbacks visible on all routers via ISIS
routers 4, 8 and 5 are setup with a ldp lsp full mesh
routers 10, 9, 3, 4, 8 are setup with a rsvp lsp full mesh

The goal is to use labeled BGP to get a lsp up between routers 5 and 10 to use for services ( 6PE, inet-vpn etc.) as we are not allowed to extend neither the ldp nor the rsvp area to the other routers.

What I have done so far:
routers 4 and 8 are route reflectors for family inet unicast, family inet labeled-unicast and family inet6 labeled-unicast
routers 5 and 10 are configured like this:

## routing-options:
interface-routes {
rib-group inet interfaces;
}
rib-groups {
interfaces {
import-rib [ inet.0 inet.3 ];
import-policy only-lo;
}
}

## bgp:
type internal;
local-address 5/10
import prefer-inet3-bgp;
family inet {
unicast;
labeled-unicast {
rib {
inet.3;
}
}
}
family inet6 {
labeled-unicast {
explicit-null;
}
}
export export-inet3-lo
neighbor 4
neighbor 8

## policy
policy-statement only-lo:
term one {
from interface lo0.10;
then accept;
}
term two {
to rib inet.3;
then reject;
}
term three {
to rib inet.0;
then accept;
}

policy-statement prefer-inet3-bgp:
from {
protocol bgp;
rib inet.3;
route-filter 0.0.0.0/0 prefix-length-range /32-/32;
}
then {
preference 9;
accept;
}

policy-statement export-inet3-lo:
from {
protocol direct;
route-filter 0.0.0.0/0 prefix-length-range /32-/32;
}
then accept;

routers 4 and 8 are setup like this:

## bgp:
type internal;
local-address 4/8
family inet {
unicast;
labeled-unicast {
rib {
inet.3;
}
}
}
family inet6 {
labeled-unicast {
explicit-null;
}
}
export export-agg;
cluster 4/8
neighbor 10
neighbor 9
neighbor 3
neighbor 5

## policy
relevant term from policy-statement export-agg
term inet3 {
from {
protocol bgp;
rib inet.3;
route-filter 0.0.0.0/0 prefix-length-range /32-/32;
}
then {
next-hop self;
}
}

As far is I understand everything is fine. I have lsps (or at least routes in inet.3) to each other on routers 5 and 10

inet.3 on router 5
10.0.10.10/32 *[BGP/9] 1d 08:08:26, localpref 100, from 10.0.10.4
AS path: I

to 10.0.4.1 via ge-1/0/3.110, Push 300752

[BGP/9] 1d 08:08:27, localpref 100, from 10.0.10.8
AS path: I

to 10.0.4.13 via ge-1/0/3.107, Push 300272

inet.3 on router 10
10.0.10.5/32 *[BGP/9] 1d 03:02:33, localpref 100, from 10.0.10.4
AS path: I

to 10.0.0.6 via ge-1/0/4.101, label-switched-path R4

[BGP/9] 1d 03:02:33, localpref 100, from 10.0.10.8
AS path: I

forwarding for example from router 5 to a bgp prefix learned from router 10 uses this lsp and the labels show up in a traceroute just fine.
both lsps are going via router 4 (which as far as i understand) is normal in this setup

now I want to use this lsp for 6PE and turned on ipv6-tunneling under protocols mpls on both routers. family inet6 labled-unicast on the Route reflectors is already in place (see above)
family inet6 on all core facing interfaces is also configured. Everything is as expected for all the lsps inside the rsvp area and as well for all lsps inside the ldp area. I have entries in the inet6.3 table (for the mapped v4 address) and IPv6 Traffic is passing fine.

for example inet6.3 on R10
::ffff:10.0.10.3/128
*[RSVP/7/1] 1d 10:08:33, metric 10

to 10.0.0.6 via ge-1/0/4.101, label-switched-path R3

Unfortunately the lsp (or better said the mapped address) signaled via bgp does not show up in table inet6.3

I have read a bunch of documentation the last days and have not found a lot of help for this situation. The 6PE Feature Guide [1] says "This feature can be implemented with label-switched paths (LSPs) using the Label Distribution Protocol (LDP) or Resource Reservation Protocol (RSVP)." I am not quite sure if this means that it only can work with ldp/rsvp or if this is just a lack in documentation.

The document about scaling with labeled bgp [2] has a section about 6PE but it does not help much. First of all the method explained to get interface routes to inet6.3 does not work (at least on 10.4R9 but I figured out the correct way myself) and then when I try to follow the instructions and assign a ipv6 loopback manually and try to advertise it via family inet6 labeled-unicast rib inet6.3 I get the following commit error: "BGP: ribgroup inet6.3 is not defined for this address family and nlri"

Are there any steps I am missing to get this to work? Is 6PE even supported on lsps signaled via bgp? I also setup a static lsp between router 5 and router 10 but this did not change anything. still no route in inet6.3 for router 5 or router 10
If anybody can push me into the right direction or point out a part of the documentation I may have missed it would be appreciated.

[1] http://www.juniper.net/techpubs/en_US/junos11.2/information-products/topic-collections/fg-ipv6-islands-to-ipv4-mpls-tc/fg-ipv6-islands-to-ipv4-mpls-tc.pdf
[2] http://www.juniper.net/us/en/local/pdf/design-guides/8020013-en.pdf

regards
Tobias

[Per Granath]

Is there any reason why you are not running LDP-tunneling to/from R4/R8 and R10?

[Tobias Heister]

Post by Per Granath
Is there any reason why you are not running LDP-tunneling to/from R4/R8 and R10?

This woule be a viable solution, but as mentioned per definition it is not allowed (or for a better term wanted) in this scenario to extend ldp beyond 4, 8 and 5 (not even via rsvp tunneling) routers 9, 3 and 10 are not allowed to run ldp on any interface.
In a real world network i would probably consider ldp tunneling but this is more a "can it be done this way" scenario.

regards
Tobias

[Per Granath]

Post by Per Granath

Post by Per Granath
Is there any reason why you are not running LDP-tunneling to/from R4/R8

and R10?
This woule be a viable solution, but as mentioned per definition it is not
allowed (or for a better term wanted) in this scenario to extend ldp beyond
4, 8 and 5 (not even via rsvp tunneling) routers 9, 3 and 10 are not allowed to
run ldp on any interface.
In a real world network i would probably consider ldp tunneling but this is
more a "can it be done this way" scenario.

R10 would need to run LDP only on lo0 ...

[Tobias Heister]

Post by Per Granath

Post by Per Granath

Post by Per Granath
Is there any reason why you are not running LDP-tunneling to/from R4/R8

and R10?
This woule be a viable solution, but as mentioned per definition it is not
allowed (or for a better term wanted) in this scenario to extend ldp beyond
4, 8 and 5 (not even via rsvp tunneling) routers 9, 3 and 10 are not allowed to
run ldp on any interface.
In a real world network i would probably consider ldp tunneling but this is
more a "can it be done this way" scenario.

R10 would need to run LDP only on lo0 ...

I know, but in this scenario any interface means any interface including lo0 :)
or just to put it another way: no ldp process on router 10

regards
Tobias

[Antti Ristimäki]

Post by Tobias Heister
The document about scaling with labeled bgp [2] has a section about 6PE but it does not help much. First of all the method explained to get interface routes to inet6.3 does not work (at least on 10.4R9 but I figured out the correct way myself) and then when I try to follow the instructions and assign a ipv6 loopback manually and try to advertise it via family inet6 labeled-unicast rib inet6.3 I get the following commit error: "BGP: ribgroup inet6.3 is not defined for this address family and nlri"

Have you tried to configure a named rib-group where you specify inet6.3
as an import RIB?

AR

[Tobias Heister]

Post by Tobias Heister
The document about scaling with labeled bgp [2] has a section about 6PE but it does not help much. First of all the method explained to get interface routes to inet6.3 does not work (at
least on 10.4R9 but I figured out the correct way myself) and then when I try to follow the instructions and assign a ipv6 loopback manually and try to advertise it via family inet6
labeled-unicast rib inet6.3 I get the following commit error: "BGP: ribgroup inet6.3 is not defined for this address family and nlri"

Have you tried to configure a named rib-group where you specify inet6.3 as an import RIB?

If I understood you correctly you want me to do something like this:

show routing-options rib-groups 6PE
import-rib inet6.3;

show protocols bgp group XXX family inet6
labeled-unicast {
rib-group 6PE;
explicit-null;
}

Unfortunately this gives a similar error: "BGP: ribgroup 6PE: inet6.3 not a valid primary rib for this nlri."

What is funny too is that the document about scaling with labeled bgp [1] does not mention the explicit-null parameter in the family inet6- labeled-unicast.
But when removing it the router says: "Missing mandatory statement: 'explicit-null'" Also it states the parameter under labeled-unicast is rib and not rib-group
I am starting to believe that this guide is for different JunOS Version where they have changed the syntax and behaviour.

[1] http://www.juniper.net/us/en/local/pdf/design-guides/8020013-en.pdf

regards
Tobias

[Antti Ristimäki]

Post by Tobias Heister

Post by Tobias Heister
The document about scaling with labeled bgp [2] has a section about 6PE but it does not help much. First of all the method explained to get interface routes to inet6.3 does not work (at
least on 10.4R9 but I figured out the correct way myself) and then when I try to follow the instructions and assign a ipv6 loopback manually and try to advertise it via family inet6
labeled-unicast rib inet6.3 I get the following commit error: "BGP: ribgroup inet6.3 is not defined for this address family and nlri"

Have you tried to configure a named rib-group where you specify inet6.3 as an import RIB?

show routing-options rib-groups 6PE
import-rib inet6.3;
show protocols bgp group XXX family inet6
labeled-unicast {
rib-group 6PE;
explicit-null;
}
Unfortunately this gives a similar error: "BGP: ribgroup 6PE: inet6.3 not a valid primary rib for this nlri."

Defining "import-rib [inet6.0 inet6.3]" would probably pass the commit
check, but that wouldn't probably help in this issue, where the
BGP-learned inet.3 routes are not converted into inet6.3. I thought that
"ipv6-tunneling" knob would magically convert inet.3 routes into inet6.3
routes but as you said earlier, it seems to do so only for LDP/RSVP
learned routes...

[Tobias Heister]

Dear All,

First of all thank you for all message on and off list. I think I got this working now. I have simplified the topology (but it works on the initial topology as well)

R1 - R2 - R3 - R4 - R5

rsvp full mesh on R1, R2 and R3
ldp full mesh on R3, R4 and R5
we want to establish a bgp signaled lsp from R1 to R5 and use it for 6PE or any kind of VPN

There were a few points I was missing or at least was not totaly clear about.

1. R3 as a route reflector must have family inet6 enabled on its "core" facing interfaces as it terminates 6PE tunnels and receives packets labeled with implicit ipv6 null label (label 2) and therefore must be able to process ipv6 packets after popping the label

2. You do not need to add the "rib-groups" knob to the family inet6 labeled unicast statement it is the "rib" knob. Unfortunately the statement is hidden in my version (10.4R8) or at least does not autocomplete so I did not notice my error right away.

set family inet6 labeled-unicast rib?

rib-group Routing table group

=> only completion is rib-group

set family inet6 labeled-unicast rib inet.3

=> does work and gives us

family inet6 {
labeled-unicast {
rib {
inet.3;
}
explicit-null;
}
}

Also note that it reads "rib inet.3" although it places the routes in inet6.3 rib and also is referenced as inet6.3 in the documentation i posted in my first mail [1]

3. I got the RR to announce the mapped v4 (::ffff:x.x.x..x) prefixes which were locally generated via labeled bgp to the RR clients. Unfortunately I was not able to use this next-hops for forwarding (even with nex-hop self policy in the edges). All Routing and forwarding entries looked fine across all routers but I could not get traffic across.

What I did instead was assigning an inet6 loopback manually to R1 and R5 and advertising these loopbacks via inet6 labeled-unicast. In the next step I applied a policy to change the next hop for all inet6 prefixe to map to this loopbacks and everything was working fine.

As R1/R5 were (for some reason) not generating the mapped loopback on their own my guess is that it just does not work in a labeled bgp scenario. Also this is what is done in the config example in scaling with labeled bgp guide [1]

4. Unfortunately there is no inet6 equivalent to icmp-tunneling for mpls. These makes troubleshooting a bit hard as you have no clue where a packet might get stuck while going through the mpls cloud.

So putting it all together i now use the following configuration. With this configuration i can now use the bgp signaled lsp for plain IP, 6PE and any kind of VPNs (I have tested L2VPN, L2circuit, L3VPN and VPLS)

## R1/R5

# bgp
group ibgp {
type internal;
local-address R1/R5
import prefer-inet.3;
family inet {
unicast;
labeled-unicast {
rib {
inet.3;
}
}
}
family inet6 {
unicast;
labeled-unicast {
rib {
inet.3;
}
explicit-null;
}
}
export [ export-inet3 nhs nhs6 ];
neighbor R3
}

#policy
policy-statement export-inet3 {
term one {
from {
protocol direct;
route-filter 0.0.0.0/0 prefix-length-range /32-/32;
}
then accept;
}
term two {
from {
protocol direct;
route-filter ::/0 prefix-length-range /128-/128;
}
then accept;
}
}
policy-statement nhs {
then {
next-hop self;
}
}
policy-statement nhs6 {
from {
family inet6;
rib inet6.0;
}
then {
next-hop inet6_loopback_of_R1/R3
}
}
policy-statement prefer-inet.3 {
term inet {
from {
rib inet.3;
route-filter 0.0.0.0/0 prefix-length-range /32-/32;
}
then {
preference 9;
accept;
}
}
term inet6 {
from {
rib inet6.3;
route-filter ::/0 prefix-length-range /128-/128;
}
then {
preference 9;
accept;
}
}
}
policy-statement only-lo {
term one {
from interface lo0.5;
then accept;
}
term two {
to rib inet.3;
then reject;
}
term three {
to rib inet6.3;
then reject;
}
term four {
to rib inet6.0;
then accept;
}
term five {
to rib inet6.0;
then accept;
}
}

#routing options
interface-routes {
rib-group {
inet interfaces;
inet6 v6-interfaces;
}
}
rib-groups {
interfaces {
import-rib [ inet.0 inet.3 ];
import-policy only-lo;
}
v6-interfaces {
import-rib [ inet6.0 inet6.3 ];
import-policy only-lo;
}
}


##R3

#bgp
group ibgp {
type internal;
local-address R3
family inet {
unicast;
labeled-unicast {
rib {
inet.3;
}
}
}
family inet6 {
unicast;
labeled-unicast {
rib {
inet.3;
}
explicit-null;
}
}
export nhs;
cluster R3
neighbor R1
neighbor R5
}
}

#policy

policy-statement nhs {
term one {
from {
rib inet.3;
route-filter 0.0.0.0/0 prefix-length-range /32-/32;
}
then {
next-hop self;
accept;
}
}
term two {
from {
rib inet6.3;
route-filter ::/0 prefix-length-range /128-/128;
}
then {
next-hop self;
accept;
}
}
}

##R2/R4

only mpls + igp config no need to add anything ipv6 or bgp related

[1] http://www.juniper.net/us/en/local/pdf/design-guides/8020013-en.pdf