[j-nsp] KB20870 workaround creates problems with Hub and Spoke downstream hubs?

[Sebastian Wiesinger]

Hi,

we configured the workaround mentioned in KB20870 to prevent unwanted
VPN BGP session flaps when configuring eBGP/route-reflector clients. A
problem we noticed is that when using a Hub&Spoke hub on the affected
router and when a downstream hub is used as well, it seems that the
downstream hub stops exporting any VRF routes to other PEs.

Has anyone else noticed this and maybe even have a workaround? We'll
probably try and replicate it in the lab, but it looks strange. This
occured with JunOS 13.3 and 16.1.

Background for KB20870:
https://kb.juniper.net/InfoCenter/index?page=content&id=KB20870
https://www.juniper.net/documentation/en_US/junos/topics/example/bgp-vpn-session-flap-prevention.html

Downstream Hub:
https://www.juniper.net/documentation/en_US/junos/topics/example/vpn-hub-spoke-topologies-one-interface.html

Regards

Sebastian

[Olivier Benghozi]

Hi Sebastian,

This is an old workaround by the way.
Simpler workaround: use advertise-from-main-vpn-tables knob available since 12.3 (required if you have NSR anyway):

https://www.juniper.net/documentation/en_US/junos/topics/reference/configuration-statement/advertise-from-main-vpn-table-edit-protocols-bgp.html <https://www.juniper.net/documentation/en_US/junos/topics/reference/configuration-statement/advertise-from-main-vpn-table-edit-protocols-bgp.html>
https://www.juniper.net/documentation/en_US/junos/topics/reference/requirements/nsr-system-requirements.html#nsr-bgp <https://www.juniper.net/documentation/en_US/junos/topics/reference/requirements/nsr-system-requirements.html#nsr-bgp>
And NSP-J archives https://lists.gt.net/nsp/juniper/56263#56263 <https://lists.gt.net/nsp/juniper/56263#56263>

So you might add this knob and remove the fantom session.


Now, if you see some VPN routes no longer advertised toward other PEs, it probably means that your VRF export policies must be modified (and of course the doc is silent about that).
What we observed is that you can no longer rely on the classic "routing policies accept BGP routes by default", translated here to "(e)BGP routes are exported by default to other i-MP-BGP neighbors", probably since they are now exported to another table bgp.l3vpn.0, not directly to other neighbors.
So one must instead explicitly "accept" BGP routes in the VRF export policies (in addition to setting RT ext-community).


Olivier

Post by Sebastian Wiesinger
we configured the workaround mentioned in KB20870 to prevent unwanted
VPN BGP session flaps when configuring eBGP/route-reflector clients. A
problem we noticed is that when using a Hub&Spoke hub on the affected
router and when a downstream hub is used as well, it seems that the
downstream hub stops exporting any VRF routes to other PEs.
Has anyone else noticed this and maybe even have a workaround? We'll
probably try and replicate it in the lab, but it looks strange. This
occured with JunOS 13.3 and 16.1.
https://kb.juniper.net/InfoCenter/index?page=content&id=KB20870
https://www.juniper.net/documentation/en_US/junos/topics/example/bgp-vpn-session-flap-prevention.html
https://www.juniper.net/documentation/en_US/junos/topics/example/vpn-hub-spoke-topologies-one-interface.html

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Olivier Benghozi]

Note that our observations (and found workaround) were when using sub-policies applied within export-policies.

Post by Olivier Benghozi
Now, if you see some VPN routes no longer advertised toward other PEs, it probably means that your VRF export policies must be modified (and of course the doc is silent about that).
What we observed is that you can no longer rely on the classic "routing policies accept BGP routes by default", translated here to "(e)BGP routes are exported by default to other i-MP-BGP neighbors", probably since they are now exported to another table bgp.l3vpn.0, not directly to other neighbors.
So one must instead explicitly "accept" BGP routes in the VRF export policies (in addition to setting RT ext-community).

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Sebastian Wiesinger]

Post by Olivier Benghozi
Hi Sebastian,
This is an old workaround by the way.
Simpler workaround: use advertise-from-main-vpn-tables knob

So, I'm still stuck on this.

When using 'advertise-from-main-vpn-tables' Hub&Spoke VRFs with a
downstream hub[1] break.

In my mind the problem is that the downstream hub instance does not
advertise the hub routes to the bgp.l3vpn.0 table. Route
redistribution should work like this:

a) without advertise-from-main-vpn-tables

[Hub instance] -> [Downstream hub instance] -> MP-BGP neighbors

This *works*



b) with advertise-from-main-vpn-tables

[Hub instance] -> [Downstream hub instance] -XXXX-> [bgp.l3vpn.0] -> MP-BGP neighbors

And there it breaks. Routes from the hub instance that get import into
the downstream hub instance are not exported to the bgp.l3vpn.0 table
and thus do not get advertised to other MP-BGP neighbors.

I tried various options with auto-export and explicit rib-groups but I
can't find any working scenario.

If anyone has a working config for this please let me know.

Regards

Sebastian

[1] https://www.juniper.net/documentation/en_US/junos/topics/example/vpn-hub-spoke-topologies-one-interface.html

[Tobias Heister]

Hi,

Post by Sebastian Wiesinger
So, I'm still stuck on this.
When using 'advertise-from-main-vpn-tables' Hub&Spoke VRFs with a
downstream hub[1] break.
In my mind the problem is that the downstream hub instance does not
advertise the hub routes to the bgp.l3vpn.0 table. Route
a) without advertise-from-main-vpn-tables
[Hub instance] -> [Downstream hub instance] -> MP-BGP neighbors
This *works*
b) with advertise-from-main-vpn-tables
[Hub instance] -> [Downstream hub instance] -XXXX-> [bgp.l3vpn.0] -> MP-BGP neighbors
And there it breaks. Routes from the hub instance that get import into
the downstream hub instance are not exported to the bgp.l3vpn.0 table
and thus do not get advertised to other MP-BGP neighbors.
I tried various options with auto-export and explicit rib-groups but I
can't find any working scenario.
If anyone has a working config for this please let me know.

I had similiar problems a while ago when we tried to leak routes from inet.0 into a VPN Instance. This works fine on a router which is not a RR (so no bgp.l3vpn.0). As soon as you add RR to that scenario it will no longer announce the leaked routes via L3VPN and/or leak/export them to bgp.l3vpn.0

The reason for it not working is that routes are only allowed to be "leaked" once in Junos architecture. In Scenario b they would need to be exported/leaked twice before being send out to L3VPN peers.

Our workaround back in the day was a BGP Session between inet.0 and vpn.inet.0 via lt- interface + BGP Policy to rewrite next-hop directly to inet.0. This way the routes are exchanged via the lt- but forwarding is done directly. Not sure if that is feasible/applicable for your scenario.

[a***@netconsultings.com]

Of Sebastian Wiesinger
Sent: Tuesday, May 29, 2018 10:15 AM

Post by Olivier Benghozi
Hi Sebastian,
This is an old workaround by the way.
Simpler workaround: use advertise-from-main-vpn-tables knob available

So, I'm still stuck on this.
When using 'advertise-from-main-vpn-tables' Hub&Spoke VRFs with a
downstream hub[1] break.
In my mind the problem is that the downstream hub instance does not
advertise the hub routes to the bgp.l3vpn.0 table. Route redistribution
a) without advertise-from-main-vpn-tables
[Hub instance] -> [Downstream hub instance] -> MP-BGP neighbors
This *works*
b) with advertise-from-main-vpn-tables
[Hub instance] -> [Downstream hub instance] -XXXX-> [bgp.l3vpn.0] -> MP-
BGP neighbors
And there it breaks. Routes from the hub instance that get import into the
downstream hub instance are not exported to the bgp.l3vpn.0 table and thus
do not get advertised to other MP-BGP neighbors.
I tried various options with auto-export and explicit rib-groups but I

can't find

any working scenario.
If anyone has a working config for this please let me know.
Regards
Sebastian
[1]
https://www.juniper.net/documentation/en_US/junos/topics/example/vp
n-hub-spoke-topologies-one-interface.html
--

Theirs is this split horizon rule in VPNs,
If VPN-A leaks prefix X to VPN-B then VPN-B should not advertise this prefix
X to other PEs.
-because if you'd allow this to happen then VPN-A and VPN-B could
potentially both advertise prefix X -what would the receiving PEs think then
-did prefix X come from VPN-A or VPN-B? should I overwrite the RTs/label for
the prefix? As you can see it would create a mayhem at the receiving end.

Now, this Downstream HUB VRF feature with "no-vrf-advertise" allows you to
break this split-horizon rule if you pinky swear that VPN-A will never
advertise prefix X to any other PE.
This way then VPN-B would become the solely advertiser of prefix X to the
rest of the PEs, hence no loops are possible.
-although the feature description seems to suggest that one can leak VPN-A's
prefix into multiple local PE VPNs -which potentially leads to the same
problem outlined above.
-so hopefully there are controls in Junos allowing you to import routes from
VPN-A only to one downstream hub VRF.

Introducing Cisco style VPNs into the mix,
My understanding is that the Junos feature "advertise-from-main-vpn-tables"
-is how VPN routing works in all Cisco platforms.
I haven't yet played with this feature so don't know if it allows VPNs to
exchange prefixes with one another cisco-style i.e. via the MP-BGP table on
a common PE, or whether the junos-style direct inter-VRF route leaking is
maintained and used instead.
However if the former is the case and VPNs are allowed to exchange routes
between each other (a free trade) via the MP-BGP table, then I can see how
the controls of Downstream HUB feature might get circumvented -as the MP-BGP
table would not enforce any such rules as to which VRF can or can not leak a
given prefix to other VRFs -other than based on matching RTs.

And I think that's the reason why the "Downstream HUB VRF" feature is
disabled in presence of "advertise-from-main-vpn-tables" feature.
Just thinking out loud.

adam

netconsultings.com
::carrier-class solutions for the telecommunications industry::

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Olivier Benghozi]

I guess you have an explicit match for those routes in your VRF export policy for the downstream VRF instance ?

Post by Sebastian Wiesinger
b) with advertise-from-main-vpn-tables
[Hub instance] -> [Downstream hub instance] -XXXX-> [bgp.l3vpn.0] -> MP-BGP neighbors
And there it breaks. Routes from the hub instance that get import into
the downstream hub instance are not exported to the bgp.l3vpn.0 table
and thus do not get advertised to other MP-BGP neighbors.

_______________________________________________
juniper-nsp mailing list juniper-***@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

[Sebastian Wiesinger]

Post by Olivier Benghozi
I guess you have an explicit match for those routes in your VRF
export policy for the downstream VRF instance ?

I don't know what you mean by "explicit match" exactly, but we have an
vrf-export policy that matches these routes.

Regards

Sebastian